Back from the Brink:
the story of
the remarkable resurrection of the Canada Land Inventory data
This is a story of triumph. A story of courage and conviction, of facing and mastering odds. Not the highest mountain or the widest river, but odds just as daunting, and victory just as sweet. This is the story of how one of Canada's legacies was returned from the ashes to be accessible and useful today.
To understand the story, you will have to know something about high technology, about computers, and something called a Geographic Information System.
Long ago, before I was born, the ARDA project was started. At the time (1958), it was realized that in order to manage the land resources in our country, we would have to understand the characteristics of our land, particulary in terms of its capability for agriculture, forestry, and other land uses. A simple topographic map displays information about the presence of roads, hills, and forests, but not the inherent capability of the land. The major accomplishment of the ARDA project was the development of the Canada Land Inventory (CLI) concept, and the production of hundreds of maps showing the land divided into classes of potential for agriculture and 4 other themes. 7 principal classes were identified, and a process of mapping the southern portion of Canada was begun that ultimately employed hundreds of people, and took almost two decades to complete. To this day, land use zoning decisions take into account where prime agricultural land is located (Classes 1,2, and 3), using the concepts and terminology developed during the ARDA project, and in most cases, the CLI Agriculture maps which were printed during the 1960's and 70's.
Dominant CLI classification for Mapsheet 31F

This story might have ended here if it had not been for another development which was taking place at the same time. During the early 1960's, the Department of the Environment started to play with the idea of using computers to make maps, and most importantly, analyze mapped information using those computers. One of the principal driving forces behind the Canadian Geographic Information System (CGIS) was the idea that the CLI maps could be interpreted and analyzed in a myriad of ways if the information could be manipulated by computers.
This was a truly revolutionary idea, and from its ambitious beginnings, CGIS ultimately grew to contain the equivalent of thousands of maps. It also spawned an industry that today is worth billions of dollars, and technology that is used for everything from deciding on locations of hamburger joints to the dispatching ambulances and other emergency vehicles. In fact, GIS technology is now so commonplace that simple versions are even bundled with every copy of software such as Excel, and web mapping has made it available to the masses.
CGIS was a home grown software. Bill Gates was in public school when Environment Canada's programmers were busy developing software and functionality that was decades ahead of its time. CGIS was used to produce thousands of analyses and hundreds of government reports, and as more CLI maps were created, the database grew and grew. You see, CGIS was two things. It was electronic maps, but it was also the computer programs that allowed users to input, manipulate, analyze, and output those maps. For over two and a half decades, CGIS lead the way in geographic information technology. But all good things must come to an end, and so it was with CGIS. There were perhaps many reasons why CGIS began to languish, and was eventually decommissioned. Changes in technology were certainly one factor, shifting priorities in the Department of the Environment were definitely another. Whatever the reason, by the time I joined the Agriculture Canada in 1992 and began looking for CLI data, the CGIS system hadn't been run for a number of years.
The process that lead up to this situation can best be explained in the words of Vee Neimanis. When I phoned him in 1992, Vee answered the phone with his characteristic drawl. I met him in person a few weeks later, and his outrageous moustache completed the image of a western gunslinger. There were many people who saved the CLI data, but it was perhaps Vee, with his no-nonsense manner, that played the most crucial role. "When I look back at the demise of CGIS, I can't tell you when it happened. You see, no-one actually made the decision to scrap CGIS. It happened a little at a time. Reorganizations; budget cuts; over the years, they all played a role." By the late 1980s, CGIS was no longer being used. Vee was the last person responsible for the old CGIS, and at the end, Vee was left holding the bag. Almost literally. Boxes and boxes of tapes, and racks of documentation that was useless without a computer to run it on. Only a few computers were left in Ottawa that were capable of even reading the dusty old 9 track tapes let alone running all those programs. More than once he had been told to dispose of the entire pile, since space was in short supply. Vee grimly hung on to the bag.
My call in December 1992 must have been like reaching out with a straw to a drowning man. I could almost feel his fingers grasp me across the phone. He assured me the data was available, and useable, and he was convinced that we could get the system operational again and extract the data. Extract it, and put it in a modern GIS. I knew many people wanted the data, so after our face-to-face meeting, we bagan our first attempt to recover the data.
Joanne Boivert, a bright young programmer/analyst, was still reporting to Vee. She was one of the CGIS programmers in its twilight years, and seemed the perfect person for the job. We arranged to transfer the tapes to my ownership, and mount them on our old IBM mainframe. Coreen (one of my employees) and Joanne spend almost three months resurrecting the old programs. We were hoping to restore the old pipeline from the CGIS format to a more widely readable ARC/INFO Export format. We had every reason to hope for the best, since merely 5 years earlier, we had successfully converted all our old proprietary CanSIS databases into ARC/INFO, and my own programmers were confident that we could repeat our success.
It was not to be so. One thing after another went wrong. That fabulous security on the old IBM got in our way at every step. Although we owned the tapes, the computer recognized we were not dialing in from a registered phone number. We fixed that, but then the ownership of every program had to be changed. And every program we tried to run had to be debugged and recompiled because of the system changes that had taken place since the demise. Five years is a long time in which to forget arcane computer languages, and time was running out. Joanne was leaving the government and moving to Montreal. We had other work we were neglecting. The IBM gurus we depended on were moving or being reassigned, and the computer itself was due to be moth-balled within the year. The problems seemed insurmountable, and eventually we had to admit defeat when the time came for Joanne to move away. CGIS and all its data were languishing again, and its fate seemed more certain than ever. The only difference: I was now the proud owner of boxes and boxes of tapes and documentation.
It was almost two years later, in 1995, when I saw Vee again. He stopped by my office to introduce David Brown from the National Archives. Vee, obviously afraid that I would trash the tapes, had arranged that Dave would take over ownership of the system, and Dave was keen to get it running again. I set him straight. But things were moving again. We were aware of others that wanted the data, and were prepared to pay. Michael Bordt and Doug Trant of Statistics Canada, and Jean Thie, of the National Atlas (a former director of CGIS), and a score of others. We sat around the table, Michael Bordt tall and unassuming, with money in his pocket. Jean Thie, with his twinkling blue eyes and that white goatee that always reminds me of Colonel Sanders. And most importantly, Mike Comeau, tanned dark by his frequent contracts in the Philippines. Wiry and confident, Mike had been one of the principal CGIS programmers, and he was convinced we could get the data out. Not by resurrecting the old system, but by building a suite of programs that would take the data off the old tapes and convert them into our modern systems.
It was worth a try; it was worth the risk. We would have to find many thousands of dollars, but the odds were good, and the value of the data and the work was in the millions of dollars. This might be our last and best chance. We all chipped in. We all contributed something. Time, experience, expertise, enthusiasm, money. Dave and Mike tried to get the data off the tapes. They had to rebuild the indexes to even find out what was on each tape. Some tapes had lost information due to their lengthy storage. Other tapes broke when we tried to read them on the drives. Some tapes were missing. Fortunately, the CGIS programmers had seen fit to back everything up at least twice, and we were able to find virtually all the data.
During these many months, Mike and Dave worked in relative isolation. I might have given up hope, but occasionally I would hear something. They were having trouble with the drives. They had rebuilt the tape indexes. They were looking for tape 23. They had almost all the CLI data. They had all the CLI agriculture data. They were still looking for tape 23. Did we want the CLUMP data? Are we sure we didn't have tape 23? (Tape 23 turned up eventually... when we vacated a room after Coreen had retired). Eventually, after more than a year, the first step was completed. We now had the CLI data on a modern DAT tape. The data was safe, but still in the old format.
CGIS was a unique GIS system. In today's systems, the location of points, lines and areas is identified by a series of geographic coordinates, most commonly latitude and longitude, where the precise location of each point is described independently of its neighbors. In contrast, CGIS encoded each point as a relative offset (distance and direction) from the previous point. Furthermore, CGIS didn't deal in discrete tiles; instead, the entire country was coded as one enormous database. It was this format that Mike had to convert into coordinates that a modern system could incorporate.
Michael Bordt and his team from Statistics Canada became instrumental at this point. While Mike programmed the conversion, they imported the data into ARC/INFO. The programs were debugged, and a production system was developed. This process was much larger than anticipated, primarily because of glitches in the data. Eventually, the shapes of the maps were coming across correctly, but the attributes (the CLI class information) was being scrambled. Statistics Canada started to run out of money, but we were getting so close that I was happy to provide support from Agriculture and Agri-Food Canada. The first maps were coming out, and already Indian and Northern Affairs had a rush project that simply had to have access to the data.
Another year passed, and the project seemed to sputter along on shear grit. Mike contributed untold hours of programming for free. Temporary employees came and went. Michael Bordt moved on to new responsibilities, so did another lynchpin at Statistics Canada, Joe Filoso. Jean Thie, retired some time earlier, returned looking for the data as a consultant. The project continued. No official status. No steering committee. No ministerial directive. Just empowered employees and one private citizen determined to see it through.
Finally, June 18, 1998, I received a phone call from Mike. Would I like my own copy of the CLI dataset? Within hours, Mike arrives, with all of the CLI potential for Agriculture on one CD. The others themes are to follow. After he leaves, I look at the fragile piece of plastic in my hand. I pull it up on the screen. It is flawless. The analytical tools I have built in anticipation work perfectly. I realize that I am the first person in more than a decade to be able to use this information. And what a decade it has been. Computing power at my fingertips that those many CGIS programmers and analysts probably never could have dreamed of. I can do more analyses in a minute than their whole team could have done in a year. Analyses that could never be done if we didn't have the data in electronic form.
I look at the CD case open on my desk. Millions of dollars... countless hours of work by so many people, some of whom work with me to this day. Programmers spending almost their entire careers. All that work so near to oblivion. Sure, the paper copies survived, but who today can afford to analyze things using paper. The negatives that produced the paper were recycled years ago, and to digitize the data by hand would be an almost insurmountable task for us today. So many land use planning decisions have been made with this data. It has shaped our heritage.
Months pass, and ever so often I hear again from Mike. Would I like a copy of the CLI Potential for Forestry? CLI Potential for Wildlife? How about the CLUMP dataset? Most datasets I didn't want or need, but a few are within my sphere of responsibility, and so they followed the CLI into the NSDB. Another piece of the puzzle. Another legacy of knowledge left to us by a previous generation.
Whither CLI? The data is now in the public domain. When NRCAN started to distribute the data for free on their geogratis web site, it rapidly became their most popular product. I've restructured the data so that it can be used more easily in a GIS, and archived it in our National Soil DataBase. The updated database and documentation are put on the CanSIS web site, and NRCAN hosts the original data along with its documentation and simple interpetations of each map. The CLI is safe. For now.
How did we come so close to losing this information, and could it happen again? I have begun to realize just how very vulnerable our databases are, even in my own organization. Sure, we follow industry standards: regular backups, tapes stored off site, critical knowledge documented and replicated. But what nearly happened to the CGIS data could happen so easily again. When I think about it, I realize that this is the third time that I have been involved in rescuing some important, highly relevant, and expensive databases. Important data either lost or on the brink of extinction, and critical expertise which has retired or moved on.
Perhaps the only comfort is that when people spend a good portion of their lives building something, they are unwilling to let it go, and so many other people are capable fo recognizing things of value, and doing something about it. Thank you Vee. Thank you Jean, Dave, Mike, Michael, and all the others who contributed, secure in the knowledge that they were doing the right thing. It was a pleasure working with you.
PHS. Ottawa, 2000.