Projections and Legacy Formats

Projections are used in maps for two main purposes:

· To provide a more natural looking map, and

· To enable measurements of areas and lengths in printed maps.

Projections can be extremely simple to use, with little knowledge required if one works exclusively within Manifold using digital maps saved in modern formats. Using Manifold, one can make pretty projections with a point and a click. It's fast and it's fun.

To project a map, use the Edit - Assign Projection command to choose whatever projection you like. The map will show its contents in that projection no matter what the "native" projections of the components it contains. To change the projection used in a drawing, image or surface permanently, open the component in its own window and use the Edit - Change Projection dialog to re-project it. If that's all you would like to do, read the Projections topic and then jump straight to Projecting a Map

The situation becomes considerably more complex when importing data into Manifold that was created using older GIS systems and saved in formats that are not well suited for publishing maps. Complexity also rears its ugly head when importing geographic images and surfaces from formats that do not save projection data. We can use such data in Manifold, but to do so we will have to learn more about projections and follow a more complex import process.

Most digital map data is published in unprojected form to assure the greatest possibility of accurate interchange between various GIS systems and to reduce hassles for users. Users can then cast the map into whatever projection is desired using the facilities of their GIS package. Occasionally one encounters digital map data published in projected form. Dealing with projected maps that are published in old formats can be a real nightmare.

If you must deal with maps in legacy formats such as ESRI .shp or AutoCAD .dxf you probably will have to learn a lot about projections and coordinate systems. At a minimum, you must learn enough to be able to tell whether the digital map being imported was saved as a projected or as an unprojected map. Because legacy formats are geographically mute when it comes to projection information, users must acquire a considerable level of understanding just to be able to tell if they are dealing with a projected or unprojected map.

The situation becomes even more complex if a legacy format is used to save maps using old-fashioned projections such as UTM. In such cases one can be pulled into a confusing sequence of tasks requiring deep technical knowledge of the inner workings of map projections cast within ancient GIS formats. Not only is it a conceptual mess, it requires learning a quasi-technical lexicon of obsolete words, such as "false easting" and "false northing." It is as if to purchase a hamburger at McDonald's one needed to learn the farming methods and jargon of 1940's cattlemen.

Learning about projections in detail requires learning about coordinates and numerous other technical topics. This is not easy for non-mathematical people. It is especially difficult if the overall concepts are muddied by the tactical hassles involved in importing data created in older software using older formats. For this reason, it is a good idea to begin by learning to use projections exclusively with maps from the Manifold DVD (start at the Projecting a Map topic and continue with the Projections Tutorial topic).

Once you understand how to use projections within Manifold without the complications of dealing with legacy formats or old-fashioned methods you can dive into extended reading on projections and coordinates topics, beginning with the Coordinates topic.

Older File Formats and Projections

Manifold saves maps in a project file together with all technical parameters required to manipulate the data regardless of what projection is used. One may combine data from such maps in a completely free-form way and Manifold will automatically make all adjustments necessary. In a modern system such as Manifold, everything will always appear in the expected latitude / longitude coordinates everyone understands are used to identify locations on the Earth's surface.

Older GIS software was not always so flexible, and formats used to create maps in older systems are often unable to store important information about the projection in their file formats. Part of the user-unfriendly nature of older software and formats is that the user is expected to manually keep track of important information such as projection parameters. Quite literally, the user is expected to make notes on paper or in some sort of readme.txt file that accompanies the digital map files so that the GIS can be told what projection parameters to use with that file.

In modern times we recognize that requiring users to manually keep track of which projection parameters were used to create a particular file is a formula for endless hassles and errors. However, older GIS formats that require this of users continue in service within organizations that produce many digital maps we would like to use. In at least one case (ESRI .shp format) a format unsuited for publication of projected maps continues in use as the main format behind the most widely distributed GIS software within US government and universities. In another case, a CAD format, AutoCAD .dxf, is widely used to publish both projected and unprojected maps even though .dxf is highly unsuited for geographic map publication of any kind.

When governments and universities publish maps in unprojected form using ESRI .shp or other old formats there is usually little harm done except for the ordinary limitations of old formats such as limited file names. Even the difficulties presented by AutoCAD .dxf usually involve no more than minor hassles when unprojected maps are involved. In contrast, major problems arise when formats unsuited to the saving of projected maps are used to publish maps in projected form.

When formats like .shp and .dxf are used to save maps in projected form, one must know technical details about the projections used because one must manually provide information the format is unable to save. This introduces another conceptual minefield for users, the need to know enough about the technical internal details of various projections to be able to recognize and write down the necessary key details. This is not always easy, since some projections are themselves relics from the dawn of time. The Stereographic projection, for example, was known to the ancient Egyptians.

Many maps have been created using projections that are reflections of the limitations of the technology of their day and not because they are the best projections to use for a given purpose or are the most user-friendly. The notorious Universal Transverse Mercator (UTM) system, for example, was created in part in recognition of the limits of computation when doing arithmetic with hand-cranked calculating machines. Even though UTM was created before desktop computers were imagined it persists into the present day to wreak havoc among GIS novices. Because many maps were created in such systems and because technologically static governments have adopted some obsolete systems (like UTM) as the basis for legal norms they persist as living fossils into modern times.

If we use digital maps from many different sources or if we wish to exchange mapping information with colleagues using legacy systems, sooner or later we will need to understand projections and the coordinates within them at a seriously detailed technical level. This will be very frustrating for the new user. At times, as with UTM, the bizarre technical limitations of a living fossil projection presented within a living fossil data format such as .dxf will try the technical patience of even professional cartographers. However, if we persist and gain the technical understanding required we will be able to decode ancient formats we find on contemporary government and university web sites. We will be able to get many free maps and we will then be able to combine map data from such maps with other maps. We will be able to communicate with organizations still stuck in the technological amber of an earlier day.

It must be emphasized that there is no reason why in modern times most users of a GIS system should need to know the internals of how projections and coordinates work. This is all a result of legacy data and legacy formats. Manifold provides a very rich set of tools for dealing with such relics when they are encountered, or for the expert usage of cartographers who desire direct control over all aspects of their data. However, knowledge of internals is only required when working with legacy map data and not because of anything intrinsic to Manifold.

Problems with Drawings from Legacy Formats

Failing to import a projected drawing correctly will lead to unpredictable effects when the drawing is used in a map or other usage within Manifold. If we import a projected drawing from a geographically unaware format such as .shp or .dxf we must tell Manifold the correct projection to use to interpret that data since the format does not save this important information.

Manifold will remind us of this need by always raising an info bar the first time a new component that was imported from a geographically unaware format is opened. The info bar reminds us to verify the projection assigned to that component.

Note: The rule of thumb used by Manifold to decide if a component has been imported from a format that does not store projection information is to simply raise the info bar whenever a new component's projection is Orthographic, the default used for imports from formats that do not provide coordinate information, or if the component has been imported in Latitude / Longitude with coordinate locations outside the expected range (+/- 90 latitude and +/- 180 longitude).

To import a projected drawing from a geographically unaware format:

1. Import the drawing with File - Import. Use default settings.

2. Open the drawing.

3. Use the Edit - Assign Projection dialog to specify the projection information that should be used.

The Edit - Assign Projection dialog allows us to manually provide the correct projection information into the coordinates properties. Manifold will then be able to make sense of the data just imported. The Edit - Assign Projection dialog is normally used only once immediately after importing the drawing to specify correct projection information that the drawing's original format was too stupid to provide. It is not used to re-project the drawing. To re-project the drawing, use the Edit - Change Projection dialog.

Do not confuse the use of a chosen projection view in a map window (via Edit - Assign Projection) with the use of Edit - Assign Projection in a drawing window to specify projection information missing from legacy formats. If a projected drawing is imported from, say, .shp format and you fail to tell Manifold the correct projection to use, that drawing has not yet been correctly imported. Subsequent use of the drawing in maps will cause bizarre effects, lengthy delays or other incorrect operation as the system attempts to deal with the drawing based on fundamentally inaccurate coordinates.

Also, do not confuse the one-time use of Edit - Assign Projection with the regular use of Edit - Change Projection. Changing the internal understanding of a drawing's coordinate numbers by using Edit - Assign Projection will not re-project the drawing. To re-project the component, use Edit - Change Projection.

Notes

The classic "ancient" formats are ESRI .shp and AutoCAD .dxf. Neither saves projection information as commonly used. Use these formats for unprojected data only. There is a recent hack to the .shp format that will save projection information; however, it is so recent it does not help with the great mass of legacy data already published in .shp format. If you have a choice when getting projected legacy data, use MapInfo .mid/.mif format. Mid/mif is a reasonably modern format that saves projection information within the map files. Avoid .dxf at all costs.

When importing from .shp format, even when importing an unprojected shapefile keep on the lookout for any information that tells you what datum was used in that file. Importing an unprojected shapefile will by default import the shapefile using Latitude / Longitude projection using the WGS 84 centroid. Such imports will be acceptable for many GIS purposes; however, for full accuracy one should find out what centroid / ellipsoid / datum (three words with slightly different meanings but often used as synonyms) was used for the shapefile being imported and use the Edit - Assign Projection dialog to specify it for the imported drawing.

The classic "living fossil" projections that confuse new users are UTM , State Plane Coordinate System and Gauss Kruger . If you have a choice between receiving projected data in these systems or receiving unprojected data, always choose the unprojected data.

If you have no choice but to receive data in old formats remember to stay alert for any "readme.txt" file or any other documentation that accompanies the data files. With luck, the author of the data will have provided some notes on projection parameters or other information used within the map.

If you are not sure whether or not a .shp format or other old format file is a projected or unprojected map, an easy way to find out is to import it using default settings for the importer. One can then overlay the imported drawing as a layer in a map with a layer imported from a "known good" drawing from the Manifold DVD. If the imported drawing appears where it should geographically, it almost certainly imported OK and was quite likely an unprojected map. If it does not appear in the correct geographic context then the original file most likely contained a projected map. In such cases, the projection parameters to be used with the file must be tracked down.

Using .shp for projected maps and .dxf for anything causes such hassles during interchange that one is tempted to say bad things about publishers who do such things. However, it is often the case that the publishers themselves are trapped by organizational politics into using inappropriate formats. In some cases they might not have enough time to convert data from old formats and are simply trying to get data out to the public for better or for worse. If you find yourself in this position as a publisher please do your best to publish unprojected data sets. If you must publish projected data sets, please try to provide clear documentation for the projections that were used.

In this topic we refer to "drawings" because most projected data sets saved in legacy formats are drawings. Images and surfaces are also occasionally found as projected data as well. For that matter, images are often encountered that are "whole Earth" images in latitude / longitude projection and published using non-geographic formats such as .jpg. See the Cookie Cutter a Large Image with Transfer Selection topic for an example of how one uses Edit - Assign Projection to complete the import of such images.

Some images are accompanied by "world files" that are said (by the uninformed) to contain full projection information allowing automatic import of georegistered images. Regrettably, that is not true: world files contain only partial projection information and still require manual intervention. See the Importing Images topic for a complete discussion.

The use of .prj files together with shapefiles (as a sort of machine-readable "readme" file) to specify projections is a recent development that, unfortunately, has come into play without any definitive standards being published to precisely specify how the .prj should code its contents. Although the use of .prj files works most of the time, it is something of a haphazard system. If a shapefile is accompanied by a .prj file Manifold will analyze the .prj to try to determine what projection and projection parameters are intended for the shapefile.

Manifold recognizes over 500 projection types used in .prj files and so will be able to extract the projection information in almost all cases, reading more different types of .prj than almost all other software packages that can read shapefiles. However, because .prj files do not use standardized nomenclature for projections it is possible that in some cases it will not be possible to read the desired projection even if the shapefile is accompanied by a .prj file. manifold.net is always on the lookout for .prj files containing previously-unknown standards to add to our "zoo" of strange examples. If you encounter a shapefile using a .prj that does not import into Manifold correctly, please contact tech support for instructions on how to FTP it to manifold.net for examination.