Most of the questions I receive about map making have to do with proprietary software. That is, while some software seems “universal,” the truth is that files produced in one version are not compatible in a competing version. The success, therefore, is learning to minimize proprietary formats.

The obvious…

The simplest solution is to limit products that only save data in proprietary formats. The government-funded GPS satellite cluster is normally interpreted in a standardized, public format established by the US Coast Guard called “NMEA.” The GPS receiver will read the satellite information and transmit NMEA lines as ASCII strings. Most important is the summary line, or ‘fix data’ line, known as the GGA line. This line transmits the time, latitude, longitude, quality of fix, number of satellites, horizontal dilution of precision, altitude, and some other information. Because it is a simple ASCII line, it is easy to write software that seeks out this transmitted line and parses out the needed information. The most common commercial GPS receivers are produced by Garmin and Magellan. However, to keep customer’s cash, Garmin and Magellan use mapping software that relies on proprietary transmitted information so you cannot use other mapping software and you cannot use other brands of receivers. In summary, make sure your software can read NMEA format. In fairness, both Garmin and Magellan produce excellent products and more and more third party software can allow for Garmin and Magellan input. Read the fine print!

In a similar circumstance, it seems that every GIS program uses its own proprietary format. Rather than regurgitate a litany of acronyms, let’s just assume any product you use will have its own file format. While ESRI shape files (explained in another blog) are the most commonly shared, not all GIS programs allow for shape files. My advice: if you are not sure, save your data as a text (.txt) or comma-separated values (.csv) format. These are more universally read.

The subtle…

Have you ever been frustrated because you thought you were using a standard file format and the program would not read the file? For starters, have you received a .docx yet? I recently worked on an interesting problem with a colleague of mine that was a model for subtlety. Todd wanted to use SOLO Field on a Nomad Computer for collecting GIS data. His background map was a trimmed image from a much larger air photo. Both the original and trimmed map were saved in the standard geotiff (.tif) format which is read by SOLO Field. However, every time he loaded the map, the program would crash. We thought the problem was that the file was too big (~30 MB), but that wasn’t the case. It ended up that he used an ArcGIS product to trim the image and for some reason, the new image could not be read in a non-ESRI GIS. He went back and trimmed the original image in SOLO Office and everything worked smooth. Moral of the lesson: just because the file format is on the software list, doesn’t mean it will work.

No. However, the ESRI shape-file format with the multitude of files is the standard for most industries and academia. So when you collect your data, make sure you can output as standard shape files, composed of the minimum triad of .shp, .sbx, and .dbf files. (The .prj is valuable if you have an assigned projection.) I should also chime in that ESRI continues to dominate the GIS market because they do make amazing products.

So why go with someone else? Try cost, ease of use, memory drain, and the fighting spirit to stick it to the man. Most software is essentially a trimmed down and specialized version of ArcGIS, tailored toward a specific industry. I have written previous blogs about the amazing and free MapWindow and also the very powerful Manifold GIS system. Of course, I continue to be amazed with the power of Google Earth…

Google Earth in the field? Yes!

…which leads me to today’s blog. Recently, I was doing field work in the Boundary Waters area of northern Minnesota. Without a doubt, the Boundary Waters is the most lovely mosquito and wood tick hatchery in the lower 48. But I digress. Our fearless field party got lost under my direction. After a while, all the ponds and mossy outcrops look the same. In desperation, I sent one of the folks (we will call him “CyberKnife”) back to the car to dig up my ToughBook and GPS receiver. I popped up Google Earth though we did not have an internet connection and was amazed by how much detail was stored in the cache. Years ago, I would use the program ExpertGPS for mapping because it had a deep cache, and I still find the program fantastic for flat, back maps and airphotos. I haven’t used Google Earth in the field because I didn’t think it would work. Well, it worked fantastic! The only issue was that the COM port assigned to the GPS was too high for Google Earth so that will need to be addressed.

OK, so what to use for collecting data in the field? The industry standard is still dominated by ESRI and ArcPad is there solution for data collecting. If you have a PC and want to plot lines or points on topo maps, I am still a fan of National Geographic TOPO. Need road, airphotos, and other layers? Well, jump on the Google Earth bandwagon. Once MapWindow allows for higher COM port numbers, I’ll recommend MapWindow for all ‘real’ GIS needs in the field.

(*As long as I can claim some bandwidth on the global web, I’d like to wish my pal Brian and the rest of the GIS crew on the edge of the northern California fires good luck. Soot-covered fireman get all the thanks on the local radio, but there are lots of other folks working long shifts behind the scenes.)

Recently, three colleges converged in western Nevada for four days of field paleontology. Over thirty students participated in a series of field tasks that represented different kinds of field scientific work. The tasks ranged from classic description to modern collection techniques. For the latter, the students mapped a trilobite-rich area using a Nomad computer and SX Blue II sub-meter GPS from Walcott Scientific.

Students found the gear not too cumbersome to wear and use in the field.

Trilobites are extinct arthropods, related to modern insects, crabs, and lobsters. In the Cambrian period, about 500 million years ago, they ruled the seas. Deep water deposits of that age are rich in trilobite fossils. The field exercise was carried out in Esmeralda County, Nevada in glorious weather. The students divided into teams of four members and each team had a guide to the common trilobites. Then, they students scurried about the hills looking for fossil-rich zones. When they found a particularly rich area, the students logged their location with an SX Blue II sub-meter GPS that communicated to a TDS Nomad via Bluetooth.

Nevadia weeksi, common trilobite in the lower field area.

Data was collected as a point-based shape file in ArcPad, set against a topographic map downloaded from a state of Nevada GIS website. In addition to the location information, the students listed the dominant fossil genus and estimated the age. Both the Nomad and GPS worked without a hitch and there was plenty of battery life at the end of the project.

Back at the college, the data was transferred to MapWindow GIS and sent via Shape2Earth to GoogleEarth as a .kml file. This process took less then five minutes. Each student was then e-mailed the .kml file so they could see for themselves just how the fossil bugs were distributed in the field area.

This assignment, and similar ones I have done with my courses, really shows the students both how easy it is to collect data in the field, and how GIS adds value to a project. The Nomad proved an excellent tool in the Nevada desert with the bright screen and rugged design. While many researchers still use tablets and PCs in the field, the use of small PDA data collectors is more efficient on cost, weight, and battery life.

Recently, ABC put out a news snippet about the “child killing” inaccuracies of GPS vehicle navigation units. Glossing over the semantically misleading title, the report did a decent job of quickly summarizing the obvious—navigational software is prone to errors. It is not that the receivers are inaccurate, but rather the GIS programming behind the directional software is not foolproof. Having driven sub-par vehicles all my life, I am well cautioned to not blindly trust ‘foolproof’ gauges like gas meters, speedometers, turn signal indicators, etc. I do trust billowing white steam as a statistically likely indicator that my car is overheating. So, where is the fault with the navigational systems and what can you do to lighten your chances of becoming a GPS child killer?

The various driving programs packaged by TomTom, Magellan, Garmin, DeLorme, and others, rely on a base map of roads. Already, there is going to be a spread of accuracy based on the detail of the map and the age of the data. Roads change; in some areas, a map can be outdated within a year. I live on a street that dead-ends against a creek and then shows up on the other side with the same name. It is not uncommon for folks to knock on our door wondering where the rest of the street lies. Some software allows for upgrades but it is up to you to stay on top of things. And, it is unlikely that daily, weekly, or monthly closures will appear. Secondly, in the better mapping packages, the roads are ‘weighted’ for traffic speed and other flow parameters. That way, you can mathematically deduce which is the best route even if it is not the shortest. We all know how this works whenever we try to explain to a friend or spouse that our way is “better.” Herein lies the problem. How many software packages are going to be more accurate than your knowledge, experience, and intuition for your neighborhood? Then why do you trust it for other neighborhoods you have never been to?

I blame the problem for the accidents and lost drivers on losing our ability to read a map and growing up in an increasingly virtual world. At some point in the Late Atarian, we evolved into a race that trusts virtual landscapes more than what is outside our window. Or, we just would rather inhabit that space. Why else would so many minivans and airplanes have TV sets instead of looking out the blessed window?

At this point, you may be wondering why I am writing about this on a fieldwork blog. Do you think the software for backcountry work doesn’t suffer the same shortcomings? In future blogs, I would like to explore how to improve the software we use for monitoring and recording our field work.

In the last blog, I gave an introduction to dilution of precision or DOP and explained when you need to know it and when you don’t. This time, I want to show you a very cool, free program to calculate your DOP values throughout the day. All you need to know is your latitude and longitude and day. The program is called “Planning” and is from Trimble. You can download it here.

To use this program, you need to first enter your position through File > Station. You can use the map feature to get ‘close’ but you really should use the correct latitude and longitude. On this screen, you also set the date and time and duration of observation. Then, you need the satellite almanac information. This is downloaded from the Trimble site as an ‘.alm’ file or you can use an ‘.ssf’ file.

Here we see a plot that shows the total DOP at 10 minute intervals for San Diego (where it is very lovely right now). Looks like 7:30 a.m. is the time to work.

That’s it! Now, play around with the different map features, it is surprisingly easy to navigate. Notice that you can use GPS (United States system) or Glonass (Russian system). WAAS is available but since those satellites don’t affect the DOP, they are not included in the almanac.

This cool screenshot shows the visibility of each satellite throughout the time period. You can turn satellites off and on to clean up the graph.

But wait…if the DOP is based on satellite positions, what happens if you cannot see a particular satellite because of mountains or buildings? Obviously, that satellite is no longer used in calculating the DOP. The “Planning” software doesn’t directly allow you to blank out areas of the sky (but you can turn off certain satellites). However, there is another great program that allows you to show the blocked view based on topography. Stay tuned…

One of the more frustrating tasks I deal with is figuring out how mapping programs import data. Most will allow for you to hook up a commercial GPS, like a Garmin, and work live or download saved waypoints. But what if you just wanted to bring in a set of known locations? You would think the software manufacturers would make this most obviously useful task easy. The problem is that there isn’t really a standard data format for storing waypoint information.

Underlying all of the mapping software is some sort of geographical reference system. Therefore, each of the data points must have longitude and latitude (or some other referencing). Next, the points will have some identifying tag, like the name or waypoint number. Then, there might be coded information about attributes like elevation or time of record. Finally, there could be information on how the point is displayed (color, icon, etc.). The software should be able to parse this information once it is input.

The logically way for this to work is as a text (.txt) or comma-separated-values (.csv) file. In this format, the data for each site (=record) exists as a line with different information bits separated by commas. The software uses the commas when it is reading the text to gain the information. However, many applications use a proprietary format and therein lies the rub. (I have noticed, many will accept the ESRI standard shape file, so this is an option.)

So, how do you do it? Here is my method: 1) First, figure out the acceptable geographic format. For example, do you need decimal-degrees or UTM or some other standard? 2) Then, within the program make a few random waypoints. 3) Download these points to a file. Typically, the software will download as a text file. 4) Open up this file and *format your data points exactly!* Make sure you have the same number of spaces and any column-header information. 5) Save your file in the same format as the one you downloaded. 6) Go for the input!

This is a bit cumbersome, but I have used it successfully in National Geographic TOPO! and other software. Soon, I will post my favorite software page and I will link some conversion software.

We all use databases daily. At a minimum, you are probably being added to a database somewhere everytime you get on the web. But I digress. In the field, we often want to store our information in databases. That is, we might have many features we want to record and each feature might have several variables. For instance, I recently helped some students who were monitoring bubbling mud springs near a volcano. At each spring, they recorded the latitude and longitude, water temperature, air temperature, pH, and CO2 levels. In database terms, each location would be a ‘record’ and the variables are ‘attributes’.

Many folks simply input their field data into Mobile Excel and then download it when they get back to the office. That is the easy way and you can save the file as a text file (.txt) or comma-separated file (.csv), if you wish. But what if you wanted some of the fields automated? Or you want to use drop-down lists to avoid ambiguity from spelling errors? In that case, you need a database program for ease of data entry. The problem is, they don’t exist “out of the box” for Pocket PC.

The bottom line is that you have to either create your own database or purchase someone else’s pre-figured field collection database. For the latter, many options exist (some are free) and many are based out of ESRI’s ArcPad GIS data collection software. Personally, I use and highly recommend SOLO Field (from Tripod Data Systems) because of its ease of use. There are lots more options out there and let me know which one you use.

To create your own database in Windows Mobile or Win.CE or Palm OS, you need to purchase software that allows you to easily create entry sheets for your handheld computer. The software will typically run on a PC and output the files directly to a PDA via a sync program. The two components are a table where the data is stored and a form for entering or manipulating the data. Add various command options and you can create some complex data forms. Most software will allow you to actively sync to your PC when linked, storing the data in Microsoft Access (on the PC) or some other database format.

These two images are screen shots of a simple program I wrote to store waypoints with a GPS connected to a Recon handheld. The top image shows the form for imputing the data and the bottom image shows part of the underlying table.

I use Visual CE from Syware for making databases and have been very happy with it. Most of the folks I work with, however, use PenDragon. Again, if you know of others, please let me know. In the next blog, I will show you an example I have created using Visual CE.

I have a love / hate relationship with GIS (geographical information systems) software. I love what you can do with GIS and I hate that I am reduced to tears every time I try to use ArcGIS or its offshoots. Why is it so non-intuitive and difficult to use? As a professor, I want to create experiences where my students collect data in the field and then use a GIS to organize, analyze, and present their data. The problem is, you spend all the time explaining how it is used and very little time is left to do the project. (The experience reminds me of trying to play ‘Dungeons and Dragons’ in college, but that is another story.)

To summarize for the uninitiated, GIS is software that creates stackable map layers that contain different information, but are linked by the latitude and longitude of the map. For instance, you could have one map shows topography, another location of flowing rivers, one showing concentrations of nitrates in soil, and a fourth the locations of granite outcrops. Then you can use statistical methods to ask questions like, “Are the nitrates found near the granite?” or “What percentage of the rivers flow over nitrate areas?” The GIS allows you to then show the results by creating specialized maps, say, with river + nitrate areas highlighted within red circles. The possibilities are staggering, but the programs are very clumsy and difficult to use.

This is a photo of the sample .kml file I created from a shapefile in MapWindow. The initial data was a spreadsheet with latitude, longitude, fossil type, and carbon and oxygen isotope values. The icon design and color were chosen within Google Earth. Export as a .kml file was through Shape2Earth.

The industry standard is produced by ESRI. However, there are other GIS programs out there, such as Manifold, and a host of open-source options. Recently my friend Brian turned me on to MapWindow, an open-source GIS that is easy to use, not too memory intensive, and free. Plus, with some of the add-ons, you can incorporate comma-separated-values files right in the GIS so you don’t have to alter your Excel spreadsheet. Better still, one of the add-ons allows you to export your data to Google Earth as a .kml file. And you know my love for Google Earth!

In this view, I placed the cursor over one spot so you can see all the different data points pinned to that geographical reference. This allows the viewer to quickly see the types of data available.

The reason I am excited about this product is that I see it as a GIS program that students can use without any previous experience. Because it is free, they can download the program onto their home computers. As a teacher, you can write up fairly simple directions to get the students started on data manipulations, queries, etc.

By adding the GIS data to the shapefile before exporting as a .kml file, you can ‘tag’ your data points. This way, if you click the icon you can see all of the information that goes along with the point.

To demonstrate, I created a .kml file that shows the carbon isotope ratio of fossils collected in Colorado. (The data is fake, but based on a research project I am involved with). The different Google Earth file are separated by fossil type. Starting with the data in an Excel file, it took me less then 10 minutes to get the final product, and that includes changing the font and icon pattern. I cannot post a .kmz file to this blog, but if you want to see the finished product, send me an e-mail and I would be happy to send it out.