Proprietary formats and compatibility

Posted August 31, 2008 by fossilboy
Categories: GIS, Mapping, software, Uncategorized

Most of the questions I receive about map making have to do with proprietary software. That is, while some software seems “universal,” the truth is that files produced in one version are not compatible in a competing version. The success, therefore, is learning to minimize proprietary formats.

The obvious…

The simplest solution is to limit products that only save data in proprietary formats. The government-funded GPS satellite cluster is normally interpreted in a standardized, public format established by the US Coast Guard called “NMEA.” The GPS receiver will read the satellite information and transmit NMEA lines as ASCII strings. Most important is the summary line, or ‘fix data’ line, known as the GGA line. This line transmits the time, latitude, longitude, quality of fix, number of satellites, horizontal dilution of precision, altitude, and some other information. Because it is a simple ASCII line, it is easy to write software that seeks out this transmitted line and parses out the needed information. The most common commercial GPS receivers are produced by Garmin and Magellan. However, to keep customer’s cash, Garmin and Magellan use mapping software that relies on proprietary transmitted information so you cannot use other mapping software and you cannot use other brands of receivers. In summary, make sure your software can read NMEA format. In fairness, both Garmin and Magellan produce excellent products and more and more third party software can allow for Garmin and Magellan input. Read the fine print!

In a similar circumstance, it seems that every GIS program uses its own proprietary format. Rather than regurgitate a litany of acronyms, let’s just assume any product you use will have its own file format. While ESRI shape files (explained in another blog) are the most commonly shared, not all GIS programs allow for shape files. My advice: if you are not sure, save your data as a text (.txt) or comma-separated values (.csv) format. These are more universally read.

The subtle…

Have you ever been frustrated because you thought you were using a standard file format and the program would not read the file? For starters, have you received a .docx yet? I recently worked on an interesting problem with a colleague of mine that was a model for subtlety. Todd wanted to use SOLO Field on a Nomad Computer for collecting GIS data. His background map was a trimmed image from a much larger air photo. Both the original and trimmed map were saved in the standard geotiff (.tif) format which is read by SOLO Field. However, every time he loaded the map, the program would crash. We thought the problem was that the file was too big (~30 MB), but that wasn’t the case. It ended up that he used an ArcGIS product to trim the image and for some reason, the new image could not be read in a non-ESRI GIS. He went back and trimmed the original image in SOLO Office and everything worked smooth. Moral of the lesson: just because the file format is on the software list, doesn’t mean it will work.


Is ESRI the only way to go for field GIS?

Posted June 23, 2008 by fossilboy
Categories: GIS, Mapping, software

Tags: , ,

No. However, the ESRI shape-file format with the multitude of files is the standard for most industries and academia. So when you collect your data, make sure you can output as standard shape files, composed of the minimum triad of .shp, .sbx, and .dbf files. (The .prj is valuable if you have an assigned projection.) I should also chime in that ESRI continues to dominate the GIS market because they do make amazing products.

So why go with someone else? Try cost, ease of use, memory drain, and the fighting spirit to stick it to the man. Most software is essentially a trimmed down and specialized version of ArcGIS, tailored toward a specific industry. I have written previous blogs about the amazing and free MapWindow and also the very powerful Manifold GIS system. Of course, I continue to be amazed with the power of Google Earth…

Google Earth in the field? Yes!

…which leads me to today’s blog. Recently, I was doing field work in the Boundary Waters area of northern Minnesota. Without a doubt, the Boundary Waters is the most lovely mosquito and wood tick hatchery in the lower 48. But I digress. Our fearless field party got lost under my direction. After a while, all the ponds and mossy outcrops look the same. In desperation, I sent one of the folks (we will call him “CyberKnife”) back to the car to dig up my ToughBook and GPS receiver. I popped up Google Earth though we did not have an internet connection and was amazed by how much detail was stored in the cache. Years ago, I would use the program ExpertGPS for mapping because it had a deep cache, and I still find the program fantastic for flat, back maps and airphotos. I haven’t used Google Earth in the field because I didn’t think it would work. Well, it worked fantastic! The only issue was that the COM port assigned to the GPS was too high for Google Earth so that will need to be addressed.

OK, so what to use for collecting data in the field? The industry standard is still dominated by ESRI and ArcPad is there solution for data collecting. If you have a PC and want to plot lines or points on topo maps, I am still a fan of National Geographic TOPO. Need road, airphotos, and other layers? Well, jump on the Google Earth bandwagon. Once MapWindow allows for higher COM port numbers, I’ll recommend MapWindow for all ‘real’ GIS needs in the field.

(*As long as I can claim some bandwidth on the global web, I’d like to wish my pal Brian and the rest of the GIS crew on the edge of the northern California fires good luck. Soot-covered fireman get all the thanks on the local radio, but there are lots of other folks working long shifts behind the scenes.)

What is up with Bluetooth and why won’t my software run on new computers?

Posted May 13, 2008 by fossilboy
Categories: Bluetooth, Communication

Recently, I have had several conversations with folks about the changes in Bluetooth. More to the point, Bluetooth technology has changed over the past few years so that software written for the older model is not compatible with the new model. That means, the software you used on, say, an Recon computer a few years ago might not work on a new Recon running Windows Mobile 6 with integrated Bluetooth.

The nitty-gritty

“Bluetooth” is simply a wire-free way to connect devices. While most folks are familiar with Bluetooth headsets or maybe printers, I mostly deal with GPS receivers. In most cases, you can think of Bluetooth as a replacement for a USB or serial cable. However, Bluetooth devices can communicate with several other devices simultaneously through a ‘piconet.’

The latest version is 2.1 with Enhanced Data Rate (EDR). This version came out in July, 2007 and superseded version 2.0 with EDR. The older version had a completely different construction (known as a stack) and, consequently, different transmission rates (though the same 2.45 GHz frequency) and protocol. That is why software written for the older Bluetooth won’t work on the new version.

Regardless of the Bluetooth version, there are three classes of Bluetooth that vary by strength of signal and power consumption. Class 1 is the high-end model and has a range of 100 meters. Class 2 is what is found in most computers and has a range of 10 meters. Class 3 is not common, but is the lowest power radio and has a range of 1 meter.

Was the change necessary?

Yes. I loved my Recon with Windows Mobile 2003 but I was frustrated with the Socket Bluetooth card. Socket is a great company, but the technology was tenuous to me, as a user. In fact, I wrote back then that Bluetooth would die soon. Once again, I was wrong. Various applets and updates came out to make Bluetooth more stable but I found myself appreciating the serial cable for my devices. With the latest operating system change, I find the Bluetooth very stable and have finally gone wireless with my GPS devices. What to do? Change the software. Kind of like when they ditched unleaded gas. You could keep adding additive to your fuel or buckle down and replace the valve seats.

What does the logo represent?

It made no sense to me until I dug deeper into the roots of Bluetooth. Harald Bluetooth was a 10th century king of Denmark and Norwary who is noteworthy (at least in legend) for unifying tribes across Scandinavia. (Meanwhile, Hamlet whined about the castle and contemplated skulls while his sweetheart used poor judgement with herbs—but I digress.) The symbol is the Germanic runes for H and B forming a bind rune.

The Nomad goes on a fossil hunt!

Posted April 30, 2008 by fossilboy
Categories: Field analysis, GIS, Google Earth, GPS, Mapping, software

Tags: , , ,

Recently, three colleges converged in western Nevada for four days of field paleontology. Over thirty students participated in a series of field tasks that represented different kinds of field scientific work. The tasks ranged from classic description to modern collection techniques. For the latter, the students mapped a trilobite-rich area using a Nomad computer and SX Blue II sub-meter GPS from Walcott Scientific.

Students found the gear not too cumbersome to wear and use in the field.

Trilobites are extinct arthropods, related to modern insects, crabs, and lobsters. In the Cambrian period, about 500 million years ago, they ruled the seas. Deep water deposits of that age are rich in trilobite fossils. The field exercise was carried out in Esmeralda County, Nevada in glorious weather. The students divided into teams of four members and each team had a guide to the common trilobites. Then, they students scurried about the hills looking for fossil-rich zones. When they found a particularly rich area, the students logged their location with an SX Blue II sub-meter GPS that communicated to a TDS Nomad via Bluetooth.

Nevadia weeksi, common trilobite in the lower field area.

Data was collected as a point-based shape file in ArcPad, set against a topographic map downloaded from a state of Nevada GIS website. In addition to the location information, the students listed the dominant fossil genus and estimated the age. Both the Nomad and GPS worked without a hitch and there was plenty of battery life at the end of the project.

Back at the college, the data was transferred to MapWindow GIS and sent via Shape2Earth to GoogleEarth as a .kml file. This process took less then five minutes. Each student was then e-mailed the .kml file so they could see for themselves just how the fossil bugs were distributed in the field area.

This assignment, and similar ones I have done with my courses, really shows the students both how easy it is to collect data in the field, and how GIS adds value to a project. The Nomad proved an excellent tool in the Nevada desert with the bright screen and rugged design. While many researchers still use tablets and PCs in the field, the use of small PDA data collectors is more efficient on cost, weight, and battery life.

Quickie user guide to Windows Mobile

Posted April 18, 2008 by fossilboy
Categories: software

Tags: , ,

Your buddy / boyfriend / boss hands you a PDA for the first time and tells you to ‘figure it out.’ Where do you start? Can you understand your way around the computer? This short blog is meant to be a sort of triage to get you through a Windows Mobile device, like the Recon or Nomad.

It sure looks like Windows…

Windows Mobile look and feels like Windows XP even if the code below the surface is different. (Windows Mobile evolved from Win.CE so all the various operating systems share some similarities.) Many of the features, such as the Start button, Program list, menu buttons, etc., function in the same way so if you use a PC, it will be terra cognita. Here are the important differences:

1) When you click the X in the upper right corner, you usually only close the window, not the program like in Windows XP. You will need to close the program through a menu option or it will run in the background.

2) Use the File Explorer to find missing files, even when your files are on a CompactFlash or USB drive.

3) Most PDAs only have ‘USB out’ for synching to a desktop computer. That means you cannot plug in a USB device like a printer or flash (thumb) drive. The Nomad does have a ‘USB in’ option.

4) Can’t see the screen because it is dark? Try holding down the power button for a second or two and that should bring up the brightness. If you hold it down too long, you will enter a ‘soft reset mode’ which your probably don’t want to do.

My biggest advice…

Whenever you turn on a device, close a program, and just before turning off, go to Start>Settings>System>Memory>Running Programs to close all the programs running in the background. I just grabbed a Recon out of a drawer and checked and the following programs were running: ActiveSync, Internet Explorer, Wireless Manager, and File Explorer. Knowing our computers, I am surprised ‘Solitaire’ wasn’t running as well…

Finally, if you are using ActiveSync to send files back and forth, I recommend logging on as a “Guest” so you are not committed to a dedicated PC for file sharing.

Why can’t my Tablet-PC read my USB-connected GPS receiver?

Posted March 28, 2008 by fossilboy
Categories: Communication, GPS, software

Tags: , , , ,

I’ve written several times about COM ports and GPS receivers. Interestingly, in the last two days, I have spoken with four different folks about how this works, particular in regards to setting up a GPS receiver through a USB port on a tablet computer. It is valuable to understand how this works to avoid frustration in setting up software later.

What is a COM port?
A communication (COM) port streams data into the computer. Older computers had serial ports for this purpose. USB ports are not really COM ports, so you need to download a driver for the conversion. There are two facts about this process: you need a different driver for each different brand of GPS or other devices using the COM port, each USB port needs to be set-up as a COM port and each will be given a different port number. This COM port number will be used in any software that needs to connect to the GPS. (As a side note, you will also need to know the speed of the data stream from the GPS called the Baud rate.)

Setting up a COM port for an SX Blue II Sub-Meter GPS
The first step is to download the driver. I can’t post folders in the blog, but I put the driver on the Walcott Scientific website. This will comprise a folder with various .dll, security catalog, etc., files for the FTBI driver. The folder is zipped, so you need to unzip it. Then, place the folder somewhere you can find it, like the desktop. Now, power up the SX Blue II and plug it into the USB port you wish to use. The computer will open the “Found New Hardware Wizard.” No need for the internet, so click “No, not this time” and click “Next.” Then, choose “Install from a list or specific location (Advanced)” and “Next.” On this page, choose “Search for the best driver in these locations” and “Include this location in the search:” Use the Browse button to find the folder and click “OK” and “Next.” A warning box will appear that the USB Serial Converter has not passed the test. Click “Continue Anyway.” The driver is now installed and you will close the Wizard. But wait! Another Wizard will open and you will repeat the process. You’ve done it!

Finding the COM port
So, what is the COM port? You can use a ‘port sniffer’ like GPS Viewer which works pretty well for finding GPS devices. Alternatively, you can go to: Control Panel>Administrative Tools>Computer Management>Device Manager and click on the “Ports (COM & LPT)” to see the port number assigned to the SX Blue. This is the COM port you will use in your software. The SX Blue broadcasts at 9600 Baud, 8 stop bits, no parity.

Setting up an external Bluetooth GPS on a ToughBook

Posted February 29, 2008 by fossilboy
Categories: Bluetooth, Field analysis, GPS, Mapping

Tags: , , , ,

As I have written in a previous blog, I am not very pleased with the integrated GPS on the new Toughbooks. However, the Bluetooth is very stable and opens up a variety of GPS options. The following goes through the steps of setting up an external GPS on a Toughbook 19 running Windows XP Tablet. Other models should be similar; Vista has not been tested.

1) Turn on the GPS. As always, make sure the battery is charged!

2) On the Toughbook, open the SD card slot and flip on the Wireless toggle switch, Then, click on the Bluetooth logo on the bottom bar. A box will appear. Click “New Connection”. Here is an important step: use “Custom Mode” or else the computer will assign a COM port that may not work with some software! Click “Next.”

3) The computer will recognize the GPS. Highlight the name and click “Next.”

4) The computer will assign the GPS to a serial port. Click “Next.”

5) On the next screen, uncheck “Use default COM port.” Now, you can choose your own COM port. Many software packages only go up to COM 9 for GPS, so I recommend choosing COM 8 and “Auto Connect.” Click “Next.”

6) On the next screen you can select an icon if you wish. Click “Next” and you are done! The COM port you have chosen will be the same for all software you use. This also means that you cannot run two different software programs at the same time.

To save battery life in the field, you can turn off the WiFi while you are running Bluetooth. In the lower right corner of the bottom tool bar is an icon of a lightning bolt with a circle. Click it to bring up wireless options. Choose Wireless LAN OFF and Bluetooth ON. The 802.11a doesn’t matter.