Creating databases on a handheld field computer (PDA): Part 1

We all use databases daily. At a minimum, you are probably being added to a database somewhere everytime you get on the web. But I digress. In the field, we often want to store our information in databases. That is, we might have many features we want to record and each feature might have several variables. For instance, I recently helped some students who were monitoring bubbling mud springs near a volcano. At each spring, they recorded the latitude and longitude, water temperature, air temperature, pH, and CO2 levels. In database terms, each location would be a ‘record’ and the variables are ‘attributes’.

Many folks simply input their field data into Mobile Excel and then download it when they get back to the office. That is the easy way and you can save the file as a text file (.txt) or comma-separated file (.csv), if you wish. But what if you wanted some of the fields automated? Or you want to use drop-down lists to avoid ambiguity from spelling errors? In that case, you need a database program for ease of data entry. The problem is, they don’t exist “out of the box” for Pocket PC.

The bottom line is that you have to either create your own database or purchase someone else’s pre-figured field collection database. For the latter, many options exist (some are free) and many are based out of ESRI’s ArcPad GIS data collection software. Personally, I use and highly recommend SOLO Field (from Tripod Data Systems) because of its ease of use. There are lots more options out there and let me know which one you use.

To create your own database in Windows Mobile or Win.CE or Palm OS, you need to purchase software that allows you to easily create entry sheets for your handheld computer. The software will typically run on a PC and output the files directly to a PDA via a sync program. The two components are a table where the data is stored and a form for entering or manipulating the data. Add various command options and you can create some complex data forms. Most software will allow you to actively sync to your PC when linked, storing the data in Microsoft Access (on the PC) or some other database format.

These two images are screen shots of a simple program I wrote to store waypoints with a GPS connected to a Recon handheld. The top image shows the form for imputing the data and the bottom image shows part of the underlying table.

 

I use Visual CE from Syware for making databases and have been very happy with it. Most of the folks I work with, however, use PenDragon. Again, if you know of others, please let me know. In the next blog, I will show you an example I have created using Visual CE.

Easy GIS…for free!

I have a love / hate relationship with GIS (geographical information systems) software. I love what you can do with GIS and I hate that I am reduced to tears every time I try to use ArcGIS or its offshoots. Why is it so non-intuitive and difficult to use? As a professor, I want to create experiences where my students collect data in the field and then use a GIS to organize, analyze, and present their data. The problem is, you spend all the time explaining how it is used and very little time is left to do the project. (The experience reminds me of trying to play ‘Dungeons and Dragons’ in college, but that is another story.)

To summarize for the uninitiated, GIS is software that creates stackable map layers that contain different information, but are linked by the latitude and longitude of the map. For instance, you could have one map shows topography, another location of flowing rivers, one showing concentrations of nitrates in soil, and a fourth the locations of granite outcrops. Then you can use statistical methods to ask questions like, “Are the nitrates found near the granite?” or “What percentage of the rivers flow over nitrate areas?” The GIS allows you to then show the results by creating specialized maps, say, with river + nitrate areas highlighted within red circles. The possibilities are staggering, but the programs are very clumsy and difficult to use.

This is a photo of the sample .kml file I created from a shapefile in MapWindow. The initial data was a spreadsheet with latitude, longitude, fossil type, and carbon and oxygen isotope values. The icon design and color were chosen within Google Earth. Export as a .kml file was through Shape2Earth.

The industry standard is produced by ESRI. However, there are other GIS programs out there, such as Manifold, and a host of open-source options. Recently my friend Brian turned me on to MapWindow, an open-source GIS that is easy to use, not too memory intensive, and free. Plus, with some of the add-ons, you can incorporate comma-separated-values files right in the GIS so you don’t have to alter your Excel spreadsheet. Better still, one of the add-ons allows you to export your data to Google Earth as a .kml file. And you know my love for Google Earth!

In this view, I placed the cursor over one spot so you can see all the different data points pinned to that geographical reference. This allows the viewer to quickly see the types of data available.

The reason I am excited about this product is that I see it as a GIS program that students can use without any previous experience. Because it is free, they can download the program onto their home computers. As a teacher, you can write up fairly simple directions to get the students started on data manipulations, queries, etc.

By adding the GIS data to the shapefile before exporting as a .kml file, you can ‘tag’ your data points. This way, if you click the icon you can see all of the information that goes along with the point.

To demonstrate, I created a .kml file that shows the carbon isotope ratio of fossils collected in Colorado. (The data is fake, but based on a research project I am involved with). The different Google Earth file are separated by fossil type. Starting with the data in an Excel file, it took me less then 10 minutes to get the final product, and that includes changing the font and icon pattern. I cannot post a .kmz file to this blog, but if you want to see the finished product, send me an e-mail and I would be happy to send it out.

Why do I need a Passkey for my Bluetooth GPS receiver?

You too? Seems that everyone gets stuck on the ‘Passkey’ requirement for setting up a Bluetooth GPS. Here is the scoop to save you frustration. Firstly, keep in mind that Bluetooth is like a USB cable, only wireless. That means, each device can only connect to one computer (though one computer can receive multiple devices). For a GPS receiver, you need to establish a communication or COM port. Since many Bluetooth devices can be used to share private information, the option comes up when setting up a new device to establish a passkey, which is a password. Why would you want this for a GPS? You wouldn’t so you need to bypass the option. For my demonstration, I am going to use a Recon 400 rugged PDA with integrated Bluetooth, running Windows Mobile 2005 and an out-of-the-box GlobalSat Bluetooth GPS receiver. Your system should more or less be the same.

1) Turn on the GPS. Does it need to be on first? No, but you may as well start searching for satellites.

2) Turn on the computer. Then, make sure the Bluetooth communication is ON. In the menu, go to the Bluetooth Settings and make sure “Turn on Bluetooth” and “Make this device discoverable to other devices” are both checked.

3) Under the ‘Devices’ tab, click on setting up a ‘New Partnership.’ While you are waiting, save the blog so you don’t lose it like the last one.

4) Now, the new GPS should show in the box. Mine says “BT-GPS-35345E” with a question mark to the left. Let the games begin!

5) The next step is to highlight the device and click ‘Next.’ The next screen asks you for a Passkey to establish a secure connection. Don’t type anything, just click ‘Next.’

6) But the computer asks you again for a passkey. Don’t do anything and it will go away in about 10 seconds and you will be automatically sent to a new screen. Check the ‘Serial Port’ on that screen and ‘Finish.’ Of course, you aren’t really done.

7) Now you need to click the ‘COM Ports’ tab and choose (seemingly illogically) “New Outgoing Port.” Possible devices to occupy this port will appear, choose your GPS and ‘Next.’

On the next screen, you can pick an unused COM port and leave ‘Secure Connection’ unchecked. If all the available COM ports are used, you need to erase one of them. All done! This will remain the COM port even if you turn off the computer.

Open up a program to test it out. I chose to use the freeware “GPS Viewer” (downloaded from Walcott Scientific) and it popped right up when I chose the correct COM port. And I have a great signal (DOP 1.1; PDOP 2.2) from my desk with the blinds drawn on the window! That is why I love the GlobalSat GPS.

Postscript: Unfortunately, these are computers and bad things can happen. In the time it took me to spell check, the computer stopped seeing the GPS and I had to start over. It is working again but it took a few tries. (At least I am honest.)

WAAS, EGNOS, RTCM, RTK and a sea of acronyms!

Today, I received another question about the difference between differential GPS, WAAS, EGNOS, RTCM, RTK, and other “high end” tools that augment the standard GPS signal. In this blog, I will try to simplify the systems so that they are understandable.

Overview of GPS receiver function
The GPS receiver works because each satellite in the GPS system broadcasts a unique song-code (a bit too simplified; see RTK below). The receiver on Earth has the same codes programmed in. The computer compares the expected pattern to the received pattern and shifts the signal to align. The signal shift can then be converted to a distance, known as the ‘pseudorange.’ By using three or more satellites, the computer can locate where the distances meet on a projected surface of the Earth to get a position. Easy enough. However, there are lots of factors that can influence the reliability of the solution (hence, the ‘psuedo’ range). Some have to do with the position of the receiver. For example, being in a canyon will cause lots of signal bounce, or being under a tree could block the signal from a satellite. Most of the errors, though, are due to the condition of the satellites or atmospheric effects. To erase these extrinsic errors, you rely on a second GPS receiver that is in a fixed, well-located position, known as a “base”. The observed base position becomes offset from its real position because of these factors. Now, if your GPS (called a “rover”) is using the same satellites at the same time, you can apply the base error to correct your position. This involves a bit of calculus, but you can use software to do the calculations for you. Again, you need to know the position of the base, the satellites in use, and the time recorded for the position. The result is the calculation of a differentially-corrected GPS position or “DGPS.”

Differential GPS signal
There are several ways of obtaining a DGPS signal. You can rely on either a fixed base (or set of bases) or you can establish your own base. The next variable is whether the base signal is incorporated directly by the rover (“real time”) or if you use a separate computer to compare the base and rover data after you come in from the field.

In these panels, we see how base stations record the GPS errors, broadcast the information to centralized ground control, who bounces it to the two WAAS satellites. Your receiver can download this data from the WAAS satellites to make real time correction.

WAAS, EGNOS, MSAS, and OmniSTAR
The simplest way is to rely on a system where there is a large network of carefully located ground stations and several satellites. The ground stations calculate their offset and send the signal to a central processing facility which beams the information to one or more satellites who broadcast the signal out. For the acronym fanatics out there, this type of system is known as a satellite-based augmentation system (SBAS). Wide Area Augmentation System (WAAS) is used in North America and a similar system, EGNOS, is used in Europe, and the MSAS system covers Asia. There are also private systems, like OmniSTAR, that are used in parts of the world where the US-based GPS or Russian GLONASS are not available or if you need higher resolution. Most GPS receivers have the capability to read the WAAS satellites and apply the correction without any additional processing, obtaining accuracies in the several meter range. For driving a car or mapping at 1:24,000 scale, this is more than sufficient. Currently, there are 38 ground stations, known as “wide-area reference stations (WRS)” in the US, Canada, and Mexico and two satellites. The western satellite is PRN 135 and the eastern is PRN 138. This status changes more often then you think, so you can check the web to see current status of the system. The benefit is that the signal is free and has wide coverage, the downside is that you need to receive the data from a WAAS satellite and they are not always visible.

In RTCM, established beacons broadcast their position using RTCM language so you can make corrections based on the calculated differences.

RTCM
Another, unrelated system relies on carefully located beacons that broadcast information on their position so you can use the distance to more accurately locate the rover. The standard output was established by the Radio Technical Commission for Maritime Services and is called “RTCM SC-104.” Higher-end GPS receivers are able to receive the RTCM data from beacons and recalculate their position in real-time based on the distance. This usually requires the purchase of a separate beacon antenna to receive the data. A downside to RTCM usage is that the signal degrades the farther you are from the beacon source. However, there are more and more public and private fixed stations (beacons) coming online if you have the software to post-process the data. One of the most widely used systems is maintained by the U.S. Coast Guard, the CORS beacon system. In practice, if WAAS is available, it is more accurate than the beacon system.

Real-time kinematic systems utilize a base and rover who communicate via radios. The measured error at the static base station is used to correct the rover position.

RTK
For very accurate measurements (centimeter to millimeter), it is necessary to establish a close base station and compare offsets from the base station to your rover. The most common way is to use a Real-Time Kinematic (RTK) system. RTK systems rely on the carrier phase signal from the broadcasting satellites. Huh? The “song-code” I mentioned earlier runs at high frequency (known as coarse acquisition or C/A), but the carrier signal is carried on a frequency that is about 1,000 times faster, know as the L1 carrier phase. Why not just use the L1 then? You can’t because it isn’t very unique but more of a simple wave-form. Think of it as a way to fine tune the “song-code.” In an RTK system, you need two receivers. One is the base station and is usually established atop a bench mark. The other receiver acts as a rover. Both can adjust their position using the L1 signal but they need to communicate to get the real-time accuracy. This is accomplished using radios to broadcast and receive the data.

 

Useful Web Resources

http://www.navcen.uscg.gov/ US Coast Guard official site concerning beacons.

http://www.faa.gov/about/office_org/headquarters_offices/ato/service_units/techops/navservices/gnss/ The FAA website about the WAAS system.

http://www.omnistar.com/ The website for the OmniSTAR private GPS satellite system.

 

 

I am a Google Earth junkie

It has been a while since my last blog—I hope I am not breaking some sort of blog etiquette.  I’d like to use the usual excuses (busy classes, sick kids, the government), but the truth is I am addicted to Google Earth.   I like to mock people who waste hours in front of a television or who create elaborate virtual lives on the internet.  Now, I am reduced to their rank.  I sneak away to glimpse GE.  I make steamy cups of tea and pick a spot to search (latest was the Egyptian coast on the Red Sea for 45 minutes where I found an unmarked shipwreck).  

I even dream of GE.  One of my daughters walked by with a way-too-full diaper but I found a high-resolution area of ooid shoals near the Tongue of the Ocean!  Must…turn…off.

Now that I have admitted my shortcomings (one of the steps to recovery), let me tell why I have a new virtual love.  GE allows you view high resolution photos of the entire planet (and much of space now) that come to life in 3-D.  As a geologist, this allows me to quickly scan different environments for patterns, look for faults and folds on the Earth’s surface, and share the information with others.  Also, as there are ways to quickly upload stories and photos, you can become a member of a global community whose posts range from vacation photos to humbling stories of war.  The tools allow for a variety of explorations and many more are on their way from the Google team and independent developers who post their work online.  And it is all free!

What I would like to do in this blog is give a brief background to Google Earth.  Later, I will discuss the main tools useful to a field researcher, and give directions on how to share information.  This will be kept short with the idea that more detailed blogs will be posted in the future.

How Google Earth works

Google Earth is a program that overlays satellite (and sometimes aerial) photos onto an elevation profile for the planet.  The elevation information, known as a digital elevation model (DEM), is provided by NASA’s Shuttle Radar Topography Mission.  This can be as accurate as 10 meters in some locations, most areas are about 30 meters.  In some mountain areas, finer-scale DEM coverage leads to more accurate representation.  The global datum used is WGS84.  The base resolution is 15 meters per pixel, but many populated areas and other areas of interest are at 1 meter per pixel.  In a few places, like Las Vegas and Berlin, exceptionally high resolution of 0.15 meters per pixel is available.  Wow!  Some areas are lousy at 500 meters per pixel and other areas suffer from cloud cover or low light on water.  Still, the resolution is amazing.

Naturally, this leads to some concern from governments and privacy advocates and some areas, like Dick Cheney’s residence, are intentionally blurred.  The entire nation of Israel is limited to 2 meters / pixel or less. (So far, the rocks haven’t complained yet).

Next blog…tools of Google Earth.

Comparing sub-meter GPS accuracy

For the past year or so, I have been interested in sub-meter GPS receivers.  There are several on the market and they basically work in the same fashion.  I will write another blog later on how they work, but for now I want to focus on one particular aspect that is confusing to a lot of folks.  How good are the receivers?

To understand the answer, you need to know that all GPS work in the same way, using a satellite array to calculate the position of the receiver on Earth.  However, there are many factors that affect the location calculated by the receiver.  Some of these have to do with the satellites, some with the receiver, most with complexities in the Earth’s surface and atmosphere (and some errors are due to the mathematical representations of Earth positions).  The receiver is set to collect a number of positions and then present the average position.  The variables that are most important to the user are the accuracy of the average position and the precision, or ‘repeatability,’ of the collected positions.  These two factors are not the same!  A receiver might calculate the average position within 50 centimeters of the true position (good accuracy), but the spread of points might be quite dispersed (poor precision).  Alternatively, the average position might be many meters off of the true position (poorer accuracy), but each point might be within several centimeters of each other (great precision).  The accuracy can usually be shifted by choosing different models for the Earth’s surface in your software.  This is known as the ‘geoid’ and common ones in the United States are WGS84 and NAD27.  The bottom line?  The better the receiver, the better the precision.

So, how do the companies quantify the quality of the precision?  Here are some examples of similar real-time, sub-meter GPS receivers culled from data sheets available online.  The data are simplified and re-formatted for consistency.  I focused only on the differentially-corrected (using WAAS) accuracy of horizontal measurements.  All the data sheets have a footnote that these are ideal conditions with satellites near the horizon, clear of obstructions, etc.:

Geneq SX Blue II
DGPS Horizontal Accuracy:     < 60cm 2dRMS*

Topcon GMS2
DGPS Horizontal Accuracy:     < 50cm RMS

Magellan ProMark3
DGPS Horizontal Accuracy:     <100cm RMS

Trimble GeoXT
DGPS Horizontal Accuracy:     < 100cm RMS

 *Furthermore, the SX Blue II data sheet also lists < 30cm RMS and < 25cm CEP for DGPS horizontal accuracy.

 What does this mean?  Note there are three different benchmarks—RMS, 2dRMS, and CEP.  Each is a statistical measurement achieved by averaging the distances to each measured point from the average.  RMS is the most commonly used and means ‘root mean square.’  (HRMS means ‘horizontal RMS’ and is the value given above but I dropped the H as it is redundant.)  The RMS is calculated by taking the square root of the average of the squared errors along the X and Y axes.  The one-sigma level is 67% so that means that you can expect a 67% probability that your measurements will be within the distance stated.  2dRMS means “two times the RMS” or two-sigma.  Therefore, the distance stated will be achieved with a 95% probability.  That is significantly different!  Put in simplest terms, 2dRMS is likely 95% of the time and RMS is likely 67% of the time.  That is why the SX Blue claims <60cm 2dRMS and <30cm RMS.  Finally, CEP is the “circular error probable” and is a measure of the circle centered on the true location that collects 50% of the error distribution.  In practice, it is not as valuable or universally used as RMS or 2dRMS.

BOTTOM LINE:  Not all data sheets are the same so check if you are looking at RMS, 2dRMS, or CEP when comparing.  Based on the data sheets listed above, the SX Blue II is the best for single receiver, real-time differential correction.

What about post-processing or two-receiver systems?  Another day…

COM ports simplified

Communication ports, or COM ports, are one-directional routes for communication. That is, they create a path for GPS receivers and other devices to communicate with the computer. If you want the details on how they work, Google away! Here, let me tell you what you need to know in order to get your GPS to work.

First, any port can become a COM port. Bluetooth, CompactFlash, SDIO, and serial connections are already set-up as such. Generally, if you want to use a USB port, you need to “tell” the computer to make it a COM port by installing software provided by the GPS manufacturer. You plug in the GPS and then install the disk with the software. You need to do this for all USB ports you might use. Some GPS do this automatically.

Second, the COM port is only activated when a program requires the data. In other words, you turn on a mapping program and tell it to open the GPS (or start tracking, etc.) and a script opens the COM port that is assigned (see below).

Third, because you are sending information on the COM port, you can only use one program at a time. This is the usual point of frustration because you get a message saying “COM port in use.” If you see this, you need to turn off all the programs that might be communicating with the COM port. With Windows Mobile devices, you need to go to System > Settings > Memory and end the running programs. Interesting, Windows Mobile 2005 and 6.0 have an applet that allows you to set up a virtual COM port for the outgoing data so you can run multiple programs. Check out System > Settings > GPS.

Fourth, when establishing a COM port, either you or the computer has to assign a port number to use with the software. Some computers have dedicated ports, others do not. The simplest way to check is to use free software, such as GPS Viewer, that will scan the COM ports looking for the GPS. Note which COM port is being used, and then turn off the program.

Fifth, some computers will re-assign COM ports if the program is shut down. So, if you aren’t getting a signal, run a test to see which COM ports are in use. ArcPad seems to trigger this response, but I need to test more. In most cases, once you set-up a COM port, you do not need to check it again.

With all that said, I spent a half an hour trying to figure out which COM port was the serial port on my PDA. Never did figure it out, but it might be do to the GPS not being compatible. Sigh… At least it was a sunny day.

Feeling Blue…tooth

A few years ago, I predicted Bluetooth would die and pasture with video disks and the like. On the contrary, it is increasing in popularity as a type of connection between computers and computers and peripherals and phones, etc. I think of Bluetooth as a wireless USB cable, not as a type of ‘WiFi’ connection. So, you use it like you would a USB cable. And like USB, one device sends information and the other receives it. Of course, both devices have to have Bluetooth receivers. Because you are sending data, Bluetooth needs to be set-up through a COM port (*my next blog subject). Most computers will do this automatically or you will use a wizard to set up the relationship.

When I get calls about “my GPS won’t communicate via the Bluetooth” it is usually a COM port problem. Make sure both devices are on and Discoverable. On the reading computer, make note which COM port is used so you can set it in whatever software you are using.

What else? Passkey—don’t need it unless you want a false sense of security. If you can leave it blank, do. If you need to type something, I usually type “1”. You may need to type four zeros 0000. If you are linking a GPS, let the COM port be a Serial Port. On Windows Mobile 2005, you need to have the computer establish a New Outgoing Port. (Intuitively, you think you need a new incoming port, but that isn’t the case.)

Can one GPS be read by two computers? No. Again, think of Bluetooth like a USB cable, can’t split that either.

Finally, you should know there are two classes of Bluetooth (with more in the works). Class 1 is more expensive and gives a range of about 100 meters. Most computers have a Class 2 chip which is good for about 10 meters. Since Bluetooth runs on radio frequencies, you don’t have to be visible to both devices. However, you do need a well-charged battery as it is a bit of a drain and will affect the communication.

I finally entered the Blog world….

…and I don’t have a clue what I am doing! If you feel similarly, you are in the right spot. I started this blot on the advice of my friends who thought there was a place for me in the murky ethers of the Internet. In short, I am a professor of Geology who uses field computers, GPS, and GIS in my research, but I was never formerly trained in these products. (I am writing this first blog from a motel room on the northern California coast, getting ready for field work tomorrow at the beach.) I have spent a fair amount of time sorting out how things work and wading through the immense literature. In my role as tech adviser for Walcott Scientific, folks have responded enthusiastically to my clear explanations and helpful advice. I think this success is due to my teaching large undergraduate classes (which I love to do!). Well, it seemed natural to turn those answers into a blog to reach out to more folks. If you are bugged by a problem that doesn’t make sense, send me a note, and I will try to sort it out and post it on the blog. Enjoy!