Standardized Accuracy Tests for GPS Receivers

Dr. Timothy Stombaugh, Biosystems and Agricultural Engineering, tstomb@bae.uky.edu

The Global Positioning System (GPS) is one of the key technologies that have made precision agriculture (PA) possible. Research conducted as early as 1929 outlined the merits of spatially sampling field parameters and applying inputs at variable rates (Linsley and Bauer, 1929). The development of GPS over the last 2 decades is probably the key element that has made real-time spatial management possible.

There have been many GPS innovations in the last couple years. An example of one innovation that has had a significant impact on PA is the Wide Area Augmentation System (WAAS). Initiated by the Federal Aviation Administration, WAAS is a free satellite-based differential correction service that has facilitated development of very low cost receivers that have potential for use in PA.

The introduction of WAAS as well as rather bold accuracy claims from some GPS manufacturers have sparked interest in GPS testing and accuracy reporting methods. Most manufacturers report accuracy as a single number representing the results of a static test of the receiver. Nearly all PA field operations are dynamic, and there is currently no sufficient standardized test procedure for measuring dynamic GPS receiver accuracy. Consequently, there are many questions about the accuracy of the position information used to create maps or control variable rate application equipment. Dynamic accuracy issues are particularly pertinent in Kentucky agriculture where many fields are contoured or odd shaped.

The overall goals of this project are to develop an apparatus and a standardized procedure for evaluation of GPS receiver dynamic accuracy. This goal will be accomplished through the following specific objectives:

The accuracy of GPS receivers is dependant on a number of parameters including the inherent satellite signal accuracy, signal transmission errors, and receiver hardware and software limitations. The satellite signal error could be affected by factors such as satellite orbit perturbations or hardware health. The US Department of Defense, which oversees the maintenance of the GPS system, has committed to a minimum level of service availability, reliability, and accuracy (Department of Defense, 2001). These minimum commitments include a guaranteed achievable position accuracy of no worse than 36 m horizontal and 77 m vertical (95% confidence) anywhere on the globe. This means that the most inexpensive GPS receiver should be able to achieve at least these minimum position accuracies. The receivers used in PA utilize a number of hardware and software enhancements (including differential correction) to achieve accuracies close to 1 m or less.

Several considerations exist that must be addressed when evaluating GPS position accuracy. The first consideration is appropriate error reporting method. Most manufacturers report horizontal receiver accuracy as a single distance number (e.g. 1 m). That number usually represents the combined horizontal (circular) accuracy of the receiver, and is usually based on one of a number of statistical inferences or confidence intervals (Van Diggelen, 1998; Table 1). To illustrate the interpretation of these inferences, consider a receiver with a reported 1 m Circular Error Probable (CEP) accuracy. The implication is that the error in the receiver output will be less than 1 m at least 50% of the time. To compare receivers from manufacturers who use different reporting methods, the user must perform a numerical conversion (Table 1).

Accuracy Description	Probability (%)	Conversion Factors
Accuracy Description	Probability (%)	CEP	rms	R95	2drms
CEP – Circular Error Probable	50	1	1.2	2.1	2.4
rms or 1 Sigma – 1 standard deviation	63-68		1	1.7	2
R95 – horizontal 95% accuracy	95			1	1.2
2drms or 2 Sigma – 2 standard deviations	95-98				1

The second main consideration when comparing receivers is the way in which the test data were obtained. Most accuracy specifications are computed from a 24-hour stationary test of the GPS receiver. Often there is little information given about test conditions such as receiver configurations, satellite constellation configuration, signal strength or atmospheric conditions, all of which can affect receiver performance.

A third major consideration is the way that the position error is computed. Is the error specification based on the predictable (absolute) or repeatable (relative) accuracy (Fig 1)? The predictable or absolute accuracy represents how well the position solution reflects the true position of the antenna based on a survey relative to an accepted coordinate system. The repeatable accuracy represents how well the position solution relates to previous solutions at the same location (Department of Defense, 1995). In other words, the repeatable accuracy represents the error between a particular position solution and the average of many positions measured while the receiver was at the same location. Predictable error is measured relative to the true absolute location of the antenna. Repeatable error of GPS receivers is generally much smaller than predictable error, especially for shorter test periods; therefore, a receiver with small relative accuracy specification can appear to be better than another receiver with a slightly larger predictable error when the opposite is true. In short, it would be desirable if all manufacturers who market to the PA industry would utilize a single static test and reporting standard.

Figure 1. Typical receiver performance illustrating predictable (a) and repeatable (b) accuracy.

Because almost all GPS receivers used in PA are placed on moving field machinery, the question that still remains is whether a static accuracy test is indicative of the dynamic performance of a receiver. The project investigators have conducted several preliminary static and dynamic (constant acceleration) receiver tests using a 7.5-m radius rotating test apparatus (Fig. 2). In one comparison of comparable GPS receivers, receiver A exhibited very good static accuracy but rather poor dynamic accuracy while receiver B performed well in the dynamic test but poorly in the static test (Fig. 3). The obvious conclusion is that static performance is not necessarily indicative of dynamic performance.

One possible explanation of degraded dynamic performance is data filtering. A data filter is a software algorithm that attempts to filter out or correct data that are judged to be in error (Franklin et al., 1994). In simplest terms, a data filter considers the most recent computed positions to estimate where the next position should fall. If the new point does not fall where expected, the algorithm would assume that it is in error and adjust or scale the new location accordingly. A data filter can greatly increase static accuracy performance, but can hinder dynamic performance. One place where filter effects become especially evident is when a moving receiver makes an abrupt turn (Fig. 4). The filter expects the receiver to be continuing on the original path and applies “corrections” to the first several points after the turn. Depending on the nature of the data filter, it may take many data points for the algorithm to settle to the desired path once again. Since many farm fields in Kentucky do not have nice straight rows, data filtering and dynamic receiver performance have a significant impact on data collection and machine control accuracy.

The Institute of Navigation (ION) is the only known organization that has published standards for GPS receiver testing. Their document, which heavily references the original Department of Defense GPS Specification documents, outlines detailed procedures for static GPS accuracy tests. Since the ION serves a very broad clientele representing space, military, land and marine applications, their standard outlines only a skeletal framework for dynamic test standards. Specific dynamic test conditions are left to the discretion of each sub-discipline. No detailed procedures currently exist for agricultural or similar applications, and such a standard is needed.

There are several different approaches being pursued to evaluate the dynamic performance of DGPS receivers. The aforementioned rotation device developed by the project investigators subjects receivers to a constant acceleration. While this test may be quite effective in exposing data filtering effects, it may not be representative of typical agricultural field operating patterns. Other researchers have established a series of georeferenced, marked paths through an open field (Sullivan and Ehsani, 2002). They mount the GPS receiver on an agricultural tractor and drive it through the marked paths. This approach closely mimics typical field operating patterns, but the results are biased by the steering accuracy of the driver. Another approach is to mount several receivers and one highly accurate Real Time Kinematic (RTK) GPS receiver on one large machine and drive it through a series of maneuvers (Han, 2002). This approach also subjects receivers to actual field operating conditions, but the results rely on the performance of the RTK GPS receiver rather than a repeatable path. Researchers at Kansas State University have gained permission to conduct studies on a section of abandoned railroad track. The advantage of the railroad track is that it establishes a highly repeatable path. Unfortunately, it does not necessarily simulate typical agricultural maneuvers, and the test cannot be easily repeated at other locations.

To be accepted and used, a GPS test standard must be relatively easy and inexpensive to implement, yet be robust enough to provide an equitable evaluation and comparison of GPS equipment. An ideal test facility would be comprised of a large, fixed track system that would include every typical operating pattern that an agricultural machine might encounter. This type of facility would be extremely large, expensive, and difficult to maintain. A more realistic test facility would establish a simplified test pattern that would be easier to implement yet still provide an equitable evaluation of receiver performance under conditions relevant to agricultural field operations. The three key performance parameters that must be addressed are parallel passes, headland turns, and curve or contour following.

The first task in completion of this project will be to design and build a dynamic test facility. The investigators envision two separate devices that will comprise the facility. The aforementioned rotation test apparatus provides a good evaluation of the extended curve following ability of a receiver by subjecting it to a constant acceleration. The current apparatus will be modified slightly by increasing the radius of the path to more closely represent field-scale curve following and by providing the capability to test multiple receivers simultaneously.

The second test device will be a simple track system in the shape of a racetrack oval. This track will emulate parallel tracking on straight paths as well as headland turn operations. The track will be constructed with an I-beam or similar structural material and mounted on a gravel or concrete foundation. A cart that rides on the track will carry the GPS receiver(s), power supply, and data collection equipment. A continuous cable that encircles the track will propel the cart at various speeds typical of agricultural field operations. The dimensions of the track will be approximately 60 m by 10 m.

Once the physical facility is developed, a comprehensive test procedure will be developed. This procedure will form the basis by which all receivers will be tested. The procedure will also specify a reporting standard so that users can easily find traceable information regarding the performance of a GPS receiver.

The primary outcomes of this project will be a standardized GPS test facility and reporting methodology. These outcomes will provide current PA adopters with a better understanding of the accuracy of their data collection and application equipment. Indirectly, this work will help GPS manufacturers establish performance benchmarks for receivers they plan to market to the PA industry.

The American Society of Agricultural Engineering (ASAE) is one of the primary standards development organizations for agriculture. They have initiated an effort to develop a dynamic test standard. The project investigators are leaders in that effort, and this proposed project will be the foundational work for this international standard effort.

To the end user, Kentucky producers, the realization of this work should provide equipment purchasers with impartial performance statistics that are applicable for comparing receiver performance that is consistent with the intended receiver use, that is agricultural field operations. Assuming the resulting ASAE GPS receiver test standard is adopted by industry, we hope to see manufacturers reporting performance data that is meaningful, and that can be used for comparison shopping, and to establish the value of improved receiver performance.

The primary product of this study will be a protocol for testing GPS receivers. There will be Extension-type publications describing how to properly interpret the test reports. The scientific results of this study will be disseminated through professional conference presentations and journal articles. It is also expected that when completed, the results of this investigation will be utilized as the basis for developing an engineering practice to be adopted by standards organizations such as the ASAE or ISO.