A Study on Optimal Design of Image Traverse ... - Semantic Scholar

A STUDY ON OPTIMAL DESIGN OF IMAGE TRAVERSE NETWORKS FOR MARS ROVER LOCALIZATION Kaichang Di Ron Li Department of Civil and Environmental Engineering and Geodetic Science The Ohio State University 470 Hitchcock Hall, 2070 Neil Avenue Columbus, OH 43210-1275 Tel: (614) 292-4303, Fax: (614) 292-2957 [email protected] [email protected] Larry H. Matthies Jet Propulsion Laboratory California Institute of Technology Mail Stop 125-209, 4800 Oak Grove Drive Pasadena, CA 91109 Tel: (818) 354-3722, Fax: (818) 393-4085 [email protected] Clark F. Olson University of Washington, Bothell Computing and Software Systems 18115 Campus Way NE, Box 358534 Bothell, WA 98011 [email protected]

ABSTRACT Rover localization accuracy is critical for Mars surface exploration missions. It is especially crucial for future longrange rover missions, such as MER (Mars Exploration Rover) 2003. The geometry of the rover image traverse network is one of the most important factors to achieving high accuracy. A systematic study of geometric image traverse design was carried out. Several traverses of side looking and forward-backward looking rover image networks were simulated. Factors that affect the rover localization accuracy were examined, including traverse leg length, convergence angles, tie point distribution, effectiveness of cross-station tie points, etc. Accuracy values under different configurations are computed using a least squares bundle adjustment. Image resolution and similarity between adjacent images are analyzed as considerations for the automation of rover localization. Based on experiment results, conclusions and suggestions are given for optimal rover traverse design. Experiment results using actual data from Earth and Mars are also briefly illustrated.

INTRODUCTION Mars Pathfinder (MPF) successfully landed on the Martian surface on July 4, 1997, where it deployed and navigated the rover Sojourner. This was the first time that the rover provided powerful close-range tools for microscale rock investigation, soil research and other scientific objectives within an area of about 10m x 10m around the lander (Golombek et al., 1999). Following up on the successful rover operations in the MPF mission, the MER (Mars Exploration Rover) 2003 mission will deliver two powerful rovers to two different sites on Mars. With far greater mobility, these new MER rovers will be able to travel up to 100 meters across the surface each Martian day, making a 600 to 1,000m long-range traverse possible. Accurate navigation and localization relative to the landing center are needed so that each rover can traverse the Martian surface safely, perform operations at repeated locations, and support coordinated multidisciplinary highprecision science experiments. In the 1997 MPF mission, the rover Sojourner used primarily a heading sensor plus wheel counters for rover localization. Sojourner accumulated 1m location errors in an area of 10m x 10m. The extension of the rover traverse operation area from 10m x 10m in the 1997 mission to 600m x 600m up to 1,000m x A STUDY ON OPTIMAL DESIGN OF IMAGE TRAVERSE NETWORKS FOR MARS ROVER LOCALIZATION * ACSM-ASPRS 2002 ANNUAL CONFERENCE PROCEEDINGS

1,000m in the MER 2003 mission presents a great challenge for long-range rover localization. For example, in order to restrict the accumulated rover localization error to 1m in a 1km traverse, an accuracy of 0.1% of the distance from the landing center must be achieved. Rover localization research has been carried out at the NASA Jet Propulsion Laboratory (JPL) using several different methods. These methods include one based on colored cylindrical targets (Volpe et al., 1995), a maximum likelihood estimation method using matching range maps (Olson and Matthies, 1998; Olson, 2000), and a multiresolution mapping method using surface, descent and orbital imagery (Olson et al., 2001). Since 1998, the Mapping and GIS Laboratory at The Ohio State University (OSU) in conjunction with the JPL Machine Vision Group have been developing both integrated and incremental bundle adjustment techniques for detailed landing site mapping and rover localization using descent and rover imagery. Various rover localization experiments have been carried out with simulated descent and rover images acquired in 1999 and 2000 at a Silver Lake test site. Localization accuracy of 0.1% of the distance from the landing center was achieved (Li et al., 2000a and 2000b; Li et al., 2001; Ma et al., 2001). The geometric strength of the image network is the key to achieving high accuracy of rover localization. The quality of the network is affected by many factors, including traverse leg length (distance between adjacent stations), convergence angles, tie point distribution, requirements for automatic tie point selection, effectiveness of crossstation tie points, etc. In order to optimize the image traverse network, a systematic investigation is needed to evaluate the networks and analyze the effects. Network optimal design is an important step in survey engineering that has been extensively researched in geodetic surveying (Kuang, 1996). With conventional aerial photogrammetry, design options are essentially quite limited, basically being a choice of base-to-height ratio, image scale, and control point distribution. Standardized design procedures have been established for topographic mapping (ASP, 1980). In this paper, we investigate the optimal design for the network of multi-camera stations along a traverse from the point of view of rover localization. We simulate several traverses of side looking and forward-backward looking rover image networks and examine the factors that affect rover localization accuracy. Experiment results will provide guidance for the optimal design of rover image traverse networks, and thus be valuable for practical use in future Mars rover missions. Experiment results using actual Earth and Mars data are also illustrated.

PHOTOGRAMMETRIC MODEL FOR ROVER LOCALIZATION In our methodology, rover localization is accomplished by a bundle adjustment of the image network, which is built by linking all the rover images with tie points. The basic model of the bundle adjustment is based on the collinearity equations (Wolf and Dewitt, 2000). The linearized observation equation is expressed in matrix form V = AX - L , P (1) where V is the correction vector of the image coordinates, A the coefficient matrix or design matrix, L the observation vector, X the unknown vector including exterior orientation parameters of the rover images and ground coordinates of the tie points, and P is the weight matrix of the observations. The computation of the unknowns is basically a free network adjustment because there will be no (or not sufficient) ground control in Mars landing sites. The normal equation is NX = b (2) where N = A T PA , b = A T PL , and N is a rank-deficient square matrix. Using singular value decomposition (SVD), N is decomposed into full rank matrices U and V, and a diagonal matrix D, that is

[ ( )]

N = U diag d j V T

(3)

Then the generalized inverse of N is calculated as

[ ( )]

N -1 = V diag 1 d j U T

(4)

and the solution of the bundle adjustment is

[ ( )]

X = N -1b = V diag 1 d j UT b

(5)

The standard deviation of unit weight is calculate

V T PV r where r = number of observations – rank(N). The standard deviations of adjustment quantities are σ0 =

(6)

A STUDY ON OPTIMAL DESIGN OF IMAGE TRAVERSE NETWORKS FOR MARS ROVER LOCALIZATION * ACSM-ASPRS 2002 ANNUAL CONFERENCE PROCEEDINGS

σ Xi = σ 0 Q XiXi

(7) -1

where QXiXi is the element in the ith row and ith column of N . In the above integrated bundle adjustment model, all the unknowns are calculated and adjusted at one time. We have also developed an incremental bundle adjustment model for rover localization in order to achieve high computational efficiency (Li et al., 2000b; Li et al., 2001; Ma et al., 2001). Theoretically, the incremental adjustment method should give the same computational result as the integrated adjustment method. Therefore, for simplicity we use the integrated adjustment method to evaluate the rover image traverse network. In future missions, we could obtain the positions of certain distinct landmarks (e.g. mountain peaks) on the landing site from either the orbital imagery (such as Mars Orbital Camera - MOC imagery) or from the Digital Terrain Model (DTM) that can be generated from MOC and MOLA (Mars Orbiter Laser Altimeter). If we can find the correspondent landmarks on rover images, they would be included in the bundle adjustment of rover images as control or tie points, whose weights are different from the ordinary tie points and are specified according to their accuracy. We assess the rover localization and mapping accuracy using the estimated standard deviations of the adjusted quantities, including rover positions and orientations and the estimated standard deviations of the 3-D ground positions of tie points. The tie points are generated during the image traverse network simulation. The N matrix is established based on the interior orientation parameters of the cameras, the designated rover position and orientation parameters, and image and ground coordinates of the tie points. Given a designated standard deviation of unit weight, i.e., σ0 , the standard deviations of the adjusted quantities can be directly calculated from equation (7).

IMAGE TRAVERSE NETWORK SIMULATION AND COMPUTATIONAL RESULTS We simulated traverses of side looking and forward-backward looking rover images at five stations with different scenarios by using various convergence angles and different leg lengths (See Figures 1 and 2). Tie points are generated by selecting appropriate ground points in a rectangle shape and projecting them onto rover images with the designed exterior orientation parameters. The camera focal length and stereo base are set to 28mm and 30cm, the same as the digital stereo cameras that were used in acquiring simulated rover images at the Silver Lake test site in 2000 (Ma et al., 2000).

Figure 1. Image traverses and leg lengths

Figure 2. Convergence angles

In the first experiment, we use cases with four average leg lengths (15, 30, 45 and 60 meters) and four convergence angles (180, 150 120 and 90 degrees), respectively. Here we consider the image measurement errors as a standard deviation of 1 pixel (0.01838mm). Nine tie points are selected for each image. Figure 3 illustrates the rover localization errors (average standard deviation of rover locations) of 16 rover images at five stations, with each chart representing different looking angles. Comparing the average standard deviation of rover locations with different leg lengths, it shows that the shorter the leg length, the better the result. For convergence angles of 180o and 90o , the localization error vs. leg length relations are very clear and approximately linear. For convergence angles of 150o and 120o , however, there are some fluctuations. We can also observe that the average standard deviation of the traverse of the 60m leg length is approximately 0.05m. A STUDY ON OPTIMAL DESIGN OF IMAGE TRAVERSE NETWORKS FOR MARS ROVER LOCALIZATION * ACSM-ASPRS 2002 ANNUAL CONFERENCE PROCEEDINGS

Figure 3. Rover localization accuracy vs. leg lengths with different looking angles

In the second experiment, rover localization errors with different convergence angles are compared. The 60m leg length is used in this experiment. Figure 4a shows the case where only tie point measurement error is considered as only one pixel. Given the fixed distance between two adjacent stations, the rover localization errors increases as the convergence angle decrease, because the distance between the object and the cameras increases. . In practice, tie point (or feature) identification error should also be considered. In Figure 4b, we know that the feature identification errors are large when an object is observed from two opposite directions. Therefore, 8 to 4 pixels identification errors are assumed for convergence angles from 180o to 90o .

4a

4b Figure 4. Rover localization accuracy vs. looking angles


If we do not consider the feature identification problem, larger convergence angles (180o and 150o , i.e. in which the distance between the object and the camera is shorter) give a better result. This indicates that objectcamera distance plays a significant role in accuracy improvement. However, if we consider image measurement error and feature identification error together, the smaller the convergence angle, the better the result. This indicates that in real-life situations, a smaller convergence angle (90o ) will give better results and is preferable for automatic tie point selection. In the third experiment, impacts of number and distribution of tie points in each image are examined. Different configurations of tie points are listed in Figure 5. The numbers are IDs of the distribution patterns. Figure 6 illustrates the localization errors with respect to the configurations using convergence angles of 180o and 90o .

Figure 5. Tie points and distribution patterns

Figure 6. Rover localization accuracy vs. different number and distribution of tie points The results show that tie-point distribution plays a significant role in rover localization. For well-distributed tie points (evenly distributed in the coverage area), five or six points can give good results. On the other hand, if tie points are not well distributed, adding more tie points may not significantly improve results. In order to examine the effectiveness of cross-station (e.g. Stations 1 and 3) tie points, we conducted an experiment with a traverse of 16 images at 5 stations. For rover images that have cross-station overlapping, three cross-station tie points are selected in each image. These are difficult to find, but they are expected to be effective in improving traverse strength. These three cross-station tie points along with nine adjacent-station tie points (e.g. Stations 1 and 2) in each image are used together for rover localization. Figure 7 illustrates the average standard deviation of rover locations with and without cross-station tie points. In the case of the 90o convergence angle (left-hand chart in Figure 7), accuracy appears to be improved using cross-station tie points. The image network has a triangular structure, which results in the improvement of geometric strength. In images with image IDs 3, 4, 13 and 14, there are no cross-station tie points available, therefore the localization errors at these stations remain the same. In the forward and backward looking case (right-hand chart in Figure 7), cross-station tie points do not improve the rover localization accuracy. In fact, they made the accuracy somewhat worse for some stations because the weak triangulation geometry and the feature identification errors.


Figure 7. Rover localization accuracy and cross-station tie points Rover image resolution is essential for tie point identification and measurement. Figure 8 illustrates object (e.g. rock) size in rover images with respect to object-camera distance, given a rock of 0.25, 0.5 or 1m on the ground, respectively. Figure 9 illustrates the relationship between object size and leg length with different convergence angles, assuming an object 0.5m across. The two figures show that in order to have distinct objects appear sufficiently large and clear in rover images, object-camera distance and/or leg length have to be appropriately selected. For example, if rock size is only 0.25m across, object-camera distance must be less than 38m in order for the rock to appear as 10 pixels in the image. The relationship between leg length and object size in the image is not a linear function, with bigger-sized rocks appearing larger. For example, a 1m rock at an object-camera distance of 100m would make more than 15 pixels in the image. Meanwhile, convergence angle does not have a very significant influence on object size in the image.

Figure 8. Object size in rover images vs. object-camera distance

Figure 9. Object size in rover images vs. leg length with different convergence angles

SOME RESULTS USING ACTUAL DATA FROM EARTH AND MARS We performed rover localization experiments using actual rover image data acquired in Silver Lake, CA in 2000 (Ma et al., 2001). In previous rover localization experiments, descent images were used together with rover images in the bundle adjustment to enhance the strength of the image network. However, in the MER 2003 mission no descent imagery will be available . Therefore, we use rover images here only to test the application potential for the 2003 mission. We carried out bundle adjustment of the entire traverse of 18 stations, which is approximately 1km in length. In this experiment, 217 tie points were selected in 76 rover images to form the image network.


Internal accuracy is represented by the standard deviations of the rover positions, which are shown in Figure 10. In general, they are within 1m for the 18 stations.

Figure 10. Standard deviations of rover locations To check the external accuracy, we used “rover” stations as checkpoints where there were tripods and/or flags and GPS coordinates were taken. They can be identified in the rover images. We also selected some feature points that were determined in the former bundle adjustment with descent images. We considered these features to have better quality and used them as checkpoints as well. In total, therefore, 24 checkpoints were employed. Figure 11 shows the location errors (differences between adjusted positions and known “true” positions) at the checkpoints. We observed a trend that the localization error increases when the distance from the landing center increases. At some checkpoints, errors are greater than 2m (with one reaching close to 5m). The explanation is that these checking points are at other imaging station and far from the current imaging stations. In addition, such check points are usually in stereo pairs formed by forward and backward looking images that have weaker geometry. The RMS errors at checkpoints are 1.435, 0.883 and 0.752 meters in X, Y and Z directions respectively. The 18th station is about 1km from the landing center. Therefore, this method has the potential to localize the rover with an accuracy of 1.5m within a distance of 1km from the landing center.

Figure 11. Location errors at check points vs. distance from the landing center


We also observed that the accuracy in X direction is usually greater than that in the Y and Z directions. This is because the rover image traverse is approximately along the X direction. It should be noted that the rover location accuracy was sometimes poor because of the number, distribution, and/or quality of the tie points. When we encountered such situations, we had to rework tie point selection to improve results. Compared with the bundle adjustment using descent and rover images together (Ma et al., 2001), the bundle adjustment using rover images only is more sensitive to tie points and initial values of unknowns (exterior orientation parameters of rover images and ground positions of tie points). This indicates that an image network using rover images only is weaker than one using both descent and rover images. We also conducted a rover localization experiment using actual Mars image data. We download lander (Imager for Mars Pathfinder, or IMP) and rover images of the MPF mission from the Planetary Data System (PDS) web site. Because the rover did not take images “continuously” in the MPF mission, we could not test rover localization in a rover image traverse. We selected ten stereo pairs of IMP images and two stereo pairs of rover images to test the rover localization accuracy. An image network was built by linking the IMP and rover images: 155 tie points were selected to link the IMP images, 15 tie points were selected to link IMP and rover images, and 20 tie points were selected to link the rover images. The maximum distance from a tie point to the lander was about 25 meters, and the average distance about 6 meters. The rover was about 5 meters distance from the lander. In the image network, the distribution of the tie points between rover Stereo Pair 1 and IMP images were better than that of Stereo Pair 2. For comparison purposes, we carried out two bundle adjustment experiments: the first one uses only rover Stereo Pair 1 with all the IMP images, the second one uses the two stereo pairs with all the IMP images. Below, we briefly give the computational results. A detailed description of the bundle adjustment of MPF IMP and rover imagery will be presented in a forthcoming paper. The standard deviations of the exterior orientation parameters are obtained from the bundle adjustment. From Experiment I, the average of the estimated deviations of the rover camera positions are 0.074, 0.085 and 0.031meters in X, Y and Z (height) directions, respectively. From Experiment II, the estimated deviations of the rover camera positions are 0.083, 0.081 and 0.035 meters in X, Y and Z directions, respectively. We can see that the estimated rover localization accuracy from the two experiments is very close, being about 2% of the distance from the lander. We also assess the rover localization accuracy by comparing the ground point positions of tie points calculated from rover stereo pairs with those from IMP stereo pairs. The average difference for 10 tie points from Experiment I is 0.118, 0.051, and 0.017 meters in X, Y and Z directions; the average difference of 15 tie points from Experiment II is 0.426, 0.178, and 0.134 meters in X, Y and Z directions. Apparently, the result from the second experiment is not as good as that from the first experiment, because the triangulation geometry for the check points using the second rover stereo pair and the lander images is relatively weak.

CONCLUSIONS Based on the simulation study results, we have come to the following conclusions and suggestions: 1) Leg length has an apparent (approximate linear) influence on rover localization accuracy. If a sufficient number of tie points are available and well distributed, the traverse leg length (distance between stations) should be less than 60m in order for rover localization to achieve an accuracy of 0.1% of the total traverse length. 2) Considering both feature identification difficulty and image measurement errors, smaller convergence angles give better results. Whenever possible, a 90o convergence angle is preferable. 3) Tie point distribution plays a dominant role in rover localization. For evenly distributed tie points, five to six points should give a very good result. Keeping tie points evenly distributed is critical to success. 4) Cross-station tie points improve the rover localization accuracy only when the images have good views across stations, i.e. smaller convergence angles (e.g. 90o ). Based on the rover localization experiment using rover images from earth, we conclude that if there is no descent image available, it is still feasible to locate the rover using rover images only. However, this is more difficult than using descent images together with rover images. Using rover images only, rover localization accuracy is about 1.5m for a traverse length of 1km from the landing center. The experiments using MPF data demonstrated the capability of rover localization using MPF IMP (lander) and rover images. They prove again that the geometric strength of the image network is the key to achieving high localization accuracy. In order to build a strong image network, sufficient tie points must be selected to link rover images to lander images and these tie points should be A STUDY ON OPTIMAL DESIGN OF IMAGE TRAVERSE NETWORKS FOR MARS ROVER LOCALIZATION * ACSM-ASPRS 2002 ANNUAL CONFERENCE PROCEEDINGS

evenly distributed. A similar situation will occur with the Panoramic Camera (Pamcam) and Navigation Camera (Navcam) images in the MER 2003 mission, where the consideration of rover traverse optimization seems to be important for achieving a high accuracy. In addition to panoramic images, it is preferable to use Navcam to acquire “continuous” images so that they can be linked to form an image traverse. Convergence angle and leg length should also taken into account in order to achieve a high localization accuracy and the automation in feature extraction and tie point selection.

ACKNOWLEDGMENTS We acknowledge the help of Dr. Ray Arvidson and Bethany Ehlmann of Washington University in St. Louis; and Dr. Jurgen Oberst, Monika Kuschel and Marita Waehlisch of German DLR for providing valuable information on MPF IMP and rover data processing. We would particularly like to thank Dr. Randy Kirk of USGS for his immense help with the conversion of IMP camera models. This project is funded by JPL/NASA.

REFERENCES ASP. (1980). Manual of Photogrammetry. Fourth Edition, American Society of Photogrammetry, Falls Church, VA, 1056p. Golombek, M. P. et al. (1999). Overview of the Mars Pathfinder Mission: Launch through Landing, Surface Operations, Data Sets, and Science Results. Journal of Geophysical Research, 104(E4): 8523-8553. Fraser, C. S. (1984). Network Design Considerations for Non-Topographic Photogrammetry. Photogrammetric Engineering and Remote Sensing, 50(8): 1115-1126. Fraser, C. S. (1987). Limiting Error Propagation in Network Design. Photogrammetric Engineering and Remote Sensing, 53(5): 487-493. Fraser, C. S. (1989). Optimization of Networks in Non-Topographic Photogrammetry. In: Karara, H. M., ed. NonTopographic Photogrammetry. Second Edition, American Society for Photogrammetry and Remote Sensing, Falls Church, VA, 445p. Kuang S. (1996). Geodetic Network Analysis and Optimal Design: Concepts and Applications. Ann Arbor Press, Inc., Chelsea, MI, 368p. Li, R., F. Ma, F. Xu, L. Matthies, C. Olson and Y. Xiong (2000a). Mars Rover Localization Using Descent and Rover Imagery - Result of the Field Test at Silver Lake, CA. ASPRS Annual Conference, Washington, D.C., May 22-26, 2000 (CD-ROM). Li, R., F. Ma, F. Xu, L. Matthies, C. Olson and Y. Xiong (2000b). Large Scale Mars Mapping and Rover Localization using Descent and Rover Imagery. Proceedings of ISPRS 19th Congress, Amsterdam, July 16-23, 2000 (CD-ROM). Li, R., F. Ma, F. Xu, L. Matthies, C. Olson and R. Arvidson (2001). Localization of Mars Rovers using Descent and Surface-based Image Data. Journal of Geophysical Research - Planet, Accepted. Ma, F., K. Di, R. Li, L. Matthies and C. Olson (2001). Incremental Mars Rover Localization using Descent and Rover Imagery. ASPRS Annual Conference, St. Louis, MO, April 25-27, 2001 (CD-ROM). Mason S. (1995). Expert System-based Design of Close-range Photogrammetric Networks. ISPRS Journal of Photogrammetric Engineering and Remote Sensing, 50(5): 13-24. Matthies, L., E. Gat, R. Harrison, B. Wilcox, R. Volpe and T. Litwin (1995). Mars Microrover Navigation: Performance Evaluation and Enhancement, Autonomous Robots Journal, special issue on ``Autonomous Vehicles for Planetary Exploration'', 2(4). Olson, C. and L. Matthies (1998). Maximum Likelihood Rover Localization by Matching Range Maps, Proceedings of the IEEE International Conference on Robotics and Automation, pp.272-277. Olson, C. (2000). Probabilistic Self-Localization for Mobile Robots, IEEE Transactions on Robotics and Automation, 16(1): 55-66. Olson, C. F., L. H. Matthies, Y. Xiong, R. Li, F. Ma and F. Xu (2001). Multi-resolution Mapping using Surface, Descent and Orbital Imagery. The 2001 International Symposium on Artificial Intelligence, Robotics and Automation in Space, Montreal, June 2001. Wolf, P. R. and B. A. Dewitt (2000). Elements of Photogrammetry: with Applications in GIS, Third Edition, McGraw-Hill, 608p. A STUDY ON OPTIMAL DESIGN OF IMAGE TRAVERSE NETWORKS FOR MARS ROVER LOCALIZATION * ACSM-ASPRS 2002 ANNUAL CONFERENCE PROCEEDINGS