IMAGE PROCESSING TOOLS FOR SEMI-AUTOMATIC MINEFIELD DETECTION P. Druyts, Y. Yvinec, M. Acheroy Signal and Image Centre, Royal Military Academy, Belgium, http://www.sic.rma.ac.be, Av. de la Renaissance, 30, B-1000 Brussels, Belgium, Tel: +32-(0) 2-737-64-70, Tel: +32-(0) 2-737-64-74, Fax: +32-(0) 2-737-64-72,
[email protected],
[email protected],
[email protected]
KEYWORDS: Image processing, Humanitarian demining, Minefield detection, Airborne survey, Semi-automatic tools, Minefield indicators, image compression.
ABSTRACT Airborne surveys have proved to be a very promising way to detect the boundaries of minefields, a requisite to any mine clearance procedure. Human photo-interpreters are, to date, unchallenged in the analysis of these images. Unfortunately in order to survey all suspected areas of a country and have the resolution needed to detect relevant minefield indicators, a tremendous amount of data is to be collected. This makes the analysis a very time-consuming task. This paper shows how image-processing tools can be used to speed up the work of photo-interpreters by detecting regions of interest or making some repetitive tasks automatic. This leads to a semi-automatic procedure where image processing is used both as a help to the detection of relevant indicators and a way to enhance some given parts of the images. Based on a real campaign performed first in a test site in Belgium and then on a real mine-infested country, namely Mozambique, in the scope of a European project co-ordinated by ITC, the work presented here involves two steps. First, some indicators have been selected for their relevances in minefield detection as well as for their easiness to be automatically detected on the available images. Then detectors devoted to each of these indicators have been implemented. This paper describes some examples of these detectors. Some specifications are also given to ensure the success of such a survey from an image processing point of view. They concern the constraints that minefield indicators and indicator detectors must fulfil in order to be able to be used in such a semi-automatic procedure. Other constraints about the images and the way to collect them to improve the detection of indicators are also given. In addition, some results about compression on minefield images are presented.
1. BACKGROUND Mine clearing is a long and dangerous task. A lot of time is lost clearing areas free of mines that were selected just to be on the safe side (Craib, 1985). Deminers are looking forward to methods that can reduce the suspected areas. Minefield detection by airborne survey could be such a method. But a human analysis of all data gathered during an airborne campaign would take a tremendous time. There are two reasons for this. First the amount of data needed for minefield analysis is very important. Suspected areas may be very large, especially if for safety reasons they have been overestimated, and the resolution needed to detect indicators of minefields is very demanding: a pole or a foxhole, even a wire, can be indicators. Second, photo-interpreters must analyse closely the images to find very tiny details. Image processing, by speeding up some repetitive tasks or pointing some regions of interest to the analyst, could be useful. In the scope of the project Airborne minefield detection: pilot project, two campaigns have been performed, first in Belgium for testing and then in Mozambique, where an aircraft equipped with several sensors have been flown over suspected minefields in Buzi, Bandua, Mameme and Songo (van Genderen and Maathuis, 1998). The second section describes the problems linked with the use of image processing for minefield detection. Section 3 explains the approach chosen to build tools that can be both useful to the photo-interpret and efficient. A list of promising minefield
indicators for image processing is given in Section 4. The description of some detectors devoted to these indicators can be found in Section 5. Finally Section 6 gives some general advice for the selection of minefield indicators, automatic or semiautomatic detectors and data collection in order to achieve a successful detection by image processing.
2. PROBLEMS The close analysis of the way a human photo-interpreter works makes it clear that image processing alone could not compete with human beings who use knowledge about history, geography, hydrology and more generally a large amount of information not always obviously correlated with the problem. Moreover one part of the job of photo-interpreters is to design a procedure of detection on a case-by-case basis. Such an approach is not easy to implement in computer algorithms. Even techniques such as artificial intelligence or expert systems may lack the adaptability of a human being. Therefore we see the most promising use of image processing in speeding up the process by detecting minefield indicators to be defined by the photo-interpreters together with specialists in image processing. How to use these indicators, once detected, to infer the presence of a minefield is left to photo-interpreters. Compared to human beings, image processing faces a particular difficulty: the large sizes of the images. Disk memory alone can
be a problem. However, it is possible to reduce the sizes of the images without drawbacks. Tests have been performed with an image of 350 megabytes (minefield of Buzi in colour infrared). With a loss-less wavelet based algorithm, the compression rate was 1.5. The same algorithm reached a compression rate of 10 in a lossy configuration although the loss of quality was not perceptible visually.
3. APPROACH Two goals can be given to image processing in a semi-automatic framework for minefield detection. Image processing can be used to help the detection by selecting regions of interest that would be afterwards studied by a photo-interpreter. Image processing can also provide the photo-interpreter with enhanced images to help the interpretation. In the first case, the photo-interpreter is given some areas to study where there might be something of interest. In the second the photo-interpreter is given new information, or information displayed in a new way, to improve the understanding of a given zone and help the classification as useful minefield indicator or false alarm. In both cases, some minefield indicators must be defined. These indicators must be relevant, that is, their presence must make the possibility that a minefield is here, or nearby, larger. For that reason, minefield indicators must be provided by photointerpreters after an investigation on the country and with the help of local people or professional deminers. But these indicators must also be easily detectable by image processing. A good indicator candidate must be simple: a model must be easily designed from the original description. Since the detector should be as general as possible, the model and therefore the indicator must be as independent as possible from the environment. For instance if a soil disturbance is an indicator of the presence of a mine in the ground, then a buried mine should create more or less the same kind of disturbances as often as possible. Otherwise a detector of soil disturbances is of little help: it may detect natural disturbances and not react to a buried mine
Figure 1 Perspective projection of poles and shadows
4.3.
Elongated areas
The minefield of Buzi (Mozambique) is an elongated area fivemeter wide. Areas that are suspected minefields are not used by local population. There are no cultures and vegetation grow naturally, which may not be the case of areas just outside the suspected minefield. Since the vegetation on the minefield is different from the vegetation outside, especially in colour infrared, an elongated area is a good indicator. It is very simple and tools can be built or others can be adapted to detect it, such as detectors of roads or detectors of taxiways in an airport (Mees, 1998).
4.4.
Trenches and foxholes
The minefield of Bandua and Mameme (Mozambique) were designed to protect military camps. Remains of these camps are good indicators. If the place of a former camp is detected, then the presence of a defensive minefield may be inferred. Stereovision algorithms have proved to be efficient in detecting semi-automatically the difference in depths due to trenches or foxholes. Relevant and easy to detect, they are good candidates for minefield indicators.
5. EXAMPLES OF DETECTORS 4. EXAMPLES OF INDICATORS 4.1.
Anti-tank mines on the ground
During the first phase of the project, an airborne survey of a test site in Leopoldsburg (Belgium) was done. A model of an anti-tank mine lying on the ground was used. For more details, please refer to Druyts et al. 1998. 4.2.
Poles
The minefield of Buzi (Mozambique) is delimited by poles. These poles, actually railway tracks, are along the borders of the minefield and are at five meters from each other. These poles or their alignments are very simple minefield indicators. They are both characteristic of the minefield, at least this very one, and easy to detect automatically.
5.1.
Pole detector
A pole as seen on airborne imagery is characterised by a Vshape. One branch is the pole itself and the other is the shadow of the pole. The poles and the shadows are characterised by elongated homogeneous regions that are in general well contrasted with the background. Those regions are limited by two parallel lines. The shadows are always dark and in the case of the Buzi minefield, so are the poles. Usually, the poles are rather vertical. Therefore, all shadows are in the same direction. That direction may be computed if the orientation of the photo and the location of the sun is known (time of the day). The poles are imaged through a perspective projection (see Figure 1).
Figure 4 Pole detection (1) first stage
Figure 2 Theoritical positions of poles and shadows in an image (the poles are represented with thick lines, the shadows with thin lines)
Therefore, as shown in Figure2, all pole projection lines converge through the same point, in the image or outside the image. If the height of the poles is known, the length of the projection may be computed. This length is maximal in the corners and minimal in the middle (principal point) of the image. The length of the horizontal projection is constant on a vertical line (and vice-versa).
A pole detector has been built, using this very special configuration of directions (Figure 3). First the histogram of grey levels is computed. It is used to select the threshold. The image is then thresholded to segment the dark regions. A number of false alarms may be rejected by keeping only regions with appropriate size. The contours of those regions are then computed. Those contours are approximated by polygons to get a good estimation of the contour direction. The contours may be classified as pole, shadow or false alarm according to their direction. Amongst the shadow or pole contours, parallel lines at distance compatible with the pole section are searched and used as shadow or pole candidate. Figure 4 presents a typical candidate obtained after thresholding and size-based filtering.
Histogram based threshold selection
Figure 5 presents a typical example of remaining candidate pairs that are composed of two parallel lines with appropriate distance and appropriate direction. The number of false alarms could further be reduced by keeping only candidates for which the two parts of the V are found. One could also search for regular patterns, here aligned poles (see Lake et al., 1997).
Thresholding Edge detection
Polygonal approximation
Pole approx.
Principal
Shadow approx.
direction
direction
direction
Figure 5 Pole detection (2) final stage
5.2. Lines in pole direction
Lines in shadow direction
Pole candidates
Figure 3 Pole detector
Trenches and foxholes
Just as we are able to have an idea about the depths of objects from the two images that we receive from our two eyes, it is possible to build a three-dimensional representation of a scene from two images taken from two different points of view. The principles are given in Figure 6. A point M in space is seen on the pixel P1 on image IM1 and pixel P2 on image IM2. These pixels are the intersection on the lines between M and the optical centres O1 and O2, and the image planes. P1 and P2 are called matched pixels. If we can, for
therefore the aircraft), the height of the aircraft and the calibration parameters of the sensor. Without these data, only a projective reconstruction is possible, which is enough for the present application.
M
Figure 7 shows an example of a trench in Bandua as seen on colour infrared image. On the right, the projective reconstruction is shown. We can clearly see the shape of the trench.
O2 O1
P1
IM1
IM2
P2
Figure 6 Principles of stereovision each pixel on one image, find its matching pixels on the other images, then it is possible to find the position of M by triangulation. A three-stage process is applied. First a rectification is performed so that any couple of matched pixels are on the same row on the two images. That way, the search of the match is a one-dimensional problem (the search is done along one row of the image) and not a two-dimensional problem (we do not search in the whole image). This reduces the complexity of the task. In a manual rectification, couples of matched pixels, one on each image, are chosen manually and then a re-computation of the two images is performed so that matched pixels are on the same row on each image. In automatic rectification, the selection of these couples is automatic. These points are the basis for the weak calibration of the sensors (see Yvinec and Thiesse, 1995) that allow then a geometric transformation of the images. This transformation can be seen as a virtual modification of the sensor parameters and the relative positions of the sensors so that matched pixels are on the same row on each image. Then the matching is done for each pixel on one image. For that, we compute the correlation between a window centred on the pixel and windows centred on each pixel on the corresponding row on the other image. The estimated matched pixel is the one that gives the highest correlation value. This method implies that the images are textured. Otherwise, such as in some areas in Bandua where the ground has a very homogeneous colour, results may be poor. The method used here is based on Faugeras, 1994. If enough information about the way the images were taken is available, a three-dimensional reconstruction is possible giving heights or depths for all pixels. This reconstruction needs the parameters of the position and orientation of the camera (and
Figure 7 Projective reconstruction of a trench
The second example (Figure 8) is a foxhole in Mameme in colour. For display purposes, only half of the reconstruction is shown. There again, the presence of the foxhole is visible. In these two examples, the rectification was performed manually. This work can be expanded in two directions: a metric reconstruction (to have height, since for the moment, the reconstruction being projective, there is no meaning in the Z-axis) and an automatic rectification. As such, these detectors would be best used, not to select regions of interest, but to provide the photo-interpreter with additional information to improve the understanding of the area under investigation.
6. ADVICE FOR FURTHER IMAGE PROCESSING WORK Based on this experience, it is possible to draw some first conclusion on how to choose good minefield indicators, build a good detector and collect the data, to improve the image processing work.
6.1.
How to choose the indicators
As already mentioned, indicators must fulfil two conditions: be valid indicators for a photo-interpreter and be simple enough to allow an automatic detection. The first constraint means that the presence of the indicator must really improve the confidence in the presence of a minefield. The second means that a detector, as defined in the next paragraph, should be easily implemented. It is also a good idea, but not a necessity, to choose indicators which would take a long time to be detected by a human interpretation, because of their small sizes for instance. That way, image processing could really be a valuable addition to the tools of the photo-interpreters.
Figure 8 Projective reconstruction of a foxhole
6.2.
How to build the detectors
Once the indicators have been chosen, the detectors must be built. If the detector is to be launched prior to any human investigation, such as a pole detector as seen previously, then it must be able to work on very large images. In the examples presented here, the pole detector and the anti-tank mine detector divide large images into several smaller overlapping ones, perform the detection on all these sub-images and then merge the results. The speed of the detector is also important. If the detector is to speed up the work of a photo-interpreter, then it must be faster than a human analysis.
The project partners involved in both phases are: CAE Aviation (L), EOS (UK), Eurosense (B), VUB (B), Geograf (P), RMA (B), SSC (S) and ZEO (G). Project partners involved mainly in the first phase are: Aerodata (B), Aerosensing (G), IGI (G), Recon Optical (UK). Project partners involved in the second phase are: CND (Moz), NPA (N). The project was co-ordinated and managed by ITC. Figure 4 and Figure 5 show results from RMA, based on images from Eurosense. Left images from Figure 7 and Figure 8 are from Eurosense. 9. REFERENCES
6.3.
How to collect the data
Obviously, the data must be collected with their use kept in mind. Once the indicators have been chosen, then the most adequate sensors can be known, both in terms of principles and resolution. The detectors can also provide some constraints. For instance, the pole detector presented here uses the shadow of the poles. Images taken when the shadows are clearly visible will help the detection, typically the flights should not take place at night or at midday. For the use of stereovision, a delicate phase is the rectification, whether it is performed manually or automatically. If the two stereoscopic images could be taken in such a way that the images were originally rectified then again the quality of the result of the detector would be improved.
7. CONCLUSIONS The work performed in the scope of this project is still under way. This paper presented some minefield indicators that proved to be as promising for their relevance in inferring the presence of a minefield and their automatic detection by image processing. Some detectors were also described and their results shown. Based on this experience, it is already possible to have an idea on the way relevant indicators should be chosen and efficient detectors built. Since the collecting of the data should be linked with the use of the data, these choices of indicators and detectors imply some constraints on the flights, the types of sensors and more generally the whole process of data collecting. In addition, the large sizes of the data may be solved by use of compression tools that can reduce the sizes by 10 without any visible loss of quality.
8. ACKNOWLEDGEMENT This research project formed part of the European Pilot Project entitled: Airborne Minefield Detection Pilot Project. This project was funded by the European Commission, the governments of Belgium, Germany, Luxembourg, Portugal, the United-Kingdom and the International Institute for Aerospace Survey and Earth Sciences (ITC), The Netherlands. The project consisted of two phases: a field trial in Belgium and a test over several areas in Mozambique.
Craib. J. A., 1995, Mine detection and demining from an operator’s perspective. Workshop of anti-personnel mine detection and removal, Lausanne, 30th June and 1st July 1995 Druyts, P., Yvinec, Y., and Acheroy, M. 1998. Usefulness of semi-automatic tools for airborne minefield detection. In Clawar 98 -First International Symposium, pages 241-248, Brussels, 1998. Faugeras, O., Hotz, B., Mathieu, H., Vieville, T., Zhang, Z., Fua, P., Theron, E., Laurent, M., Berry, G., Vuillemin, J., Bertin, P., et Proy, C, 1993. Real time correlation based stereo: algorithm implementations and applications. RT 2013, INRIA, 1993. van Genderen J. L., Maathuis B. H. P., 1998, Airborne Detection of Landmines: a Review of the Techniques and some practical Results, DGP session, Disarmament and International Security, Regensburg Germany, 26th March 1998 Lake, D. E., Sadler, B. and Casey, S., 1997: Detecting regularity in minefields using collinearity and a modified Euclidean algorithm Proc. SPIE Vol. 3079, p. 500-507, Detection and Remediation Technologies for Mines and Mine like Targets II, Abinash C. Dubey; Robert L. Barnard; Eds. July 1997 Mees, W., Druyts, P., Borghys, D., Ouaghli, Y., Miravet, C., Santamaria, J., Sυß, H., Perneel, C., Acheroy, M., and Valero, J.L., 1998. Semi-automatic interpretation of airports using multisensor information. IEEE Transactions on Geoscience and Remote Sensing Yvinec, Y., Thiesse, B., 1995, Estimating the fundamental matrix from phase based 2D disparity, Second Asian Conference on Computer Vision ACCV'95 Singapore 5-8 Dec 1995.
