paring the performance of the clutter rejection systems with static and dynamic ROI extraction. 1 Introduction. ATR using forward-looking infrared (FLIR) imagery.
A Modular Clutter Rejection Technique for FLIR Imagery Using Region-Based Principal Component Analysis Syed A. Rizvi, Tarek N. Saadawi*, and Nasser M. Nasrabadi,** Department of Engineering Science and Physics College of Staten Island of City University of New York Staten Island, NY 10314 * Department of Electrical Engineering City College of City University of New York New York, NY 10031 **U.S. Army Research Laboratory, ATTN: AMSRL-SE-SE 2800 Powder Mill Road, Adelphi, MD 20783
Abstract
the clutter images (false alarms) from the potential target images provided by the detection stage, and (3) a classification stage that determines the type of the target.
I n this paper]-, a modular clutter rejection technique using region-based principal component analysis (PCA) is proposed. Our modular clutter rejection system uses dynamic R O I extraction to overcome the problem of poorly centered targets. I n dynamic R O I extraction, a representative R O I is moved in several directions with respect to the center of the potential target image to extract a number of ROIs. Each module in the proposed system applies region-based PCA to generate the feature vectors, which are subsequently used to decide about the identity of the potential target. W e also present experimental results using real-life data evaluating and comparing the performance of the clutter rejection systems with static and dynamic R O I extraction.
1
POTENnAL TARGEl
CHIPS
- 1
KCUSS
IMAGES
(TARGET DETECTION)
REJECTION
- 2
MSSlFlER
- N
“ER CHIPS
Figure 1: An ATR system. In the FLIR database at ARL, there are 10 targets, each with 72 different views (at a 5 degree separation in aspect angle). Figure 2 (a) and 2 (b) show a sample of the target tank (T72) at different orientations taken from our development set (where targets are centered using the ground truth information) and from the output of a preprocessing stage, respectively. The potential target images generated by the preprocessing stage are generally off-centered and some images have only a part of a target present. Previous research shows that the off-centered potential target images generated by the preprocessing stage are the main cause of substantially large number of false alarms passed to the classification stage from the clutter rejection stage. This ultimately results in rather poor performance of an ATR system on real-life data that, otherwise, performs very well on the development set in which targets are centered using the ground truth information. In this paper, a modular clutter rejection technique using region- based principal component analysis (PCA)
Introduction
ATR using forward-lookinginfrared (FLIR) imagery is an integral part of the ongoing research at the U.S. Army Research Laboratory (ARL) for digitization of the battlefield. FLIR ATR, however, is a challenging problem because of the highly unpredictable nature of thermal signatures. Automatic target recognition (ATR) systems generally consist of three stages as shown in Fig 1 [l]: (1) a preprocessing stage (target detection stage) that operates on the entire image and extracts regions containing potential targets, (2) a clutter rejection stage that uses a sophisticated classification technique to identify true targets by discarding ‘This research was in part supported by CUNY Collaborative Incentive Research Grant # 91916-00-06 and PCS-CUNY grant # 624680031.
1
475 0-7803-6297-7/00/$10.000 2000 IEEE
TARGET
45
90
ROI-Sell
Clutter Rejection Module 1
Ro1-Set2
Clutter Rejection Module 2
135"
---Eh--180°
270 '
225
315'
0
*
Composite Decision TargeffClutter >
7.
I
(a>
Figure 3: The proposed clutter rejection system. 225
270
315'
Figure 2: Target tank (T72) at eight different aspect angles. (a) Targets are centered in the chip using the ground truth information. (b) These target chips were generated by a preprocessing stage. Targets are well off-centered and some chips contain only a part of the target. is proposed. Our modular clutter rejection system uses dynamic ROI extraction to' overcome the problem of poorly centered targets. In dynamic ROI extraction, a representative ROI is moved in horizontal, vertical, and diagonal directions with respect to the center of the potential target image to extract a number of ROIs. These ROIs are then used by the proposed modular clutter rejection system to make a decision about the identity of a potential target. Each module in the proposed system applies region-based PCA t o generate the feature vectors, which are subsequently used t o decide about the identity of the potential target. The rest of the paper is organized as follows: Section 2 presents the proposed technique and section 3 concludes the paper presenting experimental results.
2
Modular Clutter Rejection System
Figure 3 shows the functional diagram of the proposed modular clutter rejection system and Fig. 4 shows the functional diagram of a typical clutter rejection module. The clutter rejection technique operates on potential target images extracted by the preprocessing stage. These images are referred to as chips. The performance of a clutter rejection system can be improved by minimizing the number of pixels in the background. In an attempt to minimize the background in
476
a-
Figure 4: One typical clutter clutter rejection module. a potential target chip, we follow a model based ap-
proach t o develop representative shapes for extracting ROI from the potential target chips. ROI for a given target varies with the target orientation because the target has different shapes and sizes when viewed from different angles. For example, a side view of a truck is completely different from the front view of the same truck. We have divided targets into four representative groups based on their sizes and shapes when viewed from different angles. The ROIs for each group are extracted using representative silhouettes as shown in Fig. 5. These representative silhouettes are generated by clustering together target silhouettes of similar sizes and shapes. The target silhouettes, in turn, are obtained by raytracing BRL-CAD geometric target models [2] of each
target from a number of views. Specifically, we categorize all target images by clustering tde target images with respect to their similar sizes and shapes in order to form a group; that is, we divide the set of images into subsets we call groups. Each image in a group is also divided into several regions and a PCA is performed for each region to extract feature vectors. We propose to use feature vectors of arbitrary shapes and dimensions that are optimized for the topology of a target in a particular region. One can then use these feature vectors to decide whether a potential target is a clutter or a real target.
module receives ROIs from all representative silhouettes, where all ROIs are extracted in the same mode of extraction (either static or one of the several dynamic modes). After performing ROI extraction, each target is divided into five region: left, right, top, bottom left, and bottom right. These regions generate the five feature vectors in spatial domain. A region-based PCA is then performed on the feature vectors in spatial domain. Details of region-based PCA can be found in Ref. [3]. Extracted ROI
Original Image
ROI-0 (Center)
Figure 5: The set of four representative silhouettes.
ROI-2 (Move Left)
The representative silhouettes are used to extract ROIs in two modes: (1) static and ( 2 ) dynamic. In static mode, the center of the representative silhouette coincides with the center of the potential target chip. ROI is then extracted from the potential target chip and sent to an appropriate clutter rejection module for further processing. In dynamic mode, the representative silhouette is displaced in horizontal, vertical, and diagonal directions with respect to the center of the potential target chip to extract several ROIs. These ROIs are then sent to their respective clutter rejection module for further processing. As mentioned earlier, the dynamic mode of ROI extraction allows us to extract useful features from the off-centered target chips, which is generally the case in real-life detection systems. Figure 6 shows the geometry of dynamic ROI extraction used in this paper. Specifically, we use one static and eight dynamic ROIs in the current implementation. Four of the eight dynamic ROIs are extracted by moving the representative ROI in both horizontal and vertical directions with respect to the center of the potential target chip (right, left, top, and bottom). The rest of the four dynamic ROIs are extracted by moving the representative ROI in four diagonal directions (upperright, upper-left, bottom-right, and bottom-left). It is important to note that each clutter rejection
477
ROI-3(
~D
ROI-6 (Move Upper-Left)
0~
ROI-4 (Move Up)
Bottom-Right) ~ ~ROI-7 ~ ~ (Move )
ROI-8 (Move Bottom-Left)
Figure 6: Geometry of dynamic ROI extraction. Recent research shows that combining classification decisions from several, possibly independent, classifiers can result in improved classification performance [4,5]. We adapt a similar approach in our modular clutter rejection system; that is, the classification decision by each module (matching score) is combined by using a multi-layer perceptron.
3 Experimental Results We now present experimental results of the proposed clutter rejection system with static as well as dynamic ROI extraction. We design our system in two stages. In the first stages we design the feature codebooks (for
improvement in ROC performance can be seen in Figs. 7 and 8 over clutter rejection system with static ROI extraction. 1.o c
.-.a $ .- 0.8 c
mc
0
c
0.6 m
.a .a
2
8 0.4 c
.->. ._ 4-
:0.2 13
False alarm rate
t 0.0
Figure 7: Comparison of ROC performance on the training data generated by the preprocessing stage (Gray collection) using dynamic and static ROI.
detail on designing feature codebooks see Ref. [3]). In the design of feature codebooks, we have used a total of 17,322 target images and 5,956 clutter images as our development set. The development set is further divided into a training set of 14,866 target images and 3,500 clutter images, and a testing set of 2,456 target images and the same number of clutter images. The target images in this development set have targets properly centered using the ground-truth information. In the second stage, we combine the scores generated by individual module using a neural network (multilayer perceptron). The potential target images generated by a preprocessing stage, however, may not be well centered and the performance of the clutter rejection system would deteriorate accordingly. Therefore, in the second stage we use the potential target images generated by a preprocessing stage to train the neural network. This training set that we refer to as “Gray collection” consists of 1,853 target and the same number of clutter images. We then evaluate the performance of the proposed system on another data set generated by a preprocessing stage. This testing set that we refer to as “Huli collection” consists 2,074 target and the same number of clutter images. An ROC comparison of the clutter rejection systems using static as well as dynamic ROIs on the training and testing set is shown in Figs. 7 and 8. A marked
478
0.0
0.2
0.4 0.6 False alarm rate
0.8
1.o
Figure 8: Comparison of ROC performance on the testing data generated by the preprocessing stage (Huli collection) using dynamic and static ROI.
References [l] B. Bhanu, “Automatic target recognition: state of the art survey,” IEEE Trans. Aerospace and Electronic Systems, vol. AES-22, no. 4, pp. 364-379, 1986. [2] K. Appline, M. Muss, and R. Reschly, B R L - C A D Graphic Editor MGED, U.S. Army Ballistic Research Laboratory, 1988.
[3] S. A. Rizvi, N. M. Nasrabadi, and S. Z. Der, “A clutter rejection technique for FLIR imagery using region-based principal component analysis,” in Proc. I E E E International Conference o n Image Processing (Kobe, Japan), Oct. 24-28, 1999. [4] K. Woods, W. P. Kegelmeyer Jr., and K. Bowyer, “Combination of multiple classifier using local accuracy estimates,” I E E E Trans. Pattern Analysis and Machine Intelligence, vol. PAMI-19, no. 4, pp.
405-410, 1997. [5] S. B. Cho and J. H. Kim, “Combining multiple neural networks by fuzzy integral for robust classification,” IEEE Trans. Systems, Man, and Cybernetics, vol. SMC-25, no. 2, pp. 380-384, 1995.
