LOGISTIC REGRESSION FOR DETECTING CHANGES BETWEEN DATABASES AND REMOTE SENSING IMAGES M. Chabert(1) , J.-Y. Tourneret(1) , V. Poulain(2) and J. Inglada(2) (1)
University of Toulouse, IRIT-ENSEEIHT-T´eSA, Toulouse, France ´ Centre National d’Etudes Spatiales (CNES), Toulouse, France
(2)
{marie.chabert,jean-yves.tourneret}@enseeiht.fr,
[email protected],
[email protected]
ABSTRACT
obtained on synthetic and real data sets are presented in Section 4. Conclusions and future works are finally reported in Section 5.
This paper studies database updating using optical and synthetic aperture radar images. Logistic regression is used to model the conditional probability of presence/absence of buildings given features extracted from the images. The logistic regression parameters are estimated using the maximum likelihood method. Binary hypothesis tests are then constructed from these estimates to detect changes between the optical/radar images and the existing database. The estimation and detection algorithms are evaluated using simulated and real data sets. Index Terms— Database updating, logistic regression, maximum likelihood, change detection.
2. LOGISTIC REGRESSION MODEL 2.1. Background Denote as U1 , ..., UN the N features extracted from the optical and SAR images associated to a potential object such as a potential building. Denote as M the associated database label where M = 1 (resp. M = 0) means the extracted object is an actual building (resp. is not a building). The logistic regression model between the random variables U1 , ..., UN and M is defined by π1 (β)
1. INTRODUCTION
=
Cartographic database updating is a key application of high resolution remote sensing imagery. Classically, the updating is performed by comparing the objects detected in the image to the information contained in the database [1], [2]. This paper proposes to detect changes between the database and the optical and synthetic aperture radar (SAR) images by using logistic regression. Logistic regression is an efficient tool for comparing continuous and discrete random variables [3]. It is used here to model the relationship between the database and features extracted from the optical and SAR images. Precisely, the database is transformed into a discrete binary random variable modeling the absence/presence of buildings whereas features extracted from the images are modeled as continuous random variables as in [4]. The first step of the proposed change detection strategy estimates the logistic regression model parameters between the database and the image features using a ground truth (supervised learning). The estimation is performed using the maximum likelihood (ML) method. The estimated logistic regression parameters are then used to define a measure of similarity between the database and the available images. A binary hypothesis test based on this similarity measure is finally constructed to detect changes between the database and the optical and radar images. Note that a similar method was studied in [5] for detecting built-up areas in polarimetric SAR images. The paper is organized as follows: Section 2 summarizes some results about logistic regression and defines the features used for cartographic database updating. Section 3 presents the maximum likelihood estimation method for estimating the parameters of the proposed logistic regression model. This section also defines the similarity measure between the database and the optical/SAR images that will be considered for change detection. Simulation results
978-1-4244-9566-5/10/$26.00 ©2010 IEEE
=
3198
π0 (β)
= =
P [M = 1|U1 = u1 , ..., UN = uN ] exp β T u 1 + exp β T u 1 − π1 (β) P [M = 0|U1 = u1 , ..., UN = uN ]
where u = (1, u1 , ..., uN )T , β = (β0 , ..., βN )T contains the regression model parameters (to be estimated) and π1 (β) is the probability to have a building in the database for the observed values of the image features u1 , ..., uN . Note that the proposed logistic regression model is appropriate to an arbitrary number of features and an arbitrary number of observed images. These images include optical (possibly multispectral) or SAR images as well as images resulting from appropriate pre-processing of the previous images such as the gradient of the optical image, line extraction applied to the SAR image or edge filter outputs. 2.2. Features The proposed change detection algorithm is based on features extracted from the optical and SAR images. In the case of building database updating, the following N = 5 features have shown good properties for building discrimination [6] • edges (F1 ): mean distance between building borders and optical image nearest edges, • shadow (F2 ): mean value of the shadow mask near building walls not oriented toward the sun, • vegetation (F3 ): percentage of vegetation pixels located inside the building on the optical image, • line segments (F4 ): percentage of building wall pixels containing an extracted segment in their vicinity,
IGARSS 2010
• contrast (F5 ): ratio between the layover region and shadow region mean values on the SAR image. The analyses performed in this paper will be conducted using these N = 5 features. However, other features might be included in the proposed processing chain without major modifications. The socalled significance measures to what extend the features are related to the database label [3, 5]. A feature is significant when its associated coefficient is nonzero. A statistical test, e.g., the univariate Wald test, can be used to determine which coefficient is nonzero according to its estimate mean and standard deviation ([3, p. 37]). 3. CHANGE DETECTION The proposed change detection strategy is divided into two steps detailed in this section. The first step estimates the logistic regression parameter vector β using the ML method. This step requires to have supervised database and optical/SAR images (ground truth). The second step defines a similarity measure between the database and the images for cartographic database updating. 3.1. Maximum Likelihood Estimation The ML method estimates the unknown parameter vector β by maximizing the joint likelihood of M = [M (1), ..., M (n)] conditionally upon U = [U (1), ..., U (n)], where n is the number of objects available in the learning set and U (j) = [U1 (j), ..., UN (j)] is the set of features associated to the jth object. Assuming independence between the labels M (j), j = 1, ..., n conditionally upon the features, the joint likelihood of M given U = u can be written n
L(β) =
π1 (j, β)m(j) [1 − π1 (j, β)](1−m(j))
j=1
with π1 (j, β) =
exp β T u(j) T 1 + exp β u(j)
and u(j) = [u1 (j), ..., uN (j)]. Maximizing the likelihood L(β) is equivalent to minimize the negative log-likelihood l(β)
= =
− ln L(β) n − m(j) ln [π1 (j, β)] + [1 − m(j)] ln [1 − π1 (j, β)] . j=1
Differentiating the log-likelihood with respect to β0 and βi , for i = 1, . . . , N leads to the following equations n
[m(j) − π1 (j, β)] = 0
(1)
ui (j)[m(j) − π1 (j, β)] = 0, i = 1, ..., N.
(2)
j=1 n j=1
We propose to estimate the parameter vector β by maximizing l(β) using a numerical optimization procedure. The closed-form expressions of the partial derivative given in (1) are useful for this optimization procedure.
3199
3.2. Cramer-Rao Bounds The performance of an unbiased estimator is classically measured by its variance which is lower bounded by the Cramer-Rao bound (CRB). Thus, the CRB provides an optimal estimation performance for any unbiased estimator. The CRBs of β are the diagonal terms of the (N + 1) × (N + 1) inverse Fisher information matrix (FIM). The FIM elements are classically defined as 2 ∂ l(β) [FIM(β)]ij = E (3) ∂βi ∂βj where i, j = 1, ..., N + 1. Closed form expressions for the mathematical expectations appearing in (3) are difficult to derive. However, these expectations can be estimated by averaging the results of Monte Carlo simulations. The ML estimator is known to be asymptotically efficient under mild conditions (satisfied for logistic regression). Consequently, for large values of n, the variances of the ML estimates can be approximated by the associated CRBs. 3.3. Change Detection The detection of changes between the database and a test image is performed by comparing the database with an object (a potential building) extracted from the test image1 . The resulting building detection problem can be classically formulated as the following binary hypothesis test H0 : the object is a building,
H1 : the object is not a building.
A feature vector is computed for any extracted object from the images. For any feature vector u, the corresponding object is associated to a building using the following rule T u exp β H1 T ≶ SPFA T =π
1 (β) = (4) 1 + exp β u H0 where β has been obtained in Sec. 3.1 from the ground truth. Note that T is the test statistics associated to the detector and that SPFA is a threshold depending on the probability of false alarm (PFA), i.e., the probability of rejecting hypothesis H0 when H0 is actually true [7, p. 31]. 4. SIMULATION RESULTS 4.1. Synthetic data The first experiments study the performance of the ML estimator of β using synthetic data. The optimization method used to maximize the log-likelihood is the unconstrained Nelder-Mead simplex method [8]. Figure 1 shows the mean square errors (MSEs) of the ML estimates (log scale) for parameters β0 and β1 as a function of the number of observations (size of the learning set). The actual parameter vector for this experiment is β = (1, 0.2)T . All MSEs have been computed by averaging the results of 1000 Monte Carlo runs. The MSEs of the ML estimates are also compared with the associated approximated CRBs (which provide a reference in terms of parameter MSE). Figure 1 shows that the MSEs of the ML estimator are very close to the CRB even for relatively small sample sizes, illustrating the good performance of the ML estimator. 1 Note that the proposed methodology could be used for other problems such as road detection
Fig. 1. Estimated MSEs and CRBs for β0 and β1 . Fig. 3. SAR image. 4.2. Database description Real images provided by the CNES (Toulouse, France) have been considered to evaluate the proposed cartographic database updating. Figure 2 shows an optical image acquired over a suburban area of Toulouse (France) by the airborne PELICAN sensor. The associated SAR image and building database are displayed in figures 3 and 4. Features F1 to F5 have been extracted from 111 buildings and 176 various objects (roads, shadows, vegetation, playing fields,...) from the different images.
Fig. 4. Building database.
Fig. 2. Optical image.
4.3. Building detection Figure 5 shows the normalized histograms of the conditional probability estimates π
1 (β) under hypothesis H0 and H1 using features F1 to F5 (the left figure corresponding to H0 has been obtained using the buildings from the database whereas the right figure corresponding to H1 has been obtained using other objects extracted from the images (roads, playing fields, ...). The estimated conditional probability π
1 (β) clearly allows one to detect the presence of most build-
3200
ings and to reject most objects that are not buildings. The performance of the proposed building detection strategy is also evaluated using receiver operational characteristics (ROCs) [7, p. 36] that are displayed in Figure 6. These ROCs confirm the good performance of the proposed building detection strategy. An interesting property of logistic regression is that it allows one to determine the significance of the selected features for building detection. More precisely, significant features are associated to large values of regression coefficients. In order to determine the interesting features for building detection, we have estimated logistic regression parameters using Nc = 30 different learning sets composed of 110 objects (55 buildings and 55 non buildings). The resulting average of the univariate Wald test statistics (see [3, 5] for details) for the regression coefficient associated to the 5 selected features is displayed in Fig. 7. According to Fig. 7, the two features F2 (shadow) and F3 (vegetation) are the most significant for building discrimination.
5. CONCLUSIONS This paper studied a logistic regression model appropriate to model correlations between a binary building database and an arbitrary number of continuous features extracted from optical and SAR images. The model parameters were estimated using the maximum likelihood method from a ground truth composed of a database and optical and SAR images. A change detection algorithm based on logistic regression parameter estimators was then proposed for cartographic database updating. The proposed method showed good performance on synthetic and real images. It is interesting to mention that the proposed algorithm provides a great flexibility since it can be applied to an arbitrary number of explanatory variables. Moreover, it allows one to measure the significance of any parameter for building detection. Future works include the consideration of a nonlinear feature mapping before logistic regression. This will result in a nonlinear logistic regression model that is more general than the proposed linear model. Fig. 5. Histograms of π
1 under hypotheses H0 (left) and H1 (right).
6. ACKNOWLEDGEMENTS The authors would like to thank Philippe Marthon (University of Toulouse, France) and Marc Spigai (Thales Alenia Space, Toulouse, France) for interesting discussions about cartographic database updating. They are also grateful to the CNES for its financial support. 7. REFERENCES [1] T. Bailloeul, V. Prinet, B. Serra, P. Marthon, P. Chen, and H. Zhang, “Urban building land use change mapping from high resolution satellite imagery, active contours and Hough voting,” in Proc. 9th International Symposium on Physical Measurements and Signature in Remote Sensing (ISPMSRS), Beijing, China, Oct. 2005. [2] J. Duan, V. Prinet, and H. Lu, “Building extraction in urban areas from satellite images using GIS data as prior information,” in Proc. IEEE International Geoscience and Remote Sensing Symposium, Anchorage, Alaska, USA, Sep. 2004.
Fig. 6. ROCs for building detection.
[3] D. W. Hosmer and S. Lemeshow, Applied logistice regression, Second edition, Wiley, New York, 2000. [4] J.-Y. Tourneret, V. Poulain, M. Chabert, and J. Inglada, “Similarity measures between vector data bases and optical images for change detection,” in Proc. IEEE International Geoscience Remote Sensing Symposium (IGARSS’09), Cape Town, South Africa, Jul. 2009. [5] D. Borghys, Interpretation and registration of high-resolution polarimetric SAR images, Ph.D. dissertation, University of Paris VI, Paris, France, 2001. [6] V. Poulain, J. Inglada, M. Spigai, J.-Y. Tourneret, and P. Marthon, “Fusion of high resolution optical and sar images with vector data bases for change detection,” in Proc. IGARSS’09 Cape Town, 2009. [7] Van Trees and Harry L., Detection, Estimation, and Modulation Theory: Part I, Wiley, New York, 1968. [8] J. A. Nelder and R. Mead, “A simplex method for function minimization,” Computer Journal, vol. 7, pp. 308–313, 1965.
Fig. 7. Logistic regression parameter significance.
3201
