Visual retrieval of concrete crack properties for automated post

0 downloads 0 Views 494KB Size Report
Apr 11, 2011 - Crack detection. Property retrieval. Post-earthquake reconnaissance. Image processing. The safety of post-earthquake structures is evaluated ...
Automation in Construction 20 (2011) 874–883

Contents lists available at ScienceDirect

Automation in Construction j o u r n a l h o m e p a g e : w w w. e l s ev i e r. c o m / l o c a t e / a u t c o n

Visual retrieval of concrete crack properties for automated post-earthquake structural safety evaluation Zhenhua Zhu ⁎, Stephanie German, Ioannis Brilakis School of Civil and Environmental Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA

a r t i c l e

i n f o

Article history: Accepted 5 March 2011 Available online 11 April 2011 Keywords: Crack detection Property retrieval Post-earthquake reconnaissance Image processing

a b s t r a c t The safety of post-earthquake structures is evaluated manually through inspecting the visible damage inflicted on structural elements. This process is time-consuming and costly. In order to automate this type of assessment, several crack detection methods have been created. However, they focus on locating crack points. The next step, retrieving useful properties (e.g. crack width, length, and orientation) from the crack points, has not yet been adequately investigated. This paper presents a novel method of retrieving crack properties. In the method, crack points are first located through state-of-the-art crack detection techniques. Then, the skeleton configurations of the points are identified using image thinning. The configurations are integrated into the distance field of crack points calculated through a distance transform. This way, crack width, length, and orientation can be automatically retrieved. The method was implemented using Microsoft Visual Studio and its effectiveness was tested on real crack images collected from Haiti. Published by Elsevier B.V.

1. Introduction After an earthquake occurs, entry into damaged buildings as soon as possible is necessary for a variety of reasons, including emergency search and rescue, building stabilization and repair, and salvage and retrieval of possessions [1]. There are always extensive risks associated with entering damaged buildings after an earthquake, and often, further structural collapse produces additional victims. Currently, the safety of entering damaged buildings is evaluated manually by structural specialists (e.g. structural engineers and/or certified inspectors). They follow the guidelines provided by the Federal Emergency Management Agency (FEMA) and/or the Applied Technology Council (ATC), and assess the impact of visual damage (e.g. cracks) on critical structural components to make sure that the damaged building remains stable and maintains a specific level of structural integrity. Although civil engineers are the appropriate candidates to evaluate the safety of highly engineered environments [2], several limitations were found in the current evaluation process. First, it is time-consuming. In the October 15, 2006 Hawaii Earthquake and the December 22, 2003 San Simeon Earthquake, the whole building safety evaluation processes took several weeks to complete due to the large number of buildings requested for safety assessments [3,4]. Also, the subjective inspection nature may lead to erroneous judgments [5].

⁎ Corresponding author. Tel.: +1 404 385 1276. E-mail addresses: [email protected] (Z. Zhu), [email protected] (S. German), [email protected] (I. Brilakis). 0926-5805/$ – see front matter. Published by Elsevier B.V. doi:10.1016/j.autcon.2011.03.004

The aforementioned limitations can be overcome if the current manual evaluation practices are fully automated. This not only requires load-bearing members in a structure to be automatically recognized, but also the damage lying on these structural member surfaces to be detected and further assessed based on their properties. So far, many machine vision based methods have been created to locate the damage on structural member surfaces, and their effectiveness has been validated in inspecting structures such as bridges, pipes and tunnels. As a contrast, little work was found regarding load-bearing structural members detection, damage properties retrieval from detection and damage assessment based on the retrieved properties. The authors recently proposed a research study that focuses on providing a crude but quick safety evaluation for post-earthquake concrete structures through the image/video analysis of these structures [6] (Fig. 1). In the study, concrete columns in images/ videos are first recognized with the help of the column's visual characteristics. Then, the damage inflicted on these columns is detected and its attributes are calculated. The detected columns and damage are then correlated in the form of relative damage size, orientation and position. The correlation is used to query a case based reasoning knowledge base which contains apriori classified damage states of columns. The query estimates the damage state of the column for any given damage inflicted on it. If the damage state is severe, an imminent collapse is assumed and an immediate evacuation warning will be issued. So far, the first step, concrete column detection, has been completed [7]. As the step following concrete column detection, this paper presents a novel approach of retrieving the properties of the cracks

Z. Zhu et al. / Automation in Construction 20 (2011) 874–883

Damaged Structures

Image/Video Capturing

Images/Videos

Previous work (Zhu and Brilakis, 2010)

Critical Structural Element Detection

Structural Elements

Damage Detection & Properties Retrieval

Scope of this paper

Damage Properties

Safety Evaluation

Future work

Risk Fig. 1. Machine vision enhanced structure safety assessment.

that are inflicted on concrete columns. Under the approach, a percolation-based crack detection method is first used to produce a crack map for each concrete structural element surface. Then, the topological skeletons of cracks and the distance field of the crack pixels in the map are produced through binary image thinning and a distance transform. This information can be used to calculate crack properties (width, length, orientation and location). Further, these properties are related to the dimension and orientation of the structural element to produce relative measurements for the estimation of structural element damage states. The approach is implemented in a Microsoft Visual .NET environment. A database of real concrete structural element images that were collected from Haiti was used to test the approach's validity. The results from the approach were compared with those from manual surveys to find the measurement error for crack orientation, length and width. The factor of crack detection results is considered when analyzing the measurement error. According to the test results, it is found that the properties of the cracks on the structural elements can be correctly measured using the proposed method.

875

dures and/or guidelines and make an assessment based on their experience and knowledge coupled with their visual observation of the damage inflicted on the load-bearing members of a structure. For example, in the case of evaluating the safety of post-earthquake structures for emergency responders, structural specialists are the most prepared personnel to deal with all aspects of the built environment in urban areas [8]. The involved structural specialists are responsible for identifying potential structural hazards and monitoring the structure for condition changes during rescue and recovery operations [9]. Search and rescue teams can only enter the buildings that are determined structurally stable by structural specialists. However, several collaboration-related problems between civil engineers and other organizations involved in disaster relief efforts have been recognized, including the lack of coordination, information sharing, trust and communication [10]. Also, there are not enough qualified structural specialists that can be allocated to community emergency response teams. Those that can participate must be licensed professional engineers with a minimum of 5 years of experience. In addition, they are required to take structural collapse technician courses and US&R structural specialist training. Moreover, in a large-scale earthquake where several thousand buildings may be affected, evaluating the safety of all buildings manually by structural specialists requires a significant amount of time [11]. The adverse effect is that the survival rate for trapped victims is significantly reduced. Over 91% of people trapped in collapsed structures can survive if they are rescued within 30 min; however, this value declines to 36.7% once trapped for 2 days [12]. In the case of evaluating the safety of post-earthquake structures for occupants, the ATC-20 code outlines three procedural levels: rapid evaluation, detailed evaluation, and engineering evaluation [13]. Rapid evaluation is typically based on an exterior inspection of a structure only; detailed evaluation is a thorough visual inspection of a structure inside and outside; and in an engineering evaluation, engineers/inspectors investigate the safety of a damaged structure from construction drawings and new structural calculations [14]. The purpose of the rapid evaluation is to quickly identify apparently "Unsafe" or "Safe" buildings after an earthquake. A building is regarded as unsafe if it partially collapses or its stories lean severely [13]. The buildings that cannot be determined as "Safe" or "Unsafe" are further assessed in the detailed and engineering evaluations, where the severity and extent of damage to the structural and nonstructural elements throughout a building is observed, measured and recorded. Compared with the rapid evaluation, the detailed and engineering evaluations will most often provide more accurate assessment information of a structure, but both procedures expense great resources and time, as the evaluations tend to take up to 2 h and 1 week, respectively [14]. According to Chock's report [3] about the October 15, 2006 Hawaii Earthquake, over several hundred buildings were requested to assess damage each day from October 15 to the end of October in the County of Hawaii, but approximately 100 of them can be evaluated. Similar results were reported by Johnson for the December 22, 2003 San Simeon Earthquake, where the phases of assessment work continued for a number of weeks [4]. In order to overcome the aforementioned limitations, it is necessary to automate the current manual evaluation practices. As an important part of automation, the damage lying on structural member surfaces is required to be detected, and the useful damage properties need to be retrieved for structural integrity assessment. One example of such damage is cracking. 3. Automated Crack Detection

2. Current Practices in Post-earthquake Damage Assessment The safety evaluation of buildings in the event of an earthquake is currently performed by structural specialists, such as certified inspectors and/or structural engineers. They follow existing proce-

Whether the assessment is made for emergency responders or for occupants, cracking is always an important structural damage indicator. For example, according to FEMA structural collapse technician courses, one check point for concrete frame structures is whether cracking

876

Z. Zhu et al. / Automation in Construction 20 (2011) 874–883

occurs on concrete columns at each floor line (above and below floor) [15]. Similarly, column cracking at top, bottom and wall joints is also listed as one type of the hazards that needs to be identified for precast concrete buildings [15]. In the ATC-20 code, the extent and severity of damage to the load-bearing elements of a reinforced concrete building are quantified primarily by the width and orientation of the cracks that lie on these elements [13]. A cast-inplace concrete building is immediately regarded as unsafe and prohibited to entry when there are large diagonal cracks extending though building columns [13]. So far, many machine vision based methods have been created to automatically detect cracks on structural member surfaces. These methods are generally classified into two categories. The first category recognizes only whether or not an image contains a crack (crack presence). For example, Abdel-Qader et al. proposed a principal component analysis (PCA) based algorithm for recognizing crack presence in a bridge surface image [16]. In their algorithm, an image was first segmented into sixteen square blocks. Each block was filtered by linear feature detectors (horizontal, vertical and oblique) and then projected onto dominant eigenvectors which were pregenerated from a training data set. The projection result was further compared with the projection results of training data to identify whether or not the blocks contain a crack. This way, cracks in an image can be recognized sequentially on the basis of these blocks. Similarly, Liu et al. developed a crack classification system, where a support vector machine (SVM) was used to differentiate regions in an image as "crack," "non-crack" and "intermediate" regions [17]. In addition to recognizing the presence of cracks, those methods which fall in the second category also locate crack points in an image (i.e. produce a crack map). These methods utilize cracks' special visual characteristics in images and adopt various image processing techniques, such as wavelet transforms, thresholding, and edge detection, to extract crack points from the image background. Cheng et al. detected cracks in an image by simply thresholding the concrete surface image. The threshold value was determined based on the image's mean and standard deviation values [18]. Abdel-Qader et al. compared the effectiveness of four edge detection techniques (the Canny edge detector, Sobel edge detector, Fourier transform and fast Haar transform) with respect to the detection of cracks on concrete bridges and found that the fast Haar transform was more reliable than the other three [19]. However, these methods belong to globalprocessing techniques that do not consider crack connectivity. As a result, their detection accuracy is affected by image noise [20]. To address this problem, Yamaguchi and Hashimoto proposed a type of scalable local percolation-based image processing method that considers crack connectivity among neighboring image pixels [20]. Their test results indicated that the percolation-based method can correctly detect cracks with efficient computation time even for a large-size concrete surface image [20]. Also, Sinha and Fieguth [21] introduced two crack detectors that consider relative statistical properties of adjacent image regions. These two detectors are applied in four directions (0°, 45°, 90°, and 135°) to identify crack pieces in buried concrete pipes, and then a linking and cleaning algorithm is used to connect crack pieces. Iyer and Sinha [22] designed morphology-based filters with linear structuring elements to detect cracks. A crack map is a binary image where each isolated crack point is shaded white, and non-crack points are shaded black. Directly from the map, the specific properties (length, orientation, maximum width, and average width) of each crack are all unknown. Little work has focused on automatically retrieving this information. Yu et al. [23] calculated the length, thickness and orientation of concrete cracks through a graph search; however, their method required the start and end points of the crack to be manually provided first. Chae et al. [24] relied on an artificial neural network to retrieve crack properties, but it is unclear how to form the network's input data sets and how effective the network is.

In the field of automated inspection of pavement cracking distress, researchers have developed many methods in finding pavement cracks [25,26]. The problem of the classification and quantification of the cracks still exists, and it has been a research topic for years [27]. Integrated systems, such as WiseCrax developed by Roadware Group Inc., can classify cracks as longitudinal, transverse, alligator, and block according to the protocols by state agencies [28]. However, the algorithms used in these systems to perform this procedure are proprietary [29]. Also, the protocols adopted in rating pavement cracking distress cannot be used directly in building damage assessment, which is more focused on the impact of the cracks on individual structural members. For example, a diagonal crack with maximum width of 10 mm indicates a different type of damage on a column with a width of 1 m versus that with a width of 0.5 m. 4. Method Overview This paper presents a novel method of retrieving the properties of the cracks on concrete structural elements. The method is divided into two main stages: 1) crack detection, which produced a crack map for each structural element surface; and 2) crack property retrieval, which includes length retrieval, orientation retrieval, and width retrieval using image thinning and distance transform techniques (Fig. 2). Considering that existing automated systems had no difficulty in finding cracks, the purpose of this paper is not to propose a novel, automated crack detection method. Instead, this paper adopts an

Element Surface 1

Edge Detection Edge Map Linear Percolation Crack Map

2 Image Thinning

Distance Transform

Crack Skeleton

Distance Field

Orient. Retrieval

Length Retrieval

Crack Properties 1 -Crack Detection 2 -Crack Properties Retrieval Fig. 2. Method overview.

Width Retrieval

Z. Zhu et al. / Automation in Construction 20 (2011) 874–883

Distances

877

largest eigenvalue is the gradient direction. When the precise gradient magnitude and direction at each pixel (x, y)are retrieved, the pixels with local maximum gradient magnitude in their gradient direction are marked as potential crack boundary points, and a percolation process is initiated iteratively at each of these points to identify all potential crack pixels.

Crack Pixels

Crack Skeleton

2 ∂R = ∂x Jc = 4 ∂G = ∂x ∂B = ∂x

Fig. 3. Crack reconstruction with skeleton and distance information.

existing crack detection method with slight modifications. The main focus of the paper is placed on the classification and quantification of cracks for the purpose of structural member assessment. This is also the main contribution of the paper. Details about these two stages are explained in the following sub-sections.

ð2Þ

where Q = JcT Jc

The method relies on the different image intensities of crack and non-crack pixels to simulate the percolation process. First, the pixel selected to initiate the percolation process form a region, Dp, and the image pixels neighboring to Dp form another region, Dc. The image pixels in Dc are checked, and those that have less image intensities than the image pixel in Dp are identified and percolated into Dp. As Dp grows, the new image pixels neighboring to Dp are found and included in Dc. This process continues until all image pixels in Dc have higher intensities than any image pixel in the region Dp. Then, the circularity (Fc) of Dp is measured using Eq. (3) [20]. If Fc approaches zero, the shape of Dp is linear, and all image pixels in Dp are marked as crack pixels. This way, all potential crack pixels in the image are identified, and these pixels form a crack map.

The idea of using percolation to detect cracks is from the natural phenomenon of liquid permeation through a crack on a concrete surface. Imagine water is poured at crack boundaries, and it always makes its way to fill the cracks. As a contrast, if it is poured on a concrete surface, it will be spread evenly as a circle. The crack detection work in this paper adopts the method proposed by Yamaguchi and Hashimoto [20] with a few modifications. First, instead of initiating a percolation process at each pixel of a structural element image, the percolation is performed only at the pixels that have high gradient magnitudes. This is because the crack boundaries in an image are always characterized by large first derivatives which result in high magnitudes along certain particular directions [30]. The gradient on each image pixel is calculated using the image's RGB (Red, Green and Blue) information [31]. Consider a color image as a mapping from an x–y plane to an RGB space. The Jacobian matrix (Jc) of the mapping is represented in Eq. (1) [31], which indicates the color change induced by moving any infinitesimal step (dx, dy) in the image plane. The Euclidean squared magnitude of the change can then be calculated as in Eq. (2) [31], where the largest eigenvalue of the matrix Q is the gradient magnitude and the eigenvector corresponding to the

Fc =

4 × Ccount 2 π × C max

ð3Þ

where Ccount is the number of the pixels in Dp and Cmax is the maximum length of Dp. 4.2. Crack Properties Retrieval When a crack map is produced, it is necessary to isolate each crack from the map, and retrieve the properties of each crack useful for

1

1

2

2

2

ð1Þ

M 2 = ðdx; dyÞQ ðdx; dyÞT

4.1. Percolation-based Crack Detection

1

3 ∂R = ∂y ∂G = ∂y 5 ∂B = ∂y

3

3

3

4

4

4 12 7

5 6

5 6

5 6

12 7

8 9

13

10 11

14

12 7

8 9

13

10 11

14

1

1

1 2

2

2 3

3

3 4

4

12 7

8 9

13

10 11

14 S1 (1, 5); S2 (5, 11); S3 (5,5) - Current visit

4 5 6

5 6 12 7

10 11

14 S1 (1, 5); S2 (5, 6); S3 (5,5)

S1 (1, 5); S2 (5, 5); S3 (5,5)

S1 (1, 1)

8 9

13

5 6 8 9

13

12 7 10 11

14 S1 (1, 5); S2 (5, 11); S3 (5,7)

- Non-crack skeleton points

- Crack skeleton points

Segment (A, B) where A – start point and B – end point Fig. 4. Skeleton segmentation.

8 9

13 14 S1 (1, 5); S2 (5, 11); S3 (5,14) - Next possible visit

10 11

878

Z. Zhu et al. / Automation in Construction 20 (2011) 874–883

Fig. 5. Invariant crack orientation measurement under different camera orientations.

evaluating the safety of structural elements. The properties investigated in this paper include crack length, orientation, maximum width and average width. These properties are spatially correlated with the dimension and orientation of structural elements to produce relative measurements for estimating structural element damage states. In order to retrieve these properties, a binary image thinning algorithm [32] is applied in the crack map to retrieve cracks' skeletons. Also, a Euclidean distance transform [33] is used to calculate the distance field, which supplies each crack pixel in the map with the nearest distance to its boundaries. A crack skeleton together with the distance values from the skeleton points to the crack boundaries serves as a representation of a crack since they contain all the information necessary to reconstruct the crack (Fig. 3). The topological configuration of a crack is ascertained by checking crack skeleton point connectivity. This process is illustrated in Fig. 4. Suppose one crack skeleton point is visited, and its neighboring points are checked. If there is only one crack skeleton point connected to the

point, the current crack segment grows by including that neighboring skeleton point. If there is more than one crack skeleton point connected to the point, the current crack segment stops growing, and new segments are created. The number of newly created segments depends on the number of neighboring crack skeleton points. For example, two segments (S2 and S3) are created at the crack skeleton point 5 in the figure, since there are two skeleton points (6 and 7) connected to point 5. The new segments start to grow by visiting the remaining skeleton points. When all skeleton points are visited, the segments' directions, start points and end points are checked. Any two segments are merged if they have the same direction, and the end point of one segment is specified as the start point of the other. The properties of a crack are retrieved based on its skeleton segment information and distance field. The crack length is equivalent to the crack skeleton segment length, which is approximated by the height of an object-oriented bounding box that circumscribes crack skeleton segment points. The crack orientation is the crack skeleton segment orientation, which is indicated by the direction of the objectoriented bounding box. The average of the distance values of all skeleton points is calculated, and the doubled result denotes the average crack width. Similarly, the double of the largest distance value that exists at skeleton points represents the crack's maximum width. The double of the shortest distance value that exists at skeleton points represents the crack's minimum width. The double of the shortest distance value that exists at skeleton points represents the crack's minimum width. Consider that the minimum crack width has no use in evaluating the safety of post-earthquake concrete structures. For this reason, the minimum crack width is not measured in this paper. All values are measured at the image pixel level and are of little value to estimate actual structural element damage states until they are spatially correlated with the dimension and orientation of structural elements. Spatially correlating cracks to structural elements produces relative measurements. Although specificity as to the type of relative measurements necessary for structural element damage state estimation is still under investigation, the following measurements are calculated for structural elements: 1) the angle of crack direction

Fig. 6. Collecting the images/videos of damaged structures in Haiti.

Z. Zhu et al. / Automation in Construction 20 (2011) 874–883

879

5. Implementation and Results

in Microsoft Visual Studio .NET, and it was tested by the authors to collect the images/videos of structures that were damaged in the January Haiti earthquake (Fig. 6). Intel® Open Source Computer Vision Library (OpenCV) was used as the prototype's main image processing toolbox, and EmguCV was used as a wrapper to allow OpenCV functions to be called in the prototype. Both OpenCV and EmguCV are open source [33,34]. Fig. 6 illustrates the screenshots of using the developed prototype to retrieve crack properties for a given image. Fig. 7(a) is the main interface of the prototype. A user can browse an image folder to load an image into the prototype (Fig. 7(b)). The test images here are only structural element surfaces that are manually cropped out from originally collected images/videos (Fig. 7(c)). When the user selects the "CRACK PROPERTIES RETRIEVAL" option from the menu, the corresponding crack detection result (i.e. a crack map) is displayed, while the corresponding crack properties are saved in a separate text file (Fig. 7(d)).

5.1. Implementation

6. Results

The methodology presented in this paper is implemented and integrated into the prototype that was developed by the Construction Information Technology Laboratory at the Georgia Institute of Technology as an independent module. The prototype was written

The authors went to Haiti to collect the images/videos of damaged post-earthquake structures in April 11–16, 2010 [35]. The 2010 Haiti earthquake occurred on Tuesday, January 12, 2010 and at least 52 aftershocks measuring 4.5 or greater had been recorded [36]. The

in relevance to the element's vertical edges, 2) the projection of the crack length on the element's width, and 3) the largest crack width in relevance to the element's width. The double of the shortest distance value that exists at skeleton points represents the crack's minimum width. Consider that the minimum crack width is not considered by evaluation standards when evaluating the safety of post-earthquake concrete structures. For this reason, the minimum crack width is not measured in this paper. All the retrieved crack properties are related to the dimension and/ or orientation of concrete columns. This alleviates the issues of the retrieved properties being controlled by camera settings and image distortions. For example, crack orientation is related to concrete column orientation in the method. This way, no matter how to pose a camera, the measurement of crack orientation relative to concrete column orientation still remains (Fig. 5).

(a)

(b)

(c)

(d)

Fig. 7. Screenshots using the prototype to retrieve crack properties.

880

Z. Zhu et al. / Automation in Construction 20 (2011) 874–883

earthquake caused major damage in Port-au-Prince and other settlements in the region. On this field trip of 6 days, three damaged reinforced concrete buildings (i.e. the CDTI Hospital, Union Elementary School and Learning Center, and the Digicel Building) were surveyed. Manual and image/video data of the structural damage for each building were obtained. The manual data were collected following the ATC-20-type assessment procedure. The image/video data consisted of a detailed walk-through of the damaged portions of the buildings. Over one hundred images from the collected images/videos were used to validate the effectiveness of the method proposed in this paper. The collected videos were decomposed into sequences of images for the test purpose in this study. The image resolution is 1600 by 1200. All images/videos were captured at natural light conditions. Consider the main contribution of the paper is the classification and

rating of the detected cracks rather than the ability of crack detection. Therefore, the impact of light conditions on crack detection results is not evaluated. Also, in those images/videos, most concrete columns and cracks are captured in the middle of the images/videos as inspection targets. For the concrete columns and cracks that lie at the corners of the images, although the distortion of pixels at the corners may affect the properties measurement for concrete columns and cracks, the issue has been alleviated, since the properties have been measured relatively. This point has been illustrated in the following test results. The performance of the method in crack detection is first measured. As mentioned in the previous section, the crack detection part of the method is modified from the approach proposed by Yamaguchi and Hashimoto [20]. The crack detection result is a crack map that represents all crack pixels inflicted on a structural member

(a)

(b)

(c)

(d)

(e)

(f)

Fig. 8. Crack detection example: (a) original image; (b) ground truth; (c) crack map; (d) true positive; (e) false positive; and (f) false negative.

Z. Zhu et al. / Automation in Construction 20 (2011) 874–883 Table 1 Crack detection precision and recall for 100 images.

881

Table 2 Measurement from the method for the example in Fig. 8.

Summarization

Crack properties measurement

Total # of correctly detected crack pixels (TP): Total # of incorrectly detected crack pixels (FP): Total # of real crack pixels not detected (FN): Average precision (TP/(TP + FP)): Average recall (TP/(TP + FN)):

13,0278 72,599 11,594 64.2% 91.8%

surface without any topological information. In order to measure the crack detection performance of the method, the evaluation is carried out by comparing the detected crack pixels with the cracks traced manually. The correctly detected crack pixels (true positive), the incorrectly detected crack pixels (false positive), and the undetected crack pixels (false negative) are identified with the procedure suggested by Iyer and Sinha [22]. One example of the matching procedure is illustrated in Fig. 8. Crack detection precision is defined as the percentage of correctly extracted crack pixels from the detected results, and recall is calculated as the percentage of actual crack pixels that are detected. Table 1 illustrates the detection precision and recall calculated from all test images. The performance of the method to retrieve crack properties is further measured on the cracks that are correctly detected. According to the previous tests, 225 cracks are correctly detected by the method. The properties of these cracks are automatically measured. The properties include crack length, width, and orientation. The measurement of these properties is compared with the results of the manual surveys to determine the absolute measurement errors. Fig. 9 shows one example of the intermediate results used by the method for automatic crack properties retrieval. The intermediate results include crack skeleton, skeleton segmentation, as well as the crack distance map. The corresponding measurement made by the method for this example is described in Table 2. Table 3 illustrates the statistical results of the crack properties measurement for all 225 cracks. The average errors in measuring the crack properties related to structural element dimensions are 3.29° for orientation, 2.21% for crack length and 0.35% for maximum crack width. The results indicate that the relative measurement made by the method is close to the one from the manual surveys, although it is based on image pixels.

(a)

(b)

Crack segment

Length – L (pixel)

Orientation

Max. width (pixel)

L/W*

Max. W/W*

a b c d e f g h i j k l m

127.8 73.0 69.3 76.0 60.3 287.7 23.3 300.8 24.1 418.1 13.6 29.1 235.8

103° 101° 5° 160° 106° 115° 124° 114° 85° 115° 58° 4° 110°

6.3 6.3 5.7 6.3 5.7 7.2 7.2 7.2 4.0 8.0 6.0 5.7 7.2

0.444 0.253 0.240 0.264 0.209 0.999 0.081 1.045 0.084 1.452 0.047 0.101 0.819

0.022 0.022 0.020 0.022 0.020 0.025 0.025 0.025 0.014 0.028 0.021 0.020 0.025

Orientation – the angle to x-axis. L – crack length. Max. W – maximum crack width. W⁎ – structural element width.

Table 3 Measurement error for 225 cracks.

Total Average Std

|Δ| – Orientation

|Δ| – L/W*

|Δ| – Max. W/W*

739.65° 3.29° 2.70°

498.17% 2.21% 2.90%

78.66% 0.35% 0.49%

Orientation – the angle to x-axis. L – crack length. Max. W – maximum crack width. W⁎ – Structural element width.

7. Discussion The test results indicate that the crack properties measurement from the proposed method is matched to the manual measurement, as long as crack pixels are correctly and fully retrieved with the crack

(c)

Fig. 9. Intermediate results for crack properties measurement: (a) crack skeleton; (b) segmented skeleton; and (c) crack distance map.

882

Z. Zhu et al. / Automation in Construction 20 (2011) 874–883

detection method. However, none of the existing crack detection methods has been capable of retrieving all crack pixels correctly so far, although they have no difficulty in finding cracks. Missing the detection of crack pixels (e.g. the failure to detect thin cracks) may have critical impacts on crack properties retrieval and further on crack-based safety evaluation. Even if all cracks can be correctly detected, the measurement of crack properties may have little use for structural damage assessment, unless certain post-processing steps are performed. For example, crack k and crack e in Table 2 are detected as two separate cracks in the method. These two cracks may be considered as one crack, which indicates a much more serious structural problem from the perspectives of structural engineers, since they share the same orientation and are close enough in locations. How to effectively combine the detected cracks for the purpose of structural damage assessment needs to be further considered. The knowledge in structural damage assessment from structural engineers must be integrated when performing the crack combination. This will be the focus of the next step in this research. Also, the test results have shown that no matter how accurate existing crack detection methods are, there are always some cracks that are not visible in images, but are visible to human eyes. This limitation cannot be simply overcome by the improvement of crack detection methods or with higher resolution image capturing cameras. Other sensing devices need be considered. Fusing different types of sensing data to retrieve full detailed cracking information on structural members may be one possible way to solve this problem. The proposed method is created for post-earthquake safety evaluation in structural engineering forensics, but it can also be applied for automated routine inspection in the construction or reconstruction domains in order to save inspection cost and resources. For example, highway bridges require inspection at regular intervals to determine their physical and functional conditions. The inspection is currently carried out manually by certified inspectors following the established standards and manuals, which is a time-consuming process associated with high cost. Millions of dollars are spent in highway bridge inspection every year in local Departments of Transportation (DOT) [37]. This limitation could be alleviated with the introduction of automated inspection toolkits to replace manual practices. The automatic retrieval of cracking information on bridge components, including crack detection, crack properties measurement, and crack classification and rating, is one important part in these automated inspection toolkits. The detection of cracks and the retrieval of crack properties in the proposed method can be used to classify and rate cracks under bridge inspection protocols. The results of crack classification and rating will be further input into a bridge management system to help transportation agencies make systemwide prioritization decisions in allocating limited construction maintenance and rehabilitating resources. In addition to bridge inspection, the proposed method can also be applied in concrete pavement inspection to automatically classify and rate pavement cracking distress. 8. Conclusion After an earthquake, structural specialists manually evaluate the visible damage that is inflicted on critical structural members to make sure the building is safe for entry. The process is time-consuming especially when thousands of buildings are involved. Although many damage detection methods have been created to help the specialists locate the damage, few of them can automatically retrieve useful damage properties for structural member assessment. This paper presented an automated method for retrieving the properties of the cracks that are inflicted on concrete structural member surfaces. The method starts with a percolation-based crack detection method to locate crack points on each concrete structural element surface. Then,

the crack properties, such as length, orientation and width, are retrieved from the topological skeletons of cracks and the distance field of crack pixels in the map. The properties are further related to the dimension and orientation of the structural element to produce relative measurements. The method was implemented into a prototype developed by Microsoft Visual Studio .NET. Over 100 structural member images were used to test the performance of the method. These images were collected from structures damaged by the January 2010 earthquake in Haiti. The crack properties measured by the method were compared with manual measurements, and the absolute and relative errors in the measurement of length, orientation and width are calculated. The average measurement error (3.29° for orientation, 2.21% for relative crack length and 0.35% for relative maximum crack width) indicated that the proposed method can automatically and correctly retrieve crack length, orientation and width. 9. Future Work Future work will focus on 1) the improvement of crack detection precision, and 2) the automated assessment of structural members based on the retrieved crack properties. According to the test performed in this paper, it is found that the results from crack detection (i.e. the crack map) play an important role in correctly retrieving crack properties. The method relies on the crack connectivity information in the map to figure out the crack topological configuration. Missing the detection of critical crack pixels connecting two crack segments leads to the error of identifying one single crack as two separate cracks. The linking operation proposed by Sinha and Fieguth [21] may help to solve this problem; however, the operation is case-dependent and empirical. Several practical issues, such as how properly choose the gap searching space and how to fill a gap between two segments, need to be resolved before applying the operation in the crack map to link two crack segments. The method considers the merging of multiple crack segments when they connect each other and have similar directions. How to combine the properties of two separate cracks for the perspective of structural element integrity assessment also needs to be investigated. The ultimate goal of this research is to use the retrieved crack properties for automated structural element integrity assessment. In the assessment, two cracks may indicate the same damage on a structural element, even if they are separated in a crack map. Therefore, the properties of the cracks that show the same damage need to be considered together instead of separately from the assessment perspective. Acknowledgement The NSF grant #0933931 and 0904109 financially support our work. We would like to thank our sponsor. Any opinions, findings, and conclusions or recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of the National Science Foundation. References [1] Applied Technology Council (ATC), Earthquake aftershocks – entering damaged buildings, ATC-35 TechBrief 28 last visit: https://www.atcouncil.org/atc-pdf/ atc35tb2.pdf 19998 (June 2008). [2] R. Prieto, The three R's: lessons learned from September 11, 2001, Royal Academy of Engineering, London, 2002. [3] G. Chock, ATC-20 post-earthquake building safety evaluations performed after the October 15, 2006 Hawai'i Earthquakes summary and recommendations for improvements (updated)8 last visit, http://www.scd.state.hi.us/HazMitPlan/chapter_6_appM. pdf 20078 (July 2009). [4] K. Johnson, San Simeon Earthquake, City of Paso Robles emergency response report8last visit: http://www.prcity.com/government/pdf/EQResponseRpt.pdf 20048 (March 2009).

Z. Zhu et al. / Automation in Construction 20 (2011) 874–883 [5] V. Kamat, S. El-Tawil, Evaluation of augmented reality for rapid assessment of earthquake-induced building damage, Journal of Computing in Civil Engineering 21 (4) (2007) 247–258. [6] Z. Zhu, I. Brilakis, G. Parra-Montesinos, Real time concrete damage visual assessment for first responders, Proceedings of the ASCE Construction Research Congress, 5–7 April 2009, Seattle, WA, 2009, pp. 1204–1213. [7] Zhu, Z. and Brilakis, I. (2010) Concrete Column Recognition in Images and Videos, Journal of Computing in Civil Engineering, American Society of Civil Engineers, (in print). [8] R. Aldunate, S.F. Ochoa, F. Pena-Mora, M. Nussbaum, Robust mobile ad hoc space for collaboration to support disaster relief efforts involving critical physical infrastructure, Journal of Computing in Civil Engineering 20 (1) (2006) 13–27. [9] Federal Emergency Management Agency (FEMA), National urban search and rescue response system – structure specialist position description8 last visit: http://www. disasterengineer.org/library/Struct%20Spec%20PD%20July%202006.pdf 20068 (Dec. 2008). [10] D. Kostoulas, R. Aldunate, F. Peña-Mora, S. Lakhera, A decentralized trust model to reduce information unreliability in complex disaster relief operations, Proc. 13th EG-ICE Workshop on Intelligent Computing in Engineering and Architecture, Ascona, Switzerland, June 25–30, 2006, LNCS 4200, 2006, pp. 383–407. [11] S. Mehta, F. Pena-Mora, Expediting assessment of damaged buildings during disaster response using mobile ad-hoc networks, Proceedings of the ASCE International Workshop on Computing in Civil Engineering, Austin, Texas, June 24–27, 2009, pp. 217–226. [12] United Kingdom Fire Service Search and Rescue Team (UKFSSART), Structure collapse – a guide for emergency personnel8 last visit: http://www.ukfssart.org. uk/index.htm 20078 (Jan, 2009). [13] ATC-20, Procedures for postearthquake safety evaluations of buildings, Report ATC-20, Applied Technology Council, Redwood City, CA, 1989. [14] NASA, Disaster assistance and rescue team — structural assessment team8 last visit: http://dart.arc.nasa.gov/SAT/SAT.html 20098 (Dec, 2009). [15] Federal Emergency Management Agency (FEMA), Module 1C structural engineering, structural collapse technician course – student manual8 last visit: http:// www.fema.gov/emergency/usr/sctc.shtm 20098 (Dec, 2009). [16] I. Abdel-Qader, S. Pashaie-Rad, O. Abudayyeh, S. Yehia, PCA-based algorithm for unsupervised bridge crack detection, Advances in Engineering Software, 2006 37 (12) (2006) 771–778. [17] Z. Liu, A. Shahrel, T. Ohashi, E. Toshiaki, Tunnel crack detection and classification system based on image processing, Proc. SPIE, 4664, 2002, pp. 145–152, Machine Vision Applications in Industrial Inspection X. [18] H. Cheng, X. Shi, C. Glazier, Real-time image thresholding based on sample space reduction and interpolation approach, Journal of Computing in Civil Engineering 17 (4) (2003) 264–272. [19] I. Abdel-Qader, O. Abudayyeh, M.E. Kelly, Analysis of edge-detection techniques for crack identification in bridges, Journal of Computing in Civil Engineering 17 (4) (2003) 255–263.

883

[20] T. Yamaguchi, S. Hashimoto, Fast crack detection method for large-size concrete surface images using percolation-based image processing, Machine Vision and Applications 11 (5) (2009) 797–809. [21] S.K. Sinha, P.W. Fieguth, Automated detection of cracks in buried concrete pipe images, Automation in Construction 15 (1) (2006) 58–72. [22] S. Iyer, S.K. Sinha, Segmentation of pipe for crack detection in buried sewers, Computer-Aided Civil and Infrastructure Engineering 21 (2006) 395–410. [23] S. Yu, J. Jang, C. Han, Auto inspection system using a mobile robot for detecting concrete cracks in a tunnel, Journal of Computation in Civil Engineering 17 (4) (2006) 255–263. [24] M.J. Chae, T. Iseley, D.M. Abraham, Computerized sewer pipe condition assessment, Proceedings of the ASCE International Conference on Pipeline Engineering and Construction, 2003, pp. 477–493, July 13–16, 2003, Baltimore, MD. [25] Y. Hu, C. Zhao, H. Wang, Automatic pavement crack detection using texture and shape descriptors, The Institution of Electronics and Telecommunication Engineers Technical Review 2010 27 (2010) 398–405. [26] Y. Huang, B. Xu, Automatic inspection of pavement cracking distress, Journal of Electronic Imaging 15 (1) (2006) 013017. [27] K. Wang, R. Elliott, Investigation of image archiving for pavement surface distress surface distress survey, A final report submitted to Mack-Blackwell Transportation Center, 1999, pp. 357–405. [28] NCHRP (National Cooperative Highway Research Program)., Automated Pavement Distress Collection Techniques, Transportation Research Record, NCHRP Synthesis 334, 2004. [29] J. Groeger, P. Stephanos, P. Dorsey, M. Chapman, Implementation of automated network-level crack detection processes in Maryland, Transportation Research Record (1860) (2003) 109–116. [30] G. Sun, Q. Liu, C. Ji, X. Li, A novel approach for edge detection based on the theory of universal gravity, Pattern Recognition 40 (10) (2007) 2766–2775. [31] Y. Liu, T. Ikenaga, S. Goto, An MRF model-based approach to the detection of rectangular shape objects in color images, Journal of Signal Processing 87(11) (2007) 2649–2658. [32] Momma, OpenCV Thinning, last visit: http://www.eml.ele.cst.nihon-u.ac.jp/~momma/ wiki/wiki.cgi/OpenCV/ 2008, (March 1, 2011). [33] G. Bradski, A. Kaehler, Learning OpenCV: computer vision with the OpenCV, 20088 ISBN–13: 978–0596516130. [34] C.V. Emgu, Emgu CV: OpenCV in .NET (C#, VB, C++ and More)8 last visit: http:// www.emgu.com/wiki/index.php/Main_Page 20108 (Jun, 2010). [35] C.E.E. News, Machine vision-based damage assessment, last visit: http://www. ce.gatech.edu/news/435/74/Machine-Vision-Based-Damage-Assessment/8 (July, 2010). [36] M. Melia, J. Katz, M. Faul, P. Haven, F. Jordans, As Haiti mourns, quake survivor found in rubble8Last visit: http://savannahnow.com/latest-news/2010-01-24/ haiti-mourns-quake-survivor-found-rubble 20108 (Dec. 21, 2010). [37] E. Wasserman, Are Tennessee bridges safe?8Last visit: http://www.tdot.state.tn. us/documents/Wasserman_editorial.pdf 20078 (Sep. 21, 2009).