Multimed Tools Appl https://doi.org/10.1007/s11042-017-5310-9
Guided 3D point cloud filtering Xian-Feng Han1 Wei Jiang1
· Jesse S. Jin1 · Ming-Jie Wang1 ·
Received: 23 May 2017 / Revised: 5 October 2017 / Accepted: 13 October 2017 © Springer Science+Business Media, LLC 2017
Abstract 3D point cloud has gained significant attention in recent years. However, raw point clouds captured by 3D sensors are unavoidably contaminated with noise resulting in detrimental efforts on the practical applications. Although many widely used point cloud filters such as normal-based bilateral filter, can produce results as expected, they require a higher running time. Therefore, inspired by guided image filter, this paper takes the position information of the point into account to derive the linear model with respect to guidance point cloud and filtered point cloud. Experimental results show that the proposed algorithm, which can successfully remove the undesirable noise while offering better performance in feature-preserving, is significantly superior to several state-of-the-art methods, particularly in terms of efficiency. Keywords 3D point cloud · Guided filtering · Noise removal · Efficiency
1 Introduction The increasing use of affordable 3D data acquisition devices, such as structured light scanners [27], Kinect [4] and time of flight [11], makes 3D point cloud become a new primitive for object representation. However, raw point clouds obtained from these devices inevitably
Jesse S. Jin
[email protected] Xian-Feng Han
[email protected] Ming-Jie Wang
[email protected] Wei Jiang
[email protected] 1
Tianjin University, Tianjin, China
Multimed Tools Appl
suffer from noise due to physical factors [12], limitations of the sensors [11] and illumination situations or artifact in the scene. Therefore, it is necessary to perform a filtering operation on the raw point clouds before further processing [5] (e.g. object recognition [24], 3D reconstruction [23] or vehicles detection [31]). The development of robust filtering algorithms for point cloud has become a currently subject of considerable research. The purpose of such filtering algorithms is to eliminate noise from point cloud while preserving geometric features. Compared with triangular mesh denoising methods, filtering point cloud demonstrates lower overhead in terms of computation time and memory cost as the polygonal-mesh connectivity information [32] or topological consistency [15] needs to be maintained for triangular mesh [8]. The bilateral filter for point cloud has been extensively used in noise reduction recently. It can be considered as a method that builds a filter kernel using guidance point cloud. Although this method can offer good performance for feature-preserving when denoising a point cloud, it has a high time consuming. Motivated by the remarkable results of guided image filtering algorithm introduced by He et al. [7], this paper extends this efficient and effective method to 3D point cloud to propose a novel point cloud filtering approach, called guided point cloud filtering, by taking into account the position of point instead of the intensity attribute. In our method, we first find the nearest neighbors for each point based on K-nearest neighbor or neighbors within radius method, then the filtered output point can be represented as a linear model of the corresponding point of guidance point cloud (Here we use input point cloud itself) using its neighborhood, and finally we minimize a cost function to determine the linear coefficients. Our method is fast and easy to implement. The experimental results show that our approach can obtain impressive results and a better performance than many state-of-the-art methods (e.g. bilateral filter). The contributions of our work are list as follows: (1) the idea of linear transformation is introduced to design a novel point cloud filtering approach with remarkable efficiency and effectiveness. (2) a comprehensively experimental evaluation and comparison between the devised algorithm (Guided 3D point cloud filtering) and several state-of-the-art filtering methods is conducted. The remainder of this paper is structured as follows. A brief overview of point cloud filtering techniques is given in Section 2. Section 3 describes our proposed approach in detail. Experimental results are demonstrated in Section 4 and conclusions are discussed in Section 5.
2 Related work In the field of computer graphics, point cloud filtering has been an on-going research problem and many filtering technologies have been proposed in recent years. Here, we only give attention to these works associated with ours. The bilateral filter, originally introduced by Tomasi and Manduchi [28], is an edge-preserving [22] smoothing filter, which is extended to 3D mesh denoising [3, 13, 16]. However, these methods require mesh generation processing which itself suffers from noise [19]. Therefore, in order to address this problem, the bilateral filter is applied directly on the point cloud [18, 25, 27], based on point position and intensity.
Multimed Tools Appl
To further reduce calculation time, Xu et al. [30] replaced the weight of grey domain in the bilateral filter with a binary function (1) to achieve better performance of improved filter, while this filter demands the point cloud must contain intensity component. As a consequence, normal [14, 19–21, 29], being as one of the important attributes of point cloud, is considered to be utilized in the processing of the bilateral filter, where the weights are defined as a function of spatial location and normal information of point, shown in (2). 1 |I (x, y) − I (i, j )| ≤ 3, I (x, y) = 0 wr = (1) 0 |I (x, y) − I (i, j )| > 3, I (x, y) = 0 (2) w = f [d (p, q)] × g c np , nq Where f and g are the Gaussian function with σf and σg . d(p, q) defines certain distance between point p and its neighbor q, mostly used the Euclidian distance. c(np ,nq ) indicates the normal relation at points p and q, such as inner product of normal vector np , nq [19], (p − q) · np nq [14], np − nq 2 [29] and np · np − nq [20]. Han et al. [6] proposed an iterative guidance normal filter for 3D point cloud based on normal information and bilateral filter. However, it can be seen that efficiency becomes a principal concerning issue of bilateral filter or its improvement methods, especially dealing with point cloud containing large amount of points. All the above methods take a relatively long time to achieve satisfactory results.
3 Guided point cloud filtering The guided image filtering is proposed as a time-efficient feature-preserving smoothing operator [17]. The fundamental idea of the guided image filtering is that the output image is a linear transform of guidance image in a window. However, it is not a straightforward task that directly applying this approach to 3D point cloud due to the differences between image and point cloud. From the perspective of 3D point cloud, the 3D position of a point can be considered as the spatial information. Furthermore, many point clouds do not contain intensity attribute. Therefore, we have to consider the position information of a point to extend the guided filtering technology to point cloud forming a versatile filter, called guided point cloud filtering. The computation of our approach is based purely on local neighborhood. The first step therefore is to determine a neighborhood for each point in point cloud. Given a raw 3D point cloud P={pi ∈ R3 }, a kdtree structure is constructed to represent this point cloud and the neighborhood N(pi ) = {pij ∈ P} (including pi ) for each point pi is searched by using the k-nearest neighbor (KNN) method or r-ball (neighbors within radius) strategy, where pij denotes the jth neighboring point around pi . From the set of neighbors, we can calculate one quantity, named the centroid pi of the neighborhood of point pi , which is estimated by 1 p¯ i =
(3) p ij
N pi
pij ∈N (pi )
where N (pi ) represents the cardinality of N(pi ).
Multimed Tools Appl
Once the neighborhood of pi is found, we then begin to make an assumption that the filtered output point cloud has a locally linear model with respect to guidance point cloud (Here, we regard the input point cloud P as the guidance point cloud) in the neighborhood N(pi ), that is, a linear transform of each point pij in N(pi ) is performed to yield the corresponding output point. p ij = ai pij + bi
(4)
Where pij indicates the filtered point. ai and bi are the coefficients of our linear model, which are computed by minimizing the following function in the N(pi ).
2 ai pij + bi − p ij + ai2 J (ai , bi ) = (5) p ij ∈N (p i ) Where is a parameter controlling the filtered effects that will be discussed in Section 4. We get the solution to (5) as below.
1 ¯ ¯ p · p − p · p ij ij i i |N (pi )|
ai = (6) 1 ¯ ¯ p · p − p · p + ij ij i i |N (pi )| bi = p¯ i − ai · p¯ i
(7)
Finally, we need to perform the linear transformation in every neighborhood computed for each point of the entire point cloud model. However, there exists one problem that the output point pij in (5) may correspond to many different values, resulting from the fact that point pij may be contained in many neighborhoods. Unlike guided image filtering, after computing ai and bi in the neighborhood N(p i ), we compute the filtered output point for p i in its own neighborhood N(pi ). pi = ai pi + bi Details of guided point cloud filtering algorithm are described as follows.
(8)
Multimed Tools Appl
Fig. 1 Filtering results using different values of radius. a noisy office chair model; (b) r = 0.003; (c) r = 0.01; (d) r = 0.03; (e) r = 0.05; (f) r = 0.1
Fig. 2 Filtering results using different values of k. a k = 10; (b) k = 30; (c) k = 50; (d) k = 100; (e) k = 200; (f) k = 300
Multimed Tools Appl Table 1 Running times using different raidus on Office Chair Model
Radius
0.003
0.01
0.03
0.05
0.08
0.1
Time(ms)
931
1486
6891
18372
46012
70431
Table 2 Running times using different k for KNN on Office Chair Model
k
10
30
50
100
200
300
Time(ms)
858
2126
3441
7512
17400
81494
Fig. 3 Filtering results using different values of . a = 0.01; (b) = 0.04; (c) = 0.16
Table 3 Timings for our method and other filtering methods on different models (in ms) Model
Points
Bilateral filter
MLS
WLOP
EAR
Ours(k=50)
Office chair
29,148
8,269
7,776
14,962
71,082
2,828
i-H Bunny
34,834
5,428
8,015
17,735
87,920
3,053
Julius
36,201
11,516
9,875
21,125
73,998
3,450
Sofa
48,668
14,836
15,863
24,923
98,276
4,125
Coffee-mug
49,017
12,750
11,161
25,096
122,659
4,265 5,599
Bowl
53,166
13,853
13,662
27,285
101,910
Iron
85,574
24,144
22,558
46,702
175,409
7,927
Armadillo
172,974
20,225
23,803
99,650
365,790
15,361
Dragon
437,645
514,836
80,013
227,273
940,147
40,609
Table
460,400
590,851
89,597
261,235
995,090
57,844
Multimed Tools Appl
4 Experimental results The algorithm introduced in Section 3 is implemented using C++ language and Point Cloud Library(PCL) [1, 26]. We demonstrate the experimental results of our approach and conduct a comparative evaluation of the performance tested on a variety of point clouds corrupted with Gaussian noise. The noise is generated using a zero-mean Gaussian function with a standard deviation. And we use the PC with Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz and 16GB memory to carry out these experiments.
4.1 Parameters As for our algorithm, the selector of the radius parameter r or the k of KNN, which is utilized for calculating a geometrical neighborhood, is a trade-off between computational time and filtering effect. If r or k is too small, less points are involved in the neighborhood, which leads to fast computation speed. However, some noise may not be removed; if r or k is too large, it increases the computation cost due to the number of neighbors becoming larger. In addition, the results may lose features and produce shrinkage obviously. Figures 1 and 2 illustrate the results of applying guided point cloud filtering on office chair model using
Table 4 Error metrics for different methods and the best error metrics values are highlighted in bold Model
Error
Bilateral filtering
MLS
WLOP
EAR
Ours
Chair
δ
20.336
8.962
8.904
8.785
8.527
Dmean
0.307
0.133
0.134
0.132
0.030
δ
6.021
5.862
5.710
5.553
2.654
Dmean
0.092
0.089
0.089
0.085
0.041
i-H bunny Sofa Julius Coffee-mug Bowl Iron Armadillo Dragon Table
δ
10.126
10.388
5.334
5.186
5.955
Dmean
0.169
0.167
0.087
0.087
0.095
δ
10.647
3.409
2.166
3.378
5.240
Dmean
0.177
0.058
0.038
0.058
0.091
δ
12.729
7.655
6.447
6.498
6.339
Dmean
0.201
0.120
0.100
0.101
0.100
δ
21.676
6.149
7.667
7.311
6.132
Dmean
0.312
0.086
0.107
0.102
0.083
δ
7.513
5.844
6.182
6.180
5.101
Dmean
0.116
0.091
0.097
0.096
0.079
δ
3.616
3.599
5.124
5.139
3.401
Dmean
0.052
0.052
0.097
0.075
0.050
δ
23.036
20.816
20.816
17.646
16.095
Dmean
0.359
0.299
0.310
0.262
0.239
δ
29.313
17.010
28.540
21.918
14.436
Dmean
0.502
0.277
0.463
0.358
0.198
Multimed Tools Appl
Fig. 4 Execution times for filtering, as a function of the size of input point cloud model
Fig. 5 a noisy bowl model; (b) filtering result with bilateral filter; (c) MLS; (d) WLOP; (e) EAR; (f) filtering result with Ours(k = 50)
Multimed Tools Appl
different values of r and k respectively and the corresponding running times are presented in Tables 1 and 2. It is clearly demonstrated from the observation that r = 0.03 or k = 50 provides a desirable result. The second key parameter is , simply controlling the filtering quality. Smaller values of will result in much noise unremoved. Filtering effects applying different values of to the same noisy office chair model are given in Fig. 3. Both of these two important parameters require users to tune to adjust the filtering effect and efficiency depending on experimental requirements and point cloud data.
4.2 Results and comparison After a comprehensive analysis of the effect of the parameters in our proposal, we now proceed to carry out experiments to compare our method with several state-of-the-art algorithms aiming at exhibiting the performance and effectiveness of our approach.
Fig. 6 a noisy iron model; (b) filtering result with bilateral filter; (c) MLS; (d) WLOP; (e) EAR; (f) filtering result with Ours(k = 50)
Multimed Tools Appl
4.2.1 Selected methods In our experiments, we choose four different filtering methods, namely, Normal-based Bilateral Filter(NBF), Moving Least Square(MLS) [2], Weighted Locally Projection(WLOP) [9], Edge Aware Resample(EAR) [10]. And these algorithms are widely cited and commonly used for comparison in related literature. Moreover, all these filtering proposals, implemented using C++ and run using the same computer mentioned above, are tested on different point cloud models corrupted with Gaussian noise.
4.2.2 Efficiency Since point cloud generally contains a huge number of points, which brings noticeable impact on the running efficiency of filtering approaches, so we here conduct an experiment to give a comprehensive evaluation of the considered filtering algorithms and our proposal in terms of computational efficiency.
4.2.3 Measures To evaluate the quality of the denoising results, two error metrics δ and Dmean are used. The ground truth shapes of all models used in our paper are known.δ represents the averaged
Fig. 7 a noisy sofa model; (b) filtering result with bilateral filter; (c) MLS; (d) WLOP; (e) EAR; (f) filtering result with Ours(k = 50)
Multimed Tools Appl
angle over all angles between the ground truth point normals and the resulting point normals.Dmean the average distant from the resulting points to the corresponding ground truth points.
4.2.4 Results and discussion The remarkable advantage of guided point cloud filter is its ability to lower time consuming significantly due to the linear filtering model we adopted. Table 3 demonstrates a summary of computational timings for our method and others over different point cloud models with different number of points. Figure 4 also illustrates the corresponding running times as a function of size of input point cloud model. And it can be obviously reported that our guided point cloud filter achieves a dramatically efficient result, which is one or nearly two orders of magnitude faster compared with other algorithms. In addition, with the increase of the number of points, although our approach becomes time-consuming, it still remarkably outperforms other methods. The error metrics for these methods are shown in Table 4. According to these values, it can be seen evidently that our method can achieve better filtering effectiveness on most point cloud models as expected, with the exception of sofa model and Julius model. Figures 5, 6, 7, 8 and 9 are the filtering results of guided point cloud filtering and other filtering methods. Here, the size of neighborhood k=50 is adopted for other methods that is the same as ours (k=50). Considering feature preservation, it can be obviously seen that our
Fig. 8 a noisy table model; (b) filtering result with bilateral filter; (c) MLS; (d) WLOP; (e) EAR; (f) filtering result with Ours(k = 50)
Multimed Tools Appl Fig. 9 a noisy bunny model; (b) filtering result with bilateral filter; (c) MLS; (d) WLOP; (e) EAR; (f) filtering result with Ours(k = 50)
method can yield a smoother result whist preserving shape features more accurately, like Fig. 9. Furthermore our method can also work on real-world 3D point cloud as well. Overall, as for point clouds, especially these with a large number of points, our filtering algorithm can obtain more satisfactory results than many other filtering strategies in terms of effectiveness and efficiency. The relatively accurate models yielded by our algorithm may be more suitable for further processing.
5 Conclusion In this paper, we proposed an effective and efficient filtering method, called guided point cloud filtering, which can be regarded as an extension of guided image filtering by taking into account position of point being as critical information. The main idea of our method is that the filtered output point can be deduced as a linear model of the corresponding point of guidance point cloud using its neighborhood. Experimental results tested on different point clouds demonstrated that our approach not only performs feature-preserving filtering, but also can be computed efficiently and accurately. Furthermore, it is conceptually simple and easy to implement our algorithm. However, one limitation is that our method does not recover the sharp features such as corner well. Therefore, in the future, we will further investigate our method to address this problem. And meanwhile we will also take geometric attributes, such as normal and
Multimed Tools Appl
curvature into account to modify our algorithm to increase the performance of filtering scheme.
References 1. Aldoma A, Marton ZC, Tombari F, Wohlkinger W, Potthast C, Zeisl B, Rusu RB, Gedikli S, Vincze M (2012) Tutorial: point cloud library: three-dimensional object recognition and 6 DOF pose estimation. IEEE Robot Autom Mag 19(3):80–91 2. Alexa M, Behr J, Cohenor D, Fleishman S, Levin D, Silva CT (2003) Computing and rendering point set surfaces. IEEE Trans Vis Comput Graph 9(1):3–15 3. Fleishman S, Drori I, Cohen-Or D (2003) Bilateral mesh denoising. Acm Trans Graph 22(3):950–953 4. Han J, Shao L, Xu D, Shotton J (2013) Enhanced computer vision with Microsoft Kinect sensor: a review. IEEE Trans Cybern 43(5):1318–1334 5. Han XF, Jin JS, Wang MJ, Jiang W, Gao L, Xiao L (2017) A review of algorithms for filtering the 3D point cloud. Signal Process Image Commun 57:103–112 6. Han XF, Jin JS, Wang MJ, Jiang W (2017) Iterative guidance normal filter for point cloud. Multimed Tools Appl. https://doi.org/10.1007/s11042-017-5258-9 7. He K, Sun J, Tang X (2013) Guided image filtering. IEEE Trans Pattern Anal Mach Intell 35(6):1397– 1409 8. Hu G, Peng Q, Forrest AR (2006) Mean shift denoising of point-sampled surfaces. Vis Comput 22(3):147–157 9. Huang H, Li D, Zhang H, Ascher U, Cohen-Or D (2009) Consolidation of unorganized point clouds for surface reconstruction. Acm Trans Graph 28(5):89–97 10. Huang H, Wu S, Gong M, Cohen-Or D, Ascher U, Zhang H (2013) Edge-aware point set resampling. Acm Trans Graph 32(1):60–61 11. Huhle B, Schairer T, Jenke P, Strasser W (2008) Robust non-local denoising of colored depth data. In: IEEE Cvpr workshop on time of flight camera based computer vision, pp 1–7 12. Jenke P, Wand M, Bokeloh M, Schilling A et al (2006) Bayesian point cloud reconstruction. Comput Graph Forum 25(3):379–388 13. Jones TR, Durand F, Desbrun M (2003) Non-iterative, feature-preserving mesh smoothing. Acm Trans Graph 22(3):943–949 14. Jones TR, Durand F, Zwicker M (2004) Normal improvement for point rendering. IEEE Comput Graph Appl 24(4):53–56 15. Kobbelt L, Botsch M (2004) Point-based computer graphics. Comput Graph 28(6):801–814 16. Lee KW, Wang WP (2005) Feature-preserving mesh denoising via bilateral normal filtering. In: International conference on computer aided design and computer graphics, pp 275–280 17. Liu S, Chan KC, Wang CCL (2012) Iterative consolidation of unorganized point clouds. IEEE Comput Graph Appl 32(3):70–83 18. Ma S, Zhou C, Zhang L, Hong W (2014) Depth image denoising and key points extraction for manipulation plane detection. In: World Congress on intelligent control and automation, pp 3315– 3320 19. Miropolsky A, Fischer A (2004) Reconstruction with 3D geometric bilateral filter. In: ACM Symposium on solid modeling and applications, pp 225–229 20. Moorfield B, Haeusler R, Klette R (2015) Bilateral filtering of 3D point clouds for refined 3d roadside reconstructions? Lect Notes Comput Sci 9257:394–402 21. Nasab SE, Ghaleh SF, Ramezanpour S, Kasaei S (2014) Permutohedral lattice in 3D point cloud processing. In: International symposium on telecommunications 22. Paris S (2007) A gentle introduction to bilateral filtering and its applications. In: SIGGRAPH. ACM, pp 853–879 23. Park J, Kim H, Tai YW, Brown MS, Kweon I (2011) High quality depth map upsampling for 3D-TOF cameras. In: CVPR. IEEE, pp 1623–1630 24. Rangel JC, Morell V, Cazorla M, Orts-Escolano S, Garca-Rodrguez J (2016) Object recognition in noisy RGB-D data using GNG. Formal Pattern Anal Appl 1–16 25. Rosli NAIM, Ramli A (2014) Mapping bootstrap error for bilateral smoothing on point set. 21st Nat Symp Math Sci 1605(1):149–154 26. Rusu RB, Cousins S (2011) 3D is here: point cloud library (PCL). In: IEEE International conference on robotics and automation, pp 1–4
Multimed Tools Appl 27. Shi BQ, Liang J, Liu Q (2011) Adaptive simplification of point cloud using K means clustering. ComputAided Design 43(8):910–922 28. Tomasi C, Manduchi R (1998) Bilateral filtering for gray and color images. In: CVPR. IEEE, pp 839–846 29. Wand M, Berner A, Bokeloh M, Jenke P, Fleck A, Hoffmann M, Maier B, S-taneker D, Schilling A, Seidel HP (2008) Processing and interactive editing of huge point clouds from 3D scanners. Comput Graph 32(2):204–220 30. Xu W, Lee IS, Lee SK, Lu B, Lee EJ (2015) MultiView-based hand posture recognition method based on point cloud. Ksii Trans Int Inf Sys 9(7):2585–2598 31. Yan C, Xie H, Yang D, Yin J, Zhang Y, Dai Q (2017) Supervised hash coding with deep neural network for environment perception of intelligent vehicles. IEEE Trans Intell Transp Syst 32. Zwicker M, Pauly M (2004) Point-based computer graphics. Comput Graph 28(24):22–23
Xian-Feng Han is a Ph.D. candidate at the High-dimensional Information Processing Laboratory, Tianjin University. His research interests include image processing, computer vision and computer graphics.
Prof. Jesse S. Jin graduated with a B.Eng from Shanghai Jiao Tong University and a Ph.D. from University of Otago, New Zealand. He is a recruit professor in Tianjin University under the China Talent Program. He held the Chair Professor of IT and other academic positions in many universities in Australia. He also chaired the Academic Board of the College of Design and Commerce, and was an independent board member of Raffles University. He has published 343 articles and 13 books. His research interests include image processing, computer vision, multimedia, medical imaging, etc.
Multimed Tools Appl
Ming-Jie Wang is a master candidate at the High-dimensional Information Processing Laboratory, Tianjin University. His research interests include image processing, computer vision and machine learning.
Wei Jiang is a Ph.D. candidate at the High-dimensional Information Processing Laboratory, Tianjin University. His research interests include image processing, computer vision and machine learning.
