Ultramicroscopy 151 (2015) 62–67
Contents lists available at ScienceDirect
Ultramicroscopy journal homepage: www.elsevier.com/locate/ultramic
A nonlinear filtering algorithm for denoising HR(S)TEM micrographs$ Hongchu Du a,b,c,n a b c
Ernst Ruska-Centre for Microscopy and Spectroscopy with Electrons, Jülich Research Centre, Jülich, 52425, Germany Central Facility for Electron Microscopy (GFE), RWTH Aachen University, Aachen 52074, Germany Peter Grünberg Institute, Jülich Research Centre, Jülich 52425, Germany
art ic l e i nf o
a b s t r a c t
Article history: Received 25 August 2014 Received in revised form 1 November 2014 Accepted 6 November 2014 Available online 17 November 2014
Noise reduction of micrographs is often an essential task in high resolution (scanning) transmission electron microscopy (HR(S)TEM) either for a higher visual quality or for a more accurate quantification. Since HR(S)TEM studies are often aimed at resolving periodic atomistic columns and their non-periodic deviation at defects, it is important to develop a noise reduction algorithm that can simultaneously handle both periodic and non-periodic features properly. In this work, a nonlinear filtering algorithm is developed based on widely used techniques of low-pass filter and Wiener filter, which can efficiently reduce noise without noticeable artifacts even in HR(S)TEM micrographs with contrast of variation of background and defects. The developed nonlinear filtering algorithm is particularly suitable for quantitative electron microscopy, and is also of great interest for beam sensitive samples, in situ analyses, and atomic resolution EFTEM. & 2014 Elsevier B.V. All rights reserved.
Keywords: HR(S)TEM Noise reduction Denoising Filtering Nonlinear filter Image processing
1. Introduction Aberration-corrected high resolution (scanning) transmission electron microscopy (HR(S)TEM) enables quantitatively imaging atomic structures of condensed matters at sub-angstrom resolution [1,2]. Nowadays, electron microscopes are widely equipped with CCD cameras. HR(S)TEM micrographs recorded from a CCD camera tend to be degraded by noise. A low signal-to-noise ratio (SNR) often makes the accurate quantification difficult. The SNR can be improved by either increasing the signal intensity or decreasing the noise level. When it is possible, the increase of signal intensity is of higher priority than the decrease of noise level in achieving high SNR. However, for the majority of beam sensitive samples or in situ analyses, the increase of signal intensity may not be practical. Under these circumstances, any improvement of the SNR through noise reduction by filtering is incredibly valuable. There are several important sources of noise in a micrograph [3]: (i) quantum noise (shot-noise) of electron beam; (ii) dark current noise from thermally generated electrons; (iii) the socalled Fano noise from electron–photon and photon–electron conversions; (iv) read-out noise from electronic devices to read ☆ It is a great pleasure to dedicate this paper to Prof. Harald Rose in celebration of his 80th birthday. n Correspondence address: Ernst Ruska-Centre for Microscopy and Spectroscopy with Electrons, Jülich Research Centre, Jülich 52425, Germany. fax: þ49 2461 61 6444. E-mail address:
[email protected]
http://dx.doi.org/10.1016/j.ultramic.2014.11.012 0304-3991/& 2014 Elsevier B.V. All rights reserved.
the image from a CCD. The Poisson statistics would apply for the first three sources of noise, whereas Gaussian for the read-out noise. The dark current noise and the read-out noise are independent of beam intensity. The Fano noise is empirically described as a function of photon energy, while the shot noise is proportional to the square root of the number of recorded electrons per pixel. At low to moderate electron dose, the shot-noise tends to dominate the noise in electron micrographs because of its dependence on the beam intensity. For more than 10 electrons detected, the Poisson distributed shot-noise appears to approach a Gaussian distribution with its standard deviation proportional to the square root of the number of detected electrons in each pixel. Therefore it usually is valid to assume an experimental electron micrograph (Iexp ) as a certain theoretically predicted image (Ith ) from a specific model plus uncorrelated additive Poisson or Gaussian (Inoise , σ 2 = Ith ) noise for simplification, so that
Iexp = Ith + Inoise
(1)
Noise reduction can be considered as an inverse process to obtain an estimated theoretical micrograph (Iest ) by applying a filter to the Iexp :
Iest = Iexp ⊗ F
(2)
The noise reduction algorithms in general can be categorized into spatial and temporal filtering [4]. Frequently used spatial filters for denoising HR(S)TEM images include Bragg filter [5], Wiener filter
H. Du / Ultramicroscopy 151 (2015) 62–67
[6–9], and Gaussian filter [10,11]. Whereas the simplest method of temporal filtering is frame averaging [4]. Registration of frames to one another is often essential before frame averaging to account for drift of the sample between frames. Both rigid [12] and nonrigid [13,14] registration methods have been reported for frame averaging to obtain high SNR images. Because HR(S)TEM studies are often aimed at resolving periodic atomistic columns and their non-periodic deviation at defect areas, it is important to develop a noise reduction algorithm that can simultaneously handle both periodic and non-periodic features properly. It is possible, often justified, to find out how complete the noise is removed and how well the periodic and non-periodic atomistic information is preserved by inspecting the difference (Idiff ) between the recorded Iexp and the denoised Iest images:
Idiff = Iexp − Iest
63
and its estimation Iest (Fig. 2b). An approximate solution was given in the literature [7] as
F wiener ≈
∣Ith ∣2 − ∣Inoise ∣2 ∣Iexp ∣2
(5)
However, artifacts are found in the Wiener filtered image when the intensity of background varies and non-periodic feature is present (Fig. 2d). To solve these problems, a nonlinear filtering algorithm has been developed, which can efficiently reduce noise in HR(S)TEM micrographs without noticeable artifacts even for contrast of variation of background and defects. Moreover, peak position and intensity can be more accurately determined from the nonlinear filtered images.
(3)
The Idiff actually is estimated noise. In this work, simulated instead of experimental images were used as the first testing ground in order to justify the performance of filters by exactly knowing the true signal, Ith , so that an error image (Ierr ) can be obtained:
Ierr = Ith − Iest
(4)
2. Tests of Gaussian and Wiener filters A super cell of MgO with terrace and an edge dislocation with ¯ 〉 was constructed (Fig. 1a). A HRTEM Burgers vector of a/2 〈110 image of the super cell with size of 512 512 pixels (0.01 nm/ pixel) was simulated using the optimized FEI Titan 80–300 parameters at 300 kV under a negative spherical aberration (Cs) imaging (NCSI) condition (Fig. 1b), which is considered to be Ith . The maximum intensity of the noise-free image was normalized to 100. Poisson noise was included so that at each pixel the standard deviation of noise is proportional to the square root of the intensity, which results in the maximum SNR that is about 10. The image with Poisson noise included (Fig. 1c) is used as experimental image (Iexp ). Gaussian low-pass filtering is one of the simplest ways to reduce high spatial frequency noise in HR(S)TEM micrographs [15,16]. Fig. 2a shows a filtered image of the testing HRTEM image of the MgO super cell (Fi. 1c) by convoluting a Gaussian kernel (s ¼2 pixels and kernel size of 5 5 pixels). The peak attenuation is evidently seen in the error image (Fig. 2c), which hinders faithful quantification of the peak intensity. Wiener filter can effectively reduce peak attenuation by minimizing the summed square differences between the true signal Ith
a
b
3. Nonlinear filtering algorithm Reduction of peak attenuation of 1D spectra has been reported by adding low-pass filtered residuals to the original low-pass filter output [17]. The nonlinear filtering algorithm described in this paper is by adding residuals of Wiener filtering (Fw ) to the output of low-pass filtering (Flp ), or vice versa through an iterative process. Fig. 3 shows the flowchart of the nonlinear filtering algorithm. At each iteration, the output is used as input for the next iteration. The mathematic form of the nonlinear filter (Fnl ) can be described as N
Fnl =
∏ (Flp, i + F w, i
− Flp, i ·F w, i )
i=1
(6)
Provided that all Flp, i and Fw, i are known, the algorithm can be time-efficiently designed and implemented so that the Fnl is directly calculated without an iterative process. The cutoff frequency of the nonlinear filter is defined by the cutoff frequencies of lowpass filters.
4. Performance of the nonlinear filter 4.1. Simulated images Tests were made to investigate the performance of the nonlinear filter in noise reduction and quantification of the simulated HR(S)TEM micrographs. Fig. 4a shows the nonlinear filtered image of the testing HRTEM image of the MgO super cell (Fig. 1c). No noticeable peak attenuation and artifacts are seen in the error
c
c a
b
¯ 〉 (Mg: orange, O: blue), (b) simulated noise-free HRTEM image under a Fig. 1. (a) A super cell of MgO with terrace and an edge dislocation with Burgers vector of a/2 〈110 negative-Cs-imaging condition with the maximum of intensity normalized to 100 (512 512 pixels), and (c) with Poisson noise included. (For interpretation of the references to color in this figure caption, the reader is referred to the web version of this paper.)
64
H. Du / Ultramicroscopy 151 (2015) 62–67
a
b
c
d
Fig. 2. (a) Gaussian (s ¼2 pixels and kernel size of 5 5 pixels) and (b) Wiener filtered images of the simulated HRTEM image with Poisson noise included (Fig. 1c), (c) and (d) are error images of (a) and (b), respectively.
Start
XIN i=i+1 i
