SPATIAL-VARIANT MORPHOLOGICAL FILTERS WITH NONLOCAL-PATCH-DISTANCE-BASED AMOEBA KERNEL FOR IMAGE DENOISING

Filters of the Spatial-Variant amoeba morphology can preserve edges better, but with too much noise being left. For better denoising, this paper presents a new method to generate structuring elements for Spatially-Variant amoeba morphology. The amoeba kernel in the proposed strategy is divided into two parts: one is the patch distance based amoeba center, and another is the geodesic distance based amoeba boundary, by which the nonlocal patch distance and local geodesic distance are both taken into consideration. Compared to traditional amoeba kernel, the new one has more stable center and its shape can be less influenced by noise in pilot image. What’s more important is that the nonlocal processing approach can induce a couple of adjoint dilation and erosion, and combinations of them can construct adaptive opening, closing, alternating sequential filters, etc. By designing the new amoeba kernel, a family of morphological filters therefore is derived. Finally, this paper presents a series of results on both synthetic and real images along with comparisons with current state-of-the-art techniques, including novel applications to medical image processing and noisy SAR image restoration.


INTRODUCTION
Currently, one very active research area in mathematical morphology is on the construction of Spatially-Variant (SV) or adaptive morphological operators, which are based on SV structuring elements (SEs) that adjust their shape and size according to the local context of the image.A number of different methods for the construction of adaptive SEs have been proposed, for example, morphological amoebas (Lerallut et al., 2007), Adaptive Neighborhood Morphology (Debayle and Pinoli, 2005), bilateral structuring functions (Angulo, 2011) and salience adaptive structuring elements (Curic, et al., 2012).These examples are practical special cases of SV mathematical morphology and each of them corresponds to a special choice of the SE mapping θ that is application oriented.This paper mainly deals with amoeba morphology for denoising.As we know, for denoising, morphological filters based on amoeba kernel have good performance in preserving the detailed structures while removing noise compared with classical morphological filters (SE is fixed and basic morphological operations are invariant under translation).However, the amoeba structuring elements are derived from a smoothed version (pilot image) of the original noisy image (Lerallut et al., 2007).Generally, there is a large amoeba distance between the noise point (not removed in the pilot image) and its surrounded pixel points; thereby the growth of amoeba body is limited.Consequently, the filter based on morphological amoebas preserves edges better, but it is relatively poor in denoising, especially in the case of images with serious noise.
Recently, nonlocal schemes for image processing have received a lot of attention, starting from the initial paper by Baudes et al. (2005).Salembier (2009) proposed a straightforward generalization of nonlocal means filter to morphological filters.Ta et al. (2011) introduced a formalism of graph-based nonlocal morphology by generalizing the PDE of dilation and erosion.It is obvious that such approach does not induce a couple of adjoint dilation and erosion, and consequently their products do not involve openings and closings.Velasco-Forero and Angulo (2013) dis-cussed the necessary algebraic properties of nonlocal morphology and introduced an algorithm of sparse nonlocal morphology.
In this paper, a new weighted distance to generate SE for amoeba morphological filters is defined.The local geodesic distance and nonlocal patch distance are naturally combined in the proposed weighted amoeba kernel.Indeed, the new amoeba kernel is innovative for the following reason that the SE is divided into two parts: one is the amoeba center, where nonlocal patch distance are used by computing the similarity between the central pixel and the pixel in its 8-neighbourhoods, another is the amoeba boundary, where the geodesic distance is used by computing the similarity between two adjacent pixels.This new amoeba distance naturally combine local and nonlocal configurations and make nonlocal patchbased processing method becomes local processing on amoeba structuring elements, which guarantees the SE could be a connected component.Therefore， what distinguish the methods from the nonlocal morphology (Salembier, 2009;Ta et al., 2011;Velasco-Forero and Angulo, 2013) is that the new amoeba morphology in this paper is still a study of the local SV morphology.Compared with traditional amoeba morphology, new amoeba morphological filters have better properties when applied to vehicle license plate (VLP) image, medical image and synthetic aperture radar (SAR) image restoration.

METHODS LOCAL GEODESIC DISTANCE BASED AMOEBA MORPHOLOGY
Amoeba morphology (Lerallut et al., 2007) is a particular case of SV morphology.Amoeba structuring elements take into account the image contour variations to adapt their shape.Let D be a subset of the Euclidean space ℤ 2 that corresponds to the support of the image, and let T ⊂ ℝ be a set that corresponds to the gray level values in the image.Then, a gray level image can be represented by a function f : D → T. A 2D image can be represented by a surface, S , embedded in 3D space, with two spatial coordinates and one coordinate that represents the gray level value in the image.A geodesic distance between two points ( , ( )),( , ( )) , , x f x y f y S x y D ∈ ∈ is the cost to travel from one point to the other.In other words, a geodesic distance corresponds to the shortest time required to travel from a point (x, f(x)) to a point (y, f(y)) along the surface S. A path ρ xy that connects two points x and y can be considered as a set {x 0 ,x 1 ,...x n }, where x = x 1 , y = x n , and x i , x i+1 , i = 1,..., n-1 are two adjacent pixels in the path.The rule of judging the pixel belonging to an amoeba is designed as: The amoeba distance with parameter λ is defined by d λ (x, y) = min L(ρ xy ), while a parameter λ regulates a difference between the two incommensurate domains, and as such has a strong influence on the size of the morphological amoebas (given a fixed threshold r).Define the SE mapping A as , ( ) { : ( , ) } (2) Similar work was done by Grazzini and Soillev (2009) that proposed spatially variable neighborhoods by utilizing geodesic distances, used the following costs between two adjacent pixels i x and 1 and The difference between Eq. 3 and Eq. 4 is in the range of the gradient they use.For these methods of SV morphology based on local geodesic distance, for convenience, are referred to as traditional amoeba morphology or local geodesic distance-based amoeba morphology.The traditional amoeba morphological filter preserves edges better while removing noise compared with classical morphological filters.However, it is noteworthy that the structuring elements are derived from the pilot image.As we know, there is still too much noise in the pilot image and the geodesic distance value between the noise point (central pixel) and the pixels surrounding it is quite large.Consequently, noise in pilot image limits the growth of amoeba body and has a large impact on the construction of amoeba SEs.

NEW SPATIALLY-VARIANT MORPHOLOGICAL FILTERS Designing new amoeba kernels
To solve the aforementioned issues in the traditional amoeba morphology and an elicitation got from the research of nonlocal morphology (Salembier, 2009;Ta et al., 2011;Velasco-Forero and Angulo, 2013), nonlocal patch distance can be considered for computing the similarity between two adjacent pixels on the construction of amoeba kernel.First, in order to understand how and why nonlocal filter works, we will begin with a brief description of nonlocal mean filter, which have been proposed by Baudes et al. (2005) mainly for denoising applications.The filtering idea consists in computing a weighted average of the input signal in a neighborhood Where the weights w(x, y) are defined by computing the similarity between a patch P centered around the pixel x to process and a patch around the pixel at position y .If the patch around the current pixel x is very similar to the patch centered around y, then the weight w(x, y) should be close to one.On the contrary, if both patches are very different, then the weight w(x, y) should be close to zero.As can be seen, the weighted average takes into account mainly pixels that are surrounded by a patch that is similar to the one surrounding the pixel being processed.This is the key point explaining the robustness of this filter.The weights are defined as follows: In this formula, the similarity between the patches centered around x and y is computed through a weighted Euclidean distance where m represents the indexes used to scan the patch P and g(m) is a Gaussian kernel (α > 0is the standard deviation of the Gaussian kernel), Z is a normalizing constant ensuring that ( , ) 1 It is important to note that, for every pixel point in the amoeba neighborhood, if the traditional geodesic distance is completely replaced by the patch distance, the amoeba's body would be overgrowth at place with strong gradient and hence, the main property of the amoeba filter in better detailed structures preservation will be weakened.Indeed, the originality of our approach lies in combining both local geodesic distance and nonlocal patch distance in an amoeba shape kernel.the key point of the new amoeba kernel is that the SE is divided into two parts: one is the amoeba center, where nonlocal patch distance are used by computing the similarity between the center pixel and the pixel in its 8-neighbourhoods; the other is the amoeba boundary, where the geodesic distance is used by computing the simi-larity between two adjacent pixels.
In our implementation, we use 8-neighbourhoods that include horizontal, vertical, and diagonal neighbors, let 1 2 ( , ,... , ) xy n x x x x y ρ = = = be a path between the points x and y .At amoeba center, the cost between the central points (origin) 1 x and 2 In this formula, the similarity between the patches centered around 1 x and 2 x is computed through a weighted distance, where m represents the indexes used to scan the patch l P ( 0 l > is the size of patch l P ) and ( ) g m is a Gaussian kernel ( 0 α > is the standard deviation of the Gaussian kernel) giving higher (lower) weights to the central (outer) pixels of the patches.The parameter l and α are related to the noise type, noise statistics and control the degree of filtering of the obtained solution.At amoeba boundary, the cost between the points i x and 1 This definition is similar to the traditional amoeba distance in Eq. 1 and λ is a weighted parameter.
Usually, tonal distance (gray-value difference) x x + − , nevertheless the Euclidean distance or a weighted distance, i.e., 3,4 distance are often used (Ikonen and Toivanen, 2005).The 3,4 distance is defined as The chamfer 3/4 distance can be achieved via a serial scanning algorithm using a mask M 1 and mask M 2 in predefined neighborhood, as shown in Fig. 1.According to the above definition, the cost, C, of the path ρ xy is And the new amoeba distance between the points x and y is designed as: Minimal distance d(x, y) can be achieved via a Dijkstra shortest path algorithm, and then new amoeba SE with the origin x is defined as Where r is the distance threshold in pilot image, Fig. 2a shows an example of new amoeba distance definition.Square windows of size l = 3 (patch) centered at origin pixel x 1 and its adjacent pixel x 2 is used by computing the similarity of this two pixel points at amoeba center region.For the adjacent pixels x 2 and x 3 , at amoeba boundary region, the geodesic distance is used by computing the similarity of them.
We propose new amoeba structuring elements that are derived using two different distances on the constructed amoeba kernel.It is straightforward to validate the following properties of amoeba SE: 1) Reflexivity: , ( ) 2) Monotonicity with respect to radius: Eq. 14 is useful for multi-scale filtering, such as a family of alternate sequential filters.The concern with this approach is that the symmetry is not satisfied for adaptive amoeba neighborhood, i.e., the following equivalence is not valid Such an asymmetric property can be precisely used to avoid the large influence on the growth of normal pixel SE by noisy points, as illustrated in Fig. 2b.Besides, from Eq. 7, it is easily seen that That means that if the value of α is close to zero, the patch distance would become to a single pixel gray distance, so, in this case, we can say that the traditional amoeba is a special form of our newly proposed amoeba.Therefore, in this paper the nonlocal patchbased method is combined with the traditional amoeba approach to form a new framework.
Once the new amoeba structuring elements are computed for each point in the image, we can compute morphological operators using these SEs.A Morphological erosion for the SEs can be computed by taking the infimum of the values over the SE.The dilation is obtained by replacing the infimum with the supremum.If an erosion and a dilation are defined in this way they are dual operators.However, these two operators are not adjunct operators.Two operators δ and ε are where , : f g D → Τ .It is necessary that erosion and dilation are adjunct operators in order to compute the morphological opening and closing.Only if an erosion and a dilation are adjunct operators, their combination satisfies properties of openings and closings.The morphological operators, erosion and dilation with new amoeba structuring elements, which satisfy adjunction Eq.17, can be defined, respectively, as Where ( ) r x θ ( is the reflected neighborhood.Then, the corresponding opening and closing are defined by respectively.The method to compute the reflected neighborhood of amoeba structuring elements was proposed by Lerallut et al. (2007), further similar work was done by Roerdink (2009) and Curic, et al. (2012).It is stressed that morphological operators δ and ε satisfy an adjunction Eq. 17 only if adaptive structuring elements are derived once for the input image.

SV discrete filters based on new amoeba
Once the new amoeba structuring elements are derived from the pilot image, the SV discrete filters can be computed as well.We shall mainly consider SV alternating sequential filters (SVASF) and SV mean/median filters for image restoration and aims to provide a general framework for the SV discrete fil-ters based on new amoeba kernel.Classical alternating sequential filters were first introduced by Sternberg (1986).These filters in mathematical morphology are a combination of iterative morphological filters with increasing size of SEs.In this section, we shall extend the class of ASF to the SV amoeba morphology.Basically, a SVASF is composed of SV morphological openings and closings whose primitive morphological operations are SV dilation and erosion.
where the two products θ γ and θ φ are the opening and closing operators.A SVASF is an iterative application of SVAF 1 ( )( ) ( ) The two products of TYPE-I SVASF and TYPE-II SVASF yield an interesting operator, the SV averaged alternate sequential filter (SVAASF), which is defined as SVAASF presents skillful properties for denoising, especially for SAR image.
Not limited to morphological filters, we also propose here to consider the SV version of the mean filter and SV median filter (SVMF), where the amoeba structuring elements used for each pixel, ( ) x x θ → .More precisely, the SV mean filter is defined as where | ( ) | x θ is the cardinal (number of pixels) of the window centered at x ; and the corresponding SV me- dian is given by . A SV alternating sequential median filter (SVASMF) is an iterative application of SV median filters.
where N is the order of the filter and the sequence 1 { } N i i θ = is increasing i.e., for all1 i N ≤ ≤ .

RESULTS
In this section, we applied the SV discrete filters to image processing applications: restoration of noisy images, the improvement of SV operators are compared with classical morphological filters, traditional amoeba filters (Lerallut et al., 2007) and nonlocal mean filters (Buades et al., 2005).For fair comparison, the following key parameters are used for traditional amoebas and the geodesic distance definition in new amoeba boundary: 3, 4 distance and parameter λ = 1.For new amoeba center, the standard deviation of g(m) is assigned a relatively large value α = 1 in order to show the power of the patch distance.In addition, parameter R is the maximal radius of increasing window to be used for traditional and new amoebas.
The first is an application of VLP denoising problem in noisy environments.This paper takes a car image corrupted by a 30 percent salt and pepper noise for example to simulate snowflakes covering on the VLP and discuss the method to denoise the snowflakes, as shown in Fig. 3a.Note that in this simulation, pilot image is the result of a median filtering (fixed square window of size 3×3) of the original noisy image.The performance of the filter is measured by the peak signal-to-noise ratio (PSNR) and signal-to-noise ratio (SNR).Let f o denotes the original image of size M × N and f r the restored image (Bouaynaya et al., 2006).The PSNR is defined by The SNR is defined by In the first experiment, we applied the classical ASF.Fig. 3b shows the classical ASF output, removes most of the noise, but the image is overly smoothed.In the second experiment, we applied the SVASF (TYPE-I) based on traditional amoeba, the output image is shown in Fig. 3c, traditional amoeba filters are effective for preserving details in image while removing noise, but some noise still remains, the ability of removing noise is limited to severely noisy image.Then we applied the SVASF (TYPE-I) based on new amoeba, as shown in Fig. 3d, Most of the noise has been removed in the image without altering the topological characteristics of the noisefree image, especially the image in the strong gradient.In the third experiment, we applied the SVASMF based on traditional amoeba and new amoeba.Fig. 3e,f are obtained by using them, respectively.From the results we can see, the SVASMF also have better performance in removing the noise in the image while adaptively preserving the detailed structures.Fig. 4 and Fig. 5 give the second example of restoring the medical CT image corrupted by Gaussian noise (Fig. 4a) and medical MRI image corrupted by Rician noise (Fig. 5a).For Gaussian and Rician noise, a large Gaussian filter works fairly well, and in this simulation, pilot image is the result of a Gaussian filtering of the original noisy image.The noisy medical image is filtered with different mean filters.To assess the quality of the filtering for medical image, the same as previous researchers, we also use the Root Mean Square Error (RMSE) defined as According to the RMSE, The SV mean filter based on the new amoebas (Figs.4d, 5d) does yield a lower RMSE and has better performance in removing the noise while adaptively preserving the detailed structures compared with other mean filters (Figs.4b, 5b ) and (Figs.4c, 5c).To further quantitatively measure the performance of new amoeba filters versus traditional amoeba filters, the results, measured with PSNR and SNR, for the cases of 10% to 60% salt and pepper noise are shown in Figs.6a,b.It is seen that the performance of new amoeba filters definitely are better than traditional amoeba filters when the salt and pepper noise ratio is higher than about 25%.Besides, morphological filters built upon new amoeba structuring elements performs better capabilities than new amoeba median filters when the noise ratio is higher than about 35%, This can also be seen in the results of Figs.6a,b.Similar result, measured with RMSE, for the cases of different Gaussian and Rician noise levels are shown in Figs.6c,d, respectively.According to the RMSE, the mean filter based on the traditional amoebas has better properties than the one using new amoebas, when noise standard deviation is small.However, the SV mean filter based on the new amoebas performs better when noise level increases.Based on the above quantitative analysis, when there is little noise existing, a relatively small α would be a better choice for achieving the best performance of the new amoeba under any noise level.
Finally, in order to better analyze the behavior of new amoeba and compare its performance with stateof-the-art morphological amoeba filters, we present results with a collection of real images.We provide examples on a real MRI image (Fig. 7a) (http://www.ece.ncsu.edu/imaging/MedImg/Stevestuff.html) and a real SAR image (Fig. 8a) (www.es.northropgrumman.com/solutions/starlite/). Fig. 7a illustrates a very noisy MRI image in this case because the slice was very thin.The noisy SAR image is corrupted by a strong specific noise, called "speckle noise".As we know, assessing the capability of the filters to remove actual noise and preserve the edges of the real images is quite difficult, lacking a "clean" reference image, we use no-reference/blind image spatial quality evaluator (BRISQUE) suggested by Mittal et al. (2012): quality score.It is a comprehensive evaluation index and the score typically has a value between 0 and 100 (0 represents the best quality, 100 the worst).A software release of BRISQUE is available online provided by Mittal et al. (2012)

CONCLUSION
This paper presents a new type of amoeba SE that can be used in many SV morphological filters and SV mean/median filters.The originality of our approach lies in combining the local geodesic distance and nonlocal patch distance and the nonlocal patch-based method is combined with the traditional amoeba approach to form a new framework.By taking advantage of nonlocal patch distance at central region, the new amoeba kernel is more stable than the traditional amoeba and its shape is less affected by noise in the pilot image, and what is more, the local kernel boundary configuration keep the advantage of traditional amoeba kernel in preserving the detailed structures.In this paper, SV discrete filters built upon those new amoeba structuring elements have been derived for image restoration.Results on noisy vehicle license plate image, real noisy medical image and real noisy SAR image show the ability of new amoeba filters have better performance in removing the noise while adaptively preserving the detailed structures.This article only shows the new filters for the gray scale image restoration, like most morphological tools, they can also be used on color images (2D, 3D . ..), image reconstruction and future work will provide more results.
The SV alternating filter by the new amoeba SE mapping θ is defined as the compound SV openclose filter (TYPE-I SVAF) or SV close-open filter (

Fig. 4 .
Fig. 4. Medical CT image restoration, size of patch l = 3, global distance threshold r = 20 (a) CT image degraded by Gaussian noise with zero mean and standard deviation σ = 20 (b) image after Non-local mean filter (size of search window N = 11, size of patch l = 5, degree of filtering h = 20) (c) image after SV mean filter with traditional amoeba with R = 5 (d) image after SV mean filter with new amoeba with R=5 (e) and (f) show Zoomed insets of (c) and (d), respectively.

Fig. 5 .
Fig. 5. Medical MRI image restoration, size of patch l = 3, global distance threshold r = 20 (a) MRI image degraded by Rician noise with standard deviation σ = 20 (b) image after Non-local mean filter (size of search window N = 11, size of patch l = 5, degree of filtering h = 20) (c) image after SV mean filter with traditional amoeba with R = 5 (d) image after SV mean filter with new amoeba with R = 5 (e) and (f) show Zoomed insets of (c) and (d), respectively.

Fig. 6 .
Fig. 6.Comparison for traditional amoeba kernel and new amoeba kernel for different filters.(a) PSNR values operating on Fig. 3a for morphological filters and SV median filters; (b) SNR values operating on Fig. 3a for morphological filters and SV median filters; (c) RMSE values operating on Fig. 4a for SV mean filters.(d) RMSE values operating on Fig. 5a for SV mean filters.

Fig. 7 .
Fig. 7. Real noisy MRI image restoration, size of patch l = 3, global distance threshold r = 20, (a) Original MRI image (b) image after Non-local mean filter (size of search window N = 11, size of patch l = 5, degree of filtering h = 25) (c) image after SV mean filter with traditional amoeba with R = 5 (d) image after SV mean filter with new amoeba with R = 5.

Fig. 8 .
Fig. 8. Real noisy SAR image restoration, size of patch l = 3, global distance threshold r = 15 (a) Original SAR image (b) image after SVAASF 4 with traditional amoeba with R = 2~5 (c) image after SVAASF 4 with new amoeba with R = 2~5 (d) image after SV mean filter with new amoeba with R = 5.