Region homogeneity in the Logarithmic Image Processing framework: application to region growing algorithms

In order to create an image segmentation method robust to lighting changes, two novel homogeneity criteria of an image region were studied. Both were defined using the Logarithmic Image Processing (LIP) framework whose laws model lighting changes. The first criterion estimates the LIP-additive homogeneity and is based on the LIP-additive law. It is theoretically insensitive to lighting changes caused by variations of the camera exposure-time or source intensity. The second, the LIP-multiplicative homogeneity criterion, is based on the LIP-multiplicative law and is insensitive to changes due to variations of the object thickness or opacity. Each criterion is then applied in Revol and Jourlin's (1997) region growing method which is based on the homogeneity of an image region. The region growing method becomes therefore robust to the lighting changes specific to each criterion. Experiments on simulated and on real images presenting lighting variations prove the robustness of the criteria to those variations. Compared to a state-of the art method based on the image component-tree, ours is more robust. These results open the way to numerous applications where the lighting is uncontrolled or partially controlled.


INTRODUCTION
Segmentation of images acquired with different lighting conditions is a challenging task in image analysis that can occur in many settings such as visual inspection for industry (Cord et al., 2010;Noyel, 2011;Noyel et al., 2013;Parra-Denis et al., 2011), medical images (Noyel et al., 2014;, security (Foresti et al., 2005), driving assistance (Hautière et al., 2006), etc. Lighting variations obviously represent a substantial obstacle for the development of reliable image processing algorithms. To overcome this difficulty, we can determine the two following approaches: (i) Firstly, a practical approach consisting of circumventing the problem, e.g. by a learning process. This is often the case for face recognition, where a database is created by varying together the acquisition angle and the intensity of the lighting source (Ramaiah et al., 2015). An alternative way to learning processes aims at extracting from images some characteristics independent of lighting conditions (Wang et al., 2004;Shah et al., 2015). Another alternative aims at attenuating the lighting variations effects by means of various methods like shadow suppression (Zhang et al., 2019).
(ii) Secondly, a fundamental approach consists of creating models and algorithms almost insensitive to such variations. In that respect, Chen et al. (2006) define the Logarithmic Total Variation (LTV) model in order to remove varying illumination in face images. Yu and Fan (2017) regard an image in the threedimensional space as a grey wave composed of peaks and troughs which divide the image into many local sub-regions. The Retinex model takes into account the human perception of colours and contrasts. It has been extensively reviewed by Elad et al. (2003) alongside some related illumination compensation methods. We can also mention the representation methods of the image level sets by trees of their connected components, such as: the component-tree, also named max-tree (Salembier et al., 1998), or the tree of shapes, also named inclusion tree (Monasse and Guichard, 2000). They are invariant to contrast changes due to a continuous and increasing function applied to the image (Monasse and Guichard, 2000). Recently, Passat et al. (2011) introduced a segmentation method by component-tree presenting a certain robustness to intensity inhomogeneity provided that the maximal image contrast is in an intermediate range: not too low and not too huge. The connected filtering on tree-based shape spaces of Xu et al. (2016) is also theoretically invariant to those contrast changes.
Nevertheless, only a few papers focus on the causes of lighting variations. For this purpose, Jourlin and Noyel (2018) have defined two new homogeneity criteria in the LIP (Logarithmic Image Processing) framework, namely the LIP-additive and LIP-multiplicative homogeneity criteria. Each criterion is based either on the LIP law of addition + + + of two images or the multiplication × × × of an image by a scalar. Such laws are now well-known to model the lighting variations affecting the image acquisition step (Jourlin, 2016). The LIP-additive law + + + especially models changes of source intensity or exposure-time whereas the LIP-multiplicative law × × × models the thickness (or opacity) changes of the observed object.
The aim of this paper is to study these homogeneity criteria in details. At the theoretical level, we will demonstrate their insensitivity to the lighting variations which are modelled by the corresponding LIP-laws. At the application level, we will prove their robustness to lighting changes on both simulated and on real images. For segmentation purpose we will apply them within the Revol and Jourlin's (1997) algorithm which is in the class of Region Growing methods. This algorithm has the specificity of evaluating the homogeneity of an image region at each step. Its segmentation results will be compared to those obtained by the component-tree method of Passat et al. (2011).

MATERIALS AND METHODS
In this section, we will present the theoretical methods and their validation by the experimental material. First, we will summarise the Logarithmic Image Processing model. Second, we will present the specific lighting variations which are modelled by each LIP law. Third, we will introduce the LIP-additive and LIP-multiplicative homogeneity criteria. Fourth, we will recall the Revol and Jourlin's (1997) region growing algorithm based on the homogeneity of an image region. Fifth, for comparison purpose, we will recall the sate-of-the-art method of Passat et al. (2011). Finally, we will present the experimental validation with simulated and real images.

LOGARITHMIC IMAGE PROCESSING
Introduced by Jourlin and Pinoli (1988;2001), the LIP model possesses strong physical properties. Let f and g be two grey level functions defined on the spatial support D ⊂ R n , with values in the grey scale [0, M[ where M ∈ R. In fact, the addition f + + + g of f and g and the scalar multiplication λ × × × f of f by a real number λ are both deduced from the optical Transmittance Law according to: Let us note that in the LIP framework, the grey scale [0, M[ is inverted in comparison with the classical one. Indeed, due to the transmittance law, 0 represents the "white" extremity of the scale, which corresponds to the observation of the source intensity. The reason of such an inversion is that 0 appears as the neutral element of the addition + + + , i.e. when no obstacle is placed between the source and the sensor.
From equation 1, the subtraction between two grey level functions is easily deduced: In such a case, the subtraction f − − − g is a grey level image and the difference f (x) − − − g(x) represents the Logarithmic Additive Contrast between the grey levels f (x) and g(x) (Jourlin and Pinoli, 2001). This difference is the grey level which must be added to the brightest one (here g(x)) in order to obtain the darkest one (here f (x)).
Remark. Let us note that for 8-bit digitised images, M is equal to 256 and the grey levels vary from 0 to 255.
At the mathematical level, it has been demonstrated (Jourlin and Pinoli, 2001) that the laws + + + and × × × give a Vector Space structure to the set I(D, [0, M[) of images defined on a same spatial support D. This allows the use of many mathematical tools specific to this kind of space. In order to complete the outstanding properties of the LIP Model, let us recall that Brailean et al. (1991) have established its consistency with the Human Visual System, which opens the way to process images as a human eye would do.

WHICH LIGHTING VARIATIONS ARE MODELLED BY EACH LIP LAW?
Carre and Jourlin (2014) and Deshayes et al. (2015) have shown that the LIP-addition (respectively subtraction) of a constant to an image (resp. from an image) perfectly models a decrease (resp. an increase) of the camera exposure-time or of the source intensity. By definition of the scalar multiplicative law, the scalar λ appears as a "thickness" parameter: in fact, if f is a grey level image, one can associate to it a virtual half-transparent object producing f in transmission. λ × × × f represents then the image we would obtain by stacking f on itself λ times. The scalar multiplicative law models therefore the "opacity" changing of a half-transparent object or the thickness changing of a homogeneous object.

HOMOGENEITY
CRITERIA: LIP-ADDITIVE HOMOGENEITY AND LIP-MULTIPLICATIVE HOMOGENEITY Let f be a grey level function defined on the spatial support D ⊂ R n , with values in the grey scale [0, M[ and consider a region R of D. There exist some classical parameters evaluating the homogeneity of the image f in R, like the standard deviation σ f (R) or the "diameter" of R in the mathematical sense defined by sup x∈R f (x) − inf x∈R f (x) which is nothing but the dynamic of the image region. Such homogeneity parameters are obviously sensitive to lighting variations, which justifies the introduction of the two new criteria: the LIP-additive and the LIPmultiplicative homogeneity.
Remark. When there is no ambiguity, we will consider that the homogeneity of an image f over a region R is equivalent to the homogeneity of the region R. The region R is indeed supposed to be associated to the image f .

LIP-additive homogeneity
The homogeneity H + + + f (R) of the region R is defined as its dynamic in the LIP sense: Let us note that H + + + f (R) lies in [0, M[ and represents thus a grey level which can be interpreted as the maximal Logarithmic Additive Contrast observed in the region (Jourlin and Pinoli, 2001).
The most important property of this homogeneity criterion is its insensitivity to exposure time variations modelled by LIP-addition/subtraction of a constant C: Proof. Let us first remark that In fact, equation 1 gives: In the same way, we have: In such conditions, equation 4 can be written: The following computation shows that ( The two previous results prove equation 5.

LIP-multiplicative homogeneity
Now, we propose a homogeneity criterion based on the LIP scalar multiplication and more precisely on the notion of Logarithmic Multiplicative Contrast (LMC) (Jourlin et al., 2012). Let us recall that the LMC associated to a pair of grey levels g 1 and g 2 , where g 1 and g 2 are strictly positive, is the real number defined by the formula: In the context of images acquired in transmission, this means that the LMC of two grey levels represents the number of times the brightest of them must be stacked upon itself to get the darkest one. In such conditions, the LIP multiplicative homogeneity of a region R is computed according to: Remark. In the case where inf x∈R f (x) = 0, one can replace this value by 1 in order to avoid an infinite value for H × × × f (R).
This homogeneity criterion possesses the fundamental property to be insensitive to variations of lighting modelled by the LIP-multiplicative law × × × , like opacity or thickness variations: . We have to prove the equality: Given λ ≥ 0, it is easy to establish that: In the same way, inf x∈R λ × × × f (x) = λ × × × inf x∈R f (x). Thus, equation 10 is established, which ends the proof. Revol and Jourlin (1997) presented a region growing method which minimises the region variance. In the sequel, we propose to replace the variance by one of the homogeneity criteria H + + + or H × × × . Let us present the principles of this algorithm. The details are given in (Revol and Jourlin, 1997).

A REGION GROWING ALGORITHM BASED ON REGION HOMOGENEITY
Let dyn f (R) be the dynamic range of the image f over the region R. It is defined by dyn At each step n, a region R n is dilated by a unitary structuring element N, e.g a square of length side 3 pixels (Minkowski, 1903;Matheron, 1967;Serra and Cressie, 1982;Najman and Talbot, 2013). This gives a region D n+1 = R n ⊕ N whose homogeneity H f (D n+1 ) is measured. Depending on a threshold value t, the region is considered as homogeneous or not.
1.If H f (D n+1 ) ≤ t, the region D n+1 is then considered as homogeneous. The new region R n+1 becomes D n+1 .
2.If H f (D n+1 ) > t, the region D n+1 is then inhomogeneous. A reduction process Rdct f (D n+1 ) truncates the extremal classes of the histogram of D n+1 by keeping only the pixels belonging to the dynamic range of R n , dyn f (Rdct f (D n+1 )) = dyn f (R n ). This gives a region D n+1 .
2.a If H f (D n+1 ) ≤ t, the region D n+1 is then modified by an extension process which adds the neighbouring pixels to the region according to both following conditions. i) The new pixel values have a LIP-difference less or equal than 1 with the maximum or the minimum values of f over D n+1 . ii) The new pixels give an homogeneous region. The new region R n+1 becomes the extension of D n+1 : E(D n+1 ).

2.b If
a contraction process reduces the dynamic range of the region D n+1 by removing the extremal classes of its histogram until the region becomes homogeneous. LIPaddition and LIP-subtraction replace the classical addition and subtraction used by Revol and Jourlin (1997). The new region R n+1 is the contraction of D n+1 : Ctrct(D n+1 ).
The algorithm stops when the region R n+1 does not grow any more.
For a fully automatic segmentation method, the seed points could be chosen as the minima of a function such as: a gradient of the image -like for the watershed transform (Beucher and Meyer, 1992) -or a map of Asplund's distances of the image (Jourlin et al., 2012;Noyel and Jourlin, 2015;2017b;a). The seed points could also be automatically determined by a previous classification (Noyel et al., 2007;. As the homogeneity criterion is invariant to contrast changes with an optical cause, the threshold could be learnt from a training database (Lu et al., 2017). It could also be automatically chosen as the most frequent contrast on every pixel neighbourhoods (Jourlin, 2016).

A STATE OF THE ART METHOD BASED ON THE IMAGE COMPONENT-TREE
Our method will be compared to the state-of-theart one of Passat et al. (2011) which is based on level sets and presents a certain robustness to lighting variations. It consists of an interactive segmentation by component-tree. A component-tree is the tree formed by the inclusion relation between the binary components of the successive image level sets. A node N of the component-tree is a connected component of the image level set X v ( f ) = {x ∈ D| f (x) ≥ v}. Let K be the set of all connected components of all level sets. The subset K ⊂ K of the connected components which best fits the user selected region G in D is automatically determined by the following minimisation problem: where α ∈ [0, 1]. d 0 (X,Y ) = |Y \ X| and d 1 (X,Y ) = |X \ Y | are respectively the amount of false-negatives and false-positives in X with respect to Y . A solution to this minimisation problem can be found by dynamic programming. The details are given in (Passat et al., 2011). In the sequel, we have used the publicly available code of Naegel and Passat (2014).

EXPERIMENTAL VALIDATION
We have just established that the LIP-additive homogeneity criterion H + + + f (R) is theoretically insensitive to a LIP-addition + + + or to a LIP-subtraction − − − of a constant to or from an image, respectively. These operations model the lighting changes caused by a variation of the source intensity or of the exposuretime of the camera. As for the LIP-multiplicative homogeneity criterion H × × × f (R), it is insensitive to a LIP-multiplication of an image f by a scalar which models the variations of the object thickness or opacity. Let us perform an experimental validation of these insensitivities through simulated and real images. Let us also compare the presented approach to the state-of-the-art method of Passat et al. (2011).

Simulated images
For this experiment, we have selected a colour image of a butterfly (Fig. 1a) from the dataset YFCC100M (Yahoo Flickr Creative Commons 100M) (Thomee et al., 2016;Butterfly, 2010). We have extracted its luminance as a grey-level image f (Fig. 1b). In order to verify the respective insensitivities of the criteria H + + + and H × × × to lighting variations simulated by a LIP-addition + + + or a LIPmultiplication × × × , we have darkened and brightened this luminance image f by using each of the LIPoperations.
For the LIP-additive homogeneity criterion H + + + f (R), a dark image f dk,a is obtained by a LIPaddition of a constant k = 120 to the complement f c of the image f , f dk,a = ( f c + + + k) c (Fig. 1c). The image complement, which is defined by f c = M − 1 − f , allows to take advantage of the LIP-scale. A bright image f br,a is also simulated by a LIP-subtraction of the same constant: f br,a = ( f c − − − k) c (Fig. 1d).
For the LIP-multiplicative homogeneity criterion H × × × f (R), a dark image f dk,m is obtained by a LIPmultiplication by a scalar λ dr = 4 of the image complement f c , f dk,m = (λ dr × × × f c ) c (Fig. 1e). A bright image f br,m is also simulated by a LIP-multiplication by a scalar λ br = 0.1, f br,m = (λ br × × × f c ) c (Fig. 1f).
In all these five images f , f dk,a f br,a , f dk,m and f br,m the same region R is selected. The homogeneity criteria H + + + and H × × × are then computed in the region R of the images f , f dk,a , f br,a and of the images f , f dk,m , f br,m , respectively. Their values are given in section "Results" (p. 6).

Real images
The robustness to lighting variations of the LIPadditive homogeneity criterion H + + + and of the LIPmultiplicative criterion H × × × needs to be verified on real images by using specific experimental material.
The LIP-additive homogeneity criterion H + + + is expected to have a low sensitivity to variations of source intensity or exposure-time of the camera. In order to verify this assumption, we have conducted an experiment. An image of the same scene is acquired by a camera with significantly different exposure-times and slightly different positions. The scene is composed of a soft toy monster named "Nessie" (Fig. 2). Three colour images are captured with different exposuretimes 1/40 s (Fig. 2a, 2b), 1/400 s (Fig. 2c, 2d) and 1/800 s (Fig. 2c, 2e). They are converted to luminance images in grey levels: f 0 , f 1 and f 2 , respectively. The shorter the exposure-time is, the darker the image becomes. A segmentation is performed by Revol and Jourlin's (1997) algorithm using the LIP-additive homogeneity criterion H + + + and a threshold of 200 to decide if a region R is homogeneous (i.e. H + + + f (R) ≤ 200) or not. The initialisation of the algorithm is done by a seed point manually selected inside the letters on the body of "Nessie". To take advantage of the LIP-scale, the segmentation is performed on the complement of the three luminance images f c 0 , f c 1 and f c 2 . Three segmented regions R 0 (Fig. 2a), R 1 (Fig. 2c) and R 2 (Fig. 2e) are thereby obtained in the images f 0 , f 1 and f 2 , respectively. The LIP-additive homogeneity of those regions is then computed. For comparison purpose, Passat et al.'s (2011) segmentation method is applied to the images f 0 (Fig. 2b), f 1 (Fig. 2d) and f 2 (Fig. 2f). The user selected regions G are the same seed points which are used for our approach after a dilation by a square of length side 3 pixels. As the user selected regions G are much smaller than the letters that we want to segment, we choose a parameter α = 0 in order to minimise the number of false-negatives d 0 (∪ N∈K N, G) in those regions G. After minimisation, this number is equal to zero.
In order, to verify the low sensitivity to object opacity of the LIP-multiplicative homogeneity criterion H × × × , we have selected three X-ray images of a luggage acquired under different exposures for security purpose (Fig. 3). The images are coming from a publicly available database, namely GDXray (Mery et al., 2015;GDXray, 2015). The object to be detected is a razor blade. Depending on its orientation, the xrays pass through different thicknesses of the matter and are therefore more or less absorbed. In the first image f 0 (Fig. 3a), the razor blade is perpendicular to the x-rays coming from the source. The image of the razor blade is obviously brighter in this image f 0 than in the two others f 1 (Fig. 3c) and f 2 (Fig. 3e) where the razor blade is oblique to the x-rays. To detect this object, Revol and Jourlin's (1997) segmentation approach is used with a threshold of 2.7 on the LIPmultiplicative homogeneity criterion H × × × of a region. The algorithm is initialised by manually selected seed points inside the razor blade (Fig. 3a, 3c and 3e). Three regions R 0 (Fig. 3b), R 1 (Fig. 3c) and R 2 (Fig. 3e) corresponding to the object are then obtained in the images f 0 , f 1 and f 2 , respectively. The LIPmultiplicative homogeneity of those regions is then estimated. Its values are presented in the following section.

RESULTS
The experimental results are given for the simulated and the real images.

REAL IMAGES
Let us give the results obtained on the real images of figures 2 and 3.
The LIP-additive homogeneity criterion H + + + is illustrated in figure 2. One can notice that the segmentations of the letters on the body of "Nessie" are very similar in all the three images f 0 (Fig. 2a), f 1 (Fig. 2c) and f 2 (Fig. 2e) using this criterion H + + + within the Revol and Jourlin's (1997)  (R 2 ) = 197.0. These values are very close for the three regions R 0 , R 1 and R 2 of these images which were acquired with significantly different exposure-times. Those results prove therefore the robustness of the LIP-additive homogeneity criterion H + + + in Revol and Jourlin's (1997) segmentation method to the variations of the camera exposure-time. This approach is also compared to the one of Passat et al. (2011). One can notice that the segmentations with this latter method are different between the three images f 0 (Fig. 2b), f 1 (Fig. 2d) and f 2 (Fig. 2f). Those results show that Passat et al.'s (2011) method lacks of robustness for important variations of the camera exposure-time, whereas ours is robust to such variations. In addition, the letters are not entirely extracted with Passat et al.'s (2011) method whereas with ours, their segmentations are better (Fig. 2a, 2c and 2e).   Figure 2 proves the robustness of the LIP-additive homogeneity criterion H + + + f (R) to lighting changes caused by a variation of an exposuretime of the camera. This lighting variation is equivalent to a variation of the source intensity. Although the methods based on levels sets are theoretically robust to a contrast change caused by a continuous and increasing function applied to the image, the results obtained in figure 2 prove that the level set approach of Passat et al. (2011) lacks of robustness for important variations of the camera exposure-time. The results given in figure 3 prove the robustness of the LIP-multiplicative homogeneity criterion H × × × f (R) to lighting changes caused by variations of the object opacity.
The previous results show that the level sets methods based on component-trees (Salembier et al., 1998;Passat et al., 2011) or on inclusion trees (Monasse and Guichard, 2000;Xu et al., 2016) will therefore strengthen their robustness to strong contrast changes -due to an optical cause -by using a LIP homogeneity criterion. The optical cause can either be a variation of the source intensity, which will be corrected by the LIP-additive criterion H + + + , or a variation of the object opacity, which will be corrected by the LIP-multiplicative criterion H × × × . These findings will be studied in a future paper.

CONCLUSION
We have therefore successfully introduced two new region homogeneity criteria which are robust to lighting changes, namely the LIP-additive homogeneity criterion H + + + and the LIP-multiplicative criterion H × × × . With experiments on simulated and on real images, we have shown that the LIP-additive homogeneity criterion H + + + is robust to changes caused by variations of source intensity (or exposure-time of the camera) whereas the LIP-multiplicative criterion H × × × is robust to changes due to variations of object opacity (or thickness). The introduction of those criteria in Revol and Jourlin's (1997) segmentation method gives it the same robustness. Compared to Passat et al.'s (2011) method based on the componenttree of the image level sets, ours is more robust to strong intensity changes due to one of the previous optical cause. As far as we know, this is the first time that a segmentation method based on a region homogeneity parameter robust to those type of lighting changes has been shown. This novel method of segmentation paves the way to numerous applications presenting strong lighting variations.