COLOR SEGMENTATION OF MGG COLORED CYTOLOGICAL IMAGES USING NON LINEAR OPPONENT COLOR SPACES

This paper presents a color image segmentation method for the quantification of viable cells from samples obtained after cytocentrifugation process and May Grunwald Giemsa (MGG) coloration and then observed by optical microscopy. The method is based on color multi-thresholding and mathematical morphology processing using color information on human visual system based models such as CIELAB model, LUX (Logarithmic hUe eXtension) model and CoLIP (Color Logarithmic Image Processing) model, a new human color vision based model also presented in this article. The results show that the CoLIP model, developed following each step of the human visual color perception, is particularly well adapted for this type of images.


INTRODUCTION
Classic histotechnology methods (e.g.cytocentrifugation) are widely used in clinical medicine and biological research (De Brauwer et al., 2000) and lead to cell observations with morphological considerations.An important aspect is based on the cellular viability and morphological features usually highlighted by colorations such as May Grunwald Giemsa (MGG).The quantification of cells and more particularly viable cells is a determinant parameter in cytology preparations.Generally, in histology laboratories, after cell preparation and coloration, a manual determination of the viability percentage is performed by numbering each dead cell which is timeconsuming and leads to potential errors.We describe in this work an image processing algorithm dedicated to the cells numeration and classification for applications after cytocentrifugation and MGG staining.
The MGG coloration produces samples with purple colored cell nuclei and blue colored cytoplasm (see Fig. 1 typical MGG images).This coloration allows the human eye to differentiate living cells (circularly shaped and composed of a nucleus and a cytoplasm) and dead cells (composed of a scattered nuclei, usually brighter than living cell nuclei) better than on gray level sample.The color information is then useful to the human eye to cell detection.Usually, cell image extraction algorithms are based on the image segmentation of an achromatic channel, obtained by combination of the color channels.This type of image segmentation gives relatively good results but ignores the chromatic part of information used by the human visual system.In order to improve the segmentation, particularly the particles disconnection, it is necessary to add chromatic information to the achromatic information.Lezoray et al. have been working on such cytological images color segmentation and cells classification methods.Their segmentation methods are based on watershed, clustering, and mathematical morphology (Lezoray et al., 1999;Lezoray, 2003;Lezoray and Lecluse, 2007), and are most of the time based on classical RGB model.In this article we aim at demonstrating that human vision based color opponent models are also of interest with the cells classification and segmentation issue.In the case of MGG image coloration, which is widely used, we show that color opponent models are much more adapted than classical RGB model.Then we describe the segmentation method based on color multi-thresholding, mathematical morphology processing and adapted for color opponent models.We compare results on CIELAB model, which is the most known and used human vision based model, LUX (Logarithmic hUe eXtension) (Lievin and Luthon, 2004) model, and CoLIP (Color Logarihtmic Image Processing) (Gouinaud et al., 2011) model, two human vision based models derived from the LIP (Logarihtmic Image Processing) theory (Jourlin and Pinoli, 1988;2001) and the visual human color perception.

Macrophage cell culture
The RAW 264.7 cell line was provided by ATCC Cell Biology Collection (Promochem LGC) and derives from mice peritoneal macrophages transformed by the AMLV virus (Abelson Murine Leukemia Virus).Cells were cultured in DMEM medium (Dulbecco's Modified Eagle's Medium, Gibco) complemented with 10% of fetal calf serum (FCS, Gibco), 1% penicillin-streptomycin (penicillin 10 000 units/ml, streptomycin 10 mg/ml, Sigma) and incubated at 37 • C under a 5% carbon dioxide humidified atmosphere.

Cytocentrifugation
The cytocentrifuge is widely used in clinical medicine and biological research to transfer biological cells onto a microscope slide.Cells were grown in 25 cm 2 culture flasks (1 million cells in 4 ml of DMEMc).After proliferation, cells were gently removed with a scraper and centrifuged.The desired concentration of 100 000 cells per slide were then cytocentrifuged for 10 minutes at 600 rpm (Cytospin Shandon) and air dried over night at room temperature.

May Grunwald Giemsa coloration
This staining is dedicated to air dried cytology preparations such as cytocentrifugation and was performed with an inclusion automat (Leica Autostainer XL).The procedure was performed as follows: Fixation in methanol for 15 minutes, staining in May-Grunwald for 5 minutes, staining in Giemsa for 10 minutes.Rinsing in a pH 6.8 buffer, rinsing in a 50/50 buffer/acetone, dehydration in acetone X2 and clearing in xylene X3.This coloration allows the distinction between cell nuclei which appeared violet and cytoplasm which appeared in blue.Such details would not be as precise without any preparation.

Images acquisition
Samples were then observed on an optical microscope (Nikon Eclipse 80i) coupled with a digital sight Nikon camera.Acquisitions were performed using different microscope objectives (10×, 40×, 60× and 100× under oil immersion).For the rest of the study, in order to have reproductive data, only the images acquired with the 40× objective are treated.
Fig. 1 presents typical MGG images acquired following the procedure described in this paragraph.Results of the color segmentation algortithm presented below will be displayed on these example images.Due to the acquisition profile of the microscope, image coloration can be bright or dark, and a blue halo can appear (see Fig. 1c).A white balance algorithm will be added before the color segmentation in order to extract the sample background color and standardize images balance.

COLOR VISION BASED MODELS
Color human perception can be described by two complementary visual theories that model representation and processing of color information along the visual pathway.The trichromacy theory (Young-Helmoltz) states that the retina contains three types of cones, responsible for color vision and sensitive to red, green, and blue wavelength.The opponent process theory (Hering) states that the human visual system interprets information about color in an antagonistic way (Fairchild, 2005).Human vision based color spaces usually follow the two theories, and are divided in three steps which produce three antagonists channels.The first step models the trichromacy, i.e., the light absorption by cones photoreceptors denoted L, M and S according to the wavelengths of their spectral sensitivities peaks (Long, Medium, and Short wavelength, respectively).When converted to an electric signal, the light intensity signal is compressed and psychological studies have shown that the neural response from a stimulus is close to a logarithmic curve (Fairchild, 2005).This cone compression corresponds to the second steps of human vision based models.The third and last step is the opponent processing of color information that occurs on upper layers of the retina and on the brain, and produces three opposing pairs of processes, one achromatic and two chromatic red-green and yellowblue channels, which are linear combinations of the cones' neural responses (Fairchild, 2005).

CIELAB model
The CIELAB color space is a color-opponent space, where the dimension L * represents the lightness and the dimensions a * and b * represent the red-green and yellow-blue color-opponent dimensions, based on nonlinearly compressed CIE XYZ color space coordinates which take into account the illuminant or white point (X W ,Y W , Z W ) (Fairchild, 2005).
where the cone compression f is the following function: (2) The LUX model The LUX approach is based on the LIP framework too, and has also been built by the following steps of human color visual perception (Lievin and Luthon, 2004;Luthon et al., 2010).Cone intensities are represented by (R, G, B) channels and then the nonlinear response of the retinal stage is modeled by the isomorphism φ applied on each chromatic tone channel, The opponent process is modeled by the (Y,C r ,C b ) color space transformation applied on resulting logarithmic chromatic tones.The (Y,C r ,C b ) color space transformation matrix called P LUX is: Finally, in applying the inverse isomorphism φ −1 on logarithmic antagonist channels resulting from the (Y,C r ,C b ) color space transformation, the (L,U, X) channels that represent color in the LUX model can be derived from (R, G, B) intensities as follows: (5)

The CoLIP Model
The CoLIP model is the color extension of the LIP model (Jourlin and Pinoli, 1988;2001), constructed following each step of the human vision (Gouinaud et al., 2011).In the CoLIP framework, the image has first to be transformed from its original representation space to the cone pigment absorption space (L, M, S).In the case of the (X,Y, Z) space, the linear transformation matrix U from (X,Y, Z) to (L, M, S) is the M HPE matrix (Fairchild, 2005): As in the LIP framework, color images are represented by color tone vector f (x, y) = (l(x, y), m(x, y), s(x, y)) defined on a spatial domain D in R 2 and related to intensity vector F = (L(x, y), M(x, y), S(x, y)) through the following relation: where M 0 is a scaling factor, L i , M i and S i represent incident light, and L t , M t and S t represent transmitted light.The color tone vector is nothing else than a normalized inverted color intensity vector.This inverted scale has a physical meaning as images are transmitted images.The tone vector represents matter opacity.With this definition, the (0, 0, 0) vector represents a transparent point and (M 0 , M 0 , M 0 ) corresponds to a dark point.The vector tone is valued in [0, M 0 ) 3 .In the digital case, image intensities are expressed on 256 gray levels with a floor quantization.
The scaling factor M 0 is set to 256 and the tone vector f is defined as follows: The cone compression is modeled in the CoLIP model by the isomorphism ϕ Here the ϕ function is applied to each tone vector coordinates.The resulting vector is the logarithmic tone vector f = l, m, s .The opponent processing is modeled in the CoLIP framework by a linear combination of the logarithmic tone vector coordinates l, m, and s computed with the matrix P.
The resulting vector is the antagonist logarithmic tone vector f = ( ã, rg, yb) (where a stands for achromatic, rg for red-green and yb for yellow-blue).
If L = M = S, the image is achromatic (gray level image).Therefore rg = 0, yb = 0 and ã = l = m = s.The P matrix parameters satisfy the following relations: In this application, the antagonist transformation matrix P CoLIP is (Gouinaud et al., 2011):

Comparison and interests
As explained in introduction, MGG coloration produces images with blue and red dominant hues.The three interest areas, living cells' nuclei, living cells' cytoplasms, and scattered dead cells' nuclei are respectively dark purple, blue and bright purple.The aim of the segmentation is to detect disconnected living cells nuclei, in order to numerate living cells.Achromatic information on color opponent spaces do not give more information that can be found on R, G or B classical channels, since the cytoplasms and dead cells luminosity are not really different.But chromatic information, especially in color opponent spaces, could allow detecting a chromatic difference between the three interest areas.In Fig. 2 are plotted the image 1 clouds of points in the chromatic planes (R, B) in primary space RGB, (a * , b * ) in CIELAB space, (U, X) in LUX space and ( rg, yb) in CoLIP space.The three interest areas appear clearly as distinct clouds of points on each diagram.On the (R, B) chromatic plane, these areas are very close but on human vision based models, these three areas are clearly distinct and even more on both LUX and CoLIP chromatic planes.As they better model the human visual perception, human vision based models allow to compute color contrast among the interest areas and to include chromatic information in the segmentation process.
Thus, chromatic information given by human vision based models can enhance the luminosity segmentation.MGG images have a dominant blue hue, so luminosity and yellow blue channel will be redundant.But, the red-green channel should give information, particularly to detect living cells cytoplasm and nuclei frontier which could be of interest to detect and numerate living cell nuclei.In Fig. 3, the red-green channel of image (a) Fig. 1 is displayed within CIELAB, LUX and CoLIP models.We can see that it allows to differentiate cytoplasm and nuclei but also dead cells.This is particularly visible on the CoLIP yellow-blue channel.
Giving all these results, the color image segmentation method described in the following section aim at detecting living cells' nuclei in detecting image pixels with achromatic and yellow-blue channel and then sharpens the cell detection disconnection in removing cytoplasm and dead cells nuclei with yellowblue channel.Dead cells are detected on the red-green channel.

SEGMENTATION ALGORITHM
Color segmentation methods for cytological images are based on four fundamental steps.A step of pre-treatment allows checking the presence of relevant particles, smoothing the image to reduce noise, and eliminating non relevant elements.The color image is then converted in an appropriate color space.Then the algorithm main step is the relevant particles regions extraction.Here, various method can be found in the state of the art including multi-thresholding and color gradient (Lezoray et al., 1999;2003), histogram clustering (Lezoray, 2003;Lezoray and Lecluse, 2007), and pixel classification (Meurie et al., 2003).This step is achieved with segmentation refinement using mathematical morphology.Eventually a step of region growing is performed, in order to disconnect and quantify the segmented particles.This is most of the time performed by a watershed algorithm.
The color segmentation method presented in this article is composed of the same fundamental four steps.The step of pre-treatment consists in extracting an automatic red scale bar that appears on some images and induces trouble in the algorithm and in smoothing the image.It is then shown that color opponent spaces (CIELAB, LUX or CoLIP) are more convenient than classical primary space to discriminate relevant areas in this type of cytological images, due to their color antagonism and logarithmic image dynamic range.The color conversion also includes a white balance algorithm in order to extract the sample background color and standardize images balance.The third step is the phase of regions extraction and it is performed using Otsu multi-thresholding.The segmentation is then enhanced by morphological filters.Eventually, the fourth and last step quantifies living and dead cells using watershed transform and evaluate sample viability.All experimental results of this study have been computed with matlab.

PRE-TREATMENT
On some images, an automatic red scale bar can appear, and induce trouble in the algorithm.A mask image of the red scale bar is created, and then pixels on the edge of the scale bar a progressively replaced by a close pixel, like an erosion, until the scale bar disappears completely.Then, the image white point, corresponding to the sample color back ground is computed.The difficulty is to define the value to give to the white point.In order to eliminate local saturation phenomenons, the white point is calculated in taking pixels that have greatest values of images on all channels until a certain percentage p of image pixels is reached, and then taking the mean of these pixels on each channel.Here p = 1% (Gouinaud et al., 2011).The smoothing algorithm is performed using a median filter.

CONVERTING IMAGE FROM RGB TO THE COLOR OPPONENT SPACE
First of all, a white balance algorithm is performed on the image.Withim CIELAB model, the white balance is integrated since the white point W ,Y W , Z W ) is taken into account (see Eq. 1).In LUX and CoLIP model, the white balance algorithm is the ratio of R, G, B or L, M, S chanels, respectively, to the white point R W , G W , B W or L W , M W , S W , respectively.
Color conversion from RGB to the human vision based space are performed following Eqs. 1, 13, and 5.Here a scale normalization is needed as chromatic scales are different in each model (see Fig. 2).CIELAB and LUX models are rescaled in order to have the CoLIP model chromatic range: (0, M 0 ) for achromatic channel and (−M 0 , M 0 ) for chromatic channels and so that the (0, 0) value corresponds to the achromatic axe in the chromatic plane:

Regions and Marker extractions
Multi-thresholding segments each color space channel into N classes by means of the Otsu's multi-thresholding method, that uses intra-class variance optimization (Ostu, 1979).Achromatic and yellow-blue channels are divided into three areas corresponding to interest areas, in order to segment nuclei areas.Red-green channel is divided into 4 to 6 areas (depending on the model) to be more precise and segment dead nuclei areas.Morphological operations are computed to fill holes, remove small particles, smooth cells edges, in order to produce binary images that allow a cell quantification.

Morphology processing and watershed
The Euclidean distance to the object boundaries is first computed, and then the watershed is applied to the inverse of the distance map.The watershed algorithm is applied on both nuclei and dead nuclei markers images to disconnect close particles and quantify the number of particles.The sample viability rate is directly deduscted from the number of viable cells and dead cells.

DISCUSSION
Result of the color segmentation algorithm on images from Fig. 1 with CoLIP model are displayed in Fig. 4. Color segmentation and numeration of living and dead cells has been tested on a data base of 32 MGG images.Exact numeration of living and dead cells has been performed by a biologist.Then, the table 1 compares absolute error average (Av) for living and dead cells (i.e., the average error in number of cells) quantification and viability rate absolute error average (Av) and standard deviation (SD) in percentages on each color opponent space.The number of viable cells on the data base samples go from 47 to 195, depending on the sample.The number of dead cell is completely different from one sample to another, some have a viability rate of 100% (see Fig. 1c) and some have a viability rate of 60% with more than 50 dead cells.1c) induce trouble in the algoithm (see Fig. 4c) since the white balance remove the sample background color but also the contrast between relevant areas: special care is required during the image acquisition on the microscope, especially for the contrast balance.These results validate our method.The aim of cytological coloration such as MGG is to improve the human eye contrast.As color opponent models, and even more LIP-based color opponent models, follow each step of the human vision, any segmentation method on cytological images could probably be improved by using these models.Indeed, due to the color antagonism and the logarithmic image dynamic range, colors are much more discriminated before any segmentation method is performed.
As our segmentation method demonstrates the utility of human vision based color opponent models, it would be interesting in a near future to enhance this methodology with other morphological parameters.Indeed, macrophages are immune cells able to internalize foreign bodies during the phagocytic process leading to the formation of vesicles.Coupling cell death detection with a precise quantification of the amount of vesicles generated will be a powerful tool to assess biological important effects in toxicological studies (Nan et al., 2008;Leclerc et al., 2010;Song et al., 2010).

Fig. 4 .
Fig. 4. (a), (b), and (c): results of the color image segmentation with CoLIP model on corresponding images Fig.1.Living cells nuclei are in red and dead cells are in green.

Table 1 .
Average (Av) absolute error of living cells and dead cells quantification and average (Av) and standard deviation (SD) of sample viability in percentage on CIELAB, LUX and CoLIP model.models allow to enhance simple luminosity segmentation, in adding color information.The CoLIP and LUX model give much better results than CIELAB model.CoLIP results are better on living cells and much better on dead cells.This can be explained by the fact that CoLIP red-green channel discriminate dead cells much more than CIELAB and LUX red-green channels.Viability rate error is clearly better with CoLIP and LUX model, and better within CoLIP model than LUX model considering that the standard deviation is lower and thus the viability rate error really low and really stable.Images with a blue halo (see Fig.