DEEP FOCUS ; A DIGITAL IMAGE PROCESSING TECHNIQUE TO PRODUCE IMPROVED FOCAL DEPTH IN LIGHT MICROSCOPY

In light microscopy, the spatial transverse resolution is a function of the wavelength and numerical aperture. The depth resolution is another function of these parameters. The factors that enable the detection of fine detail, make the sharp focusing of more than a thin slice of the depth in an object impossible. When the examination of fracture surfaces is attempted using light reflection microscopy, the roughness will often restrict the in-focus parts of an image to a small portion of the field of view. Several authors have presented methods that enable a set of digitised images to be processed into a single composite image which contains the in-focus parts from each image. These methods are effective, unfortunately the noise present in each digital image is accumulated, resulting in increasingly noisy composite images as the number of images in a set is increased. During processing, a separate image depicting the heights in the surface, i.e. a contour map, may be produced. This image is the key that enables the production of an in focus composite image which does not accumulate noise. Image analysis under computer control will frequently require the use of automatic focusing. Several authors have published criteria which may be used to determine the state of focus of an image. Such criteria have a clear application to the above process. This paper presents an evaluation of some methods used for the processing of such images, and also some procedures used for the determination of sharpness of focus and demonstrates a sensitive method for the evaluation of such procedures. Finally, an implementation of a method which uses the one of the simplest focus criteria is presented, and a procedure for the production of deep focus images which are free from the accumulation of noise.


INTRODUCTION
The examination of fracture surfaces using optical microscopy is valuable because it provides direct non-destructive examination, colours are seen and small tilts and steps are revealed.In addition, the analysis of surface detail may be enhanced by using different illumination modes, such as interference contrast which can reveal extremely small steps.When measurement of features is required, optical methods are excellent, because magnifications may be accurately calibrated, and once determined remain fixed.Unfortunately, examination of fracture surfaces shows that the small depth of focus is a disadvantage, the observer needs to frequently refocus.Images of fractures often show a small portion sharply focused which moves across the field of view with changes in focus.To understand the whole image requires relating those parts of it that are sharp with others that are blurry.Altering the focus control rapidly may be helpful but in the case of complex detail interpretation is difficult.Using a Scanning Electron Microscope (SEM) usually solves the problem of focal depth, but SEM examinations are frequently less informative than is optical examination of the same surface.A method of producing images of rough surfaces using optical microscopy is therefore attractive for the examination of fracture surfaces.

RESOLUTION AND DEPTH OF FOCUS
The transverse resolution of the optical microscope is given by the Rayleigh Criterion, R = 0.61λ/Nsin(U), where R is the resolution in micron, λ is the wavelength of the illumination in micron, N is the refractive index of the medium in front of the lens and U is one half of the angle subtended by the objective at the object (Nsin(U) is the numerical aperture).(Born and Wolf, 1980;Conrady, 1960;Hausler and Korner, 1987;Goss and Holm, 1992) published relationships for the depth of focus of an optical system.Born and Wolf derived a relationship for a f/10 optical system using a paraxial approximation ( ) where D is depth of focus in micron.(Conrady, 1960;Hausler and Korner, 1987) derive a relationship based on the Rayleigh Limit of λ/4; Eqs. 1 and 2 behave identically for small aperture systems and start to diverge for apertures greater than § NA, see Fig. 1.The results from Eq. 2 based on the Rayleigh criterion are supported by observation.

DIGITAL COMPOSITION
Methods for extending the depth of focus in an image have been proposed by a number of authors.(Pieper and Korpel, 1983;Sugimoto and Ichioka, 1985;Häusler and Korner, 1987;Itoh et al., 1989) This enhancement of the focal range is achieved by the digital composition or assembly of an image that contains the in-focus parts of a series of images which may cover the whole depth range in an object.For example, a portion of a fracture surface that has a depth range of 50 micron imaged with a lens whose depth of focus is 1 micron would need some 50 images to cover the entire depth range.Each of these images, on the average, would only have some 1/50 in focus.
The process of digital composition must select the parts of each image in focus, and place them in a resultant composite image.The methods proposed were sensitive to noise in the image, the resultant composite image contained most or all of the noise, and when the series contained a large number images the final result was unusable because of the accumulation of noise.
Using the public domain software program NIH-Image, Rasband (1999), as the foundation for the programming, a method of digital composition has been implemented.This program provided many useful functions for handling images, and the digital composition has been accomplished with the addition of a few hundred lines of code in Pascal.In addition, a further procedure has been developed which does not suffer from the accumulation of noise pixels.The final resultant image contains the same noise pixels as the source images.

FOCUS CRITERIA
Vollath (1988) and Erteza (1976) describe various sharpness indices or criteria used to determine whether an image is in-focus, for the purpose of providing a control signal in a device which is intended to bring the image in an imaging system to focus automatically.An image is in focus when the maximum detail is visible, focus criteria are often based upon the assessment of the difference between the intensity of adjacent pixels.The finest detail will only be visible if the image is sharply focused.When the resolution limit is reached artifacts may be produced by diffraction and aliasing.
Another focus criterion is simply that when the image is focussed the Airy discs are as small as possible for the optics.The image is in focus when each part of it reaches a local maximum or minimum in intensity.Born and Wolf (1980) show the distribution of energy around the focus and it is clear that the intensity is a maximum at the focus.

EVALUATIONS OF SHARPNESS INDICES
A series of images (Fig. 3) of an obliquely mounted steel ruler, covering a range of depth, were digitized using a binocular stereoscopic microscope (Wild M5A), a video camera and an 8 bit video frame grabber.The software employed for the acquisition and display of these images was NIH-Image.These images were used for the evaluation of the various sharpness indices studied.The software was set up so that the effects of a change in the method of calculating the sharpness index could be evaluated in a few minutes.A brief description of the operation follows.A set of images is made by focusing so that each image contains an in-focus region and the series of images covers the desired depth range.Given a 3×3 or a 5×5 array of pixels as shown in Fig. 2, a sharpness index may be calculated using an algorithm such as the relatively simple and very effective algorithm, suggested by Ryall (1993), personal communication.

A REFINED METHOD FOR PRODUCING A COMPOSITE IMAGE
In an attempt to produce a smooth depth image such as 4.6 the image at 4.2 was smoothed by repeatedly applying a median filter until the image no longer changed.This image was indistinguishable from the image at 4.6.As the depth image shows the place in each image where focus is best, a new procedure was developed to make the in-focus composite image.The images are processed using a 3×3 calculation area, to produce the composite infocus image and the depth image.The depth image is then smoothed by repeatedly applying a median filter until the image no longer changes.Each grey level in the smoothed depth image is used to produce a binary mask which is then applied to the corresponding source image to copy the original pixels under the mask and these pixels are then pasted into the final composite image.The result of this operation is a composite deep-focus image made from areas from each of the original images, noise is only present if it exists in an in focus area, there is no accumulation of the noise.
Fig. 5 shows one image from a through focus series of a fracture surface and also shows the resultant composite deep focus image.
A preliminary report of some of the data was presented at the X th International Congress for Stereology, Melbourne, Australia, 1-4 November 1999.
Fig. 1 also indicates that the finest possible depth discrimination is obtained by the use of the largest numerical aperture lenses, § µm for the largest aperture (0.95 NA) dry lenses, and §P IRU WKH largest aperture (1.4 NA) immersion lenses.For comparison the corresponding transverse resolutions obtained by these lenses is 0.35 µm and 0.15 µm, respectively.

Fig. 1 .
Fig. 1.Plot of the Transverse or Vertical resolution of a lens versus its numerical aperture.The step in the plotted values at numerical aperture of 1 is caused by the transition from air (with a refractive index of 1) to an immersion fluid with a refractive index of 1.515.

Fig. 2 .
Fig. 2. Diagram of the pixel arrays used for the calculation of sharpness indices.

Fig. 3 .
Fig. 3.The image series used to test sharpness index calculation.

Fig. 4 .
Fig. 4. Pairs of composite and depth images. 1 and 2 were produced using a 3×3 pixel array, 3 and 4 using a 5×5 pixel array and 5 and 6 produced using a 7×7 pixel array.Each of the six grey tones in the images 2, 4 and 6 represents the part of the original image series (1 to 6 in Fig.3) which was in focus.Examination of the composite images 4.1, 4.3, 4.4, reveals little obvious difference, whereas the depth images(Figs.4.2, 4.4, 4.6)  clearly show that calculation over a larger area results in a smoother, less noisy image.