WO2000051081A1

WO2000051081A1 - Method of locating an object boundary in an image

Info

Publication number: WO2000051081A1
Application number: PCT/AU2000/000133
Authority: WO
Inventors: Nicola Joy Ritter
Original assignee: The Lions Eye Institute Of Western Australia Incorporated
Priority date: 1999-02-25
Filing date: 2000-02-25
Publication date: 2000-08-31
Also published as: AUPP890199A0

Abstract

A method for locating the boundary of an object in a digitised image, includes determining an initial contour by identifying a zone of the digitised image within the object, or of a modified image derived by processing the digital image, which zone has the minimum mean greyscale in the image. An active contour model is applied to locate the boundary, wherein the vertices of the contour are iteratively varied under the influence of internal forces that push the vertices towards a perfect polygon and external forces determined from the original digitised image and/or the modified image.

Description

METHOD OF LOCATING AN OBJECT BOUNDARY IN AN IMAGE

Field of the Invention

The present invention concerns a method of detecting and registering a discrete feature such as an object boundary in a digitised image. In particular the method is useful for locating objects, eg. circular objects, in a variable background. Specifically, the method of the present invention can be used to locate the pupil in images such as digital slit lamp images of an animal or human eye.

Background Art

The slit-lamp is a diagnostic tool used in ophthalmology to ascertain the health of the anterior segment of the eye. A bright slit of light is directed into the eye and the resultant reflections can provide information as to the presence or absence of disease in the lens, iris or cornea. The slit-lamp can also be used to judge the thickness of the cornea and the depth of the anterior chamber. However, in order to investigate changes or to provide a three dimensional image, multiple images or photographs taken over time need to be registered.

Slit-lamp reflections appear as three stripes on the eye, the brightest of which is reflected from the iris, the remaining stripes from the back and the front of the cornea. However, the presence of the slit-lamp reflections and other external (specular) reflections can result in bright or "washed out" patches in the image which can complicate the registration process. Movements of the eye and the surrounding tissue can also cause distortions in slit-lamp images, due to changes in the size and position of the pupil and interference from eye-lashes.

The required registration therefore needs to be based on the pupil and the iris, not on the surroundings of the eye. Traditional techniques involving template matching (See for example, IEEE Transactions on Medical Imaging 6(3): 272-278, Physics in Medicine & Biology 39: 907-915 & IEEE Conference on Vision & Pattern Recognition: June 1996, p351-357) have difficulties with the lack of detail supplied by the pupil and iris. This causes problems with the automated selection of an "interesting" template. Whole image techniques such as phase based methods (See Proceedings of the IEEE 1975 International Conference on Cybernetics and Society. Sept. 1975, p 163-165 & Proceeding 1996 IEEE Computer Society Conference on Computer Vision and Pattern Recognition: June 1996, p 532-537) and mutual information (See Medical Image Analysis 1 (1):35-51 or IEEE Transactions on Medical Imaging 16(2): 187-198) may become confused by features such as the eyelids and eyelashes, which move relative to the pupil. Eye-tracking techniques are another option in registering slit-lamp images. However, they often require complicated physical hardware or are based on the use of infra-red image differencing methods, in which details of the pupil surroundings, including the slit-lamp reflections, are lost.

The use of active contours (or 'snakes') is an alternative that has previously been applied to locate features in an image. In this method a closed contour is deformed by opposing stresses until a steady state is reached, defining the position of the area of interest. Kass et al (See International Journal of Computer Vision, 1988. Vol. 4, p321-331) first defined an active contour as a spline with energy given by:

hcontour ⁼ (Eint+ c img + Ecoπ

where:

EM is the internal energy of the contour at each vertex, enforcing the particular shape characteristics required by the current implementation;

Ejmg is the image energy at each vertex of the contour and is defined by the vertex's current position in the image and;

Econ is the image constraint applied to that vertex forcing the contour to a local minimum. The contour is placed on the image so as to encircle the object of interest. The contour forces, which minimise E_∞n_toυr, then push the contour inwards to find the edges of the object. In general, the vertices of a contour are pushed around under the influence of two opposing forces, one of these formed from constraints internal to the contour and the other from constraints external to the contour, that is from the image. Therefore each vertex is moved from time t to time t + 1 by

vι(t + 1) = v (t) + /^■"-' (t) + f ^mm°^> • (1 )

The internal forces are dependent on the characteristics that it is desired to balance for the active contour.

US patent 5,832,116 teaches the use of an active contour model to identify the edges of a road in satellite imagery, where the distance between the two edges of the road varies. The edges are found by producing two contours which follow the edges of the road. A manual operator chooses the starting point of the contours on the image. The contours are defined by a set of straight line segments which are divided up into a number of discrete points, paired between the two contours. The deformation function uses the standard energy model with an additional term E_C0Up which refers to elastic coupling forces which act between the paired points of the two contours. The elastic force works to maintain the paired points at a certain distance and to prevent slipping between the paired points.

US Patent 5,699,799 discloses a method for automatic determination of the true axis (centre point) and radius of cross section for tube-shaped objects in medical imaging. The method searches for the centre point of a cross section by maximising or minimising a correlation function about a reference point. The radius & centre points are maximized for consecutive transverse slices. The maximised centre points are then joined together to produce the curved axis of the organ of interest (e.g. left heart ventricle). Reslicing of the image using the determined axis can then be performed to achieve more accurate diagnosis. The concept of active contours has also been applied to other biomedicai imaging including ultrasonography (See US Patent 5,605,155 or Biomedicai Engineering - Applications-Basis-Communication, 1998. 10(2): 110-18) and MRI scans or echocardiograms. (See US Patent 5,239,591). The above described methods are appropriate for extracting contours of straight lines or irregular shapes in images in which lighting is not necessarily a consideration. In the case of slit-lamp images, the feature of interest, the pupil, is a regular shape and registration is complicated by reflections which may distort the boundary of interest. An object of the present invention is therefore to provide a method of locating a boundary of an object in an image that is adaptable as an automatic registration method that can locate a circular object in an image. More specifically, in one or more embodiments of the invention, it is an objective to provide a method of automatic determination of a specific contour, such as the pupil / iris border, in a real image with light stripes.

Summary of the Invention

The invention entails an appreciation that an active contour model can be successfully applied to achieve the objects of the invention. The invention uses apriori information about the form of the object - that the desired shape is a circle - for which the search is being made. This apriori information is used to provide internal forces in the active contour model that are calculated globally for the contour rather than locally for individual vertices. In other words, in a specific implementation of the invention, the internal force is provided by a push to each vertex such that the vertex positioning is correct where the whole contour forms a perfect polygon (circle) a little bigger than the current average size, while the external force is based on the actual image and a variance image, rather than an edge image as convention dictates.

In a first broad aspect of the invention, there is provided a method for locating the boundary of an object in a digitised image, including:

determining an initial contour by identifying a zone of the digitised image within the object, or of a modified image derived by processing said digital image, which zone has the minimum mean greyscale in the respective image; and

applying an active contour model to locate said boundary, wherein the vertices of the contour are iteratively varied under the influence of internal forces that push the vertices towards a perfect polygon and external forces determined from original digitised image and/or the modified image.

Advantageously, said internal forces are calculated globally for the whole contour rather than locally for individual vertices. Preferably, the internal force F, for each vertex is given by:

where:

v, is the correct position for individual vertices in a perfect polygon.

Preferably said external forces are based on the original digitised image and said modified image.

Said external force preferably pushes each vertex towards the centre of the contour with energy calculated from the difference between pixel values in the image and the modified image.

In a second broad aspect, the invention is directed to a method for locating the boundary of an object in a digitised image, including

applying an active contour model to locate said boundary, wherein the vertices of the contour are iteratively varied under the influence of internal forces determined by assuming the boundary is of circular shape, and of external forces determined from local greyscales in the image within the object.

Where the boundary is assumed to be of circular shape, the active contour is preferably pushed to form a globally expanding discrete circle, ie. a perfect or near-perfect polygon.

The invention also provides a method that combines both broad aspects of the invention. In either aspect of the method, the iterative process is advantageously repeated with said internal forces decreasing and/or increasing in size until equilibrium is reached.

The object may be a substantially circular object.

In a particularly useful application of the invention, the object is the image of a pupil in a digitised image of an eye, eg. a slit-lamp image, and the boundary to be located is the pupil/iris border. The eye may be human or animal.

It is to be understood that references herein to "digitised image" and "image" embrace, where the context requires or permits, a set of digital data indicating greyscale or other values for the pixels of a corresponding viewable image.

In a still further aspect, the invention provides a method of diagnostic investigation of an eye, including:

acquiring a plurality of slit-lamp images from the eye at predetermined intervals and/or over a predetermined period, for comparative analysis;

wherein the analysis includes locating the pupil/iris border and/or iris/sciera border in the images in accordance with the method of the first and/or second broad aspects of the invention.

In a still further aspect, the invention provides a computer program product including a set of machine readable instructions that, when installed in a computer having a suitable operating system, are operable to carry out one or more of the aforementioned methods of the invention.

Preferred Embodiments of the Invention

In order that the invention be more fully understood, a preferred embodiment will be described by way of example with reference to the following illustrations in which:

Figures 1 and 2 are digitised representations of respective slit-lamp images of an eye;

Figure 3 is a processed image derived from Figure 2;

Figure 4 is a simplified flowchart illustrating the various processing steps of a method embodying several aspects of the present invention;

Referring to Figure 1 , in slit-lamp images of the eye 10 the pupil 12 appears as a uniform, dark circle and the slit-lamp reflections as three stripes, 14, 16 & 18. The brightest reflection 14 is from the iris 20, the coloured structure made up of muscular fibres. The remaining reflections 16 and 18 are generated from the back and the front of the cornea respectively. Stripes 16, 18 are interesting because they can be used to ascertain the thickness and topological variations of the cornea. The "fuzziness" at the rear of each reflection - most noticeable on that of the iris - is caused by scatter due to imperfections in the cornea and is a major diagnostic tool for locating disease in the eye. Specular reflections 22 from photographic flash and ambient light also appear on the image. Figure 2 depicts a further sit-lamp image in which the reflection 14' from the iris has been interrupted by the pupil and lens. There is also reflected light from the front and rear of the cornea obscuring the pupil-iris border.

The presence of slit-lamp and specular reflections make images such as Figures 1 and 2 difficult to register. The present invention provides an accurate method for registering both the pupil / iris border and the iris/sclera border in slit- lamp images, without losing the desirable information provided by the slit reflections. The method of the present invention proposes the use of a novel discrete active contour that is moved under the influence of internal forces based on a globally expanding discrete circle (perfect polygon), and external forces that pushes each vertex towards the centroid of the contour with energy calculated from the difference between pixel values in the appropriate image, being the original image for the iris/sclera border and a processed image for the pupil/iris border.

Figure 4 is a flowchart showing the operation of the method of the present invention. The method takes place in stages, the initial step (100) of which involves processing the image by changing all those pixels whose values are over a threshold to the average pixel value for the image. This produces the processed image used for searching for the pupil/iris border. Figure 3 is such a processed image derived in this manner from the image of Figure 2.

To find a point interior to the pupil, an annulus is displaced over the whole image. Other shapes of zones could be adopted, but an annulus is convenient. For every position to which the annulus is moved, the mean internal to it is calculated. An iterative search is made for the annulus that has the lowest mean greyscale of pixels enclosed within the ring, step 110, indicating an area inside the pupil. This allows the active contour algorithm to have an efficient starting position (120).

An active contour is copied from the starting contour and is used to locate the pupil/iris boundary (step 130). The centre of this contour and a radius expected to be smaller than the iris/sclera boundary is used as a starting point of a contour that is formed from two opposing circular arcs. This double arced contour is then used to locate the iris/sclera border (step 140). Its result is similarly used to seed the starting point of the next pupil/iris border search and this process continues iteratively until either the last two pairs of borders agree with each other, or the number of iterations exceeds a set number (step 150). This process is shown in the flowchart of Figure 4.

In order to find the boundary of the structure of interest, each point or node on this polygon has two forces acting upon it. The contour is moved from time t to f +1 using:

v (t + l) = v(t) + βFi(t) + (l - β)fi(t) V, = l...n,

where: is the force formed from the internal structure of the contour,

t is the force produced by the information external to the contour - that is information from the images - and

βι is a constant used to balance the values, dependent on the level of contrast within the images.

The internal force F,- works to push the contour outwards from it's current position into one with the same centre point but with a radius increased by δ (i.e. to the position of the next larger-sized perfect polygon). The internal force for each vertex is found with

where:

vι is the correct position for individual vertices in a perfect polygon.

The centroid of the polygon is found by averaging the positions of the nodes that make up the polygon. That is:

Therefore if R is the average radius of the current contour and p = (r,c) is the current centroid then

Vi = r - (R + δ) * + (R + δ) * cosf — J

n being the number of vertices in the contour.

An external force opposes the internal force by pushing each vertex inwards towards the centre of the polygon with a magnitude determined by the appropriate image, being the processed image for the pupil/iris border and the original image for the iris/sclera border.

If (vι) represents the nearest neighbour greyscale value on the image to the vertex v. in processed image I:

/,(/) = d,(l(y.)- l(y. + d,))

where d, is a unit vector given by

p being the centroid of the contour.

The algorithm will move the working contour iteratively using the internal force added to the external force until either the limit on iterations M, or the edge of the image, is reached.

The algorithm will test for equilibrium, if the contour remains within the confines of the image. That is, equilibrium is determined by the centroid and average of the contour remaining steady through successive iterations. Repeated movements of the contour from its starting point, with decreasing or increasing values of δ, will continue until equilibrium occurs and the pupil / iris boundary has been located. If equilibrium has not been attained, the algorithm will continually cycle through steps 130 - 140.

Thus, the algorithm of the present invention is as follows:

Copy the starting contour to the working contour WHILE equilibrium has not been reached AND δ > 0 AND δ < δmax AND δ_c > δcthreshold

Move the working contour Check for equilibrium Alter δ and δ_c appropriately ENDWHILE In a series of experimental tests the algorithm consistently found the pupil / iris and iris/sclera border in a sequence of digitised images of the same eye with accuracy equal to that of a manual operator.

All processes within the method described herein above would be carried out automatically by a computer. One skilled in the art would be able to program a computer to perform the method as described. Modification within the spirit and scope of the aforementioned invention may be readily effected by a person skilled in the art. It is to be understood, therefore, that this invention is not limited to the particular embodiments described.

Claims

1 A method for locating the boundary of an object in a digitised image, including:

determining an initial contour by identifying a zone of the digitised image within the object, or of a modified image derived by processing said digitised image, which zone has the minimum mean greyscale in the respective image; and

applying an active contour model to locate said boundary, wherein the vertices of the contour are iteratively varied under the influence of internal forces that push the vertices towards a perfect polygon and external forces determined from the original digitised image and/or the modified image.

2 A method according to claim 1 wherein said internal forces are calculated globally for the whole contour rather than locally for individual vertices.

3 A method according to claims 1 or 2, wherein the internal force Ft for each vertex is given by:

where:

VΪ is the correct position for individual vertices in a perfect polygon.

4 A method according to claims 1 , 2 or 3, wherein said external forces are based on the original digitised image or the modified image.

5 A method according to any one of claims 1 to 4, wherein said external force pushes each vertex towards the centre of the contour with energy calculated from the difference between pixel values in the original image, and/or the modified image.

6 A method according to any one of claims 1 to 5, wherein said internal forces are determined by assuming the boundary is of circular shape, and said external forces are determined from local greyscales in the image within the object.

7 A method according to claim 6, wherein said active contour is pushed to form a globally expanding discrete circle, ie. a perfect or near- perfect polygon.

8 A method according to any one of claims 1 to 7, wherein the iterative process is repeated with said internal forces decreasing and/or increasing in size until equilibrium is reached.

9 A method according to any one of claims 1 to 8, wherein said object is a substantially circular object.

10 A method according to any one of claims 1 to 9, wherein said object is the image of a pupil and/or an iris in a digitised image of an eye, and the boundary to be located is the pupil/iris border and/or iris/sclera boundary.

11 A method according to claim 10, wherein said digitised image of an eye is a slit-lamp image.

12 A method for locating the boundary of an object in a digitised image, including

13 A method according to claim 12, wherein said active contour is pushed to form a globally expanding discrete circle, ie. a perfect or near- perfect polygon.

14 A method according to claims 12 or 13 wherein said internal forces are calculated globally for the whole contour rather than locally for individual vertices.

15 A method according to claims 12, 13 or 14, wherein the internal force E, for each vertex is given by:

where:

vi is the correct position for individual vertices in a perfect polygon.

16 A method according to any one of claims 12 to 15, wherein said external force pushes each vertex towards the centre of the contour with energy calculated from the difference between pixel values in the original image, and/or the modified image.

17 A method according to any one of claims 12 to 16, wherein the iterative process is repeated with said internal forces decreasing and/or increasing in size until equilibrium is reached.

18 A method according to any one of claims 12 to 17, wherein said object is a substantially circular object.

19 A method according to any one of claims 12 to 18, wherein said object is the image of a pupil and/or an iris in a digitised image of an eye, and the boundary to be located is the pupil/iris border and/or iris/sclera boundary.

20 A method according to claim 19, wherein said digitised image of an eye is a slit-lamp image.

21 A method of diagnostic investigation of an eye, including:

wherein the analysis includes locating the pupil/iris border and/or iris/sclera border in the images in accordance with the method of any one of claims 1 to 20.

22 A computer program product including a set of machine readable instructions that, when installed in a computer having a suitable operating system, are operable to carry out one or more of the methods of claims 1 to 21.