WO1983000972A1

WO1983000972A1 - Image analysis system

Info

Publication number: WO1983000972A1
Application number: PCT/AU1982/000152
Authority: WO
Inventors: Of Australia Commonwealth; Robert James Frederick Dow; James Dominic Quinn; Stanley Robert Silva; David William Williams; Kenneth Kwan Mow Wu
Original assignee: Commw Of Australia
Priority date: 1981-09-10
Filing date: 1982-09-08
Publication date: 1983-03-17
Also published as: EP0087442A1; IT8249094A0; EP0087442A4; IT1155238B; JPS58501445A

Abstract

Image analysing method and system, and in particular method and system for analysing X-ray radioscopic images of the contents of an article of mail to detect the presence of Improvised Explosive Devices (IED's). The system comprises a CCTV camera to scan said image and provide signals indicative of the intensity of the image at any particular point, means to process said signals and provide an indication of when the signal makes a transition through a predetermined threshold level and also the duration of any signal whose threshold varies from the, or each, said predetermined level as a measure of the extent of dark or light areas on the image, and means automatically activated in response thereto for processing said signals to apply a pattern recognition technique and including a Transition Position Encoder, a High Speed RAM Buffer Circuit, a Direct Memory Access Circuit (DMA) all incorporated with a Single Board Computer (SBC).

Description

"IMAGE ANALYSIS SYSTEM"

Technical Field

This invention relates to an image analysis system and more particularly a system for analysing an X-ray image of mail items for the purpose of detecting the pressure of Improvised Explosive Devices (IED's). Background Art

Historically there are several techniques and devices employed in the detection of IED's, the principal ones being X-ray fluoroscopy and vapour trace detection. The former technique has involved visual examination of an X-ray image, and is dependent on the sustained alertness and interpretative ability of a human operator, whilst the main problem with vapour trace detection techniques is that they can be readily countered by suitable containment of the explosive device, and furthermore they are incapable of detecting some of the types of explosives commonly used.

The use of X-ray fluoroscopy techniques are still the most feasible because of the following advantages that they offer:

1. Penetration of covering material to reveal internal contents.

2. Detection based on distinctive densitometric and geometric properties of IED components.

3. Independence from chemical characteristics of explosives, such as the escape of volatiles.

4. Potential capability for virtually certain detection combined with extremely low incidence of false alarms. 5. Additions to explosive can be considered and exploited to ensure a virtually unique response to X-ray fluorescence.

As applied to the detection of IED's the present invention involves identification and therefore detection by image analysis of explosive devices in situations where a number of apparently similar items are present, by utilising the particular characteristic qualities of the radioscopic images produced by such explosive devices. The radioscopic image of a lethal IED possesses a minimum proportion of dark area caused by the presence of X-ray absorbing material essential to the operation of the IED, and in particular a lead azide primer charge, present in the most commonly used commercially available detonators required to detonate an explosive charge. In such an application the present invention involves discerning the presence of such a material as distinct from the radioscopic images produced by other items present, which in the case of mail processing may be various types of paper clips. It is important that these articles be distinguished from an IED and the article of mail in question not diverted. As such items are present in relatively large numbers in mail the lack of such a capability would mean a relatively large amount of the mail would be necessarily diverted for special attention thus seriously inhibiting the rapidity and efficiency of the mail sorting operation.

In an earlier International Patent Application No. PCT/AU81/00068 in our name, and also entitled "IMAGE ANALYSIS SYSTEM" there is disclosed an apparatus and method for analysing images involving the use of CCTV camera to scan the image and provide video signals which were processed to provide an indication of the duration of any signal whose intensity varied from a predetermined value as the measure of the extent of dark or light areas on the image (hereinafter referred to as SYSTEM 1).

As applied to the detection of IED's, the invention of SYSTEM 1, utilised the characteristic of the fluorescent image produced by the IED by employing a closed circuit television (CCTV) camera which provided video signals from which the extent of dark areas of the images produced may be measured in terms of the duration of any signal whose intensity was less than a predetermined level (threshold level). A signal designating the category of each item displayed in the image frame as either safe, or potentially harmful, was generated based on the total area count and was immediately available on completion of a scan of one complete image frame on the CCTV camera.

In summary, SYSTEM 1, as applied to the detection of IED's, basically rested with an electronic system utilising a CCTV camera to rapidly clear any large number of articles, such as mail, as being safe, whilst if the throughput amount is relatively small a trained operator may be sufficient to clear any items detected as being possibly harmful.

SYSTEM 1 can be regarded as a "stand-alone" system. Its employment in a mail Registry situation would be directed at ensuring that mail which has been checked by the system does not contain an IED. However, articles with X-ray image densities comparable to that of an IED image are also rejected by the system. Thus, generally, a number of innocuous articles in addition to IED's would be rejected. For a small Registry situation, SYSTEM 1 may, for example, clear as safe 98% of the checked mail. The remaining suspect mail may then be examined fluoroscopically by an observer. Nevertheless, the small percentage of suspect articles could still amount to a large volume of mail in a high throughput situation which would make considerable demands upon an observer. The system of the present invention is designed to reduce this high volume by reducing the number of suspect articles, and this is achieved by the use of pattern recognition techniques by which miscellaneous or common articles such as X-ray dense large paper clips are accepted as being innocuous.

The system of the present invention may be integrated with other mail sorting equipment or procedures. These may include (in the interest of speed) an initial sorting device which diverts mail of thickness equal to or greater than that of a minimum size explosive device capable of causing significant injury. SYSTEM 1, the system of the present invention, and fluoroscopic viewing facilities may be incorporated into one unit employing one assembly of X-ray source and electro- optical equipment. Alternatively the system of the present invention may be utilised in a sequential checking system at the cost of extra assemblies.

Disclosure of the Invention

The present invention therefore envisages an apparatus for the detection of IED's comprising means to produce an X-ray image of the article under investigation, means to scan said image and provide signals indicative of the intensity of the image at any particular point, means to process said signals so as to provide an indication of at least when any signal makes a transition through at least one predetermined (threshold) level, and means activated in response thereto to process said signals and apply a pattern recognition technique to the image.

Preferably the means to scan said image is a CCTV camera.

In one preferred form of the invention, said means to process said signals is in accordance with that the subject of our aforementioned International Patent Application, that is, such as to provide an indication of not only when the signal makes a transition through a predetermined intensity level but also the duration of any signal whose (threshold level) varies from the, or each, said predetermined level as a measure of the extent of dark or light areas on the image, and said means for processing said signals to apply said pattern recognition technique is automatically activated in response thereto.

In another preferred form of the invention, said means to process said signals merely provides an indication of when any signal makes a transition through the, or each, predetermined threshold level, and said means for processing said signals to apply said pattern recognition technique is activated in response thereto.

The invention also envisages a method for analysing images, comprising the steps of scanning said image and providing signals indicative of the intensity of the image at any particular point, processing said signals so as to provide an indication when any signal makes a transition through at least one predetermined threshold level and, in response thereto, applying a pattern recognition technique to the image. The pattern recognition technique may be automatically activated to directly apply the pattern recognition technique to the image of the suspect article. Alternatively the suspect article may be dive'rted for separate and subsequent application of the pattern recognition technique.

One preferred embodiment of the invention, as applied in a system for the screening of mail, will now be described with reference to the accompanying drawings, in which:-

Brief Description of the Drawings

Figure 1 is a schematic diagram showing the basic arrangement of equipment in accordance with one preferred form of the present invention, and in particular the manner of linking the equipment to the multibus of the Single Board Computer utilised for the purposes of carrying out the pattern recognition technique of- the present invention,

Figure 2 is a block diagram of the basic circuitry of the processing system as employed in this preferred form of the invention,

Figure 3 is the procedure as carried out in a processor to provide recognition or identification of radiographic images,

Figure 4 is a flow chart of the algorithm employed to carry out part of the processing technique of the present invention, and Figure 5 is a flow chart of an example of an algorithm employed specifically for the identification of paper clips and therefore the clearance of mail containing such items.

Best Mode for Carrying out the Invention

Referring to Figures 1 to 3 of the drawings the article under investigation is positioned before an X-ray emitter, and the X-ray image produced is displayed on a fluorescent screen where it is scanned by a CCTV camera to produce a digitised signal. The digitised signal may then be processed in accordance with SYSTEM 1 for the purposes of detecting the presence of a possible IED.

In accordance with this preferred form of the invention, the "Processed Video" image which would be seen on the TV monitor with SYSTEM 1 is a single level discriminated image of the complete TV picture. The only information in the "Processed Video" image is the perimeter where the video picture makes a transition through a fixed (threshold) level. Run end coding is employed in the storage of the perimeter data in order to reduce the quantity of data and data rate. An Intel Single Board Computer 86/12A is employed. It has an 8086 CPU which has a 16 bit, one megabyte addressable microprocessor. One version of the system of this embodiment of the present invention, and as shown in

Figures 1 and 2, consists of four boards fitted into the Intel SBC 604 cardcage. The four boards are:- (a) Video Encoder Board which takes the Video signal from the TV camera and processes it for two level (i.e. above and below the threshold level), run end coded data output.

(b) High Speed RAM Buffer Board which stores the incoming run end coded data, and incorporating a high speed IKxl-Byte static ram 2115A provided by Intel.

(c) Direct Memory Access (DMA) Control Board incorporating an integrated circuit 8257-5 as also provided by Intel. RAM Buffer Board.

(d) Single Board Computer (SBC 86/12A) which consists of an Intel 8086 CPU (5 MHz clock), 16K bytes of EPROM and 32K bytes of dynamic RAM.

Data transfer rates within the system determine in one complete scan of the TV camera (odd and even lines), 23 thousand transitions which can be stored in the 86/12A memory. For a 512 line picture this is an average of 45 transitions per line. The maximum number of transitions for anv one line is 512.

Video Encoder Board

In this section any transition in the output of the comparator in SYSTEM 1 causes the position of the transition in the TV image to be encoded into a sixteen bit word, together with the direction of transition. The sixteen bit word is sent to the data acquisition and storage circuit described below.

The encoding of the position of any pixel in a 512 x 512 matrix in a sixteen bit word is performed by run end coding. The y-co-ordinate, or vertical position is not encoded into each word but a word is generated at the end of each scan line and a one bit flag inserted to indicate this. The x-co-ordinate or horizontal position of the transition is encoded into the 16 bit word as a value between 0 and 512.

High Speed Ram Buffer Board

The 16 bit data words which are generated by the video processing circuit, in a minimum time period of 120 nanoseconds, must be stored within the time interval of 120 nanoseconds. The high speed memory, capable of this performance, consists of two identical memory buffers, each 16 bits wide by 1024 words deep.

The steps of the data acquisition process are as follows:-

(1) The first buffer is filled by the asynchronously generated 16 bit data words from the video signal processing circuit. (2) The 16 bit data path is now switched from the first buffer to input to the second buffer. The data in the first buffer is now transferred by a direct memory access circuit to the memory of the central processing unit. Simultaneously the second buffer is filled by the data output from the video processing circuit.

(3) When the second buffer is full, the input data path is switched to the first buffer, by this time the data in the first buffer is transferred by the direct memory access circuit. The second buffer now has its data contents transferred to the CPU memory by direct memory access.

(4) The process of one high spe-ed memory buffer being filled by the high speed asynchronous data from the video processing circuit while the other buffer is being emptied of its data to the CPU memory via the direct memory access circuit continues until the completion of one interlaced picture scan.

DMA Control Board

The direct imemory access circuit is designed to interface with a commercial "backplane"' or bussing system marketed by the Intel Corporation as Multibus. When initiated to do so by the filling of one high speed memory buffer, the DMA circuit gains control of the multibus to become the bus " master". In this mode it can write into the memory of the CPU. The DMA circuit transfers the data in the high speed memory buffer to the memory of the CPU.

Immediately the high speed memory buffer has been emptied, the DMA circuit relinquishes control of the multibus and hence its ability to access CPU memory. This allows the CPU to immediately begin processing the data which has just been transferred into it by the DMA circuit. The CPU is thus processing the video data simultaneously with the acquisition of data by the high speed memory buffers. Analysis of the image can be partly complete before the end of the picture scan.

Single Board Computer

The central processing unit is a commercial Single Board Computer 86/12 as supplied by Intel

Corporation. The technical specification of this board computer is available commercially from Intel Corporation or its agents.

The algorithm used to recognise the processed video image is stored in electrically programmable read only memory integrated circuits and can thus be changed by the replacement or reprogramming of those integrated circuits containing the program.

Procedures for Recognition of Images

The processes involved in the pattern recognition technique are outlined in Figure 3. These include:- (1) Image acquisition - the conversion of the analogue optical image into digital form and its transfer into the computer CPU;

(2) Thresholding - the selection of relevant details based on the densitometric information of the image;

(3) Preprocessing and Enhancement - the processing of the raw image to improve its quality, rendering it more accessible to analysis and recognition;

(4) Component Labelling - the extraction of individual objects from an image;

(5) Feature analysis - the determination of geometrical features of the individual objects for shape description;

(6) Recognition/clearance - the identification of objects based on their geometrical features.

Each process will be discussed in detail in the following sections.

Image Acquisition

A standard time-saving technique for analysing an optical image is to couple a TV camera through a suitable interface to a computer. The image on the target of the camera tube is scanned by an electron beam which creates a potential difference and produces on a collector a signal proportional to the input brightness pattern. A digital image is obtained by quantizing the signal through an analogue to digital converter at fixed points along the scanning beam The transfer of fully digitised information at the standard TV scanning rate is difficult to achieve. One solution is to use a slow-scan camera. With this preferred embodiment of the invention a Hamamatsu C1000-12 SIT camera is used coupled to the minicomputer as the image acquisition and analysis system. As an alternative, in order to avoid the high cost of the camera and its slow rate of image acquisition, a standard TV camera may be used with the amount of video information to be transferred minimised. A large reduction in the quantity of video information may be achieved by classifying the video signals into two states according to predetermined intensity levels. The two states are, the black, for signals lying between intensity levels deemed to represent useful information, and, the white, for signals outside the range, deemed to be irrelevant. The compatibility of the technique to radiographic images of mail will be discussed in the next section.

A further reduction in the amount of data transfer may be achieved by the application of run-end coding technique to the binary image. A run denotes a succession of image elements on a line of scan. Run-end coding describes a scheme whereby only the positions of the two ends of each run along the line of scan are stored. The system of the present invention is designed to store images in this form. The component labelling algorithm is also designed to handle the run-end coding data. Thresholding

Thresholding involves conversion of the digital image into binary form, based on its densitometric information; thus eliminating irrelevant data from the image as per SYSTEM 1.

Enhancement

A defect usually associated with the video signal generated by a TV camera is the baseline droop. Applying thresholding techniques to this type of signal would not produce satisfactory results, however this effect can be corrected electronically by devices available commercially. The effect can also be corrected utilising suitable software, by simply subtracting a suitable background from the image.

A second type of degradation associated with a

TV image is the presence of random noise. If the original image is too noisy near the threshold, the resultant binary picture would contain scattered white and black spots. This noise poses little problem in the present system because the threshold level is usually set well outside the region affected by the noise. In cases where the noise would affect the outcome of thresholding, it may be suppressed by applying a digital filtering technique. Component Labelling

Component Labelling is the major step in the image analysis of the present invention in that it groups information extracted by thresholding into individual objects. A flow chart of a suitable labelling algorithm is shown in Figure 4. This algorithm has several useful features:-

(1) It operates on information contained in any two consecutive lines of an image at any one time. The efficient use of CPU memory enables the analysis of complex images to be carried out. It is possible to use a machine with a memory of 32K to analyse a 256 x 256 image and handle a maximum number of 256 transition points

(128 pairs) as in check board situation on any one line.

(2) The need for overlap search is reduced to a minimum. The time required for analysing an image (including feature measurements) with an average number of transition points of 1000 is less than 1 second with Intel 80/8tϊ utilised.

(3) It handles the multiple overlap situation with a simple relabelling procedure by replacing the label of each run on the two lines of scan being processed by its lowest equivalent labels when necessary. This permits feature analysis and component counting for the present study to be complete in a single pass without resorting to an equivalence table. In cases where complete relabelling of the image is required, this procedure results in a simpler equivalence table, thus the effort required for its processing may be greatly reduced.

The basis of this section of the system of the present invention is that each run of picture elements along a line of scan is specified by the coordinates of its ends on the line. In this case the most appropriate algorithm for labelling is that of run-tracking in accordance with Figure 4. The runs are located sequentially from left to right along each line, and line by line from the top to the bottom. On the first line of the image, each run is treated as belonging to a new object, thus given a new label. The labels which are usually whole numbers start with 1 then continue in ascending order according to the order in which the objects are encountered. On the second and subsequent lines, each run is subjected to an overlap search to determine whether it overlaps with any run on the preceding line. The run is assigned a new, hitherto unused label, if there is no overlap. But, if there is overlap, it is given the same label as the run it overlapped. The overlap search is continued to determine whether it also overlaps with the next run on the preceding line. If there is more overlap, and the runs are of different labels, the run with the higher number is relabelled with the lower number. As the process of overlap search and relabelling can be very time consuming, especially for complex images, it demands careful design of its computation procedure. With reference to the flow chart of Figure 4 a run is denoted by R (X-, Y^) where X and Y are the coordinates of the left- and right-end of the run

.th respectively. The subscript i indicates that it is the l th run counting from the left of the line. The r run on the preceding line is denoted by R (x"_r, Y"_r). The condition of overlap is

X_i < Y^' _r and Y_i > X '_r

The condition implies the concept of 4- adjacency for connected elements. Generally, both inequalities must be tested to determine if there is overlap. However, the procedure is so designed that, in most cases, it would be sufficient to test one of the inequalities only. In addition, it is only necessary to test P. (X_i, Y_i) for overlap with R (x '_r , Y ^' _r ) which satisfies the condition Y_r > Y_{i - 1} , and the search is terminated as soon as the condition Y_r > Y_i is fulfilled. Consequently, overlap search for R (X_i+1' Y_i+1), will automatically start with the run R (X_r , Y _r ) which terminated the search for R (X_i, Y_i). This procedure has been thoroughly tested on a large number of complex images.

In connection with the algorithm of Figure 4, the following notes are relevant to a consideration of its procedure at the points designated in the flow chart.

NOTES:

(1) Search not required, new label for (X_i , Y_i) and all subsequent runs on the line.

(2) No overlap with (X '_r , Y_r), test for overlap with next r. (3) No overlap with (x^' _r, Y "_r ) , terminate search and assign new label to (X^, Y_j_) .

(4) Overlap with (X^{^} _r, Y"_r), assign label of (X'_r, Y'_r) to (X_i, Y_i) .

(5) May also overlap with next (X_r , Y ^' _r ).

(6) Terminate overlap search and assign new labels to all subsequent (X_i, Y_i).

(7) No overlap.

(8) Multiple overlap, relabel runs as required.

Feature Analysis

It is important that shape description should be based on information which is independent of magnification and orientation. Preferably the information should be readily extracted from the image. For mail clearance application, simple geometrical features which may be computed in a single pass curing the labelling process are used to characterise shapes. The more exotic shape descriptors such as the Fourier descriptor and differential chain code are not preferred because they would require images with well defined boundaries and would involve greater computational effort. The features we have computed include area, perimeter, moments about a fixed point, vertical and horizontal Feret diameter, and secondary features such as centroid, circularity and central moments:- (1) Area. Area definition is trivial simply the total number of elements of an object. Each time a run is given a label during run tracking the register storing the area of the object with such a label is updated by the number of elements in the run. When runs with different labels merge, the area count in the register for the higher label object is emptied into one with the lower label.

(2) Perimeter. Due to the digital nature of the image, the boundary is always ragged, therefore a number of possible definitions exist. Each time a run is labelled the perimeter count for the label is increased by twice the length of the run plus two counts for the ends. The count is subsequently modified according to the result of overlap search and the adopted definition.

(3) Moments. Moments of any order may be computed in the same time as the area during labelling. These are then translated to central moments, after labelling is completed. The moments may also be normalised with respect to area.

(4) Horizontal and vertical diameter. These are respectively the maximum distance between pairs of vertical and horizontal targets of an object. They are determined by recording the left and right, upper and lower extremities of an object during labelling. These parameters may provide information about whether an object is compact or dispersed.

(5) Centroid. The Centroid is computed at the completion of labelling from the first moments and area. The centroids are particularly useful as they provide a means of deriving a relational structure among the various objects in a picture.

(6) Circularity. This is defined as 4" X Area/Perimeter.² Its magnitude tends to reflect the complexity of the boundary.

This feature takes on its maximum value of

1 for a circle, whilst more complex shapes yield lower values.

Recognition/Clearance

As an example, an algorithm for the clearance of mail containing paper clips is presented in the flow chart of Figure 5. The basis for adopting each criterion is explained below:-

Criterion 1.

Objects below a minimum size, A₁, may be disregarded. It is designed to eliminate noise and fragments of image close to the intensity threshold. The minimum size is chosen to be well below the size of items such as the priming charge of a detonator, blob of solder in electronic circuits, electric wire junctions etc. Criterion 2.

Mail containing items over a maximum size, A₂. are not cleared. A₂ is set to be the size of the largest paper clip designed to be cleared.

Criterion 3.

All paper clips are characterised by a small value of circularity, C, coupled with a large value of area normalised moment of inertia, I. In particular, the C of paper clips are much lower than those of compact items. Therefore, we may clear items with C below, say, 0.1 as paper clips.

Criterion 4.

When the image of paper clips is fragmented through thresholding, the C of the fragmented parts will be greater than the value 0.1 used in Criterion 3. These fragments often appear as bar-like items with a length to width ratio, a/b, greater than 5. In addition, the fragments are much narrower compared with images of other items, therefore, their area counts are relatively much lower than those of elongated compact items with the same a/b ratio. The values of C and I for bars of various a/b ratio are listed in the following Table. Thus, items of size below A. and with C<0.436 coupled with I>0.433, may be cleared as fragments of paper clips.

The choice of quantities A₁ , A_2' A₃ and A₄ in the algorithm is dependent on the exposure and threshold conditions, and should be subject to field trials.

The system described above is designed to meet the throughput requirement of a mail exchange. It is capable of converting an X-ray fluoroscopic image into the run-end coding form and transferring it into the CPU of a microprocessor (Intel 80/86) in one twenty-fifth of a second. The entire image analysis using the labelling procedure described may be achieved in a fraction of a second. A set of algorithms can be designed to clear various common stationery items in addition, to that described previously for paper clips.

An advantage of the system in addition to high speed is that clearance criteria may be varied according to demand. For example, if there is a demand to automatically clear mail containing audio cassette tape. one may design a set of clearance criteria based on the image of its five screws, appeared as five small circles of approximately equal size, and the sum of the distances from each screw to their centroid. Similarly, if the threat of a particular bomb design were made aware to the postal authority, the system may be programmed to search for this particular design.

Claims

1. An apparatus for the detection of IED's comprising means to produce an X-ray image of the article under investigation, means to scan said image and provide signals indicative of the intensity of the image at any particular point, means to process said signals so as to provide an indication of at least when any signal makes a transition through at least one predetermined threshold level, and means activated in response thereto to process said signals and apply a pattern recognition technique to the image.

2. An apparatus according to Claim 1, wherein the means to scan said image is a CCTV camera.

3. An apparatus according to Claim 1 or 2, wherein said means to process said signals is such as to provide an indication of when the signal makes a transition through a predetermined threshold level and also the duration of any signal whose threshold varies from the, or each, said predetermined level as a measure of the extent of dark or light areas on the image, and said means for processing said signals to apply said pattern recognition technique is automatically activated in response thereto.

4. An apparatus according to Claim 1 or 2, wherein said means to process said signals provides an indication of when any signal makes a transition through the, or each, predetermined threshold level, and said means for processing said signals to apply said pattern recognition technique is activated in response thereto.

5. A method for analysing X-ray images to detect the presence of IED's, comprising the steps of scanning said image and providing signals indicative of the intensity of the image at any particular point, processing said signals so as to provide an indication when any signal makes a transition through at least one predetermined threshold level and, in response thereto, applying a pattern recognition technique to the image.

6. A method according to Claim 5, wherein the pattern recognition technique is automatically activated to directly apply the pattern recognition technique to the image of the suspect article.

7. A method according to Claim 5, wherein the suspect article is diverted for separate and subsequent application of the pattern recognition technique.