CA2013378C - Technique for object orientation detection using a feed-forward neural network - Google Patents
Technique for object orientation detection using a feed-forward neural networkInfo
- Publication number
- CA2013378C CA2013378C CA002013378A CA2013378A CA2013378C CA 2013378 C CA2013378 C CA 2013378C CA 002013378 A CA002013378 A CA 002013378A CA 2013378 A CA2013378 A CA 2013378A CA 2013378 C CA2013378 C CA 2013378C
- Authority
- CA
- Canada
- Prior art keywords
- text
- orientation
- features
- symbol
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/24—Aligning, centring, orientation detection or correction of the image
- G06V10/242—Aligning, centring, orientation detection or correction of the image by image rotation, e.g. by 90 degrees
Abstract
A TECHNIQUE FOR OBJECT ORIENTATION DETECTION
USING A FEED-FORWARD NEURAL NETWORK
Abstract The present invention relates to a technique in the form of an exemplary computer vision system for detecting the orientation of text or features on an object of manufacture. In the present system, an image of the features or text is used to extract lines using horizontal bitmap sums, and then individual symbols using vertical bitmap sums, using thresholds with each of the sums. The separated symbols are then appropriately trimmed and scaled to provide individual normalized symbols. A Decision Module comprising a Feed-Forward Neural network and a sequential decision arrangement determines the "up", "down" or "indeterminate" orientation of the text after a variable number of symbols have been processed. The system can then compare the determined orientation with a database to further determine if the object is in the "right-side up" "upside-down" or "indeterminate" orientation.
USING A FEED-FORWARD NEURAL NETWORK
Abstract The present invention relates to a technique in the form of an exemplary computer vision system for detecting the orientation of text or features on an object of manufacture. In the present system, an image of the features or text is used to extract lines using horizontal bitmap sums, and then individual symbols using vertical bitmap sums, using thresholds with each of the sums. The separated symbols are then appropriately trimmed and scaled to provide individual normalized symbols. A Decision Module comprising a Feed-Forward Neural network and a sequential decision arrangement determines the "up", "down" or "indeterminate" orientation of the text after a variable number of symbols have been processed. The system can then compare the determined orientation with a database to further determine if the object is in the "right-side up" "upside-down" or "indeterminate" orientation.
Description
:~ ` 2~-~ 337~
A TECHNIQUE FOR OBJECT ORIENTATION DETECTION
USING A FEED-FORWARD NEURAL NETWORK
Technical Field The present invention relates to a technique for detecting the 5 orientation of features, such as printed text, on an object using a network including a decision arrangement in the form of a feed-forward neural network to determine the orientation of the object.
De~cription of the Prior Art As assembly processes move towards "Just In Time" operation, 10 automatic inspection becomes a more necessary technology. For example, a tight loop between an operation and its inspection can be created, thereby ensuring that when errors occur due to a manufacturing setup, no more than a minimum of defective items are produced. This contrasts with traditional batch manufacturing, where a lot or batch of a product may be 15 made before an error is detected. Rapid detection offers other advantages as well. For example, in surface mounted assembly of circuit packs, components are first placed and held with an adhesive before being soldered. Therefore, an inspection system placed in-line after the placement operation can catch errors before the soldering process is performed, thereby 20 also reducing the repair cost to a minimum.
Various arrangements have been devised for inspecting circuit boards for defects. For example, U. S. patent 4,028,827 issued to B. H.
Sharp on June 7, 1977, discloses a video system for selectively ascertaining the presence and absence of, and discriminating between, at least two 25 different types of light reflecting surface areas on articles. The system has use for inspecting circuit paths and solder connections. Another arrangement is disclosed in U.S. patent 4,578,810 issued to J. W.
MacFarlane et al. on March 25, 1986 wherein an automatic visual tester detects printed wiring board (PWB) defects. The detector comprises an 30 array of optical sensors that forms a binary image pattern of the PWB for optically inspecting the printed wire circuit.
An article by S. P. Denker et al. in Proceedin~s of International Test Conference. October 1~84 in Philadelphia, Pa. at pages 558-563 discloses an automatic visual tester that detects printed circuit board (PCB) 35 assembly errors using rnachine vision technology. In this tester, a camera isused to capture an imagP of the PCB, and transmits an electrical ~
~7r representation of this image to a computer which compares the features of the PCB image with an ideal image stored in memory to detect any assembly errors. An alternative arrangement was disclosed in the article by D. J. Svetkoff et al. in Hybrid Circuits (GB), No. 13, May 1987 at pages 5-8 5 wherein a technique is disclosed for the automatic inspection of component boards using both a three-dimensional map of a circuit board under test and Gray-Scale vision data. The technique is described as usable for the detection of components such as solder paste volume, and measurements of orientation.
The prior art, although inspecting various locations of elements and whether connections and all components are properly placed on a circuit board, is limited to situations where the non-defective item always appears identical to a stored image. Absent from the prior art is a method to provide a low-cost, reliable and complete inspection system that will 15 determine the orientation of the components themselves in various stages of manufacture, such as the loading of hoppers, component placement on a board before soldering, placement of a chip on a circuit board, etc.
Furthermore, a problem with the prior art systems is that these inspection systems require the stored ideal images to match the inspected components, 20 whereas the markings on components may vary considerably, for example, due to the use of date stamps or varying printing positions and styles.
Summary of the In~ention The foregoing deficiency and problem in the prior art have been solved in accordance with the present invention which relates to a system 25 for determining the orientation of features or markings, such as letters, numerals, trademark symbols, etc., on an object, e.g., an electrical component, to determine the feature orientation, and, in turn, the orientation of the object itself. In the present system, an image of all or part of the object is used to extract lines of symbols and individual symbols.
30 The separated symbols are then each appropriately trimmed and scaled to provide normalized symbols before, for example, an Optimal Bayesian Decision method, in the form of a feed-forward neural network, determines the "right-side up", "upside-down" or "indeterminate" orientation of the text after a predetermined number of symbols are processed. Because the 35 present invention can determine the orientation of features or markings without requiring a stored ideal image, it is insensitive to changes in content, e.g, date stamps and font stylc, while reLaining ~he needed sensitivity to orientation.
In accordance with one aspect of the invention there is provided a method of determining the orientation of features or text on an object, the method S comprising the steps of (a) extracting a prcdc~ermined sized image of each of at least one of the features, or at least one symbol ol lhe text, disposed on the object; (b) normalizing each oE the at least one fea~ure or symbol obtained in step (a) within a second predetermined sized image when the al leasL one feature or symbol extracted in step (a) does not already correspond to a normalized feature or symbol in the10 second predetermined sized image; (c) processing the predetermined sized image for each of the at least one feature or symbol ob~aincd in stcp (a) or (b) for determining the similarity measure between vectors and dcLerlllining Lherefrom the probability that each of the at least one feature or symbol hus a prcdcLcrmined "up" or "down"
orientation; (d) determining from the probability ror each of the at least one feature 15 or symbol obtained in step (c) whether the fea~ure or text is in the "up" or "down" or "indeterminate" orientation; and (e) determining from the indication of the orientation of the features or text obtained in step (d) whether the object is in the "right-side up", "upside-down" or "indeterminate" orienLalion.
In accordance with anoLhcr aspccL ol the invention there is provided 20 apparatus for determining the orienta~ion ol a reature or text on an object comprising:
means for extracting a feature or symbol of the text Erom an image, and generating an output signal representative of a normalized sized bitmap for each of the extracted symbols; and a decision arrangement comprising first means responsive to the output signal from the extracting and generating means ror determining a similarity measure 25 between vectors of the normalized sizcd bi~map lor each extracted feature or symbol, and generating an output signal for each exLracLcd lcaLure or symbol indicating the probability that each of the extracted featurc,s or symbols is in the "up", "down" or "indeterminate" orientation, and seconcl means for generating an output signal indicating that the overall features or text are in the "up", "down" or "indeterminate"
30 orientation in response to the output signals of the first means Other and further aspecLs Or Lhe present invention will become apparent during the course of the following descripLioll and by reference to the accompanying drawings ~9 .
- 3a -Brief Description of the Drawin s FIG. 1 is a block diagram ol' a prcl'crrcd embodiment of an object orientation detection system in accordancc wiLh ~hc prcscnL invention; and FIG. 2 is a block diagram ol' an cxcml)lary arrangement for a decision 5 module for use in the system of FIG. 1.
Detailed DescriPtion FIG. 1 is a block diagram of a prererred arrangement for an object orientation detection system in accordance with the present invention, which system is able to read information, such as prin~cd tcxl or markings, on a component e.g., an 10 electrical device or chip, an~i determinc Llle orientaLion ol' the information. Such system can be used, for example, in elcc~ronic asscmbly lines where sometimes hundreds of components are placed on a circui~ pack. In such assembly lines, thecomponents must be, for example, first loaded into their hoppers manually, and the symmetry of the component allows an orientation error to occur, either at setup or 15 replenishment time. While many items havc an oricnlation mark such as a notch, bevel or dot, there is at present no s~andardi~a~ion ol' lhese marks and, where marks are used, they are often very hard lo dc~ccL wi~h a computer vision system. Some of the key requirements of a font orientalion dctcclion problem are (1) that there is no advance knowledge of the exact nature or conlenl ot' the features, for example, font 20 style or size; (2) the features are often poorly visible, for example, the printing is often of poor quality; (3) detection must be carried O~lt quickly while the probability of (a) detection of incorrect origination should be as high as possible, and (b) a "false reject"
be very low; and (4) there are many l'calurcs, l'or cxample, characters that areinvariant, or almost invariant, to a 18() dcgrcc rolalion and would be read correctly in 25 either orientation. Regarding this lalter seclion, il should be noted that of the set of Ietters A-Z and digits 0-9, about 40% (namely BHIMOQSWXZ01689) are approximately either rotation invariant or rotation conjugate-invariant, i.e., symbols after rotation by 180 degrees appear likc otllcr symbols in the set, depending on the font used.
c..
A.
A TECHNIQUE FOR OBJECT ORIENTATION DETECTION
USING A FEED-FORWARD NEURAL NETWORK
Technical Field The present invention relates to a technique for detecting the 5 orientation of features, such as printed text, on an object using a network including a decision arrangement in the form of a feed-forward neural network to determine the orientation of the object.
De~cription of the Prior Art As assembly processes move towards "Just In Time" operation, 10 automatic inspection becomes a more necessary technology. For example, a tight loop between an operation and its inspection can be created, thereby ensuring that when errors occur due to a manufacturing setup, no more than a minimum of defective items are produced. This contrasts with traditional batch manufacturing, where a lot or batch of a product may be 15 made before an error is detected. Rapid detection offers other advantages as well. For example, in surface mounted assembly of circuit packs, components are first placed and held with an adhesive before being soldered. Therefore, an inspection system placed in-line after the placement operation can catch errors before the soldering process is performed, thereby 20 also reducing the repair cost to a minimum.
Various arrangements have been devised for inspecting circuit boards for defects. For example, U. S. patent 4,028,827 issued to B. H.
Sharp on June 7, 1977, discloses a video system for selectively ascertaining the presence and absence of, and discriminating between, at least two 25 different types of light reflecting surface areas on articles. The system has use for inspecting circuit paths and solder connections. Another arrangement is disclosed in U.S. patent 4,578,810 issued to J. W.
MacFarlane et al. on March 25, 1986 wherein an automatic visual tester detects printed wiring board (PWB) defects. The detector comprises an 30 array of optical sensors that forms a binary image pattern of the PWB for optically inspecting the printed wire circuit.
An article by S. P. Denker et al. in Proceedin~s of International Test Conference. October 1~84 in Philadelphia, Pa. at pages 558-563 discloses an automatic visual tester that detects printed circuit board (PCB) 35 assembly errors using rnachine vision technology. In this tester, a camera isused to capture an imagP of the PCB, and transmits an electrical ~
~7r representation of this image to a computer which compares the features of the PCB image with an ideal image stored in memory to detect any assembly errors. An alternative arrangement was disclosed in the article by D. J. Svetkoff et al. in Hybrid Circuits (GB), No. 13, May 1987 at pages 5-8 5 wherein a technique is disclosed for the automatic inspection of component boards using both a three-dimensional map of a circuit board under test and Gray-Scale vision data. The technique is described as usable for the detection of components such as solder paste volume, and measurements of orientation.
The prior art, although inspecting various locations of elements and whether connections and all components are properly placed on a circuit board, is limited to situations where the non-defective item always appears identical to a stored image. Absent from the prior art is a method to provide a low-cost, reliable and complete inspection system that will 15 determine the orientation of the components themselves in various stages of manufacture, such as the loading of hoppers, component placement on a board before soldering, placement of a chip on a circuit board, etc.
Furthermore, a problem with the prior art systems is that these inspection systems require the stored ideal images to match the inspected components, 20 whereas the markings on components may vary considerably, for example, due to the use of date stamps or varying printing positions and styles.
Summary of the In~ention The foregoing deficiency and problem in the prior art have been solved in accordance with the present invention which relates to a system 25 for determining the orientation of features or markings, such as letters, numerals, trademark symbols, etc., on an object, e.g., an electrical component, to determine the feature orientation, and, in turn, the orientation of the object itself. In the present system, an image of all or part of the object is used to extract lines of symbols and individual symbols.
30 The separated symbols are then each appropriately trimmed and scaled to provide normalized symbols before, for example, an Optimal Bayesian Decision method, in the form of a feed-forward neural network, determines the "right-side up", "upside-down" or "indeterminate" orientation of the text after a predetermined number of symbols are processed. Because the 35 present invention can determine the orientation of features or markings without requiring a stored ideal image, it is insensitive to changes in content, e.g, date stamps and font stylc, while reLaining ~he needed sensitivity to orientation.
In accordance with one aspect of the invention there is provided a method of determining the orientation of features or text on an object, the method S comprising the steps of (a) extracting a prcdc~ermined sized image of each of at least one of the features, or at least one symbol ol lhe text, disposed on the object; (b) normalizing each oE the at least one fea~ure or symbol obtained in step (a) within a second predetermined sized image when the al leasL one feature or symbol extracted in step (a) does not already correspond to a normalized feature or symbol in the10 second predetermined sized image; (c) processing the predetermined sized image for each of the at least one feature or symbol ob~aincd in stcp (a) or (b) for determining the similarity measure between vectors and dcLerlllining Lherefrom the probability that each of the at least one feature or symbol hus a prcdcLcrmined "up" or "down"
orientation; (d) determining from the probability ror each of the at least one feature 15 or symbol obtained in step (c) whether the fea~ure or text is in the "up" or "down" or "indeterminate" orientation; and (e) determining from the indication of the orientation of the features or text obtained in step (d) whether the object is in the "right-side up", "upside-down" or "indeterminate" orienLalion.
In accordance with anoLhcr aspccL ol the invention there is provided 20 apparatus for determining the orienta~ion ol a reature or text on an object comprising:
means for extracting a feature or symbol of the text Erom an image, and generating an output signal representative of a normalized sized bitmap for each of the extracted symbols; and a decision arrangement comprising first means responsive to the output signal from the extracting and generating means ror determining a similarity measure 25 between vectors of the normalized sizcd bi~map lor each extracted feature or symbol, and generating an output signal for each exLracLcd lcaLure or symbol indicating the probability that each of the extracted featurc,s or symbols is in the "up", "down" or "indeterminate" orientation, and seconcl means for generating an output signal indicating that the overall features or text are in the "up", "down" or "indeterminate"
30 orientation in response to the output signals of the first means Other and further aspecLs Or Lhe present invention will become apparent during the course of the following descripLioll and by reference to the accompanying drawings ~9 .
- 3a -Brief Description of the Drawin s FIG. 1 is a block diagram ol' a prcl'crrcd embodiment of an object orientation detection system in accordancc wiLh ~hc prcscnL invention; and FIG. 2 is a block diagram ol' an cxcml)lary arrangement for a decision 5 module for use in the system of FIG. 1.
Detailed DescriPtion FIG. 1 is a block diagram of a prererred arrangement for an object orientation detection system in accordance with the present invention, which system is able to read information, such as prin~cd tcxl or markings, on a component e.g., an 10 electrical device or chip, an~i determinc Llle orientaLion ol' the information. Such system can be used, for example, in elcc~ronic asscmbly lines where sometimes hundreds of components are placed on a circui~ pack. In such assembly lines, thecomponents must be, for example, first loaded into their hoppers manually, and the symmetry of the component allows an orientation error to occur, either at setup or 15 replenishment time. While many items havc an oricnlation mark such as a notch, bevel or dot, there is at present no s~andardi~a~ion ol' lhese marks and, where marks are used, they are often very hard lo dc~ccL wi~h a computer vision system. Some of the key requirements of a font orientalion dctcclion problem are (1) that there is no advance knowledge of the exact nature or conlenl ot' the features, for example, font 20 style or size; (2) the features are often poorly visible, for example, the printing is often of poor quality; (3) detection must be carried O~lt quickly while the probability of (a) detection of incorrect origination should be as high as possible, and (b) a "false reject"
be very low; and (4) there are many l'calurcs, l'or cxample, characters that areinvariant, or almost invariant, to a 18() dcgrcc rolalion and would be read correctly in 25 either orientation. Regarding this lalter seclion, il should be noted that of the set of Ietters A-Z and digits 0-9, about 40% (namely BHIMOQSWXZ01689) are approximately either rotation invariant or rotation conjugate-invariant, i.e., symbols after rotation by 180 degrees appear likc otllcr symbols in the set, depending on the font used.
c..
A.
- 2 ~ 1 3 ~ 7 ~3 For purposes of illustration, it is assumed hereinafter that the present system will be used to determine the orientation of a chip, but it should be understood that the present system could be adapted for use in determining the orientation of any object having single or multiple features thereon, 5 such as, for example, boxes on a conveyor belt, labels on bottles, etc.
Furthermore, the system is equally applicable for use in determining the orientation of features or markings other than text. The system relies only on having as an input a set of reference images which could be arbitrary.
In the orientation detection system of FIG. 1, a video imaging 10 device 10 is provided including, for example, (1) a video camera, (2) a framegrabber to capture the picture signals associated with a frame of the video picture; and (3) analog-to-digital (A/D) hardware to provide, for example, 512-by-512 pixel frames with a real object scale of approximately 0.002 inches per pixel, and 256 gray levels per pixel. A reference design database 15 11 is used to store and provide an accurate footprint or position and size for each device being viewed, such as a chip relative to a board onto which it is assembled, and the feature orientation for a correctly placed or inserted device. The resultant images of the exemplary chips from the video image device 10 are provided as one input to an Adaptive Threshold Module 20 (ATM) 12.
In ATM 12, the footprint of the chip from design database 11 is used to direct attention to the particular component or area of interest on the image of the chip from video imaging device 10, and such resultant component or area image may now only include, for example, 200-by-130 25 pixels instead of the 256-by-256 pixel image provided by video imaging device 10. ATM 12 transforms the resultant component or area image into a binary image where each pixel includes only a 0 or 1 level instead of one of the exemplary 256 gray levels. Although it is possible to omit this binarization step and process the image entirely with gray scale data, the 30 ATM binarization step is present in the preferred embodiment, because it provides a much faster and lower cost implementation. In general, automatic evaluation of a suitable threshold between the original 256 gray levels and the two binary levels is a nontrivial problem, but in the present system a simple histogram method will work well because in most cases the 35 text appears as one color on a contrasting background, and the gray level histogram has two fairly easily resolved pcaks. These peaks are resolved 7 ~
adaptively using iteration of two parameters, namely the radial circumspection and mass threshold, until a satisfactory solution is obtained.
The radial circumspection is the radius of a neighborhood in the histogram which is examined to verify that a value is a local maximum. To con~titute S a peak, a gray level must be both a local maximum and have a mass exceeding the mass threshold. This technique also allows correction so that the foreground is represented as a "1" and the background as a "0", regardless of whether the printing is light-on-dark or dark-on-light, etc.
The binary image produced by ATM 12 is provided as an input 10 to a Framer Module 13 where the symbols (letters, numbers, etc.) of the image are captured. More particularly, such symbol capture can be performed by, for example, first taking horizontal sums of the bitmap matrix of O's and 1's of the binary image to extract lines of symbols, and then taking vertical sums within lines to extract the individual symbols.
15 For such purpose, preset quantile thresholds can be used. More particularly, for each scanline i of the binary image, the row sum r[i] is computed of the "1" bits. Thus rli] contains peaks for rows of text, and valleys, or gaps, between rows of text. These peaks and gaps can be obscured by noise and variation of the text. Therefore, a threshold d can be 20 used to separate the peaks from the gaps where, for example, d=~C rmax~ ~ is a predetermined constant as, for example 0.07, and rmaX is the largest entry in the row sum vector r[]. The process essentially starts at the top scanline and proceeds through the subsequent scanlines looking for the beginning of a peak above a certain threshold and then continues over a peak area 25 looking for a valley below a certain threshold, etc. to separate lines of symbols. Lines or gaps which appear too small are rejected as being attributed to noise. The process is then repeated using the column sums of the binary image to extract individual symbols. It should be noted that the technique of horizontal and vertical histograms can also disclose, and allow 30 immediate correction for, a 90 degree rotation. To filter out isolated dots which may flnd their way through the process of ATM 12, the Framer Module 13 can be implemented to also ignore areas with a "connected dot mass" less than a predetermined constant. While such process may bring a risk of losing part of a broken valid symbol, no untoward effects on a 35 recognizer procedure is found to occur. It is to be understood thal the Framer Module 13 can also be omitted when the symbols are a!ways in ~-` 2 ~ ~ G ~ ~ ~7 ,!3 predetermined positions, but is included in the preferred embodiment because this module allows conventionally marked electronic components, on which the printing varies considerably in position, to be suitably inspected. The Framer Module 13, therefore, provides a displacement-5 invariance capability to the overall system.
A Normalization Module (NM) 14 accepts the output fromFramer Module 13 and "trims and scales" each extracted symbol. Trimming is performed because the extracted symbols can have accompanying white, or almost white, spaces at the sides, top and/or bottom. Scaling is 10 performed to scale up the symbol image to occupy a predetermined standard size of, for example, 24 rows by 16 columns. In certain cases this normalization process produces distortion to the original image as, for example, thin characters such as the letter "I" will be "fattened" to occupy the 16 columns. However, this is not undesirable as both the input data 15 and the reference vectors described below are transformed in the same manner. The alternative of not fattening an exemplary "I" has the potential disadvantage of permitting a mismatch due to a vertical misalignment of a reference and sample image. It is to be understood that the Normalization module 14 can be omitted when symbols are always of a predetermined size, 20 but is provided in the preferred embodiment, since NM 14 provides a scale-invariance capability to the system.
The signals representing the normalized symbols are then presented to a decision module 15 where a determination is made, using the design database 11 information, as to whether the symbols are disposed in 25 the "up" or "down" orientation, or is "indeterminate" as to its orientation.
As will be described hereinafter, a preferred arrangement for decision module 15 using a preferred Optimal Detection (OD) method, which, in its particular form, can also be termed a "Feedforward" (FF) neural network method, computes the likelihood of the symbol being oriented "up" or 30 "down", or being "indeterminate", by computing a "similarity measure", as for example the Hamming distance, between bitmaps of the captured symbol and reference images, using a lookup table. A symbol, such as a letter, will be referred to in its normal orientation as "up" and in the inverted orientation as "down". An object, such as a chip, may have its 35 text oriented correctly in either thè "up" or "down" orientation. A correctlyoriented chip will be referred to herein as "right-side up" and an incorrectly oriented chip as "upside down".
More particularly, observed images are bitmaps of length N, i.e., vectors in Q = {o,1}N. It is assumed that there is a collection of reference images or symbols (i.e., letters, numerals, trademark logos, etc. in various 5 fonts sizes, etc.) which appear in the "up" orientation as ul, u2,..., uS~ andin the "down" orientation as d1, d2,...,d5. It is also assumed that there is a distortion process which represents both noise inherent in the images, noise due to the image capture process, and variation due to the use of font styles, sizes, etc., which are not in the reference set. This distortion process10 is represented hereinafter by p(xly), meaning that reference vector y is distorted into observed vector x with a probability p(xly). Also defined is that p(xlu) or p(xld) is the probability that the vector x is observed, given that vector x is a distortion of a randomly chosen symbols from a reference set of ul, u2,...,uS or d1, d2,...,d5, respectively. Then the probability 15 p(xlu) = ~ p(x~j)p(uj) and p(xld) = ~ p(xldj)p(dj) where p(u;) or p(di) is i--1 i-- 1 the a Priori probability that symbol uj or dj, respectively, is used, given thatthe symbol has orientation "up" or "down". The symbols of the above reference set may, for example, comprise the letters A-Z, and numbers 0-9 in various sizes, font styles, and orientations. It should be noted that 20 although a scaling operation is carried out in Normalization Module 14, the presence of multiple sizes in the reference set is desirable because fonts are not typically scale-invariant.
One formulation of the problem is then to flnd a partition of 52 into regions Qu, Qd ~ Qi representing decisions "up", "down" or 25 "indeterminate". This problem is formulated as maximizing the probability of correct determination of orientation subject to a limit on the probability of a "false reject". This results in the following practical technique for optimal determination of the orientation of a single symbol.
The technique outputs:
d ("down") if p(Xld)/p(xlu) >
u ("up") if p(xld)/p(xlu) <
i ("indeterminate") otherwise.
1 33~
The parameter ~>1 is then adjusted from analysis or experiment to be as small as possible, but not so small that there are an excessive number of false rejects.
Determination of orientation occurs for a chip consisting of 5 multiple symbols: x(l), x(2), ..., x(L). Thus a conditional independence assumption:
P{x(1)~ x(2) ,---, x(L)I u} = P {x(l)l u}. P{x(2)1 u}.... P{x(L)I u}, P{x(1), x(2), ..., x(L)I d} = P{x(1)l d}. P{x(2)1 d}- ... P{x(L)I d}
is adopted.
10 Then the optimal test becomes to output:
d ("down") if Il p(x(i) I d)/ p(x(i)l u) > ~ (L)~
u ("up") if 1~ p(x(i) Id)/p(x(i)l u) < )~(L)-i ("indeterminate") otherwise.
If the time allocated to inspecting a board is sufficient, the best 15 possible results are obtained by examining every symbol found on the object. However, it is often possible to obtain very high certainty about the orientation of the object before all of the symbols are read. This suggests the use of a Sequential Testing procedure as disclosed in the book "Sequential Analysis" by A. Wald, Dover Publications, 1947, at pages 34-43, 20 wherein symbols are read until the cumulative product of likelihood ratios exceeds some upper bound ~ or falls below a lower bound ~ , at which time a determination of orientation is made. This may offer the potential of a significant speedup of the process, as measured by mean time per chip.
Nevertheless, in cases where bias or poor knowledge of the prior 25 distributions exists, it may still be desirable to use a slowly growing ~\
function, or to limit the contribution to the product of any one observation.
An exemplary arrangement for implementing the feed-forward neural network method in order for decision module 15 to carry out the above described process is shown in FIG. 2. In FIG. 2, the inputs from 3~
Normalization Module 14 comprises separate elements of the bitmap (matrix) for a normalized symbol and designated xl to XN. Each of the bitmap elements x; are provided as an input to a separate Input Unit (IU) 20l to 20N. More particularly, if each normalized symbol is arranged to be 5 disposed within an exemplary bitmap matrix of 24-by-16 elements, then N
would equal 384 elements and decision module 15 would include 384 Input Units 20. The output from each Input Unit 20j is distributed to each of M
Pattern Units 21l to 21M, where, for example, a first half of the M pattern units 21 is used to determine the likelihoods 10 p(xlu) = ~ p(xluj) p(u;) and p(xld) = ~ p(xldj)p(dj). The values p(uj) j_l i_1 and p(dj) are a priori probabilities which can be obtained, for example, from field studies, or by using uniform probabilities throughout, i.e., taking every symbol equally likely in either orientation. To obtain the values p(xluj) and p(xldj), some assumptions must be made about the distortion process. A
15 simple exemplary model is adopted (1) that with probability g(k), k bits of a bitmap are altered, and (2) that those bits are altered according to independent and identical distributed trials. Then p(xluj) = g(h(x,uj))/ [h(~;N ~¦ where h(x, u; )is the Hamming distance between vectors x and u;. A neural network view of this procedure is shown 20 in FIG. 2. The first active layer, consisting of pattern units, 21, computes p(x l u;) and p( x l dj), in accordance with the above equation. The next layer, consisting of summation units 22l and 222, forms the likelihoods p(xlu) and p(xld), and Sequential Decision Unit 23 gives the final decision for a chip. Parts of FIG. 2, and the terminology used, are similar to that 25 used in the article by D. F. Specht in The Proceedin~s of the IEEE
International Conference On Neural Networks, July, 1988, San Diego, Calif., Vol. 1, at pages I-525 to I-532; however, the internal structure of the pattern unit 21 is quite different. FIG. 2 immediately suggests a parallel implementation. It should be understood that the algebraic quantities 30 shown on the interconnections in FIG. 2 are values being passed between the elements and that quantities enclosed in parenthesis are multiplicative weights.
3 r l ~ 3 The internal structure of the pattern units can be serial or parallel. The remainder of this paragraph describes a method for efflciently implementing the pattern units 21 when the "similarity measure" is a Hamming distance. In the absence of special purpose hardware for 5 computing Hamming distances, there is a technique that can be employed to compute them very quickly. The Hamming distances between bitmaps can be computed a word at a time by taking the bitwise "exclusive or," and then using a precomputed table lookup which returns the number of one bits in the resulting word. This reduces the Hamming distance calculation 10 to several machine instructions per word, and one can calculate distances in the order of a million words per second on a contemporary microprocessor.
After the operation of elements 20 through 22 of FIG. 2 are completed, an output probability value is obtained for each symbol. These are combined in the FF method using multiplication in the Sequential 15 Decision Unit 23. The results can then be compared against an upper and lower threshold in Sequential Decision Unit 23 to generate an overall decision for the chip, or an indecision. This procedure can proceed according to a "stopping" rule whereby the processing of further symbols on the chip is discontinued when a prescribed degree of certainty has been 20 obtained. Finally, the result for the chip is compared in Comparator 24 against the correct orientation for the chip, which orientation is stored in "the design database", and depending upon the setting, e.g., "stop on errors", or "stop on errors plus indecisions", an output signal is provided to, for example, operate an alarm or not.
It is to be understood that the above description of the Feed-Forward Neural Network method was for purposes of explanation, and not for purposes of limitation since any other suitable method could be used to provide the appropriate decision. For example, it is possible to apply the Learning Vector Quantization (LVQ) method similar to that described by 30 T. Kohonen in the book "Self-Organization and Associative Memory", Second Edition, Springer-Verlag at pages 19~-209. For the LVQ method, the result obtained in Decision Module 15 would merely be a "vote", i.e., the number of "up" scores divided by the total number of observations, and not an explicit probability vælue as found in the FF method.
,.,. , .. , , ~ , . , . ~
Furthermore, the system is equally applicable for use in determining the orientation of features or markings other than text. The system relies only on having as an input a set of reference images which could be arbitrary.
In the orientation detection system of FIG. 1, a video imaging 10 device 10 is provided including, for example, (1) a video camera, (2) a framegrabber to capture the picture signals associated with a frame of the video picture; and (3) analog-to-digital (A/D) hardware to provide, for example, 512-by-512 pixel frames with a real object scale of approximately 0.002 inches per pixel, and 256 gray levels per pixel. A reference design database 15 11 is used to store and provide an accurate footprint or position and size for each device being viewed, such as a chip relative to a board onto which it is assembled, and the feature orientation for a correctly placed or inserted device. The resultant images of the exemplary chips from the video image device 10 are provided as one input to an Adaptive Threshold Module 20 (ATM) 12.
In ATM 12, the footprint of the chip from design database 11 is used to direct attention to the particular component or area of interest on the image of the chip from video imaging device 10, and such resultant component or area image may now only include, for example, 200-by-130 25 pixels instead of the 256-by-256 pixel image provided by video imaging device 10. ATM 12 transforms the resultant component or area image into a binary image where each pixel includes only a 0 or 1 level instead of one of the exemplary 256 gray levels. Although it is possible to omit this binarization step and process the image entirely with gray scale data, the 30 ATM binarization step is present in the preferred embodiment, because it provides a much faster and lower cost implementation. In general, automatic evaluation of a suitable threshold between the original 256 gray levels and the two binary levels is a nontrivial problem, but in the present system a simple histogram method will work well because in most cases the 35 text appears as one color on a contrasting background, and the gray level histogram has two fairly easily resolved pcaks. These peaks are resolved 7 ~
adaptively using iteration of two parameters, namely the radial circumspection and mass threshold, until a satisfactory solution is obtained.
The radial circumspection is the radius of a neighborhood in the histogram which is examined to verify that a value is a local maximum. To con~titute S a peak, a gray level must be both a local maximum and have a mass exceeding the mass threshold. This technique also allows correction so that the foreground is represented as a "1" and the background as a "0", regardless of whether the printing is light-on-dark or dark-on-light, etc.
The binary image produced by ATM 12 is provided as an input 10 to a Framer Module 13 where the symbols (letters, numbers, etc.) of the image are captured. More particularly, such symbol capture can be performed by, for example, first taking horizontal sums of the bitmap matrix of O's and 1's of the binary image to extract lines of symbols, and then taking vertical sums within lines to extract the individual symbols.
15 For such purpose, preset quantile thresholds can be used. More particularly, for each scanline i of the binary image, the row sum r[i] is computed of the "1" bits. Thus rli] contains peaks for rows of text, and valleys, or gaps, between rows of text. These peaks and gaps can be obscured by noise and variation of the text. Therefore, a threshold d can be 20 used to separate the peaks from the gaps where, for example, d=~C rmax~ ~ is a predetermined constant as, for example 0.07, and rmaX is the largest entry in the row sum vector r[]. The process essentially starts at the top scanline and proceeds through the subsequent scanlines looking for the beginning of a peak above a certain threshold and then continues over a peak area 25 looking for a valley below a certain threshold, etc. to separate lines of symbols. Lines or gaps which appear too small are rejected as being attributed to noise. The process is then repeated using the column sums of the binary image to extract individual symbols. It should be noted that the technique of horizontal and vertical histograms can also disclose, and allow 30 immediate correction for, a 90 degree rotation. To filter out isolated dots which may flnd their way through the process of ATM 12, the Framer Module 13 can be implemented to also ignore areas with a "connected dot mass" less than a predetermined constant. While such process may bring a risk of losing part of a broken valid symbol, no untoward effects on a 35 recognizer procedure is found to occur. It is to be understood thal the Framer Module 13 can also be omitted when the symbols are a!ways in ~-` 2 ~ ~ G ~ ~ ~7 ,!3 predetermined positions, but is included in the preferred embodiment because this module allows conventionally marked electronic components, on which the printing varies considerably in position, to be suitably inspected. The Framer Module 13, therefore, provides a displacement-5 invariance capability to the overall system.
A Normalization Module (NM) 14 accepts the output fromFramer Module 13 and "trims and scales" each extracted symbol. Trimming is performed because the extracted symbols can have accompanying white, or almost white, spaces at the sides, top and/or bottom. Scaling is 10 performed to scale up the symbol image to occupy a predetermined standard size of, for example, 24 rows by 16 columns. In certain cases this normalization process produces distortion to the original image as, for example, thin characters such as the letter "I" will be "fattened" to occupy the 16 columns. However, this is not undesirable as both the input data 15 and the reference vectors described below are transformed in the same manner. The alternative of not fattening an exemplary "I" has the potential disadvantage of permitting a mismatch due to a vertical misalignment of a reference and sample image. It is to be understood that the Normalization module 14 can be omitted when symbols are always of a predetermined size, 20 but is provided in the preferred embodiment, since NM 14 provides a scale-invariance capability to the system.
The signals representing the normalized symbols are then presented to a decision module 15 where a determination is made, using the design database 11 information, as to whether the symbols are disposed in 25 the "up" or "down" orientation, or is "indeterminate" as to its orientation.
As will be described hereinafter, a preferred arrangement for decision module 15 using a preferred Optimal Detection (OD) method, which, in its particular form, can also be termed a "Feedforward" (FF) neural network method, computes the likelihood of the symbol being oriented "up" or 30 "down", or being "indeterminate", by computing a "similarity measure", as for example the Hamming distance, between bitmaps of the captured symbol and reference images, using a lookup table. A symbol, such as a letter, will be referred to in its normal orientation as "up" and in the inverted orientation as "down". An object, such as a chip, may have its 35 text oriented correctly in either thè "up" or "down" orientation. A correctlyoriented chip will be referred to herein as "right-side up" and an incorrectly oriented chip as "upside down".
More particularly, observed images are bitmaps of length N, i.e., vectors in Q = {o,1}N. It is assumed that there is a collection of reference images or symbols (i.e., letters, numerals, trademark logos, etc. in various 5 fonts sizes, etc.) which appear in the "up" orientation as ul, u2,..., uS~ andin the "down" orientation as d1, d2,...,d5. It is also assumed that there is a distortion process which represents both noise inherent in the images, noise due to the image capture process, and variation due to the use of font styles, sizes, etc., which are not in the reference set. This distortion process10 is represented hereinafter by p(xly), meaning that reference vector y is distorted into observed vector x with a probability p(xly). Also defined is that p(xlu) or p(xld) is the probability that the vector x is observed, given that vector x is a distortion of a randomly chosen symbols from a reference set of ul, u2,...,uS or d1, d2,...,d5, respectively. Then the probability 15 p(xlu) = ~ p(x~j)p(uj) and p(xld) = ~ p(xldj)p(dj) where p(u;) or p(di) is i--1 i-- 1 the a Priori probability that symbol uj or dj, respectively, is used, given thatthe symbol has orientation "up" or "down". The symbols of the above reference set may, for example, comprise the letters A-Z, and numbers 0-9 in various sizes, font styles, and orientations. It should be noted that 20 although a scaling operation is carried out in Normalization Module 14, the presence of multiple sizes in the reference set is desirable because fonts are not typically scale-invariant.
One formulation of the problem is then to flnd a partition of 52 into regions Qu, Qd ~ Qi representing decisions "up", "down" or 25 "indeterminate". This problem is formulated as maximizing the probability of correct determination of orientation subject to a limit on the probability of a "false reject". This results in the following practical technique for optimal determination of the orientation of a single symbol.
The technique outputs:
d ("down") if p(Xld)/p(xlu) >
u ("up") if p(xld)/p(xlu) <
i ("indeterminate") otherwise.
1 33~
The parameter ~>1 is then adjusted from analysis or experiment to be as small as possible, but not so small that there are an excessive number of false rejects.
Determination of orientation occurs for a chip consisting of 5 multiple symbols: x(l), x(2), ..., x(L). Thus a conditional independence assumption:
P{x(1)~ x(2) ,---, x(L)I u} = P {x(l)l u}. P{x(2)1 u}.... P{x(L)I u}, P{x(1), x(2), ..., x(L)I d} = P{x(1)l d}. P{x(2)1 d}- ... P{x(L)I d}
is adopted.
10 Then the optimal test becomes to output:
d ("down") if Il p(x(i) I d)/ p(x(i)l u) > ~ (L)~
u ("up") if 1~ p(x(i) Id)/p(x(i)l u) < )~(L)-i ("indeterminate") otherwise.
If the time allocated to inspecting a board is sufficient, the best 15 possible results are obtained by examining every symbol found on the object. However, it is often possible to obtain very high certainty about the orientation of the object before all of the symbols are read. This suggests the use of a Sequential Testing procedure as disclosed in the book "Sequential Analysis" by A. Wald, Dover Publications, 1947, at pages 34-43, 20 wherein symbols are read until the cumulative product of likelihood ratios exceeds some upper bound ~ or falls below a lower bound ~ , at which time a determination of orientation is made. This may offer the potential of a significant speedup of the process, as measured by mean time per chip.
Nevertheless, in cases where bias or poor knowledge of the prior 25 distributions exists, it may still be desirable to use a slowly growing ~\
function, or to limit the contribution to the product of any one observation.
An exemplary arrangement for implementing the feed-forward neural network method in order for decision module 15 to carry out the above described process is shown in FIG. 2. In FIG. 2, the inputs from 3~
Normalization Module 14 comprises separate elements of the bitmap (matrix) for a normalized symbol and designated xl to XN. Each of the bitmap elements x; are provided as an input to a separate Input Unit (IU) 20l to 20N. More particularly, if each normalized symbol is arranged to be 5 disposed within an exemplary bitmap matrix of 24-by-16 elements, then N
would equal 384 elements and decision module 15 would include 384 Input Units 20. The output from each Input Unit 20j is distributed to each of M
Pattern Units 21l to 21M, where, for example, a first half of the M pattern units 21 is used to determine the likelihoods 10 p(xlu) = ~ p(xluj) p(u;) and p(xld) = ~ p(xldj)p(dj). The values p(uj) j_l i_1 and p(dj) are a priori probabilities which can be obtained, for example, from field studies, or by using uniform probabilities throughout, i.e., taking every symbol equally likely in either orientation. To obtain the values p(xluj) and p(xldj), some assumptions must be made about the distortion process. A
15 simple exemplary model is adopted (1) that with probability g(k), k bits of a bitmap are altered, and (2) that those bits are altered according to independent and identical distributed trials. Then p(xluj) = g(h(x,uj))/ [h(~;N ~¦ where h(x, u; )is the Hamming distance between vectors x and u;. A neural network view of this procedure is shown 20 in FIG. 2. The first active layer, consisting of pattern units, 21, computes p(x l u;) and p( x l dj), in accordance with the above equation. The next layer, consisting of summation units 22l and 222, forms the likelihoods p(xlu) and p(xld), and Sequential Decision Unit 23 gives the final decision for a chip. Parts of FIG. 2, and the terminology used, are similar to that 25 used in the article by D. F. Specht in The Proceedin~s of the IEEE
International Conference On Neural Networks, July, 1988, San Diego, Calif., Vol. 1, at pages I-525 to I-532; however, the internal structure of the pattern unit 21 is quite different. FIG. 2 immediately suggests a parallel implementation. It should be understood that the algebraic quantities 30 shown on the interconnections in FIG. 2 are values being passed between the elements and that quantities enclosed in parenthesis are multiplicative weights.
3 r l ~ 3 The internal structure of the pattern units can be serial or parallel. The remainder of this paragraph describes a method for efflciently implementing the pattern units 21 when the "similarity measure" is a Hamming distance. In the absence of special purpose hardware for 5 computing Hamming distances, there is a technique that can be employed to compute them very quickly. The Hamming distances between bitmaps can be computed a word at a time by taking the bitwise "exclusive or," and then using a precomputed table lookup which returns the number of one bits in the resulting word. This reduces the Hamming distance calculation 10 to several machine instructions per word, and one can calculate distances in the order of a million words per second on a contemporary microprocessor.
After the operation of elements 20 through 22 of FIG. 2 are completed, an output probability value is obtained for each symbol. These are combined in the FF method using multiplication in the Sequential 15 Decision Unit 23. The results can then be compared against an upper and lower threshold in Sequential Decision Unit 23 to generate an overall decision for the chip, or an indecision. This procedure can proceed according to a "stopping" rule whereby the processing of further symbols on the chip is discontinued when a prescribed degree of certainty has been 20 obtained. Finally, the result for the chip is compared in Comparator 24 against the correct orientation for the chip, which orientation is stored in "the design database", and depending upon the setting, e.g., "stop on errors", or "stop on errors plus indecisions", an output signal is provided to, for example, operate an alarm or not.
It is to be understood that the above description of the Feed-Forward Neural Network method was for purposes of explanation, and not for purposes of limitation since any other suitable method could be used to provide the appropriate decision. For example, it is possible to apply the Learning Vector Quantization (LVQ) method similar to that described by 30 T. Kohonen in the book "Self-Organization and Associative Memory", Second Edition, Springer-Verlag at pages 19~-209. For the LVQ method, the result obtained in Decision Module 15 would merely be a "vote", i.e., the number of "up" scores divided by the total number of observations, and not an explicit probability vælue as found in the FF method.
,.,. , .. , , ~ , . , . ~
Claims (10)
1. A method of determining the orientation of features or text on an object, the method comprising the steps of:
(a) extracting a predetermined sized image of each of at least one of the features, or at least one symbol of the text, disposed on the object;
(b) normalizing each of the at least one feature or symbol obtained in step (a) within a second predetermined sized image when the at least one feature or symbol extracted in step (a) does not already correspond to a normalized feature or symbol in the second predetermined sized image;
(c) processing the predetermined sized image for each of the at least one feature or symbol obtained in step (a) or (b) for determining the similarity measure between vectors and determining therefrom the probability that each of the at least one feature or symbol has a predetermined "up or "down" orientation;
(d) determining from the probability for each of the at least one feature or symbol obtained in step (c) whether the feature or text is in the "up", "down" or "indeterminate" orientation; and (e) determining from the indication of the orientation of the features or text obtained in step (d) whether the object is in the "right-side up", "upside-down" or "indeterminate" orientation.
(a) extracting a predetermined sized image of each of at least one of the features, or at least one symbol of the text, disposed on the object;
(b) normalizing each of the at least one feature or symbol obtained in step (a) within a second predetermined sized image when the at least one feature or symbol extracted in step (a) does not already correspond to a normalized feature or symbol in the second predetermined sized image;
(c) processing the predetermined sized image for each of the at least one feature or symbol obtained in step (a) or (b) for determining the similarity measure between vectors and determining therefrom the probability that each of the at least one feature or symbol has a predetermined "up or "down" orientation;
(d) determining from the probability for each of the at least one feature or symbol obtained in step (c) whether the feature or text is in the "up", "down" or "indeterminate" orientation; and (e) determining from the indication of the orientation of the features or text obtained in step (d) whether the object is in the "right-side up", "upside-down" or "indeterminate" orientation.
2. The method according to claim 1 wherein in performing step (a), performing the substeps of:
(a1) summing the binary values for each of the rows of a bitmap of the features or text and applying a threshold value to the sums of the rows for determining the location of lines of features or text; and (a2) summing the binary values for each of the columns of the bitmap of the features or text and applying a second threshold value to the sums of the columns for determining, in conjunction with the result of step (a1), the location of the individual features or symbols of the text.
(a1) summing the binary values for each of the rows of a bitmap of the features or text and applying a threshold value to the sums of the rows for determining the location of lines of features or text; and (a2) summing the binary values for each of the columns of the bitmap of the features or text and applying a second threshold value to the sums of the columns for determining, in conjunction with the result of step (a1), the location of the individual features or symbols of the text.
3. The method according to claim 1 or 2 wherein in performing step (c), performing the substeps of:
(c1) determining, from a bitmap of each normalized extracted feature or symbol of the text obtained in step (a) or (b), the probability that an input vector is observed, by measuring the Hamming distance using bitwise "exclusive OR" and table lookup operations between the input vector and each of reference set of features or symbols in a particular "up"
or "down" orientation, where the input vector comprises a distortion of a randomly chosen feature or symbol from the reference set; and (c2) in response to step (c1), determining the probability that the extracted symbol has the orientation "up", "down" or "indeterminate".
(c1) determining, from a bitmap of each normalized extracted feature or symbol of the text obtained in step (a) or (b), the probability that an input vector is observed, by measuring the Hamming distance using bitwise "exclusive OR" and table lookup operations between the input vector and each of reference set of features or symbols in a particular "up"
or "down" orientation, where the input vector comprises a distortion of a randomly chosen feature or symbol from the reference set; and (c2) in response to step (c1), determining the probability that the extracted symbol has the orientation "up", "down" or "indeterminate".
4. The method according to claim 1 or 2 wherein in performing step (d):
multiplying the probability determined for each of the sequential symbols, processed in step (c) with the product of the probabilities of all other prior processed symbols until a certain threshold is attained indicating that the feature or text is in the "up", "down" or "indeterminate"
orientation.
multiplying the probability determined for each of the sequential symbols, processed in step (c) with the product of the probabilities of all other prior processed symbols until a certain threshold is attained indicating that the feature or text is in the "up", "down" or "indeterminate"
orientation.
5. The method according to claim 1 or 2 wherein in performing step (d), performing the substeps of:
(d1) providing a determination of whether each symbol of the text is in the "up", "down" or "indeterminate" orientation; and (d2) from a consensus of the determinations made in step (d1), generating a determination of whether the text is in the "up", "down" or "indeterminate" orientation.
(d1) providing a determination of whether each symbol of the text is in the "up", "down" or "indeterminate" orientation; and (d2) from a consensus of the determinations made in step (d1), generating a determination of whether the text is in the "up", "down" or "indeterminate" orientation.
6. Apparatus for determining the orientation of a feature or text on an object comprising:
means for extracting a feature or symbol of the text from an image, and generating an output signal representative of a normalized sized bitmap for each of the extracted symbols; and a decision arrangement comprising first means responsive to the output signal from the extracting and generating means for determining a similarity measure between vectors of the normalized sized bitmap for each extracted feature or symbol, and generating an output signal for each extracted feature or symbol indicating the probability that each of the extracted features or symbols is in the "up", "down" or "indeterminate" orientation, and second means for generating an output signal indicating that the overall features or text are in the "up", "down" or "indeterminate"
orientation in response to the output signals of the first means.
means for extracting a feature or symbol of the text from an image, and generating an output signal representative of a normalized sized bitmap for each of the extracted symbols; and a decision arrangement comprising first means responsive to the output signal from the extracting and generating means for determining a similarity measure between vectors of the normalized sized bitmap for each extracted feature or symbol, and generating an output signal for each extracted feature or symbol indicating the probability that each of the extracted features or symbols is in the "up", "down" or "indeterminate" orientation, and second means for generating an output signal indicating that the overall features or text are in the "up", "down" or "indeterminate"
orientation in response to the output signals of the first means.
7. Apparatus according to claim 6 wherein the decision arrangement further comprises:
third means for determining the orientation of the object from the output signal of the second means.
third means for determining the orientation of the object from the output signal of the second means.
8. Apparatus according to claim 6 or 7 wherein the extracting and generating means comprises:
first means for generating a sum of the binary values for each of the rows of a bitmap of the image of the features or text and applying a threshold value to the sums of the rows for generating an output signal indicating the location of lines within the features or text;
second means for generating a sum of each of the columns of a bitmap of the image of the features or text and applying a second threshold value to the sums of the columns for determining, from both the output signal from the first means and the sums of the columns, the location of individual features or of the symbols within the text.
first means for generating a sum of the binary values for each of the rows of a bitmap of the image of the features or text and applying a threshold value to the sums of the rows for generating an output signal indicating the location of lines within the features or text;
second means for generating a sum of each of the columns of a bitmap of the image of the features or text and applying a second threshold value to the sums of the columns for determining, from both the output signal from the first means and the sums of the columns, the location of individual features or of the symbols within the text.
9. Apparatus according to claim 6 or 7 wherein the second means of the decision apparatus comprises:
means responsive to the output signals from the first means of the decision apparatus for multiplying the probability determined for each of the sequential signals with the product of the probabilities of all other prior processed features or symbols of the text until a certain threshold is attained indicating the features or text are in the "up", "down" or "indeterminate" orientation and generating an output signal representative of such orientation determination.
means responsive to the output signals from the first means of the decision apparatus for multiplying the probability determined for each of the sequential signals with the product of the probabilities of all other prior processed features or symbols of the text until a certain threshold is attained indicating the features or text are in the "up", "down" or "indeterminate" orientation and generating an output signal representative of such orientation determination.
10. Apparatus according to claim 6 or 7 wherein the second means of the decision apparatus comprises:
means for generating an output signal indicating whether each feature or symbol of the text is in the "up", "down" or "indeterminate"
orientation, and means for forming a consensus from the output signals from the generating means of the second means for determining whether the features or text are in the "right-side up", "upside-down" or "indeterminate"
orientation.
means for generating an output signal indicating whether each feature or symbol of the text is in the "up", "down" or "indeterminate"
orientation, and means for forming a consensus from the output signals from the generating means of the second means for determining whether the features or text are in the "right-side up", "upside-down" or "indeterminate"
orientation.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US07/359,296 US5060276A (en) | 1989-05-31 | 1989-05-31 | Technique for object orientation detection using a feed-forward neural network |
| US359,296 | 1989-05-31 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CA2013378A1 CA2013378A1 (en) | 1990-11-30 |
| CA2013378C true CA2013378C (en) | 1993-10-12 |
Family
ID=23413214
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002013378A Expired - Fee Related CA2013378C (en) | 1989-05-31 | 1990-03-29 | Technique for object orientation detection using a feed-forward neural network |
Country Status (8)
| Country | Link |
|---|---|
| US (1) | US5060276A (en) |
| EP (1) | EP0400881B1 (en) |
| JP (1) | JPH0726836B2 (en) |
| KR (1) | KR930003403B1 (en) |
| CA (1) | CA2013378C (en) |
| DE (1) | DE69029427T2 (en) |
| ES (1) | ES2095237T3 (en) |
| HK (1) | HK45997A (en) |
Families Citing this family (204)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6067379A (en) * | 1988-12-09 | 2000-05-23 | Cognex Corporation | Method and apparatus for locating patterns in an optical image |
| FR2662526B1 (en) * | 1990-05-22 | 1994-12-09 | Thomson Csf | METHOD FOR FAST RECOGNITION OF OBJECTS LIKELY TO BE PART OF A COLLECTION COMPRISING A LARGE NUMBER OF OBJECTS WITH DIFFERENT CHARACTERISTICS. |
| JPH04195485A (en) * | 1990-11-28 | 1992-07-15 | Hitachi Ltd | Image information input device |
| US5263107A (en) * | 1991-01-31 | 1993-11-16 | Sharp Kabushiki Kaisha | Receptive field neural network with shift-invariant pattern recognition |
| DE69226103T2 (en) * | 1991-02-22 | 1998-11-19 | Canon Kk | Image processing system |
| EP0506483B1 (en) * | 1991-03-29 | 1999-02-17 | Canon Kabushiki Kaisha | Information recording apparatus and image recording method |
| US5239593A (en) * | 1991-04-03 | 1993-08-24 | Nynex Science & Technology, Inc. | Optical pattern recognition using detector and locator neural networks |
| US10361802B1 (en) | 1999-02-01 | 2019-07-23 | Blanding Hovenweep, Llc | Adaptive pattern recognition based control system and method |
| US8352400B2 (en) | 1991-12-23 | 2013-01-08 | Hoffberg Steven M | Adaptive pattern recognition based controller apparatus and method and human-factored interface therefore |
| US5742702A (en) * | 1992-10-01 | 1998-04-21 | Sony Corporation | Neural network for character recognition and verification |
| US5319722A (en) * | 1992-10-01 | 1994-06-07 | Sony Electronics, Inc. | Neural network for character recognition of rotated characters |
| US5594815A (en) * | 1992-10-19 | 1997-01-14 | Fast; Bruce B. | OCR image preprocessing method for image enhancement of scanned documents |
| JPH06203005A (en) * | 1992-10-27 | 1994-07-22 | Eastman Kodak Co | High speed partitioned neural network and building-up method thereof |
| US6259446B1 (en) | 1992-12-23 | 2001-07-10 | Object Technology Licensing Corporation | Menu state system |
| US5550563A (en) * | 1992-12-23 | 1996-08-27 | Taligent, Inc. | Interaction framework system |
| JP3793226B2 (en) * | 1992-12-23 | 2006-07-05 | オブジェクト テクノロジー ライセンシング コーポレイション | Atomic command system |
| US5530864A (en) * | 1992-12-23 | 1996-06-25 | Taligent | Command object system for an object-oriented software platform |
| US5434965A (en) * | 1992-12-23 | 1995-07-18 | Taligent, Inc. | Balloon help system |
| US5551055A (en) * | 1992-12-23 | 1996-08-27 | Taligent, Inc. | System for providing locale dependent user interface for presenting control graphic which has different contents or same contents displayed in a predetermined order |
| US5315703A (en) * | 1992-12-23 | 1994-05-24 | Taligent, Inc. | Object-oriented notification framework system |
| US5390325A (en) * | 1992-12-23 | 1995-02-14 | Taligent, Inc. | Automated testing system |
| JPH08505721A (en) * | 1992-12-23 | 1996-06-18 | タリジェント インコーポレイテッド | Object-oriented framework system |
| US5859923A (en) * | 1992-12-29 | 1999-01-12 | Cognex Corporation | Mark quality inspection apparatus and method |
| AU6019094A (en) * | 1993-01-22 | 1994-08-15 | Taligent, Inc. | Business card system |
| US5428718A (en) * | 1993-01-22 | 1995-06-27 | Taligent, Inc. | Tessellation system |
| WO1994017479A1 (en) * | 1993-01-22 | 1994-08-04 | Taligent, Inc. | Flexible network system |
| AU6081994A (en) * | 1993-01-22 | 1994-08-15 | Taligent, Inc. | Flexible system |
| US5394523A (en) * | 1993-01-22 | 1995-02-28 | Taligent, Inc. | Polymorphic graphic device |
| AU6161594A (en) * | 1993-02-26 | 1994-09-14 | Taligent, Inc. | Collaborative work system |
| US5519862A (en) * | 1993-02-26 | 1996-05-21 | Taligent, Inc. | Concurrent processing apparatus with incremental command objects |
| US5369766A (en) * | 1993-03-25 | 1994-11-29 | Taligent, Inc. | Object-oriented loader system with support for different load formats |
| US5485373A (en) * | 1993-03-25 | 1996-01-16 | Taligent, Inc. | Language-sensitive text searching system with modified Boyer-Moore process |
| EP0672278B1 (en) * | 1993-03-25 | 1996-05-22 | Taligent, Inc. | Multi-level interrupt system |
| US5439160A (en) * | 1993-03-31 | 1995-08-08 | Siemens Corporate Research, Inc. | Method and apparatus for obtaining reflow oven settings for soldering a PCB |
| WO1994023379A1 (en) * | 1993-04-05 | 1994-10-13 | Taligent, Inc. | Font selection system |
| CA2145670A1 (en) * | 1993-04-05 | 1994-10-13 | Mark Edward Davis | Text input font system |
| US5459865A (en) * | 1993-04-05 | 1995-10-17 | Taligent Inc. | Runtime loader |
| WO1994025921A1 (en) * | 1993-04-26 | 1994-11-10 | Taligent, Inc. | Text transliteration system |
| US5432948A (en) * | 1993-04-26 | 1995-07-11 | Taligent, Inc. | Object-oriented rule-based text input transliteration system |
| JPH09500744A (en) * | 1993-06-03 | 1997-01-21 | タリジェント インコーポレイテッド | Place object system |
| US5544302A (en) * | 1993-06-03 | 1996-08-06 | Taligent, Inc. | Object-oriented framework for creating and using container objects with built-in properties |
| CA2141931A1 (en) * | 1993-06-03 | 1994-12-22 | Robert David Dickinson | Place object system |
| US5524190A (en) * | 1993-06-04 | 1996-06-04 | Taligent, Inc. | Command object logging system for restoring documents |
| US5471568A (en) * | 1993-06-30 | 1995-11-28 | Taligent, Inc. | Object-oriented apparatus and method for scan line conversion of graphic edges |
| US5487145A (en) * | 1993-07-09 | 1996-01-23 | Taligent, Inc. | Method and apparatus for compositing display items which minimizes locked drawing areas |
| US5379432A (en) * | 1993-07-19 | 1995-01-03 | Taligent, Inc. | Object-oriented interface for a procedural operating system |
| US5455951A (en) * | 1993-07-19 | 1995-10-03 | Taligent, Inc. | Method and apparatus for running an object-oriented program on a host computer with a procedural operating system |
| US5404529A (en) * | 1993-07-19 | 1995-04-04 | Taligent, Inc. | Object-oriented interprocess communication system interface for a procedural operating system |
| US6684261B1 (en) | 1993-07-19 | 2004-01-27 | Object Technology Licensing Corporation | Object-oriented operating system |
| US5473777A (en) * | 1993-07-19 | 1995-12-05 | Moeller; Christopher P. | Wrapper for enabling an object otented application to maintain virtual memory using procedural function calls |
| CN1110064A (en) * | 1993-07-19 | 1995-10-11 | 塔里根特公司 | Dynamic linking system |
| US5519867A (en) * | 1993-07-19 | 1996-05-21 | Taligent, Inc. | Object-oriented multitasking system |
| JPH0737103A (en) * | 1993-07-23 | 1995-02-07 | Olympus Optical Co Ltd | Inclination angle detection device |
| FR2708365B1 (en) * | 1993-07-27 | 1995-09-01 | Thomson Csf | Method and device for correcting the positioning of an electronic component. |
| US5471675A (en) * | 1993-07-27 | 1995-11-28 | Taligent, Inc. | Object oriented video framework system |
| WO1995004330A1 (en) * | 1993-07-27 | 1995-02-09 | Taligent, Inc. | Object-oriented rendering system |
| US5396626A (en) * | 1993-08-04 | 1995-03-07 | Taligent, Inc. | Object-oriented locator system |
| US5479589A (en) * | 1993-08-04 | 1995-12-26 | Taligent, Inc. | Object-oriented system for selecting a graphic image on a display |
| US5379430A (en) * | 1993-08-04 | 1995-01-03 | Taligent, Inc. | Object-oriented system locator system |
| US5621434A (en) * | 1993-08-11 | 1997-04-15 | Object Technology Licensing Corp. | Cursor manipulation system and method |
| US5586236A (en) * | 1993-08-11 | 1996-12-17 | Object Technology Licensing Corp. | Universal color look up table and method of generation |
| US5566278A (en) * | 1993-08-24 | 1996-10-15 | Taligent, Inc. | Object oriented printing system |
| US5481666A (en) * | 1993-08-25 | 1996-01-02 | Taligent, Inc. | Object-oriented navigation system |
| US5500929A (en) * | 1993-08-30 | 1996-03-19 | Taligent, Inc. | System for browsing a network resource book with tabs attached to pages |
| US5428744A (en) * | 1993-08-30 | 1995-06-27 | Taligent, Inc. | Object-oriented system for building a graphic image on a display |
| US5511002A (en) * | 1993-09-13 | 1996-04-23 | Taligent, Inc. | Multimedia player component object system |
| DE69403806T2 (en) | 1993-09-13 | 1998-01-22 | Taligent Inc | MULTIMEDIA DATA CONTROL SYSTEM |
| US5388264A (en) * | 1993-09-13 | 1995-02-07 | Taligent, Inc. | Object oriented framework system for routing, editing, and synchronizing MIDI multimedia information using graphically represented connection object |
| US5390138A (en) * | 1993-09-13 | 1995-02-14 | Taligent, Inc. | Object-oriented audio system |
| CA2153964A1 (en) * | 1993-09-13 | 1995-03-23 | Steven H. Milne | Object-oriented audio record/playback system |
| DE69403915T2 (en) | 1993-09-13 | 1998-01-15 | Taligent Inc | OBJECT-ORIENTED VIDEO SYSTEM |
| US5583977A (en) * | 1993-10-21 | 1996-12-10 | Taligent, Inc. | Object-oriented curve manipulation system |
| US5522025A (en) * | 1993-10-25 | 1996-05-28 | Taligent, Inc. | Object-oriented window area display system |
| US5455854A (en) * | 1993-10-26 | 1995-10-03 | Taligent, Inc. | Object-oriented telephony system |
| EP0712513B1 (en) * | 1993-10-29 | 1997-06-04 | Taligent, Inc. | Graphic editor framework system |
| DE69406462T2 (en) * | 1993-11-02 | 1998-05-28 | Taligent Inc | OBJECT-ORIENTED GRAPHIC SYSTEM AND RELATED METHOD |
| US6040838A (en) * | 1993-11-05 | 2000-03-21 | Obejct Technology Licensing Corporation | Graphic state processing |
| US5428722A (en) * | 1993-11-05 | 1995-06-27 | Taligent, Inc. | Object-oriented painter maker |
| US5537526A (en) * | 1993-11-12 | 1996-07-16 | Taugent, Inc. | Method and apparatus for processing a display document utilizing a system level document framework |
| AU6018294A (en) * | 1993-12-02 | 1995-06-19 | Taligent, Inc. | Method and apparatus for displaying hardware dependent graphics in an object-oriented operating system |
| US5594921A (en) * | 1993-12-17 | 1997-01-14 | Object Technology Licensing Corp. | Authentication of users with dynamically configurable protocol stack |
| US5548726A (en) * | 1993-12-17 | 1996-08-20 | Taligeni, Inc. | System for activating new service in client server network by reconfiguring the multilayer network protocol stack dynamically within the server node |
| US5499343A (en) * | 1993-12-17 | 1996-03-12 | Taligent, Inc. | Object-oriented networking system with dynamically configurable communication links |
| US5530799A (en) * | 1993-12-17 | 1996-06-25 | Taligent Inc. | Rendering cache in an object oriented system |
| WO1995017064A1 (en) * | 1993-12-17 | 1995-06-22 | Taligent, Inc. | Object-oriented distributed communications directory service |
| US5515508A (en) * | 1993-12-17 | 1996-05-07 | Taligent, Inc. | Client server system and method of operation including a dynamically configurable protocol stack |
| US5548723A (en) * | 1993-12-17 | 1996-08-20 | Taligent, Inc. | Object-oriented network protocol configuration system utilizing a dynamically configurable protocol stack |
| US5491800A (en) * | 1993-12-20 | 1996-02-13 | Taligent, Inc. | Object-oriented remote procedure call networking system |
| US5574915A (en) * | 1993-12-21 | 1996-11-12 | Taligent | Object-oriented booting framework |
| US5546595A (en) * | 1993-12-21 | 1996-08-13 | Taligent, Inc. | Object-oriented system using objects representing hardware devices, physical connectors and connections between the physical connectors for configuring a computer |
| US5566346A (en) * | 1993-12-21 | 1996-10-15 | Taligent, Inc. | System for constructing hardware device interface software systems independent of operating systems including capability of installing and removing interrupt handlers |
| US5548779A (en) | 1993-12-21 | 1996-08-20 | Taligent | System for providing system services for a device to a client using stack definition and stack description of a stack having top, intermediate, and bottom service objects |
| US5680624A (en) * | 1993-12-21 | 1997-10-21 | Object Licensing Corporation | Object oriented interrupt system |
| AU6814594A (en) * | 1993-12-21 | 1995-07-10 | Taligent, Inc. | Automatic hardware configuration |
| US5371884A (en) * | 1993-12-21 | 1994-12-06 | Taligent, Inc. | Processor fault recovery system |
| WO1995017729A1 (en) * | 1993-12-22 | 1995-06-29 | Taligent, Inc. | Input methods framework |
| AU7872494A (en) * | 1993-12-30 | 1995-07-17 | Taligent, Inc. | Object-oriented view coordinate space system |
| WO1995018413A1 (en) * | 1993-12-30 | 1995-07-06 | Taligent, Inc. | Object-oriented view hierarchy framework |
| US5555368A (en) * | 1993-12-30 | 1996-09-10 | Taligent | Object-oriented multi-tasking view framework |
| US5544301A (en) * | 1993-12-30 | 1996-08-06 | Taligent, Inc. | Object-oriented view layout system |
| US5615326A (en) * | 1993-12-30 | 1997-03-25 | Taligent, Inc. | Object-oriented viewing framework having view grouping |
| US5524199A (en) * | 1993-12-30 | 1996-06-04 | Taligent | Object-oriented view system with background processing of update request |
| US5524200A (en) * | 1993-12-30 | 1996-06-04 | Taligent, Inc. | Object-oriented non-rectilinear viewing framework |
| US5465362A (en) * | 1993-12-30 | 1995-11-07 | Taligent, Inc. | Object-oriented view-system for displaying information in a windowing environment |
| US5832219A (en) | 1994-02-08 | 1998-11-03 | Object Technology Licensing Corp. | Distributed object networking service |
| WO1995025999A1 (en) * | 1994-03-21 | 1995-09-28 | Taligent, Inc. | Document proxy framework |
| US5680563A (en) * | 1994-07-25 | 1997-10-21 | Object Technology Licensing Corporation | Object-oriented operating system enhancement for filtering items in a window |
| US5912666A (en) * | 1994-08-23 | 1999-06-15 | Object Technology Licensing Corp. | Object-oriented global cursor tool |
| US5504892A (en) * | 1994-09-08 | 1996-04-02 | Taligent, Inc. | Extensible object-oriented file system |
| US5519818A (en) * | 1994-09-19 | 1996-05-21 | Taligent, Inc. | Object-oriented graphic picking system |
| DE69509406D1 (en) * | 1994-10-25 | 1999-06-02 | Otl Corp | OBJECT-ORIENTED WINDOW MANAGEMENT SYSTEM |
| US5652884A (en) * | 1994-11-14 | 1997-07-29 | Object Technology Licensing Corp. | Method and apparatus for dynamic update of an existing object in an object editor |
| US5630131A (en) * | 1994-11-14 | 1997-05-13 | Object Technology Licensing Corp. | Method and apparatus for importing and exporting archive files for a graphical user interface |
| US5613122A (en) * | 1994-11-14 | 1997-03-18 | Object Technology Licensing Corp. | Object-oriented operating system |
| US5659735A (en) * | 1994-12-09 | 1997-08-19 | Object Technology Licensing Corp. | Object-oriented system for program version and history database management system for various program components |
| US5752245A (en) * | 1994-12-09 | 1998-05-12 | Object Technology Licensing Corporation | Object-oriented system for configuration history management with a project workspace and project history database for draft identification |
| US5553282A (en) * | 1994-12-09 | 1996-09-03 | Taligent, Inc. | Software project history database and method of operation |
| US5642106A (en) * | 1994-12-27 | 1997-06-24 | Siemens Corporate Research, Inc. | Visual incremental turn detector |
| JP3727971B2 (en) * | 1995-02-01 | 2005-12-21 | キヤノン株式会社 | Document processing apparatus and document processing method |
| US5713045A (en) * | 1995-06-29 | 1998-01-27 | Object Technology Licensing Corporation | System for processing user events with input device entity associated with event producer which further links communication from event consumer to the event producer |
| US6026176A (en) * | 1995-07-25 | 2000-02-15 | Cognex Corporation | Machine vision methods and articles of manufacture for ball grid array inspection |
| US5815653A (en) * | 1995-11-13 | 1998-09-29 | You; Lawrence L. | Debugging system with portable debug environment-independent client and non-portable platform-specific server |
| US5778230A (en) * | 1995-11-13 | 1998-07-07 | Object Technology Licensing Corp. | Goal directed object-oriented debugging system |
| US6158045A (en) * | 1995-11-13 | 2000-12-05 | Object Technology Licensing Corporation | Portable debugging services utilizing a client debugger object and a server debugger object with flexible addressing support |
| US5956479A (en) * | 1995-11-13 | 1999-09-21 | Object Technology Licensing Corporation | Demand based generation of symbolic information |
| US5787245A (en) * | 1995-11-13 | 1998-07-28 | Object Technology Licensing Corporation | Portable debugging service utilizing a client debugger object and a server debugger object |
| US5812850A (en) * | 1995-11-13 | 1998-09-22 | Object Technology Licensing Corp. | Object-oriented symbolic debugger using a compiler driven database and state modeling to control program execution |
| US5790132A (en) * | 1995-12-07 | 1998-08-04 | Object Technology Licensing Corp. | Image rendering system with extensible mechanism for providing visual user feedback when an image graphic is selected |
| US5822580A (en) * | 1996-01-19 | 1998-10-13 | Object Technology Licensing Corp. | Object oriented programming based global registry system, method, and article of manufacture |
| US5872870A (en) * | 1996-02-16 | 1999-02-16 | Cognex Corporation | Machine vision methods for identifying extrema of objects in rotated reference frames |
| US5909504A (en) * | 1996-03-15 | 1999-06-01 | Cognex Corporation | Method of testing a machine vision inspection system |
| US6298149B1 (en) | 1996-03-21 | 2001-10-02 | Cognex Corporation | Semiconductor device image inspection with contrast enhancement |
| US6259827B1 (en) | 1996-03-21 | 2001-07-10 | Cognex Corporation | Machine vision methods for enhancing the contrast between an object and its background using multiple on-axis images |
| US5978502A (en) * | 1996-04-01 | 1999-11-02 | Cognex Corporation | Machine vision methods for determining characteristics of three-dimensional objects |
| TW329409B (en) * | 1996-04-15 | 1998-04-11 | Nakabayashi Kk | Retaining construction for film box and box photo album |
| US5877768A (en) * | 1996-06-19 | 1999-03-02 | Object Technology Licensing Corp. | Method and system using a sorting table to order 2D shapes and 2D projections of 3D shapes for rendering a composite drawing |
| US6137893A (en) * | 1996-10-07 | 2000-10-24 | Cognex Corporation | Machine vision calibration targets and methods of determining their location and orientation in an image |
| US5960125A (en) * | 1996-11-21 | 1999-09-28 | Cognex Corporation | Nonfeedback-based machine vision method for determining a calibration relationship between a camera and a moveable object |
| DE19652925C2 (en) * | 1996-12-18 | 1998-11-05 | Hans Dr Geiger | Method and device for the location and size-independent detection of features from an image |
| US5953130A (en) * | 1997-01-06 | 1999-09-14 | Cognex Corporation | Machine vision methods and apparatus for machine vision illumination of an object |
| US6075881A (en) * | 1997-03-18 | 2000-06-13 | Cognex Corporation | Machine vision methods for identifying collinear sets of points from an image |
| US5974169A (en) * | 1997-03-20 | 1999-10-26 | Cognex Corporation | Machine vision methods for determining characteristics of an object using boundary points and bounding regions |
| US6141033A (en) * | 1997-05-15 | 2000-10-31 | Cognex Corporation | Bandwidth reduction of multichannel images for machine vision |
| US6608647B1 (en) | 1997-06-24 | 2003-08-19 | Cognex Corporation | Methods and apparatus for charge coupled device image acquisition with independent integration and readout |
| US5978080A (en) * | 1997-09-25 | 1999-11-02 | Cognex Corporation | Machine vision methods using feedback to determine an orientation, pixel width and pixel height of a field of view |
| US6025854A (en) * | 1997-12-31 | 2000-02-15 | Cognex Corporation | Method and apparatus for high speed image acquisition |
| US6236769B1 (en) | 1998-01-28 | 2001-05-22 | Cognex Corporation | Machine vision systems and methods for morphological transformation of an image with zero or other uniform offsets |
| US6282328B1 (en) | 1998-01-28 | 2001-08-28 | Cognex Corporation | Machine vision systems and methods for morphological transformation of an image with non-uniform offsets |
| US6381375B1 (en) | 1998-02-20 | 2002-04-30 | Cognex Corporation | Methods and apparatus for generating a projection of an image |
| US6215915B1 (en) | 1998-02-20 | 2001-04-10 | Cognex Corporation | Image processing methods and apparatus for separable, general affine transformation of an image |
| US6324299B1 (en) * | 1998-04-03 | 2001-11-27 | Cognex Corporation | Object image search using sub-models |
| US6411734B1 (en) | 1998-04-03 | 2002-06-25 | Cognex Corporation | Object image search using validated sub-model poses |
| US6424734B1 (en) | 1998-04-03 | 2002-07-23 | Cognex Corporation | Fiducial mark search using sub-models |
| US6252986B1 (en) | 1998-04-03 | 2001-06-26 | Cognex Corporation | Locating an image of an object having a concentric profile |
| US6516092B1 (en) | 1998-05-29 | 2003-02-04 | Cognex Corporation | Robust sub-model shape-finder |
| US7016539B1 (en) | 1998-07-13 | 2006-03-21 | Cognex Corporation | Method for fast, robust, multi-dimensional pattern recognition |
| US6963338B1 (en) | 1998-07-31 | 2005-11-08 | Cognex Corporation | Method for refining geometric description models using images |
| US6459820B1 (en) | 1998-08-25 | 2002-10-01 | Cognex Corporation | Gauging using sub-model poses |
| US6687402B1 (en) | 1998-12-18 | 2004-02-03 | Cognex Corporation | Machine vision methods and systems for boundary feature comparison of patterns and images |
| US6381366B1 (en) | 1998-12-18 | 2002-04-30 | Cognex Corporation | Machine vision methods and system for boundary point-based comparison of patterns and images |
| US7904187B2 (en) | 1999-02-01 | 2011-03-08 | Hoffberg Steven M | Internet appliance system and method |
| US6614926B1 (en) | 1999-06-29 | 2003-09-02 | Cognex Corporation | Methods and apparatuses for generating from an image a model of an object |
| US6813377B1 (en) | 1999-08-06 | 2004-11-02 | Cognex Corporation | Methods and apparatuses for generating a model of an object from an image of the object |
| US6898333B1 (en) | 1999-08-06 | 2005-05-24 | Cognex Corporation | Methods and apparatus for determining the orientation of an object in an image |
| US6684402B1 (en) | 1999-12-01 | 2004-01-27 | Cognex Technology And Investment Corporation | Control methods and apparatus for coupling multiple image acquisition devices to a digital data processor |
| GB2404733B (en) * | 2000-03-14 | 2005-03-23 | Intel Corp | Estimating text color and segmentation of images |
| US6748104B1 (en) | 2000-03-24 | 2004-06-08 | Cognex Corporation | Methods and apparatus for machine vision inspection using single and multiple templates or patterns |
| US6993205B1 (en) | 2000-04-12 | 2006-01-31 | International Business Machines Corporation | Automatic method of detection of incorrectly oriented text blocks using results from character recognition |
| US6950548B1 (en) | 2000-05-17 | 2005-09-27 | Cognex Corporation | Creating geometric model descriptions for use in machine vision inspection systems |
| US7006669B1 (en) | 2000-12-31 | 2006-02-28 | Cognex Corporation | Machine vision method and apparatus for thresholding images of non-uniform materials |
| US6804416B1 (en) | 2001-03-16 | 2004-10-12 | Cognex Corporation | Method and system for aligning geometric object models with images |
| US6954551B2 (en) * | 2001-12-04 | 2005-10-11 | The Boeing Company | Method for determining attitude of an object |
| US7020338B1 (en) * | 2002-04-08 | 2006-03-28 | The United States Of America As Represented By The National Security Agency | Method of identifying script of line of text |
| US7190834B2 (en) | 2003-07-22 | 2007-03-13 | Cognex Technology And Investment Corporation | Methods for finding and characterizing a deformed pattern in an image |
| US8081820B2 (en) | 2003-07-22 | 2011-12-20 | Cognex Technology And Investment Corporation | Method for partitioning a pattern into optimized sub-patterns |
| FR2861524A1 (en) * | 2003-10-23 | 2005-04-29 | Thomson Licensing Sa | Method for detecting orientation of image taken by digital camera, involves detecting lines in image, calculating attributes that characterize lines, for each detected line, and detecting orientation of image according to attributes |
| US7349567B2 (en) * | 2004-03-05 | 2008-03-25 | Electro Scientific Industries, Inc. | Method and apparatus for determining angular pose of an object |
| JP4420447B2 (en) * | 2004-06-14 | 2010-02-24 | キヤノン株式会社 | Color processing apparatus and color processing method |
| US7639861B2 (en) | 2005-09-14 | 2009-12-29 | Cognex Technology And Investment Corporation | Method and apparatus for backlighting a wafer during alignment |
| US8111904B2 (en) | 2005-10-07 | 2012-02-07 | Cognex Technology And Investment Corp. | Methods and apparatus for practical 3D vision system |
| US8162584B2 (en) | 2006-08-23 | 2012-04-24 | Cognex Corporation | Method and apparatus for semiconductor wafer alignment |
| US8208725B2 (en) * | 2007-06-21 | 2012-06-26 | Sharp Laboratories Of America, Inc. | Methods and systems for identifying text orientation in a digital image |
| US8144989B2 (en) * | 2007-06-21 | 2012-03-27 | Sharp Laboratories Of America, Inc. | Methods and systems for identifying text orientation in a digital image |
| US8340430B2 (en) * | 2007-07-10 | 2012-12-25 | Sharp Laboratories Of America, Inc. | Methods and systems for identifying digital image characteristics |
| US8023770B2 (en) * | 2008-05-23 | 2011-09-20 | Sharp Laboratories Of America, Inc. | Methods and systems for identifying the orientation of a digital image |
| US8023741B2 (en) * | 2008-05-23 | 2011-09-20 | Sharp Laboratories Of America, Inc. | Methods and systems for detecting numerals in a digital image |
| US8437576B2 (en) * | 2008-06-23 | 2013-05-07 | Samsung Electronics Co., Ltd. | Method of correcting orientation of an image in automatic printing, and an image forming method and image forming apparatus thereof |
| US8160365B2 (en) * | 2008-06-30 | 2012-04-17 | Sharp Laboratories Of America, Inc. | Methods and systems for identifying digital image characteristics |
| US8218904B2 (en) * | 2008-08-27 | 2012-07-10 | Lockheed Martin Corporation | Method and system for circular to horizontal transposition of an image |
| US8620080B2 (en) * | 2008-09-26 | 2013-12-31 | Sharp Laboratories Of America, Inc. | Methods and systems for locating text in a digital image |
| US20110185353A1 (en) * | 2010-01-27 | 2011-07-28 | Jack Matthew | Mitigating Problems Arising From Incompatible Software |
| US9091628B2 (en) | 2012-12-21 | 2015-07-28 | L-3 Communications Security And Detection Systems, Inc. | 3D mapping with two orthogonal imaging views |
| US9076058B2 (en) | 2013-01-29 | 2015-07-07 | Sharp Laboratories Of America, Inc. | Methods, systems and apparatus for determining orientation in a document image |
| US9679224B2 (en) | 2013-06-28 | 2017-06-13 | Cognex Corporation | Semi-supervised method for training multiple pattern recognition and registration tool models |
| US10074042B2 (en) | 2015-10-06 | 2018-09-11 | Adobe Systems Incorporated | Font recognition using text localization |
| US9875429B2 (en) * | 2015-10-06 | 2018-01-23 | Adobe Systems Incorporated | Font attributes for font recognition and similarity |
| US10007868B2 (en) | 2016-09-19 | 2018-06-26 | Adobe Systems Incorporated | Font replacement based on visual similarity |
| CN107179495B (en) * | 2017-06-06 | 2023-10-20 | 上海传英信息技术有限公司 | Circuit board welding detection method and device |
| US10616443B1 (en) * | 2019-02-11 | 2020-04-07 | Open Text Sa Ulc | On-device artificial intelligence systems and methods for document auto-rotation |
| US10950017B2 (en) | 2019-07-08 | 2021-03-16 | Adobe Inc. | Glyph weight modification |
| US11295181B2 (en) | 2019-10-17 | 2022-04-05 | Adobe Inc. | Preserving document design using font synthesis |
| CN111402284B (en) * | 2020-03-17 | 2023-07-25 | 中国人民解放军国防科学技术大学 | Image threshold value determination method and device based on three-dimensional connectivity |
| CN113845064B (en) * | 2021-08-05 | 2023-05-09 | 乐仓信息科技有限公司 | Positioning method and system for material bearing device with round support legs |
| WO2023148850A1 (en) * | 2022-02-02 | 2023-08-10 | Nec Corporation | Training apparatus, angle estimation apparatus, training method, and angle estimation method |
Family Cites Families (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US3801775A (en) * | 1972-08-07 | 1974-04-02 | Scanner | Method and apparatus for identifying objects |
| US3847346A (en) * | 1972-08-30 | 1974-11-12 | Scanner | Data field recognition and reading method and system |
| US4028728A (en) * | 1976-04-02 | 1977-06-07 | Western Electric Company, Inc. | Method of and video system for identifying different light-reflective surface areas on articles |
| JPS55119782A (en) * | 1979-03-09 | 1980-09-13 | Daihen Corp | Pattern automatic inspection method |
| JPS605881B2 (en) * | 1979-08-09 | 1985-02-14 | 富士電機株式会社 | Tilt detection device |
| DE3204021A1 (en) * | 1982-02-05 | 1983-08-18 | Gebhardt Fördertechnik GmbH, 6920 Sinsheim | DISTRIBUTION SYSTEM FOR MOVING PARTS |
| US4500202A (en) * | 1982-05-24 | 1985-02-19 | Itek Corporation | Printed circuit board defect detection of detecting maximum line width violations |
| JPS5924386A (en) * | 1982-08-02 | 1984-02-08 | Hitachi Ltd | Input device of optical pattern |
| US4578810A (en) * | 1983-08-08 | 1986-03-25 | Itek Corporation | System for printed circuit board defect detection |
| JPS6054565A (en) * | 1983-09-06 | 1985-03-29 | Canon Inc | Ducument processor |
| JPS6177707A (en) * | 1984-09-25 | 1986-04-21 | Mitsubishi Electric Corp | Direction judging device of semiconductor integrated circuit device |
| JPH0810132B2 (en) * | 1986-06-04 | 1996-01-31 | 富士電機株式会社 | Target pattern rotation angle detection method |
| JP2667435B2 (en) * | 1987-05-01 | 1997-10-27 | 株式会社リコー | Region extraction method |
-
1989
- 1989-05-31 US US07/359,296 patent/US5060276A/en not_active Expired - Lifetime
-
1990
- 1990-03-29 CA CA002013378A patent/CA2013378C/en not_active Expired - Fee Related
- 1990-05-23 DE DE69029427T patent/DE69029427T2/en not_active Expired - Fee Related
- 1990-05-23 EP EP90305587A patent/EP0400881B1/en not_active Expired - Lifetime
- 1990-05-23 ES ES90305587T patent/ES2095237T3/en not_active Expired - Lifetime
- 1990-05-30 KR KR1019900007838A patent/KR930003403B1/en not_active IP Right Cessation
- 1990-05-30 JP JP2138656A patent/JPH0726836B2/en not_active Expired - Fee Related
-
1997
- 1997-04-10 HK HK45997A patent/HK45997A/en not_active IP Right Cessation
Also Published As
| Publication number | Publication date |
|---|---|
| JPH0321811A (en) | 1991-01-30 |
| JPH0726836B2 (en) | 1995-03-29 |
| EP0400881B1 (en) | 1996-12-18 |
| HK45997A (en) | 1997-04-18 |
| ES2095237T3 (en) | 1997-02-16 |
| EP0400881A2 (en) | 1990-12-05 |
| CA2013378A1 (en) | 1990-11-30 |
| US5060276A (en) | 1991-10-22 |
| DE69029427T2 (en) | 1997-04-03 |
| KR900018851A (en) | 1990-12-22 |
| KR930003403B1 (en) | 1993-04-26 |
| EP0400881A3 (en) | 1992-12-30 |
| DE69029427D1 (en) | 1997-01-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA2013378C (en) | Technique for object orientation detection using a feed-forward neural network | |
| CN111179251B (en) | Defect detection system and method based on twin neural network and by utilizing template comparison | |
| US5642442A (en) | Method for locating the position and orientation of a fiduciary mark | |
| US5299269A (en) | Character segmentation using an associative memory for optical character recognition | |
| CN108548820A (en) | Cosmetics paper labels defect inspection method | |
| US4872024A (en) | Print inspection method, print inspection apparatus and automatic print sorting system | |
| US6549683B1 (en) | Method and apparatus for evaluating a scale factor and a rotation angle in image processing | |
| US7110591B2 (en) | System and method for recognizing markers on printed circuit boards | |
| EP0437273A2 (en) | Method of matching patterns and apparatus therefor | |
| JP2787151B2 (en) | Article quantity inspection method and device | |
| Kim et al. | A practical pattern recognition system for translation, scale and rotation invariance | |
| US6694054B1 (en) | Pattern recognition process | |
| US5058177A (en) | Method for inspection of protruding features | |
| CN110288040A (en) | A kind of similar evaluation method of image based on validating topology and equipment | |
| Nagarajan et al. | A real time marking inspection scheme for semiconductor industries | |
| CN116993725B (en) | Intelligent patch information processing system of flexible circuit board | |
| CA2307439C (en) | Method and apparatus for evaluating a scale factor and a rotation angle in image processing | |
| EP0434871B1 (en) | Character recognition method | |
| CA2036274A1 (en) | Document processor including method and apparatus for identifying and correcting errors | |
| JPH07175894A (en) | Neural network, character recognition method, electronic parts mounting inspecting device and managing method using the same | |
| EP4358020A1 (en) | Method of detecting errors in the placement of elements in the pcb | |
| Luo et al. | FPC surface defect detection based on improved Faster R-CNN with decoupled RPN | |
| CN112891907B (en) | Billiard box with billiard number recognition function | |
| Wu et al. | A system for automated BGA inspection | |
| JP3388918B2 (en) | String reader |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request | ||
| MKLA | Lapsed |