CA2083733A1 - Word disambiguation methods and apparatus - Google Patents
Word disambiguation methods and apparatusInfo
- Publication number
- CA2083733A1 CA2083733A1 CA002083733A CA2083733A CA2083733A1 CA 2083733 A1 CA2083733 A1 CA 2083733A1 CA 002083733 A CA002083733 A CA 002083733A CA 2083733 A CA2083733 A CA 2083733A CA 2083733 A1 CA2083733 A1 CA 2083733A1
- Authority
- CA
- Canada
- Prior art keywords
- word
- sense
- words
- pair
- text
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/253—Grammatical analysis; Style critique
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/40—Processing or translation of natural language
- G06F40/42—Data-driven translation
- G06F40/44—Statistical methods, e.g. probability models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/40—Processing or translation of natural language
- G06F40/42—Data-driven translation
- G06F40/45—Example-based machine translation; Alignment
Abstract
WORD DISAMBIGUATION APPARATUS AND METHODS
Abstract Apparatus and methods for determining whether a word/sense pair is proper for a context. Wide contexts (100 words) are employed for both training and testing, and testing is done by adding the weights of vocabulary words from the context. The weights are determined by Bayesian techniques which interpolate between the probability of occurrence of a vocabulary word in a conditional sample of the training text and the probability of its occurrence in the entire training text. A
further improvement in testing takes advantage of the fact that a word is generally used in only a single sense in a single discourse. Also disclosed are automated training techniques including training on bilingual bodies of text and training using categories from Roget's Thesaurus.
Abstract Apparatus and methods for determining whether a word/sense pair is proper for a context. Wide contexts (100 words) are employed for both training and testing, and testing is done by adding the weights of vocabulary words from the context. The weights are determined by Bayesian techniques which interpolate between the probability of occurrence of a vocabulary word in a conditional sample of the training text and the probability of its occurrence in the entire training text. A
further improvement in testing takes advantage of the fact that a word is generally used in only a single sense in a single discourse. Also disclosed are automated training techniques including training on bilingual bodies of text and training using categories from Roget's Thesaurus.
Description
2 ~ ~J) ?J ? ~ ~
WORD DISAMBIGUATION APPARATUS AND h~ETHODS
Back~round of the Invenffon Field of the InYention The invention relates to computerized text analysis generally and more specifically 5 to the problem of determining whether a given word-sense pair is proper for a given context.
Description f the Prior Art Machine translation of natural language texts has long been a goal of researchers in computer science and linguistics. A major barrier to high-quality machine 10 translation has been the difficulty of disambiguating words. Word disarnbiguation is necessary because many words in any natural language have more than one sense.
For example, the English noun sentence has two senses in common usage: one relating to grammar, where a sentence is a part of a text or speech, and one relating to punishment, where a sentence is a punishment imposed for a crime. Human 15 beings use the context in which the word appears and their general knowledge of the world to determine which sense is meant, and consequently do not even have trouble with texts such as:
l~e teacher gave the student the sentence of writing the sentence "I will not throw spit wads" 100 times.
20 Computers, however, have no general knowledge of the world, and consequently,have had a great deal of trouble translating sentences such as the above into languages such as French, where the word used to translate sentence when it is employed in the grammatical sense is phrasc and the word used to translate sentence when it is employed in the sense of punishment is peine.
The ability to determine a probable sense of a word from the contex~ in which the word is uséd important in other areas of text analysis as well. For example, optical character recognition systems and speech recognition systems often can only resolve a printed or spoken word into a small set of possibilities; one way of making a choice arnong the words in the small set is to determine which word has 30 a sense which best fits the context. Other examples in this area are determining whether characters such as accen~s or umlauts should be present on a word or whether the word should be capitalized. Additionally, there are tex~ editing tools :, ' ' .: :
r~ g'~
such as spelling checkers or inreractive thesauri which present the user with a set of suggested alternatives for a word. These tools, too, are improved if the set of alternatives is limited to words whose senses ISt the context.
Another area of text analysis that will benefit from good techniques for 5 determining the probable sense of a word from its context is data base searching.
Word searches in data bases work by simply matching a search term with an occurrence of the terrn in the data base, without regard to the sense in which the term is used in the data base. The only way to restrict a search to a given sense of a term is to provide other search terms which the searcher expects to find in conjunction 10 with the first search tenn. Such a search strategy will, however, miss occurrences of the first term where the first terrn has the proper sense but is not found in conjunction with the other search terrns. Given a useful way of determining what sense of a word best fits a conte~t, it will be possible to search by specifying not only the search term, but also the sense in which it is being used.
Past researchers have used three different general approaches to the word disarnbiguation problem sketched above:
Qualitative Methods, e.g., Hirst (1987) Dictionary-based Methods, e.g., Lesk (1986) Corpus-based Methods, e.g., Kelly and Stone (1975) In each case, the work has been limited by a knowledge acquisition 20 bottleneck. For example, there has been a tradition in parts of the AI community of building large experts by hand, e.g., Granger (1977), Rieger (1977), Small and Rieger (1982), Hirst (1987). Unfortunately, this approach is not very easy to scale up, as many researc}lers have observed:
"The expert for THROW is currently six pages long, ... but it should be 2S 10 times that size." (Small and Reiger, 198X) Since this approach is so difficult to scale up, much of the work has had to focus on "toy" domains (e.g., Winograd's Blocks World) or sublanguages (e.g., Isabelle (19~4), Hirschman (1986)). Currently, it is not possible to find a semantic network with the kind of broad coverage that would be required for unrestricted text.
Others such as Lesk (1986), Walker (1987), Ide (1990, Waterloo Meeting) have turned to machine-readable dictionarys (MRD) such as Oxford's Advanced Learner's Dictionary of Curren~ English (OALDCE) in the hope ~hat MRDs might provide a way out of the knowledge acquisition bottleneck. These ..
.
WORD DISAMBIGUATION APPARATUS AND h~ETHODS
Back~round of the Invenffon Field of the InYention The invention relates to computerized text analysis generally and more specifically 5 to the problem of determining whether a given word-sense pair is proper for a given context.
Description f the Prior Art Machine translation of natural language texts has long been a goal of researchers in computer science and linguistics. A major barrier to high-quality machine 10 translation has been the difficulty of disambiguating words. Word disarnbiguation is necessary because many words in any natural language have more than one sense.
For example, the English noun sentence has two senses in common usage: one relating to grammar, where a sentence is a part of a text or speech, and one relating to punishment, where a sentence is a punishment imposed for a crime. Human 15 beings use the context in which the word appears and their general knowledge of the world to determine which sense is meant, and consequently do not even have trouble with texts such as:
l~e teacher gave the student the sentence of writing the sentence "I will not throw spit wads" 100 times.
20 Computers, however, have no general knowledge of the world, and consequently,have had a great deal of trouble translating sentences such as the above into languages such as French, where the word used to translate sentence when it is employed in the grammatical sense is phrasc and the word used to translate sentence when it is employed in the sense of punishment is peine.
The ability to determine a probable sense of a word from the contex~ in which the word is uséd important in other areas of text analysis as well. For example, optical character recognition systems and speech recognition systems often can only resolve a printed or spoken word into a small set of possibilities; one way of making a choice arnong the words in the small set is to determine which word has 30 a sense which best fits the context. Other examples in this area are determining whether characters such as accen~s or umlauts should be present on a word or whether the word should be capitalized. Additionally, there are tex~ editing tools :, ' ' .: :
r~ g'~
such as spelling checkers or inreractive thesauri which present the user with a set of suggested alternatives for a word. These tools, too, are improved if the set of alternatives is limited to words whose senses ISt the context.
Another area of text analysis that will benefit from good techniques for 5 determining the probable sense of a word from its context is data base searching.
Word searches in data bases work by simply matching a search term with an occurrence of the terrn in the data base, without regard to the sense in which the term is used in the data base. The only way to restrict a search to a given sense of a term is to provide other search terms which the searcher expects to find in conjunction 10 with the first search tenn. Such a search strategy will, however, miss occurrences of the first term where the first terrn has the proper sense but is not found in conjunction with the other search terrns. Given a useful way of determining what sense of a word best fits a conte~t, it will be possible to search by specifying not only the search term, but also the sense in which it is being used.
Past researchers have used three different general approaches to the word disarnbiguation problem sketched above:
Qualitative Methods, e.g., Hirst (1987) Dictionary-based Methods, e.g., Lesk (1986) Corpus-based Methods, e.g., Kelly and Stone (1975) In each case, the work has been limited by a knowledge acquisition 20 bottleneck. For example, there has been a tradition in parts of the AI community of building large experts by hand, e.g., Granger (1977), Rieger (1977), Small and Rieger (1982), Hirst (1987). Unfortunately, this approach is not very easy to scale up, as many researc}lers have observed:
"The expert for THROW is currently six pages long, ... but it should be 2S 10 times that size." (Small and Reiger, 198X) Since this approach is so difficult to scale up, much of the work has had to focus on "toy" domains (e.g., Winograd's Blocks World) or sublanguages (e.g., Isabelle (19~4), Hirschman (1986)). Currently, it is not possible to find a semantic network with the kind of broad coverage that would be required for unrestricted text.
Others such as Lesk (1986), Walker (1987), Ide (1990, Waterloo Meeting) have turned to machine-readable dictionarys (MRD) such as Oxford's Advanced Learner's Dictionary of Curren~ English (OALDCE) in the hope ~hat MRDs might provide a way out of the knowledge acquisition bottleneck. These ..
.
- 3 ~ 3 (~ J ~
researchers seek to develop a program that could read an arbitrary text and tag each word in the text with a pointer to a particular sense number in a particular dictionary.
Thus, for example, if Lesk's program was given the phrase pine cone, it ought to tag pine with a pointer to the first sense underpine in OALDCE (a kind of evergreen tree), and it ought to tag cone wi~h a pointer to the third sense under cone in OALDCE (fruit of certain evergreen trees). Lesk's program accomplishes this taskby looking for overlaps between the words in the definition and words in the text "near" the ambiguous word.
Unfornatuately, the approach doesn't seem to work as well as one might 10 hope. Lesk (1986) reports accuracies of 50-70% on short samples of Pride and Prejudice. Part of the problem may be that dictionary definitions are too short to mention all of the collocations (words that are often found in the context of a particular sense of a polysemous word). In addition, dictionaries have much lesscoverage than one might have expected. Walker (1987) reports that perhaps half of 15 the words occurring in a new text cannot be related to a dictionary entry.
Thus, like ~he AI approach, the dictionary-based approach is also limited by the knowledge acquisition bottleneck; dictionaries simply don't record enough of the relevant information, and much of the inforrnation that is stored in the dictionary is not in a format that computers can easily digest, at least at present.
A third line of research makes use of hand-annotated corpora. Most of these studies are limited by the availability of hand-annotated text. Since it is unlikely that such text will be available in large quantities for most of the polysemous words in the vocabulary, there are serious questions about how such an approach could be scaled up to handle unrestricted text. Kelly and Stone (1975) built 1815 disarnbiguation models by hand, selPcting words with a frequency of at least 20 in a half million word corpus. They started from key word in context (KWIC) concordances for each word~ and used these to establish the senses they perceived as useful for content analysis. The models consisted of an ordered set of rules, each giving a sufficient condition for deciding on one classification, or for jumping to 30 another rule in the same model, or for jumping to a rule in the model for another word. The conditions of a given rule could refer to the context within four words of the target word. They could test the morphology of the target word, an exact context word, or the part of speech or semantic class of any of the context words. The sixteen semantic classes were assigned by hand.
, . ..
.
researchers seek to develop a program that could read an arbitrary text and tag each word in the text with a pointer to a particular sense number in a particular dictionary.
Thus, for example, if Lesk's program was given the phrase pine cone, it ought to tag pine with a pointer to the first sense underpine in OALDCE (a kind of evergreen tree), and it ought to tag cone wi~h a pointer to the third sense under cone in OALDCE (fruit of certain evergreen trees). Lesk's program accomplishes this taskby looking for overlaps between the words in the definition and words in the text "near" the ambiguous word.
Unfornatuately, the approach doesn't seem to work as well as one might 10 hope. Lesk (1986) reports accuracies of 50-70% on short samples of Pride and Prejudice. Part of the problem may be that dictionary definitions are too short to mention all of the collocations (words that are often found in the context of a particular sense of a polysemous word). In addition, dictionaries have much lesscoverage than one might have expected. Walker (1987) reports that perhaps half of 15 the words occurring in a new text cannot be related to a dictionary entry.
Thus, like ~he AI approach, the dictionary-based approach is also limited by the knowledge acquisition bottleneck; dictionaries simply don't record enough of the relevant information, and much of the inforrnation that is stored in the dictionary is not in a format that computers can easily digest, at least at present.
A third line of research makes use of hand-annotated corpora. Most of these studies are limited by the availability of hand-annotated text. Since it is unlikely that such text will be available in large quantities for most of the polysemous words in the vocabulary, there are serious questions about how such an approach could be scaled up to handle unrestricted text. Kelly and Stone (1975) built 1815 disarnbiguation models by hand, selPcting words with a frequency of at least 20 in a half million word corpus. They started from key word in context (KWIC) concordances for each word~ and used these to establish the senses they perceived as useful for content analysis. The models consisted of an ordered set of rules, each giving a sufficient condition for deciding on one classification, or for jumping to 30 another rule in the same model, or for jumping to a rule in the model for another word. The conditions of a given rule could refer to the context within four words of the target word. They could test the morphology of the target word, an exact context word, or the part of speech or semantic class of any of the context words. The sixteen semantic classes were assigned by hand.
, . ..
.
- 4 - ~ , '?J ~,t Most subsequent work has sought automatic methods becau~se it is quite labor intensive to construct theæ rules by hand Weiss (1973) first built mle sets by hand for five words, then developed automatic procedures for building similar rule sets, which he applied to an additional three words, Unfortunately, the system was 5 tested on the training set, so it is difficult to know how well it actually worked, Black (1987, 1988) studied five 4-way polysemous words using about 2000 hand tagged concordance lines for each word Using 1500 training examples for each word, his program constructed decision trees based on the presence or absence of 81 "contextual ca~egories" within the context of the ambiguous word.
10 He used three different types of contex~ual categories: (1) subject categories from LDOCE, the Longman Dictionary of Contemporary English (Longman, 1978), (2) the 41 vocabulary items occurring most frequently within two words of the ambiguous word, and (3) the 40 vocabulary items excluding function words occurring most frequently in the concordance line. Black found that the dictionary 15 categories produced the weakest performance (47 percent correct), while the other two were quite close at 72 and 75 percent correct, respectively.
There has recently been a flurry of interest in approaches based on hand-annotated corpora. Hearst (1991) is a very recent example of an approach somewhat like Black (1987, 1988), Weiss (1973) and Kelly and Stone (1975), in this 20 respect, though she makes use of considerably more syntactic information ~han the others did. Her performance also seems to be somewhat better than the others', though it is difficult to compare performance across systems.
As may be seen from the foregoing, the lack of suitable techniques for determining which word-sense pair best fits a given context has been a serious 25 hindrance in many areas of text analysis. It is an object of the apparatus and methods disclosed herein to provide such technlques.
Summary of the Invention In one aspect, the invention is a method of automatically determining that a word/sense pair has a sense which suits a given position in a text. The method 30 includes the steps of: determining a sequence of words in the text which includes the given position and is substantially longer than a single line of the text; and determining whether the word/sense pa~r has the suitable sense by automatically analyzing the sequence.
In another aspect, the invention is a method of automatically determining a 35 probability that a word/sense pair has a sense which suits a given position in a text.
. . .. .
.
:
~c~ ",i? '~
The method includes the steps of: determining a se~quence of words in the text which includes the given position; and automatically employing a Bayesian discrimination technique involving the words in the æqwence and the sen~se of the word-sense pair to determine the probability that the word/sense pair has a sense which suits the S given position.
In s~ill another aspect, the invention is a method of automatically determining whether a given occurrence of a word in a text has a given sense. The method includes the steps of: making a first determination of the sense of the given occurrence of the word; and making a final determination of the sense of the given 10 occurrence of the word by comparing the first determination with a determination of the sense of a neighboring occurrence of the word.
The foregoing and other objects, aspects, and advantages of the invention will be apparent to one of ordinary skill in the art who peruses the following Drawing and Detailed Description~ wherein:
15 Brief Descrip~on o~ ~e Drawin~
FIG. 1 is a block diagram of apparatus for determining the probability that a word/sense pair is proper for a context;
FIG. 2 is a table of data from which table 107 of FIG. 1 may be constructed;
FIG. 3 is an example of part of a conditional sample; and FIG. 4 is an example of weights computed using Roget's categories.
The reference numbers employed in ~he Drawing and the Detailed Description have three or more digits. The two least significant digits are a number within a figure:
the remaining digits are the figure number. Thus, the element with the reference25 number "305" is first shown in FIG. 3.
Detailed Descrip~iorl The following Detailed Description will first provide an overview of Ihe theoretical aporach to the disambiguation problem in a preferred embodiment. will then describe appara~us for solving the disambiguation problem, and will finally30 discuss how the apparatus for solving the disambiguation problem is trained.
, .
, ~ ~
' .
?~Jr~ 3 Bayesian Disambi~ua~on Techniques The word-sense disambiguation problem is a discrimination problem, not very different from problems such as author identification and information retrieval. In author identification and information retrieval, it is customary to split the problem up into a testing phase and a training phase. During the training phase, we are given two (or more) sets of documents and are asked ~o construct a discriminator which can distinguish between the two (or more) classes of documents. These discriminators are then applied to new documents during the testing phase. In the author identification task, for example, the training set consists 10 of several documents written by each of the two (or more) authors. The resulting discriminator is then tested on documents whose authorship is disputed. In the information retrieval application, the training set consists of a set of one or more relevant documents and a set of zero or more irrelevant documents. The resultingdiscriminator is then applied to all documents in the library in order to separate the 15 more relevant ones from the less relevant ones. In the sense disambiguation case, the 100-word context surrounding instances of a polysemous word (e.g., du~) are treated very much like a document.
It is natural to take a Bayesian approach to these discrimination problems. Mosteller and Wallace (1964, section 3.1) used the following formula to 20 combine new evidence (e.g., the term by document matrix) with prior evidence (e.g., the historical record) in their classic authorship study of the Federalist Papers.
final odds-(initial odds)x(likelihood ratio) For two groups of documents, the equation becomes P(clasi 1 ) p(class 1 ) x L(class 1 ) P(class2) p(cla~ss2) L(class2) 25 where P represents a final probability, p represents an initial probability, and L
represents a likelihood. Similar equations can be found in textbooks on information retrieval (e.g., Salton (1989), equation 10.17).
The initial odds depend on the problem. In the author identificalion problem, for example, the initial odds are used to model what we know about thc 30 documents from the various conflicting his~orical records. In the informationretrieval application, the user may have a guess about the fraction of the library that he or she would expect to be relevant; such a guess could be used as the prior. It is - 7 ~ J ~,'J '.~ 7 ~, ~
o~ten the case that the prior probability will not have very much influence on the outcome, which is fortunate, since the prior can sometimes be difficult to estimate.
It is common practice to decompose the likelihoods into a product of likelihoods over tokens (occurrences of words) in the document (under appropriate 5 independence assumptions):
L(class 1 ) ~ Il Pr(tok class 1 ) L(class2) tok in doc Pr(tok class2) The crucial ingredients for this calculation are the probabilities of each term in the vocabulary conditional on the document being from a given class. These conditional probabilities have been computed in a number of different ways 10 depending on the application and the study.
For two senses, the Bayesian equation mentioned above becomes:
P(sensel) p(sensel) x L(sensel) P(sense2) p(sense2) L(sense2) where p, P and L are the initial probab;lity, the final probability and likelihood, as before. The initial probabilities are deterrnined from the overall probabilities of the 15 two senses in the corpus. AS in other large dimension discrimination problems, the likelihoods are decomposed into a product over tokens:
L(sense 1 ) - ln[ Pr(tok sense l ) L(sense2) ~ok in conte~t Pr(tok sense2) Apparatus îor IJse in Word~sense Disambi~uation: FIG. t FIG. 1 shows apparatus 101 which deteImines the likelihood that a 20 word/sense pair has the sense required by a given context. In a preferred embodiment, the output of apparatus lOl is the logarithm of the likelihood. Log word/sense pair likelihood (WSPL) 117 is computed by sense likelihood calcula~or103. Inputs to sense likelihood calculator 103 come from text 111, word/sense pair probabili~ ~able 107, and word/sense pair 105. Word/sense pair probability tabl~25 107 is a table which contains a subtable 108 for each word/sense pair which is of interest. Each subtable 108 contains entries (W~PPIE) 123 for at least all of the words in text 111 which give useful indications whether the word/sense pair 1()5 [o which subtable 108 corresponds has a sense suitable for position 116. Each entry for a text word 121 contains an indication of the weight 119 of that word for detennining ': ~ ' ', ' . ~ - .' -.
10 He used three different types of contex~ual categories: (1) subject categories from LDOCE, the Longman Dictionary of Contemporary English (Longman, 1978), (2) the 41 vocabulary items occurring most frequently within two words of the ambiguous word, and (3) the 40 vocabulary items excluding function words occurring most frequently in the concordance line. Black found that the dictionary 15 categories produced the weakest performance (47 percent correct), while the other two were quite close at 72 and 75 percent correct, respectively.
There has recently been a flurry of interest in approaches based on hand-annotated corpora. Hearst (1991) is a very recent example of an approach somewhat like Black (1987, 1988), Weiss (1973) and Kelly and Stone (1975), in this 20 respect, though she makes use of considerably more syntactic information ~han the others did. Her performance also seems to be somewhat better than the others', though it is difficult to compare performance across systems.
As may be seen from the foregoing, the lack of suitable techniques for determining which word-sense pair best fits a given context has been a serious 25 hindrance in many areas of text analysis. It is an object of the apparatus and methods disclosed herein to provide such technlques.
Summary of the Invention In one aspect, the invention is a method of automatically determining that a word/sense pair has a sense which suits a given position in a text. The method 30 includes the steps of: determining a sequence of words in the text which includes the given position and is substantially longer than a single line of the text; and determining whether the word/sense pa~r has the suitable sense by automatically analyzing the sequence.
In another aspect, the invention is a method of automatically determining a 35 probability that a word/sense pair has a sense which suits a given position in a text.
. . .. .
.
:
~c~ ",i? '~
The method includes the steps of: determining a se~quence of words in the text which includes the given position; and automatically employing a Bayesian discrimination technique involving the words in the æqwence and the sen~se of the word-sense pair to determine the probability that the word/sense pair has a sense which suits the S given position.
In s~ill another aspect, the invention is a method of automatically determining whether a given occurrence of a word in a text has a given sense. The method includes the steps of: making a first determination of the sense of the given occurrence of the word; and making a final determination of the sense of the given 10 occurrence of the word by comparing the first determination with a determination of the sense of a neighboring occurrence of the word.
The foregoing and other objects, aspects, and advantages of the invention will be apparent to one of ordinary skill in the art who peruses the following Drawing and Detailed Description~ wherein:
15 Brief Descrip~on o~ ~e Drawin~
FIG. 1 is a block diagram of apparatus for determining the probability that a word/sense pair is proper for a context;
FIG. 2 is a table of data from which table 107 of FIG. 1 may be constructed;
FIG. 3 is an example of part of a conditional sample; and FIG. 4 is an example of weights computed using Roget's categories.
The reference numbers employed in ~he Drawing and the Detailed Description have three or more digits. The two least significant digits are a number within a figure:
the remaining digits are the figure number. Thus, the element with the reference25 number "305" is first shown in FIG. 3.
Detailed Descrip~iorl The following Detailed Description will first provide an overview of Ihe theoretical aporach to the disambiguation problem in a preferred embodiment. will then describe appara~us for solving the disambiguation problem, and will finally30 discuss how the apparatus for solving the disambiguation problem is trained.
, .
, ~ ~
' .
?~Jr~ 3 Bayesian Disambi~ua~on Techniques The word-sense disambiguation problem is a discrimination problem, not very different from problems such as author identification and information retrieval. In author identification and information retrieval, it is customary to split the problem up into a testing phase and a training phase. During the training phase, we are given two (or more) sets of documents and are asked ~o construct a discriminator which can distinguish between the two (or more) classes of documents. These discriminators are then applied to new documents during the testing phase. In the author identification task, for example, the training set consists 10 of several documents written by each of the two (or more) authors. The resulting discriminator is then tested on documents whose authorship is disputed. In the information retrieval application, the training set consists of a set of one or more relevant documents and a set of zero or more irrelevant documents. The resultingdiscriminator is then applied to all documents in the library in order to separate the 15 more relevant ones from the less relevant ones. In the sense disambiguation case, the 100-word context surrounding instances of a polysemous word (e.g., du~) are treated very much like a document.
It is natural to take a Bayesian approach to these discrimination problems. Mosteller and Wallace (1964, section 3.1) used the following formula to 20 combine new evidence (e.g., the term by document matrix) with prior evidence (e.g., the historical record) in their classic authorship study of the Federalist Papers.
final odds-(initial odds)x(likelihood ratio) For two groups of documents, the equation becomes P(clasi 1 ) p(class 1 ) x L(class 1 ) P(class2) p(cla~ss2) L(class2) 25 where P represents a final probability, p represents an initial probability, and L
represents a likelihood. Similar equations can be found in textbooks on information retrieval (e.g., Salton (1989), equation 10.17).
The initial odds depend on the problem. In the author identificalion problem, for example, the initial odds are used to model what we know about thc 30 documents from the various conflicting his~orical records. In the informationretrieval application, the user may have a guess about the fraction of the library that he or she would expect to be relevant; such a guess could be used as the prior. It is - 7 ~ J ~,'J '.~ 7 ~, ~
o~ten the case that the prior probability will not have very much influence on the outcome, which is fortunate, since the prior can sometimes be difficult to estimate.
It is common practice to decompose the likelihoods into a product of likelihoods over tokens (occurrences of words) in the document (under appropriate 5 independence assumptions):
L(class 1 ) ~ Il Pr(tok class 1 ) L(class2) tok in doc Pr(tok class2) The crucial ingredients for this calculation are the probabilities of each term in the vocabulary conditional on the document being from a given class. These conditional probabilities have been computed in a number of different ways 10 depending on the application and the study.
For two senses, the Bayesian equation mentioned above becomes:
P(sensel) p(sensel) x L(sensel) P(sense2) p(sense2) L(sense2) where p, P and L are the initial probab;lity, the final probability and likelihood, as before. The initial probabilities are deterrnined from the overall probabilities of the 15 two senses in the corpus. AS in other large dimension discrimination problems, the likelihoods are decomposed into a product over tokens:
L(sense 1 ) - ln[ Pr(tok sense l ) L(sense2) ~ok in conte~t Pr(tok sense2) Apparatus îor IJse in Word~sense Disambi~uation: FIG. t FIG. 1 shows apparatus 101 which deteImines the likelihood that a 20 word/sense pair has the sense required by a given context. In a preferred embodiment, the output of apparatus lOl is the logarithm of the likelihood. Log word/sense pair likelihood (WSPL) 117 is computed by sense likelihood calcula~or103. Inputs to sense likelihood calculator 103 come from text 111, word/sense pair probabili~ ~able 107, and word/sense pair 105. Word/sense pair probability tabl~25 107 is a table which contains a subtable 108 for each word/sense pair which is of interest. Each subtable 108 contains entries (W~PPIE) 123 for at least all of the words in text 111 which give useful indications whether the word/sense pair 1()5 [o which subtable 108 corresponds has a sense suitable for position 116. Each entry for a text word 121 contains an indication of the weight 119 of that word for detennining ': ~ ' ', ' . ~ - .' -.
whether word/sense pair 105 has a suitable sense.
When sense likelihood calculator 103 is calculating the likelihood that a given word/sense pair 105 is suitable for a position P 116 in text 111, sense likelihood calculator 103 begins reading words of text 111 from 50 words ahead of S position 116 and continues reading 50 words following position 116. The 100 words which contain position 116 are position 116's context 115. An important aspect of the preferred embodiment is that there are substantially more words in context 115 than there are in a line of text 111, which is presumed to contain about 10 words.
For each current word (CW) 113 read from context 115, calculator 103 determines 10 whether the word has an entry 123 in table 107, and if it does, it adds the weight specified in the entry to the weights accumulated from the words read so far. The value of the accumulated weights 119 is then the likelihood 117 that word/sense pair 105 is suitable for position 116.
Of course, in most applications of apparatus 101, the issue will be which 15 of two or more wordtsense pairs best suits context 115. To find that out, one simply uses apparatus 101 as described above for each word/sense pair in turn. The one witb the highest accumulated weights is the word/sense pair best suited to position 116. For example, if apparatus 101 were being used in the word disambiguation application, there would be a separate word/sense pair for each sense of the word to 20 be disambiguated, and the sense used to translate the word would be the sense of the word/sense pair with the highest accumulated weights.
When words are being disambiguated, it will occasionally occur that the difference between the highest and the next highest accumulated weights is not enough to permit clear disambiguation. In that case, SLC 103 may take other 25 approaches. One such approach is by analyzing the discourse to whicb text 111belongs. For purposes of the present discussion, a discourse is one or more texts which concern a single subject or a set of related subjects. Within a given discourse, ambiguous words tend to be used in a single sense. For instance, if the discourse concerns grammar, sentence used in the punishment sense will rarely occur; if the 30 discourse concerns criminal trial practice, sentence used in the ~rammatical sense will rarely occur.
A simple way of analyzing a discourse using apparatus 101 is as follows: The text belonging to the discourse is marked as such and calculator 103 stores the most suitable sense and weight for every position where there was clear 35 disambiguation for the word in question. Generally, there will be a heavy preponderance of one of the possible senses, and the preponderant sense can be u~sed J 7 ~ ~
in those situations where ~he analysis of context 115 alone did not clearly disambiguate. An even simpler, albeit less precise way, takes advantage of the fact that adjacent uses of a word tend to belong to the same discourse. In this technique, if the analysis of context l lS alone does not clearly (lisambiguate, apparatus 101 will S apply the result of the examination of a neighboring context 115 which contains the word in question and determine the sense of the word in question from its sense in the neighboring context.
In a preferred embodiment, apparatus 101 is implemented in a digital computer system. Text 111, table 107, and word/sense pair 105 are stored in the 10 computer system's data storage system and sense likelihood calculator 103 is implemented by means of a program executed by the digital computer system's processor. In some embodiments, table 107 may be in read-only memory and sense likelihood calculator may be a special-purpose processor which has been adapted for rapid referencing of table 107. Such an embodiment might, for exarnple, be useful in 15 a pocket translating device or in an electronic typewriter.
Computation of Word/Sense Pair Probability Table 107 Conceptually, word/sense pair probability table 107 is a whole set of subtables 108. There is a subtable 108 for at least every word/sense pair which is of interest for the task we are interested in, and the subtable 108 for that word/sense 20 pair contains an entry for at least every word in text 111. Moreover, what is really wanted is a wordlsense pair probability table 107 which will work with virtually any text written in a given language. In the disa nbiguation context, such a table would conceptually include a subtable 108 for each wor(Vsense pair for each ambiguous word in the given language and each subtable 108 would include entries for every25 word in the given language. Of course, optimizations are possible. For example, most words will contribute little or nothing to the disambiguation, and such words may simply be left out of the table and given a default weight.
lt is apparent that large wordtsense pair probability tables 107 can only be computed by machine. In the terrns employed in the art, the training of apparatus 30 101 must be automated. Training is in many respects the reverse of the testing jUSL
descAbed. In the disarnbiguation context, a subtable 10~ is created in table 107 for a given ambiguous word by examining contexts 115 which contain known senses of the ambiguous word to find other words in the context which can be used in testing to estimate the probable sense of the ambiguous word. For example, if the subtable 35 108 is for sentence's punishment sense, the contexts will certainly show moreoccurrences of ~he word "judge" or the word "trial" than is normal in English , .
.
language texts, and these words can be weighted accordingly in table 107 The largest problem in automating training is of course determining which sense the word being trained for has in a given context 115. Por large tables 107, it is clearly impractical for human readers to flag the word being trained for as S having one sense or another in the contexts 115 being used for training. During development of apparatus 101, two techniques have been discovered for automatically determining the sense of the word being trained for in a given context 115. One of the techniques employs bilingual bodies of text; the other employs subject matter categories such as those provided by a thesaurus.
10 Trainin~ on Bilin~ual Bodies of Text Training on bilingual bodies of text takes advantage of two facts: first, a translation of a word which is ambiguous in a first languge into a word in a second language often indicates the sense of the ambiguous word. Thus, if the English word sentence is translated peine, we know that the English word is being used in the15 punishment sense. If it is translatedphrose, we know that the grammatical sense is meant. Second, there are presently available large amounts of machine-readable text for which versions in two languages are available. One example of such a bilingual body of text is the Canadian Hansards, which is the journal in both English and French of the debates in the Canadian Houses of Parliament. In the following, the 20 English version is termed the English Hansards and the French version the French Hansards.
The preferred embodiment is trained on the Canadian Hansards for one sense of a given ambiguous English word as follows: first, statistics axe gathered for the entire body of the English Hansards. The statistics include the number of tokens 25 (words and phrases treated as words) in the English Canadian Hansards and then the number of occurrences of each word in the English Hansards. From these sta~istics the probability that a given word will occur in a 100-word context of the English Hansards is computed.
Then a conditional sample of the English Hansards is made for Ihe 30 desired sense of the given ambiguous word. This is done by finding each occurr~nce of the given ambiguous word in the English Hansards. Then the French word (or phrase) which corresponds to the occurrence is located in the French Canadian Hansards. The French word which corresponds to the occurence of the arnbiguous English word is located by aligning sentences of the English text with corresponlling 35 ones of the French text, as described in Gale, W., and K. Church, "A Program lor Aligning Sentences in Bilingual Corpora," Proceedings: 29th Annual Meeting of Ihe .
.J ~ ,J
Association for Computa~ional Linguistics, pp. 177-184, 1991. When located, the French word or phrase detennines whether that occurrence of the given word has the desired sense. If it does, the 50 words on either side of the occurrence are output to the conditional sample. It should be pointed out here that the use of a 100-word5 context in training is as important as it is in the operation of apparatus 101.
Once the conditional sample is made, it is analyzed using Bayesian techniques which will be described in detail below to determine the weight of each word in the conditional sample with regard to the probability that the given arnbiguous word has the sense used to make the conditional sample. The result of10 the analysis is a subtable 108 in word/sense pair probability table 107 for the given ambiguous word and the desired sense. FIG. 2 shows an example of the data which results from the analysis. Table 201 contains the data for two senses of the English word du~. The two senses are the tax sense and the obligation sense. Table 202 shows some of the data for the tax sense and table 203 shows some of the data for 15 the obligation sense. Each table has four columns. Word column 209 contains an entry for each word found in ~he conditional sample; Frequency column 207is the number of times the word occurs in the conditional sample; Weight column 205is the weight assigned to the word for the purposes of deterrnining the desired sense;
and Weight * frequency column 204is the product for each word of the weight and 20 the frequency.
The values in column 204 are used to decide which words should be included in subtable 108 for the word/sense pair. Those having the highest product are most indicative. The size of the subtable 108 may be reduced by using a default weight for all words whose products in column 204 are less than a predeterrnined25 value. Subtable 108 is constructed from table 202 by making an entry for each word which has not been given the default weight. The entry contains the weight computed for that word.
As is apparent from the foregoing, the creation of word sense pair probability table 107 from the Canadian Hansards can be completely automated.
30 The Canadian Hansards exist in machine-readable form, there are machine techniques available for locating the French equivalent of a given occurrence of an English word in the Canadian Hansards, and the extraction of the conditional saml~le and the computation of the weight may also be done completely by computer.
.: . .
) 73 ~
Computation oî the Weig~t In a preferred embodiment, the weight, wt, of a given vocabulary word in deterrnining the sense of a given word/sense pair is defined by the equation:
wt = log E(~la) p(word in corpus) 5 where 7~ is the conditional probability of the given vocabulary word, a is thefrequency of the given vocabulary word in the conditional sample, and E is the expectation of ~ given a.
An advantage of computing the weight as set forth above is that it interpolates between word probabilities computed within the 100-word contexts of10 the conditional sample and word probabilities computed over the entire corpus. For a word that appears fairly often within the 100-word context, we will tend to believe the local estimate and will not weight the global context very much in the interpolation. Conversely, for a word that does not appear very often in the local context, we will be much less confident in the local estimate and will tend to weight lS the global estimate somewhat rnore heavily. The key obseNation behind the method is this: the entire corpus provides a set of well measured probabilities which are of unknown relevance to the desired conditional probabililies, while the conditional set provides poor estimates of probabilities that are certainly relevant. Using probabilities from the entire corpus thus in~roduces bias, while using those from the ~0 conditional set introduce random errors. We seek to detennine the relevance of the larger corpus to the conditional sample in order to make this trade off between bias and random error.
E(7~la) is computed in a preferred embodiment by dividing the entire body of text into a conditional sample of size n for the given wordJsense pair and the 25 residual corpus (the entire corpus less the conditional sarnple) of size N n. Let a be the frequency of a given vocabulary word in the conditional sarnple, and A its frequency in the residual corpus. Either of the~se frequencies may be zero, but nol both. Let ~ represent the conditional probability of the vocabulary word. Beforeknowing the frequency of the word in either sample, we could express our ignorance 30 of the value of ~ by the uninformat~ve distribution:
B- l (1/2 ,l/2 ) ~- /2 ( ~
. .
,-. , , ~l~3 ~J7~
where B(x,y) is the E~eta function. Several variations of the method can be based on variations in the uninforrnative distribution If A additional observations out of N, relevant to the determination of 7~, were made the distribution expressing our knowledge would become S B - l (A ~ I/2 ,N - A + I/2 ) ~ % ( 1 ~
While we have A out of N observations of the word in question in the residual corpus, we do not know their relevance. Thus we set as our knowledge before observing the conditional sample the distribution:
p(~) = rB-l (A+I~2,N_A+I~2~7~A-I/2~ N-A-I/2 + (1 -r)B~I (I/2,l/2)1~-/2 (1~ '/2 10 where OSr<l is the relevance of the residual corpus to the conditional sample.
When r= 0, this gives the uninformative distribution, while if r = 1, it gives the distribution after observing the residual corpus. Another way of reading this is that with probability r we are expecting an observation in line with the residual corpus, but that with probability 1 -r we won't be surprised by any value.
The joint probability of observing a out of n instances of the word in question in the conditional sample and that the conditional probability is ~ is then [n~ ¦ B - l (A ~ I/2 ~--A + I/2 ) ~A+ a- h ( 1 ~ (1-I)B-I(I/2,1/2)~a-%(1~ )n-a-'~
We then form p(a) = Ip(~,a)d~
and p(~ a) = p(~,a)/p(a) which is then integrated to give , - ~. , : , . .
:
.
.
- 14 - 2 ~ ~ f~ r~ ~J ~
E(7~ a) = 1~P'~ a)d~
B(A~a~ll/2,N-A~n-a~l/2) + (I r) B(a~ll/2,n-a~l/2) B(A+l/2,N-A~!/2) B(l/2,1/2) rB(A~a+l/2,N-A~n-a~l/2) 1 B(a+l/2,n-a~l/2) B(A+l/2,N-A+'i2) + ( -r) B(l/2,1/2) This can be approximated in various ways, but it is practical to calcula~
it directly using the relationship B(x,y)= r(x)r(~) The parameter r, which denotes the relevance of the residual corpus to the conditional sample, can be estimated in various ways. Its basic interpretation is the fraction of words that have conditional probabilities close to their global probabilities (as estimated from the residual sample). Thus given a set of estimates 10 of conditional probabilities, one can estimate r as the fraction of them which lie within a few standard deviations of the corresponding global probabilities. Thisestimate is performed using the words which are observed in the conditional sample.
Alternatively r can be regarded as a free pararneter of the method and adjusted to produce optimal results on a specific task. Although it could be varied for each15 word, we have used r = 0.8 for all words in the sense disambiguation application, and r = 0.98 for all words in the author identification application. Further, the fact that the weight calculated by the above technique is a logarithmic value rneans that the process of multiplying the probabilities that the occurrences of the vocabulary words in the sample indicate the given word/sense pair can be done as follows: for 20 each occurrence of a vocabulary word in the context, look up the word's weight in subtable 108 for the given word/sens pair and add ~he weight to the sum of the weights of the preceding occurrences of vocabulary words.
Usin~ Cate~oFies of Meanin~ to Determine Suitability of a WordJSense Pair The use of bilingual bodies of text as described above permits the 25 automation of training, and consequently represents a significant advance. However, training on such bilingual bodies of text does have its drawbacks. First, it offers no help whatever with words that do not appear in the bilingual body of text. Such bilingual bodies of text are presently often bodies of legal or political text, and therefore have quite specialized vocabularies. Second, particularly with regard to 30 languages like English and French, which have closely-related vocabulanes, the .
'` ' ~' , ~
.
: - ~
.
- 15- '~3~'3~
translation of an ambiguous word may itself be ambiguous. An example here is theEnglish word interest. Its French translation, interet, has substantially all of the senses of the English equivalent. The latter problem becomes less difficult withmultilingual bodies of text. Disambiguation should be particularly easy if one of the S languages in which the multilingual body of text exists is non-Indo-European.
These problems can be overcome by using the categories of meaning to which vocabulary words belong to determine the suitability of a word/sense pair for a context. Thesauruses and dictionaries often divide the words they give equivalents for or otherwise define into categories. For example, Chapman, Robert~ Roget's 10 International Tfiesaurus (Fourth dition), Harper and Row, New York, 1977, divides the words for which it gives equivalents into 1042 subject matter categories.
These categories can be used to automatically train on any body of English text and the relationships between vocabulary words in the text and subject matter categories can be used in the testin~g phase to determine suitable word/sense pairs for a context 15 as described above with regard to bilingual bodies of text.
The method of using categories of meaning is based on the following general observations: 1) Different classes of words, such as ANIMALS or MAC~IINES
tend to appear in recognizably different con~exts. 2) Different word senses tend to belong to different conceptual classes (cranc can be an ANlMAL or a MACHINE). 3)20 If one can build a context discriminator for the conceptual classes, one has effectively built a context discAminator for the word senses that are members ofthose classes. Furthermore, the context indicators for a Roget category (e.g. gear, piston and engine for the category TOOLS/MACHINERY) will also tend to be contextindicators for the members of that category (such as the machinery sense of crane).
We attempt to identify, weight and utilize these indicative words as follows. For each of the 1042 Roget Categories: Collect contexts which are representative of the Roget category Identify salient words in the collective con~ext and weight the words appropAately, and Use the resulting weigh-ts in the testingphase.
30 Step 1: Collect Context which is lRepresentaffve Or the Ro~et ca~ory The goal of this step is to collect a set of vocabulary words of the body of text upon which training is being done which are typically found in contexts which contain at least one of the words listed in a given Roget cateory. To do ihis, we make a conditional sample for the category. The conditional sample consists Or 35 the 100 surrounding words for each occurrence of each member of the given category in the corpus. FIG. 3 shows a sample set 301 of portions of the condi~ional .
- : ' .
6~ s sample for words in the catego~y rooLslMAc~ Ry (348). The complete conditional sample contains 30,924 lines, selected from the particula~ body of text used fortraining in the preferred embocliment, the 10 million word, June 1991 electronicversion of Grolier's Encyclopedia.
S Ideally, the conditional sample would only include references to the given category. But in practice it will unavoidably include spurious examples since many of the words are polysemous (such as drill at 303 and crane at 305 of FIG. 3).
Polysemy is dealt with in the preferred embodiment by weighting the vocabulary words in the conditional sample to minimize the effect. The weighting also serves to 10 malce the sample representative of all tools and machinery, not just the morecommon ones. The weighting is done as follows: If a word listed in a Roget's category such as drill occurs k times in the corpus, all vocabulary words in thecontext of drill contribute weight 1 /k to the frequency values for those vocabulary words in the conditional sample.
15 While the level of noise introduced through polysemy is substantial, it is tolerable because those senses of the words in the given Roget's category which do not belong to the category are distributed through the 1041 other categories, and consequently, there is no strong association between vocabulary words associatedwith the senses of the words which do not belong to the category and any one 20 category, whereas all of the vocabulary words which are associated with the senses of the words which do belong to the category are associated with that category. Only if several words listed in the given Roget's category had secondary senses in a single different Roget's category would context typical for the other category appear significant in this context.
Despite its f~aws9 the conditional sample serves as a representative, albeit noisy, sample of the typical context of TOOLSIMACH~RY in ~rolier's encyclopedia.
Step 7: Idenff~y salient words in the conditiollal sample, and weixht appropriately Intuitively, a salient word is one which appears significantly more often in the conditional sample than at other points in the body of text, and hence is a better than average indicator ror the category. We formalize this with a mutual-inforrnation like estimate: Pr(wlRCat) / Pr(w), the probability of a vocabulary word w appearing in the conditional sample for a Roget's category divided by the35 overall probability of the vocabulary word appearing in the body of text.
1 7 ~ ~ ~j 5 j , ~ ~ 5 ~
Pr(wlRCat) can be computed using the same techniques as were used to compute the weight of the vocabulary words in the case of the bilingual body of text, The only substantial difference is the u~se of 1 /k as described above to compwte the frequency of a vocabulary word in the conditional sample. FIG. 4 shows the most S important salient words for Roget categories 348 and 414, TOOLS/MA(: HINERY
and ANIMAL,INSECT respectively. I'he numbers in parentheses are the log of the salience, that is, the weight 119, for each of the vocabulary words for the given category. The vocabulary words which appear in categories 403 and 405 are those which are most important for determining whether a word/sense pair belongs to a 10 given context. As before, importance is determined by multiplying the vocabulary word's weight by its frequency in the conditional sample.
It is worth noting that the salient vocabulary words for a Roget's category includes many words which are not listed in the category. What the salient vocabulary words are is vocabulary words which are likely to co-occur with words15 which are listed in the given category. The list of salient vocabulary words for a category typically contains over 3000 words, and is far richer than any list which can be derived from a dictionary definition.
Step 3: Use the resulffn~ wei~hts in the tes~nR phase In the testing phase, the Roget's catego~y indicated by the vocabulary 20 words which surround a position in a text can be used ~o determine whether a word/sense pair is proper for the position. If the sense of the word/sense pair belongs to the sarne Roget's category as that indicated by one of the salient vocabulary words, then there is evidence that the word/sense pair is proper for the position. If several salient vocabulary words appear, the evidence is compounded.
25 As shown below, the testing phase works as described with regard to FIG. 1: having built word/sense pair probability tables 107 for all of the word/sense pairs we are interested in using the techniques just described to determine the weights of vocabulary words for indicating the various word/sense pairs, the probability of a given word/sense pair for a position in a 100-word context surrounding the posi~ion 30 is deterrnined by adding the weights of all of the words in the context. When the apparatus is being used to disambiguate, the word/sense pairs used are those for lhe word and its various senses. The word/sense pair with the largest total of weights is the pair which is best suited for the context.
~. - . -. .
- : ~
'~ 'fi '~
Por example, the word crane appears 74 times in (~roliers; 36 occurrences refer to the animal sense and 38 refer to the heavy machinery sense.The system correctly classified all but one of the machinery senses, yielding 99%
overall accuracy . The one misclassified case had a low ~score for all models, S indicating a lack of confidence in any classification.
In a preferred embodiment, the senses of a word are defined to be the Roget's categories to which the word belongs. Although it is often useful to reslrict the search in this way, the restriction does sometimes lead to trouble, especially when there are gaps in the thesaurus. For example, the category AMUSEMENT (#
10 876) lists a number of card playing terms, but for some reason, the word suit iS not included in this list. As it happens, the Grolier's Encyclopedia contains 54 instances of ~he card-playing sense of suit, all of which are mislabeled if the search is limited to just those categories of suit that are listed in Roget's. However, if we open up the search to consider all 1042 categories, then we find that all 54 instances of the card-15 playing sense of suit are correctly labeled as AMUSEMENT, and moreover, the scoreis large in all 54 instances, indicating great confidence in the assignment. It is possible that the unrestricted search mode might be a good way to attempt to fill in omissions in the thesaurus. In any case, when suit is added to the AMUSEMENT
category, overall accuracy improves from 68% to 92%.
20 Conclusion The foregoing Detailed Description has disclosed to those of ordinary skill in the text analysis arts how a context larger than a single line may be advantageously used to determine whether a word/sense pair is proper for a location in the context. It has also dislcosed how Bayesian techniques may be used to give the vocabulary words 25 of a context weights with regard to a given wordtsense pair and how these weights may be used to determine whether the word/sense pair is proper for the location. It has further disclosed how the properties of ambiguous words in discourses can beused in disambiguating the words. The Detailed Description has moreover shown the importance of large contexts and Bayesian techniques for training apparatus for 30 determining whether a word/sense pair is proper for a location and has shown how training may be automated using bilingual bodies of texts or categories of word meanings.
As set forth in the Detailed Description, the above techniques may be used separately or may be combined to particular advantage. Moreover, one training 35 technique may remedy the shortcomings of another. For example, as previously pointed out, the French Hansards cannot be used to train for the various senses of the ;, ' ' ' .
word interest; however, the method of training on bilingual texts coald be supplemented in this case by the use of the Roget's ca~egoAes.
Many variations on the techniques disclosed herein will be apparent to those of ordinary skill in the text analysis arts. For example, only two methods have S been disclosed herein for producing conditional samples, but the techniques ofBayesian analysis described herein may be applied to conditional samples produced in any fashion whatever. Similarly, while the contexts disclosed herein have a length of 100 words, contexts of any length substantially longer than a single line will have some of the advantages of those of 100 words. Finally, many techniques other than 10 those disclosed herein can be employed which take advantage of the fact that a given discourse tends to use an ambiguous word in only one sense. Additionally, those skilled in the an will know many ways of implementing word/sense pair probability table 107 and of implementing sense likelihood calculator 103. That being the case, foregoing Detailed Description is to be regarded as being in all15 respects illustrative and exemplary, and not restrictive, and the scope of the inventions disclosed herein is to be determined solely by the following claims as interpreted in light of the Specification and according to the doctrine of equivalents.
When sense likelihood calculator 103 is calculating the likelihood that a given word/sense pair 105 is suitable for a position P 116 in text 111, sense likelihood calculator 103 begins reading words of text 111 from 50 words ahead of S position 116 and continues reading 50 words following position 116. The 100 words which contain position 116 are position 116's context 115. An important aspect of the preferred embodiment is that there are substantially more words in context 115 than there are in a line of text 111, which is presumed to contain about 10 words.
For each current word (CW) 113 read from context 115, calculator 103 determines 10 whether the word has an entry 123 in table 107, and if it does, it adds the weight specified in the entry to the weights accumulated from the words read so far. The value of the accumulated weights 119 is then the likelihood 117 that word/sense pair 105 is suitable for position 116.
Of course, in most applications of apparatus 101, the issue will be which 15 of two or more wordtsense pairs best suits context 115. To find that out, one simply uses apparatus 101 as described above for each word/sense pair in turn. The one witb the highest accumulated weights is the word/sense pair best suited to position 116. For example, if apparatus 101 were being used in the word disambiguation application, there would be a separate word/sense pair for each sense of the word to 20 be disambiguated, and the sense used to translate the word would be the sense of the word/sense pair with the highest accumulated weights.
When words are being disambiguated, it will occasionally occur that the difference between the highest and the next highest accumulated weights is not enough to permit clear disambiguation. In that case, SLC 103 may take other 25 approaches. One such approach is by analyzing the discourse to whicb text 111belongs. For purposes of the present discussion, a discourse is one or more texts which concern a single subject or a set of related subjects. Within a given discourse, ambiguous words tend to be used in a single sense. For instance, if the discourse concerns grammar, sentence used in the punishment sense will rarely occur; if the 30 discourse concerns criminal trial practice, sentence used in the ~rammatical sense will rarely occur.
A simple way of analyzing a discourse using apparatus 101 is as follows: The text belonging to the discourse is marked as such and calculator 103 stores the most suitable sense and weight for every position where there was clear 35 disambiguation for the word in question. Generally, there will be a heavy preponderance of one of the possible senses, and the preponderant sense can be u~sed J 7 ~ ~
in those situations where ~he analysis of context 115 alone did not clearly disambiguate. An even simpler, albeit less precise way, takes advantage of the fact that adjacent uses of a word tend to belong to the same discourse. In this technique, if the analysis of context l lS alone does not clearly (lisambiguate, apparatus 101 will S apply the result of the examination of a neighboring context 115 which contains the word in question and determine the sense of the word in question from its sense in the neighboring context.
In a preferred embodiment, apparatus 101 is implemented in a digital computer system. Text 111, table 107, and word/sense pair 105 are stored in the 10 computer system's data storage system and sense likelihood calculator 103 is implemented by means of a program executed by the digital computer system's processor. In some embodiments, table 107 may be in read-only memory and sense likelihood calculator may be a special-purpose processor which has been adapted for rapid referencing of table 107. Such an embodiment might, for exarnple, be useful in 15 a pocket translating device or in an electronic typewriter.
Computation of Word/Sense Pair Probability Table 107 Conceptually, word/sense pair probability table 107 is a whole set of subtables 108. There is a subtable 108 for at least every word/sense pair which is of interest for the task we are interested in, and the subtable 108 for that word/sense 20 pair contains an entry for at least every word in text 111. Moreover, what is really wanted is a wordlsense pair probability table 107 which will work with virtually any text written in a given language. In the disa nbiguation context, such a table would conceptually include a subtable 108 for each wor(Vsense pair for each ambiguous word in the given language and each subtable 108 would include entries for every25 word in the given language. Of course, optimizations are possible. For example, most words will contribute little or nothing to the disambiguation, and such words may simply be left out of the table and given a default weight.
lt is apparent that large wordtsense pair probability tables 107 can only be computed by machine. In the terrns employed in the art, the training of apparatus 30 101 must be automated. Training is in many respects the reverse of the testing jUSL
descAbed. In the disarnbiguation context, a subtable 10~ is created in table 107 for a given ambiguous word by examining contexts 115 which contain known senses of the ambiguous word to find other words in the context which can be used in testing to estimate the probable sense of the ambiguous word. For example, if the subtable 35 108 is for sentence's punishment sense, the contexts will certainly show moreoccurrences of ~he word "judge" or the word "trial" than is normal in English , .
.
language texts, and these words can be weighted accordingly in table 107 The largest problem in automating training is of course determining which sense the word being trained for has in a given context 115. Por large tables 107, it is clearly impractical for human readers to flag the word being trained for as S having one sense or another in the contexts 115 being used for training. During development of apparatus 101, two techniques have been discovered for automatically determining the sense of the word being trained for in a given context 115. One of the techniques employs bilingual bodies of text; the other employs subject matter categories such as those provided by a thesaurus.
10 Trainin~ on Bilin~ual Bodies of Text Training on bilingual bodies of text takes advantage of two facts: first, a translation of a word which is ambiguous in a first languge into a word in a second language often indicates the sense of the ambiguous word. Thus, if the English word sentence is translated peine, we know that the English word is being used in the15 punishment sense. If it is translatedphrose, we know that the grammatical sense is meant. Second, there are presently available large amounts of machine-readable text for which versions in two languages are available. One example of such a bilingual body of text is the Canadian Hansards, which is the journal in both English and French of the debates in the Canadian Houses of Parliament. In the following, the 20 English version is termed the English Hansards and the French version the French Hansards.
The preferred embodiment is trained on the Canadian Hansards for one sense of a given ambiguous English word as follows: first, statistics axe gathered for the entire body of the English Hansards. The statistics include the number of tokens 25 (words and phrases treated as words) in the English Canadian Hansards and then the number of occurrences of each word in the English Hansards. From these sta~istics the probability that a given word will occur in a 100-word context of the English Hansards is computed.
Then a conditional sample of the English Hansards is made for Ihe 30 desired sense of the given ambiguous word. This is done by finding each occurr~nce of the given ambiguous word in the English Hansards. Then the French word (or phrase) which corresponds to the occurrence is located in the French Canadian Hansards. The French word which corresponds to the occurence of the arnbiguous English word is located by aligning sentences of the English text with corresponlling 35 ones of the French text, as described in Gale, W., and K. Church, "A Program lor Aligning Sentences in Bilingual Corpora," Proceedings: 29th Annual Meeting of Ihe .
.J ~ ,J
Association for Computa~ional Linguistics, pp. 177-184, 1991. When located, the French word or phrase detennines whether that occurrence of the given word has the desired sense. If it does, the 50 words on either side of the occurrence are output to the conditional sample. It should be pointed out here that the use of a 100-word5 context in training is as important as it is in the operation of apparatus 101.
Once the conditional sample is made, it is analyzed using Bayesian techniques which will be described in detail below to determine the weight of each word in the conditional sample with regard to the probability that the given arnbiguous word has the sense used to make the conditional sample. The result of10 the analysis is a subtable 108 in word/sense pair probability table 107 for the given ambiguous word and the desired sense. FIG. 2 shows an example of the data which results from the analysis. Table 201 contains the data for two senses of the English word du~. The two senses are the tax sense and the obligation sense. Table 202 shows some of the data for the tax sense and table 203 shows some of the data for 15 the obligation sense. Each table has four columns. Word column 209 contains an entry for each word found in ~he conditional sample; Frequency column 207is the number of times the word occurs in the conditional sample; Weight column 205is the weight assigned to the word for the purposes of deterrnining the desired sense;
and Weight * frequency column 204is the product for each word of the weight and 20 the frequency.
The values in column 204 are used to decide which words should be included in subtable 108 for the word/sense pair. Those having the highest product are most indicative. The size of the subtable 108 may be reduced by using a default weight for all words whose products in column 204 are less than a predeterrnined25 value. Subtable 108 is constructed from table 202 by making an entry for each word which has not been given the default weight. The entry contains the weight computed for that word.
As is apparent from the foregoing, the creation of word sense pair probability table 107 from the Canadian Hansards can be completely automated.
30 The Canadian Hansards exist in machine-readable form, there are machine techniques available for locating the French equivalent of a given occurrence of an English word in the Canadian Hansards, and the extraction of the conditional saml~le and the computation of the weight may also be done completely by computer.
.: . .
) 73 ~
Computation oî the Weig~t In a preferred embodiment, the weight, wt, of a given vocabulary word in deterrnining the sense of a given word/sense pair is defined by the equation:
wt = log E(~la) p(word in corpus) 5 where 7~ is the conditional probability of the given vocabulary word, a is thefrequency of the given vocabulary word in the conditional sample, and E is the expectation of ~ given a.
An advantage of computing the weight as set forth above is that it interpolates between word probabilities computed within the 100-word contexts of10 the conditional sample and word probabilities computed over the entire corpus. For a word that appears fairly often within the 100-word context, we will tend to believe the local estimate and will not weight the global context very much in the interpolation. Conversely, for a word that does not appear very often in the local context, we will be much less confident in the local estimate and will tend to weight lS the global estimate somewhat rnore heavily. The key obseNation behind the method is this: the entire corpus provides a set of well measured probabilities which are of unknown relevance to the desired conditional probabililies, while the conditional set provides poor estimates of probabilities that are certainly relevant. Using probabilities from the entire corpus thus in~roduces bias, while using those from the ~0 conditional set introduce random errors. We seek to detennine the relevance of the larger corpus to the conditional sample in order to make this trade off between bias and random error.
E(7~la) is computed in a preferred embodiment by dividing the entire body of text into a conditional sample of size n for the given wordJsense pair and the 25 residual corpus (the entire corpus less the conditional sarnple) of size N n. Let a be the frequency of a given vocabulary word in the conditional sarnple, and A its frequency in the residual corpus. Either of the~se frequencies may be zero, but nol both. Let ~ represent the conditional probability of the vocabulary word. Beforeknowing the frequency of the word in either sample, we could express our ignorance 30 of the value of ~ by the uninformat~ve distribution:
B- l (1/2 ,l/2 ) ~- /2 ( ~
. .
,-. , , ~l~3 ~J7~
where B(x,y) is the E~eta function. Several variations of the method can be based on variations in the uninforrnative distribution If A additional observations out of N, relevant to the determination of 7~, were made the distribution expressing our knowledge would become S B - l (A ~ I/2 ,N - A + I/2 ) ~ % ( 1 ~
While we have A out of N observations of the word in question in the residual corpus, we do not know their relevance. Thus we set as our knowledge before observing the conditional sample the distribution:
p(~) = rB-l (A+I~2,N_A+I~2~7~A-I/2~ N-A-I/2 + (1 -r)B~I (I/2,l/2)1~-/2 (1~ '/2 10 where OSr<l is the relevance of the residual corpus to the conditional sample.
When r= 0, this gives the uninformative distribution, while if r = 1, it gives the distribution after observing the residual corpus. Another way of reading this is that with probability r we are expecting an observation in line with the residual corpus, but that with probability 1 -r we won't be surprised by any value.
The joint probability of observing a out of n instances of the word in question in the conditional sample and that the conditional probability is ~ is then [n~ ¦ B - l (A ~ I/2 ~--A + I/2 ) ~A+ a- h ( 1 ~ (1-I)B-I(I/2,1/2)~a-%(1~ )n-a-'~
We then form p(a) = Ip(~,a)d~
and p(~ a) = p(~,a)/p(a) which is then integrated to give , - ~. , : , . .
:
.
.
- 14 - 2 ~ ~ f~ r~ ~J ~
E(7~ a) = 1~P'~ a)d~
B(A~a~ll/2,N-A~n-a~l/2) + (I r) B(a~ll/2,n-a~l/2) B(A+l/2,N-A~!/2) B(l/2,1/2) rB(A~a+l/2,N-A~n-a~l/2) 1 B(a+l/2,n-a~l/2) B(A+l/2,N-A+'i2) + ( -r) B(l/2,1/2) This can be approximated in various ways, but it is practical to calcula~
it directly using the relationship B(x,y)= r(x)r(~) The parameter r, which denotes the relevance of the residual corpus to the conditional sample, can be estimated in various ways. Its basic interpretation is the fraction of words that have conditional probabilities close to their global probabilities (as estimated from the residual sample). Thus given a set of estimates 10 of conditional probabilities, one can estimate r as the fraction of them which lie within a few standard deviations of the corresponding global probabilities. Thisestimate is performed using the words which are observed in the conditional sample.
Alternatively r can be regarded as a free pararneter of the method and adjusted to produce optimal results on a specific task. Although it could be varied for each15 word, we have used r = 0.8 for all words in the sense disambiguation application, and r = 0.98 for all words in the author identification application. Further, the fact that the weight calculated by the above technique is a logarithmic value rneans that the process of multiplying the probabilities that the occurrences of the vocabulary words in the sample indicate the given word/sense pair can be done as follows: for 20 each occurrence of a vocabulary word in the context, look up the word's weight in subtable 108 for the given word/sens pair and add ~he weight to the sum of the weights of the preceding occurrences of vocabulary words.
Usin~ Cate~oFies of Meanin~ to Determine Suitability of a WordJSense Pair The use of bilingual bodies of text as described above permits the 25 automation of training, and consequently represents a significant advance. However, training on such bilingual bodies of text does have its drawbacks. First, it offers no help whatever with words that do not appear in the bilingual body of text. Such bilingual bodies of text are presently often bodies of legal or political text, and therefore have quite specialized vocabularies. Second, particularly with regard to 30 languages like English and French, which have closely-related vocabulanes, the .
'` ' ~' , ~
.
: - ~
.
- 15- '~3~'3~
translation of an ambiguous word may itself be ambiguous. An example here is theEnglish word interest. Its French translation, interet, has substantially all of the senses of the English equivalent. The latter problem becomes less difficult withmultilingual bodies of text. Disambiguation should be particularly easy if one of the S languages in which the multilingual body of text exists is non-Indo-European.
These problems can be overcome by using the categories of meaning to which vocabulary words belong to determine the suitability of a word/sense pair for a context. Thesauruses and dictionaries often divide the words they give equivalents for or otherwise define into categories. For example, Chapman, Robert~ Roget's 10 International Tfiesaurus (Fourth dition), Harper and Row, New York, 1977, divides the words for which it gives equivalents into 1042 subject matter categories.
These categories can be used to automatically train on any body of English text and the relationships between vocabulary words in the text and subject matter categories can be used in the testin~g phase to determine suitable word/sense pairs for a context 15 as described above with regard to bilingual bodies of text.
The method of using categories of meaning is based on the following general observations: 1) Different classes of words, such as ANIMALS or MAC~IINES
tend to appear in recognizably different con~exts. 2) Different word senses tend to belong to different conceptual classes (cranc can be an ANlMAL or a MACHINE). 3)20 If one can build a context discriminator for the conceptual classes, one has effectively built a context discAminator for the word senses that are members ofthose classes. Furthermore, the context indicators for a Roget category (e.g. gear, piston and engine for the category TOOLS/MACHINERY) will also tend to be contextindicators for the members of that category (such as the machinery sense of crane).
We attempt to identify, weight and utilize these indicative words as follows. For each of the 1042 Roget Categories: Collect contexts which are representative of the Roget category Identify salient words in the collective con~ext and weight the words appropAately, and Use the resulting weigh-ts in the testingphase.
30 Step 1: Collect Context which is lRepresentaffve Or the Ro~et ca~ory The goal of this step is to collect a set of vocabulary words of the body of text upon which training is being done which are typically found in contexts which contain at least one of the words listed in a given Roget cateory. To do ihis, we make a conditional sample for the category. The conditional sample consists Or 35 the 100 surrounding words for each occurrence of each member of the given category in the corpus. FIG. 3 shows a sample set 301 of portions of the condi~ional .
- : ' .
6~ s sample for words in the catego~y rooLslMAc~ Ry (348). The complete conditional sample contains 30,924 lines, selected from the particula~ body of text used fortraining in the preferred embocliment, the 10 million word, June 1991 electronicversion of Grolier's Encyclopedia.
S Ideally, the conditional sample would only include references to the given category. But in practice it will unavoidably include spurious examples since many of the words are polysemous (such as drill at 303 and crane at 305 of FIG. 3).
Polysemy is dealt with in the preferred embodiment by weighting the vocabulary words in the conditional sample to minimize the effect. The weighting also serves to 10 malce the sample representative of all tools and machinery, not just the morecommon ones. The weighting is done as follows: If a word listed in a Roget's category such as drill occurs k times in the corpus, all vocabulary words in thecontext of drill contribute weight 1 /k to the frequency values for those vocabulary words in the conditional sample.
15 While the level of noise introduced through polysemy is substantial, it is tolerable because those senses of the words in the given Roget's category which do not belong to the category are distributed through the 1041 other categories, and consequently, there is no strong association between vocabulary words associatedwith the senses of the words which do not belong to the category and any one 20 category, whereas all of the vocabulary words which are associated with the senses of the words which do belong to the category are associated with that category. Only if several words listed in the given Roget's category had secondary senses in a single different Roget's category would context typical for the other category appear significant in this context.
Despite its f~aws9 the conditional sample serves as a representative, albeit noisy, sample of the typical context of TOOLSIMACH~RY in ~rolier's encyclopedia.
Step 7: Idenff~y salient words in the conditiollal sample, and weixht appropriately Intuitively, a salient word is one which appears significantly more often in the conditional sample than at other points in the body of text, and hence is a better than average indicator ror the category. We formalize this with a mutual-inforrnation like estimate: Pr(wlRCat) / Pr(w), the probability of a vocabulary word w appearing in the conditional sample for a Roget's category divided by the35 overall probability of the vocabulary word appearing in the body of text.
1 7 ~ ~ ~j 5 j , ~ ~ 5 ~
Pr(wlRCat) can be computed using the same techniques as were used to compute the weight of the vocabulary words in the case of the bilingual body of text, The only substantial difference is the u~se of 1 /k as described above to compwte the frequency of a vocabulary word in the conditional sample. FIG. 4 shows the most S important salient words for Roget categories 348 and 414, TOOLS/MA(: HINERY
and ANIMAL,INSECT respectively. I'he numbers in parentheses are the log of the salience, that is, the weight 119, for each of the vocabulary words for the given category. The vocabulary words which appear in categories 403 and 405 are those which are most important for determining whether a word/sense pair belongs to a 10 given context. As before, importance is determined by multiplying the vocabulary word's weight by its frequency in the conditional sample.
It is worth noting that the salient vocabulary words for a Roget's category includes many words which are not listed in the category. What the salient vocabulary words are is vocabulary words which are likely to co-occur with words15 which are listed in the given category. The list of salient vocabulary words for a category typically contains over 3000 words, and is far richer than any list which can be derived from a dictionary definition.
Step 3: Use the resulffn~ wei~hts in the tes~nR phase In the testing phase, the Roget's catego~y indicated by the vocabulary 20 words which surround a position in a text can be used ~o determine whether a word/sense pair is proper for the position. If the sense of the word/sense pair belongs to the sarne Roget's category as that indicated by one of the salient vocabulary words, then there is evidence that the word/sense pair is proper for the position. If several salient vocabulary words appear, the evidence is compounded.
25 As shown below, the testing phase works as described with regard to FIG. 1: having built word/sense pair probability tables 107 for all of the word/sense pairs we are interested in using the techniques just described to determine the weights of vocabulary words for indicating the various word/sense pairs, the probability of a given word/sense pair for a position in a 100-word context surrounding the posi~ion 30 is deterrnined by adding the weights of all of the words in the context. When the apparatus is being used to disambiguate, the word/sense pairs used are those for lhe word and its various senses. The word/sense pair with the largest total of weights is the pair which is best suited for the context.
~. - . -. .
- : ~
'~ 'fi '~
Por example, the word crane appears 74 times in (~roliers; 36 occurrences refer to the animal sense and 38 refer to the heavy machinery sense.The system correctly classified all but one of the machinery senses, yielding 99%
overall accuracy . The one misclassified case had a low ~score for all models, S indicating a lack of confidence in any classification.
In a preferred embodiment, the senses of a word are defined to be the Roget's categories to which the word belongs. Although it is often useful to reslrict the search in this way, the restriction does sometimes lead to trouble, especially when there are gaps in the thesaurus. For example, the category AMUSEMENT (#
10 876) lists a number of card playing terms, but for some reason, the word suit iS not included in this list. As it happens, the Grolier's Encyclopedia contains 54 instances of ~he card-playing sense of suit, all of which are mislabeled if the search is limited to just those categories of suit that are listed in Roget's. However, if we open up the search to consider all 1042 categories, then we find that all 54 instances of the card-15 playing sense of suit are correctly labeled as AMUSEMENT, and moreover, the scoreis large in all 54 instances, indicating great confidence in the assignment. It is possible that the unrestricted search mode might be a good way to attempt to fill in omissions in the thesaurus. In any case, when suit is added to the AMUSEMENT
category, overall accuracy improves from 68% to 92%.
20 Conclusion The foregoing Detailed Description has disclosed to those of ordinary skill in the text analysis arts how a context larger than a single line may be advantageously used to determine whether a word/sense pair is proper for a location in the context. It has also dislcosed how Bayesian techniques may be used to give the vocabulary words 25 of a context weights with regard to a given wordtsense pair and how these weights may be used to determine whether the word/sense pair is proper for the location. It has further disclosed how the properties of ambiguous words in discourses can beused in disambiguating the words. The Detailed Description has moreover shown the importance of large contexts and Bayesian techniques for training apparatus for 30 determining whether a word/sense pair is proper for a location and has shown how training may be automated using bilingual bodies of texts or categories of word meanings.
As set forth in the Detailed Description, the above techniques may be used separately or may be combined to particular advantage. Moreover, one training 35 technique may remedy the shortcomings of another. For example, as previously pointed out, the French Hansards cannot be used to train for the various senses of the ;, ' ' ' .
word interest; however, the method of training on bilingual texts coald be supplemented in this case by the use of the Roget's ca~egoAes.
Many variations on the techniques disclosed herein will be apparent to those of ordinary skill in the text analysis arts. For example, only two methods have S been disclosed herein for producing conditional samples, but the techniques ofBayesian analysis described herein may be applied to conditional samples produced in any fashion whatever. Similarly, while the contexts disclosed herein have a length of 100 words, contexts of any length substantially longer than a single line will have some of the advantages of those of 100 words. Finally, many techniques other than 10 those disclosed herein can be employed which take advantage of the fact that a given discourse tends to use an ambiguous word in only one sense. Additionally, those skilled in the an will know many ways of implementing word/sense pair probability table 107 and of implementing sense likelihood calculator 103. That being the case, foregoing Detailed Description is to be regarded as being in all15 respects illustrative and exemplary, and not restrictive, and the scope of the inventions disclosed herein is to be determined solely by the following claims as interpreted in light of the Specification and according to the doctrine of equivalents.
Claims (18)
1. A method of automatically determining that a word/sense pair has a sense which suits a given position in a text, the method comprising the steps ofdetermining a sequence of words in the text which includes the given position and is substantially longer than a single line of the text; and making a determination of whether the word/sense pair has the suitable sense by automatically analyzing the sequence.
2. The method set forth in claim 1 wherein:
the step of determining a sequence of words determines a sequence whose length is in the neighborhood of 100 words.
the step of determining a sequence of words determines a sequence whose length is in the neighborhood of 100 words.
3. The method set forth in claim 1 further including the steps of:
determining whether the determination indicates sufficiently strongly whether the word/sense pair has the suitable sense; and if the determination does not do so, making a final determination of the suitable sense by comparing the determination of the suitable sense with anotherdetermination of the suitable sense at a neighboring position.
determining whether the determination indicates sufficiently strongly whether the word/sense pair has the suitable sense; and if the determination does not do so, making a final determination of the suitable sense by comparing the determination of the suitable sense with anotherdetermination of the suitable sense at a neighboring position.
4. The method set forth in claim 1 wherein:
in the step of making a determination, the sequence is automatically analyzed by employing a Bayesian discrimination technique involving the words inthe sequence and the sense of the word-sense pair to determine the probability that the word/sense pair has a sense which suits the given position.
in the step of making a determination, the sequence is automatically analyzed by employing a Bayesian discrimination technique involving the words inthe sequence and the sense of the word-sense pair to determine the probability that the word/sense pair has a sense which suits the given position.
5. The method set forth in claim 1 wherein:
there is a plurality of the word/sense pairs;
the step of making a determination is performed for each of the word/sense pairs; and the method further includes the step of selecting the word/sense pair which is most suitable for the given position.
there is a plurality of the word/sense pairs;
the step of making a determination is performed for each of the word/sense pairs; and the method further includes the step of selecting the word/sense pair which is most suitable for the given position.
6. The method set forth in claim 5 wherein:
the step of selecting the word/sense pair includes the step of comparing the determination with another determination of the most suitable sense at a neighboring position.
the step of selecting the word/sense pair includes the step of comparing the determination with another determination of the most suitable sense at a neighboring position.
7. The method set forth in claim 1 wherein the step of making the determination includes the steps of:
for words in the sequence, obtaining weights from a table which indicates the weight of certain words for determining whether the word/sense pair has the suitable sense and summing the weights to determine a likelihood that the word/sense pair is suitable for the given position.
for words in the sequence, obtaining weights from a table which indicates the weight of certain words for determining whether the word/sense pair has the suitable sense and summing the weights to determine a likelihood that the word/sense pair is suitable for the given position.
8. The method set forth in claim 7 wherein:
there is a plurality of the word/sense pairs;
the steps of obtaining the weights and summing the weights are performed for each word/sense pair; and the method includes the further step of selecting the word/sense pair having the summed weights which indicate the greatest likelihood that the word/sense pair is suitable for the given position.
there is a plurality of the word/sense pairs;
the steps of obtaining the weights and summing the weights are performed for each word/sense pair; and the method includes the further step of selecting the word/sense pair having the summed weights which indicate the greatest likelihood that the word/sense pair is suitable for the given position.
9. The method set forth in claim 8 wherein the step of selecting the word/sense pair includes the step of:
comparing the selected word/sense pair with another determination of the most suitable sense at a neighboring position.
comparing the selected word/sense pair with another determination of the most suitable sense at a neighboring position.
10. The method of any of claims 1-9 wherein there is an occurrence of the word in the word/sense pair at the given position.
11. Apparatus for determining that a word/sense pair has a sense which suits a given position in a text, the apparatus comprising:
means for obtaining a sequence of words in the text which includes the given position and is substantially longer than a single line of the text; and means for analyzing the sequence to determine whether the word/sense pair has the suitable sense.
means for obtaining a sequence of words in the text which includes the given position and is substantially longer than a single line of the text; and means for analyzing the sequence to determine whether the word/sense pair has the suitable sense.
12. The apparatus set forth in claim 11 wherein:
the means for analyzing the sequence further includes a table which indicates the weights of certain words in the text for determining whether the word/sense pair has the suitable sense and the means for analyzing the sequence analyzes the sequence by obtaining weights of words in the sequence from the table and summing the weights of those words to determine a likelihood that the word/sense pair is suitable for the given position.
the means for analyzing the sequence further includes a table which indicates the weights of certain words in the text for determining whether the word/sense pair has the suitable sense and the means for analyzing the sequence analyzes the sequence by obtaining weights of words in the sequence from the table and summing the weights of those words to determine a likelihood that the word/sense pair is suitable for the given position.
13. A method of making a probability table for use in apparatus for determining that a word/sense pair has a sense which suits a given position in a text, the method of making the table comprising the steps of:
making a conditional sample of the text corpus which includes contexts which are semantically related to the sense of the word/sense pair;
employing a Bayesian technique to determine for each word which occurs in the conditional sample a weight of that word in the conditional sample with regard to the probability that the word of the word/sense pair has the sense of the word/sense pair; and making a table entry including the weight of the word for each of the occurring words which has more than a given weight.
making a conditional sample of the text corpus which includes contexts which are semantically related to the sense of the word/sense pair;
employing a Bayesian technique to determine for each word which occurs in the conditional sample a weight of that word in the conditional sample with regard to the probability that the word of the word/sense pair has the sense of the word/sense pair; and making a table entry including the weight of the word for each of the occurring words which has more than a given weight.
14. The method set forth in claim 13 wherein:
in the step of employing the Bayesian technique, the weight, wt for each word which occurs in the conditional sample is determined by the equation where .pi. is the conditional probability of the word, a is the frequency of the word in the conditional sample, and E is the expectation of .pi. given a.
in the step of employing the Bayesian technique, the weight, wt for each word which occurs in the conditional sample is determined by the equation where .pi. is the conditional probability of the word, a is the frequency of the word in the conditional sample, and E is the expectation of .pi. given a.
15. The method set forth in claim 13 wherein:
there exists a translation of the text corpus from which the conditional sample is made into another language; and in the step of making a conditional sample, a determination that the word of the word/sense pair is used in the sense of the word/sense pair is made with reference to the corresponding word in the translation.
there exists a translation of the text corpus from which the conditional sample is made into another language; and in the step of making a conditional sample, a determination that the word of the word/sense pair is used in the sense of the word/sense pair is made with reference to the corresponding word in the translation.
16. The method set forth in claim 13 wherein:
there exists a list of categorized words which belong to the same semantic category as the word/sense pair; and in the step of making a conditional sample, the contexts contain words from the list.
there exists a list of categorized words which belong to the same semantic category as the word/sense pair; and in the step of making a conditional sample, the contexts contain words from the list.
17. The method set forth in claim 16 wherein:
in the step of employing the Bayesian technique, the weight, wt for each word which occurs in the conditional sample is determined by the equation where .pi. is the conditional probability of the word, a is a value for the frequency of the word in the conditional sample, and E is the expectation of .pi. given a.
in the step of employing the Bayesian technique, the weight, wt for each word which occurs in the conditional sample is determined by the equation where .pi. is the conditional probability of the word, a is a value for the frequency of the word in the conditional sample, and E is the expectation of .pi. given a.
18. The method set forth in claim 17 wherein:
in the step of employing the Bayesian technique, a is derived by determining for each word from the list which is contained in a context the number of times k the word from the list appears in the corpus and letting all words in the context of such a word contribute a weight of ? to a.
in the step of employing the Bayesian technique, a is derived by determining for each word from the list which is contained in a context the number of times k the word from the list appears in the corpus and letting all words in the context of such a word contribute a weight of ? to a.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US814,850 | 1985-12-30 | ||
| US07/814,850 US5541836A (en) | 1991-12-30 | 1991-12-30 | Word disambiguation apparatus and methods |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CA2083733A1 true CA2083733A1 (en) | 1993-07-01 |
Family
ID=25216159
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002083733A Abandoned CA2083733A1 (en) | 1991-12-30 | 1992-11-25 | Word disambiguation methods and apparatus |
Country Status (5)
| Country | Link |
|---|---|
| US (1) | US5541836A (en) |
| EP (1) | EP0550160A2 (en) |
| JP (1) | JPH05242138A (en) |
| KR (1) | KR930014120A (en) |
| CA (1) | CA2083733A1 (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105005553A (en) * | 2015-06-19 | 2015-10-28 | 四川大学 | Emotional thesaurus based short text emotional tendency analysis method |
| CN106021224A (en) * | 2016-05-13 | 2016-10-12 | 中国科学院自动化研究所 | Bilingual discourse annotation method |
Families Citing this family (164)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6278967B1 (en) * | 1992-08-31 | 2001-08-21 | Logovista Corporation | Automated system for generating natural language translations that are domain-specific, grammar rule-based, and/or based on part-of-speech analysis |
| GB2279164A (en) * | 1993-06-18 | 1994-12-21 | Canon Res Ct Europe Ltd | Processing a bilingual database. |
| US5873056A (en) * | 1993-10-12 | 1999-02-16 | The Syracuse University | Natural language processing system for semantic vector representation which accounts for lexical ambiguity |
| US5822720A (en) | 1994-02-16 | 1998-10-13 | Sentius Corporation | System amd method for linking streams of multimedia data for reference material for display |
| US5794050A (en) * | 1995-01-04 | 1998-08-11 | Intelligent Text Processing, Inc. | Natural language understanding system |
| US5659771A (en) * | 1995-05-19 | 1997-08-19 | Mitsubishi Electric Information Technology Center America, Inc. | System for spelling correction in which the context of a target word in a sentence is utilized to determine which of several possible words was intended |
| US5680511A (en) * | 1995-06-07 | 1997-10-21 | Dragon Systems, Inc. | Systems and methods for word recognition |
| US5828991A (en) * | 1995-06-30 | 1998-10-27 | The Research Foundation Of The State University Of New York | Sentence reconstruction using word ambiguity resolution |
| US5717914A (en) * | 1995-09-15 | 1998-02-10 | Infonautics Corporation | Method for categorizing documents into subjects using relevance normalization for documents retrieved from an information retrieval system in response to a query |
| US5878386A (en) * | 1996-06-28 | 1999-03-02 | Microsoft Corporation | Natural language parser with dictionary-based part-of-speech probabilities |
| US6173298B1 (en) | 1996-09-17 | 2001-01-09 | Asap, Ltd. | Method and apparatus for implementing a dynamic collocation dictionary |
| US6119114A (en) * | 1996-09-17 | 2000-09-12 | Smadja; Frank | Method and apparatus for dynamic relevance ranking |
| JPH10198680A (en) * | 1997-01-07 | 1998-07-31 | Hitachi Ltd | Distributed dictionary managing method and machine translating method using the method |
| GB2321117A (en) * | 1997-01-09 | 1998-07-15 | Sharp Kk | Disambiguating syntactic word multiples |
| GB2323693B (en) * | 1997-03-27 | 2001-09-26 | Forum Technology Ltd | Speech to text conversion |
| DE69837979T2 (en) * | 1997-06-27 | 2008-03-06 | International Business Machines Corp. | System for extracting multilingual terminology |
| US5970483A (en) * | 1997-07-25 | 1999-10-19 | Claritech Corporation | Apparatus and methodology for submitting search queries |
| JPH11102414A (en) | 1997-07-25 | 1999-04-13 | Kuraritec Corp | Method and device for correcting optical character recognition by using bitmap selection and computer-readable record medium record with series of instructions to correct ocr output error |
| US5987448A (en) | 1997-07-25 | 1999-11-16 | Claritech Corporation | Methodology for displaying search results using character recognition |
| JPH11110480A (en) * | 1997-07-25 | 1999-04-23 | Kuraritec Corp | Method and device for displaying text |
| US6078878A (en) * | 1997-07-31 | 2000-06-20 | Microsoft Corporation | Bootstrapping sense characterizations of occurrences of polysemous words |
| WO1999017223A1 (en) * | 1997-09-30 | 1999-04-08 | Ihc Health Services, Inc. | Aprobabilistic system for natural language processing |
| US6260008B1 (en) * | 1998-01-08 | 2001-07-10 | Sharp Kabushiki Kaisha | Method of and system for disambiguating syntactic word multiples |
| US6782510B1 (en) * | 1998-01-27 | 2004-08-24 | John N. Gross | Word checking tool for controlling the language content in documents using dictionaries with modifyable status fields |
| US6115709A (en) | 1998-09-18 | 2000-09-05 | Tacit Knowledge Systems, Inc. | Method and system for constructing a knowledge profile of a user having unrestricted and restricted access portions according to respective levels of confidence of content of the portions |
| AU5910699A (en) | 1998-09-18 | 2000-04-10 | Tacit Knowledge Systems | Method of constructing and displaying an entity profile constructed utilizing input from entities other than the owner |
| WO2000017727A2 (en) | 1998-09-18 | 2000-03-30 | Tacit Knowledge Systems | Method and apparatus for querying a user knowledge profile |
| US6154783A (en) | 1998-09-18 | 2000-11-28 | Tacit Knowledge Systems | Method and apparatus for addressing an electronic document for transmission over a network |
| US6377949B1 (en) | 1998-09-18 | 2002-04-23 | Tacit Knowledge Systems, Inc. | Method and apparatus for assigning a confidence level to a term within a user knowledge profile |
| US8380875B1 (en) | 1998-09-18 | 2013-02-19 | Oracle International Corporation | Method and system for addressing a communication document for transmission over a network based on the content thereof |
| US6253202B1 (en) | 1998-09-18 | 2001-06-26 | Tacit Knowledge Systems, Inc. | Method, system and apparatus for authorizing access by a first user to a knowledge profile of a second user responsive to an access request from the first user |
| US6256629B1 (en) * | 1998-11-25 | 2001-07-03 | Lucent Technologies Inc. | Method and apparatus for measuring the degree of polysemy in polysemous words |
| DE19855137A1 (en) * | 1998-11-30 | 2000-05-31 | Honeywell Ag | Process for converting data |
| EP1266313A2 (en) | 1999-03-19 | 2002-12-18 | Trados GmbH | Workflow management system |
| US6490548B1 (en) | 1999-05-14 | 2002-12-03 | Paterra, Inc. | Multilingual electronic transfer dictionary containing topical codes and method of use |
| KR20010004404A (en) * | 1999-06-28 | 2001-01-15 | 정선종 | Keyfact-based text retrieval system, keyfact-based text index method, and retrieval method using this system |
| US20060116865A1 (en) | 1999-09-17 | 2006-06-01 | Www.Uniscape.Com | E-services translation utilizing machine translation and translation memory |
| US6405162B1 (en) * | 1999-09-23 | 2002-06-11 | Xerox Corporation | Type-based selection of rules for semantically disambiguating words |
| US6256605B1 (en) * | 1999-11-08 | 2001-07-03 | Macmillan Alan S. | System for and method of summarizing etymological information |
| US7315891B2 (en) * | 2000-01-12 | 2008-01-01 | Vericept Corporation | Employee internet management device |
| US7716163B2 (en) | 2000-06-06 | 2010-05-11 | Microsoft Corporation | Method and system for defining semantic categories and actions |
| US7712024B2 (en) | 2000-06-06 | 2010-05-04 | Microsoft Corporation | Application program interfaces for semantically labeling strings and providing actions based on semantically labeled strings |
| US7788602B2 (en) | 2000-06-06 | 2010-08-31 | Microsoft Corporation | Method and system for providing restricted actions for recognized semantic categories |
| US7421645B2 (en) | 2000-06-06 | 2008-09-02 | Microsoft Corporation | Method and system for providing electronic commerce actions based on semantically labeled strings |
| US7770102B1 (en) * | 2000-06-06 | 2010-08-03 | Microsoft Corporation | Method and system for semantically labeling strings and providing actions based on semantically labeled strings |
| US6668251B1 (en) | 2000-11-01 | 2003-12-23 | Tacit Knowledge Systems, Inc. | Rendering discriminator members from an initial set of result data |
| US20040111386A1 (en) * | 2001-01-08 | 2004-06-10 | Goldberg Jonathan M. | Knowledge neighborhoods |
| EP1490790A2 (en) * | 2001-03-13 | 2004-12-29 | Intelligate Ltd. | Dynamic natural language understanding |
| US7032174B2 (en) * | 2001-03-27 | 2006-04-18 | Microsoft Corporation | Automatically adding proper names to a database |
| US7778816B2 (en) | 2001-04-24 | 2010-08-17 | Microsoft Corporation | Method and system for applying input mode bias |
| US7191115B2 (en) * | 2001-06-20 | 2007-03-13 | Microsoft Corporation | Statistical method and apparatus for learning translation relationships among words |
| AU2002316581A1 (en) | 2001-07-03 | 2003-01-21 | University Of Southern California | A syntax-based statistical translation model |
| US7130861B2 (en) | 2001-08-16 | 2006-10-31 | Sentius International Corporation | Automated creation and delivery of database content |
| JP2003157376A (en) * | 2001-11-21 | 2003-05-30 | Ricoh Co Ltd | Network system, identification information management method, server device, program and recording medium |
| AU2003269808A1 (en) | 2002-03-26 | 2004-01-06 | University Of Southern California | Constructing a translation lexicon from comparable, non-parallel corpora |
| US7325194B2 (en) | 2002-05-07 | 2008-01-29 | Microsoft Corporation | Method, system, and apparatus for converting numbers between measurement systems based upon semantically labeled strings |
| US7707496B1 (en) | 2002-05-09 | 2010-04-27 | Microsoft Corporation | Method, system, and apparatus for converting dates between calendars and languages based upon semantically labeled strings |
| US7707024B2 (en) | 2002-05-23 | 2010-04-27 | Microsoft Corporation | Method, system, and apparatus for converting currency values based upon semantically labeled strings |
| US7742048B1 (en) | 2002-05-23 | 2010-06-22 | Microsoft Corporation | Method, system, and apparatus for converting numbers based upon semantically labeled strings |
| US7827546B1 (en) | 2002-06-05 | 2010-11-02 | Microsoft Corporation | Mechanism for downloading software components from a remote source for use by a local software application |
| US7281245B2 (en) | 2002-06-05 | 2007-10-09 | Microsoft Corporation | Mechanism for downloading software components from a remote source for use by a local software application |
| US7356537B2 (en) | 2002-06-06 | 2008-04-08 | Microsoft Corporation | Providing contextually sensitive tools and help content in computer-generated documents |
| US7716676B2 (en) | 2002-06-25 | 2010-05-11 | Microsoft Corporation | System and method for issuing a message to a program |
| US7392479B2 (en) | 2002-06-27 | 2008-06-24 | Microsoft Corporation | System and method for providing namespace related information |
| US7209915B1 (en) | 2002-06-28 | 2007-04-24 | Microsoft Corporation | Method, system and apparatus for routing a query to one or more providers |
| US7353165B2 (en) * | 2002-06-28 | 2008-04-01 | Microsoft Corporation | Example based machine translation system |
| US7158983B2 (en) | 2002-09-23 | 2007-01-02 | Battelle Memorial Institute | Text analysis technique |
| US9805373B1 (en) | 2002-11-19 | 2017-10-31 | Oracle International Corporation | Expertise services platform |
| US7249012B2 (en) * | 2002-11-20 | 2007-07-24 | Microsoft Corporation | Statistical method and apparatus for learning translation relationships among phrases |
| US7783614B2 (en) | 2003-02-13 | 2010-08-24 | Microsoft Corporation | Linking elements of a document to corresponding fields, queries and/or procedures in a database |
| US7356457B2 (en) * | 2003-02-28 | 2008-04-08 | Microsoft Corporation | Machine translation using learned word associations without referring to a multi-lingual human authored dictionary of content words |
| US7711550B1 (en) | 2003-04-29 | 2010-05-04 | Microsoft Corporation | Methods and system for recognizing names in a computer-generated document and for providing helpful actions associated with recognized names |
| US8640234B2 (en) * | 2003-05-07 | 2014-01-28 | Trustwave Holdings, Inc. | Method and apparatus for predictive and actual intrusion detection on a network |
| US7558841B2 (en) | 2003-05-14 | 2009-07-07 | Microsoft Corporation | Method, system, and computer-readable medium for communicating results to a data query in a computer network |
| US7739588B2 (en) | 2003-06-27 | 2010-06-15 | Microsoft Corporation | Leveraging markup language data for semantically labeling text strings and data and for providing actions based on semantically labeled text strings and data |
| US8548794B2 (en) | 2003-07-02 | 2013-10-01 | University Of Southern California | Statistical noun phrase translation |
| US7774333B2 (en) | 2003-08-21 | 2010-08-10 | Idia Inc. | System and method for associating queries and documents with contextual advertisements |
| US20070136251A1 (en) * | 2003-08-21 | 2007-06-14 | Idilia Inc. | System and Method for Processing a Query |
| US7509313B2 (en) | 2003-08-21 | 2009-03-24 | Idilia Inc. | System and method for processing a query |
| US7475010B2 (en) * | 2003-09-03 | 2009-01-06 | Lingospot, Inc. | Adaptive and scalable method for resolving natural language ambiguities |
| JP3856778B2 (en) * | 2003-09-29 | 2006-12-13 | 株式会社日立製作所 | Document classification apparatus and document classification method for multiple languages |
| US7487515B1 (en) | 2003-12-09 | 2009-02-03 | Microsoft Corporation | Programmable object model for extensible markup language schema validation |
| US7434157B2 (en) | 2003-12-09 | 2008-10-07 | Microsoft Corporation | Programmable object model for namespace or schema library support in a software application |
| US7178102B1 (en) | 2003-12-09 | 2007-02-13 | Microsoft Corporation | Representing latent data in an extensible markup language document |
| US7404195B1 (en) | 2003-12-09 | 2008-07-22 | Microsoft Corporation | Programmable object model for extensible markup language markup in an application |
| US7509573B1 (en) | 2004-02-17 | 2009-03-24 | Microsoft Corporation | Anti-virus security information in an extensible markup language document |
| US7983896B2 (en) | 2004-03-05 | 2011-07-19 | SDL Language Technology | In-context exact (ICE) matching |
| US20100262621A1 (en) * | 2004-03-05 | 2010-10-14 | Russ Ross | In-context exact (ice) matching |
| US8296127B2 (en) | 2004-03-23 | 2012-10-23 | University Of Southern California | Discovery of parallel text portions in comparable collections of corpora and training using comparable texts |
| US8666725B2 (en) | 2004-04-16 | 2014-03-04 | University Of Southern California | Selection and use of nonstatistical translation components in a statistical machine translation framework |
| GB2417103A (en) * | 2004-08-11 | 2006-02-15 | Sdl Plc | Natural language translation system |
| JP5452868B2 (en) | 2004-10-12 | 2014-03-26 | ユニヴァーシティー オブ サザン カリフォルニア | Training for text-to-text applications that use string-to-tree conversion for training and decoding |
| US20060206806A1 (en) * | 2004-11-04 | 2006-09-14 | Motorola, Inc. | Text summarization |
| US9195766B2 (en) | 2004-12-14 | 2015-11-24 | Google Inc. | Providing useful information associated with an item in a document |
| WO2006086179A2 (en) * | 2005-01-31 | 2006-08-17 | Textdigger, Inc. | Method and system for semantic search and retrieval of electronic documents |
| US20070073678A1 (en) * | 2005-09-23 | 2007-03-29 | Applied Linguistics, Llc | Semantic document profiling |
| US7599828B2 (en) * | 2005-03-01 | 2009-10-06 | Microsoft Corporation | Grammatically correct contraction spelling suggestions for french |
| JP4050755B2 (en) * | 2005-03-30 | 2008-02-20 | 株式会社東芝 | Communication support device, communication support method, and communication support program |
| WO2006110684A2 (en) * | 2005-04-11 | 2006-10-19 | Textdigger, Inc. | System and method for searching for a query |
| WO2006113597A2 (en) * | 2005-04-14 | 2006-10-26 | The Regents Of The University Of California | Method for information retrieval |
| US9606634B2 (en) * | 2005-05-18 | 2017-03-28 | Nokia Technologies Oy | Device incorporating improved text input mechanism |
| US20090193334A1 (en) * | 2005-05-18 | 2009-07-30 | Exb Asset Management Gmbh | Predictive text input system and method involving two concurrent ranking means |
| US8099281B2 (en) * | 2005-06-06 | 2012-01-17 | Nunance Communications, Inc. | System and method for word-sense disambiguation by recursive partitioning |
| US8886517B2 (en) | 2005-06-17 | 2014-11-11 | Language Weaver, Inc. | Trust scoring for language translation systems |
| US8676563B2 (en) | 2009-10-01 | 2014-03-18 | Language Weaver, Inc. | Providing human-generated and machine-generated trusted translations |
| US7574348B2 (en) * | 2005-07-08 | 2009-08-11 | Microsoft Corporation | Processing collocation mistakes in documents |
| US20070073745A1 (en) * | 2005-09-23 | 2007-03-29 | Applied Linguistics, Llc | Similarity metric for semantic profiling |
| US7992085B2 (en) | 2005-09-26 | 2011-08-02 | Microsoft Corporation | Lightweight reference user interface |
| US7788590B2 (en) | 2005-09-26 | 2010-08-31 | Microsoft Corporation | Lightweight reference user interface |
| US10319252B2 (en) * | 2005-11-09 | 2019-06-11 | Sdl Inc. | Language capability assessment and training apparatus and techniques |
| WO2007081681A2 (en) | 2006-01-03 | 2007-07-19 | Textdigger, Inc. | Search system with query refinement and search method |
| WO2007114932A2 (en) | 2006-04-04 | 2007-10-11 | Textdigger, Inc. | Search system and method with text function tagging |
| US8943080B2 (en) | 2006-04-07 | 2015-01-27 | University Of Southern California | Systems and methods for identifying parallel documents and sentence fragments in multilingual document collections |
| KR100785928B1 (en) * | 2006-07-04 | 2007-12-17 | 삼성전자주식회사 | Method and system for searching photograph using multimodal |
| US8886518B1 (en) | 2006-08-07 | 2014-11-11 | Language Weaver, Inc. | System and method for capitalizing machine translated text |
| US7689408B2 (en) * | 2006-09-01 | 2010-03-30 | Microsoft Corporation | Identifying language of origin for words using estimates of normalized appearance frequency |
| US8521506B2 (en) * | 2006-09-21 | 2013-08-27 | Sdl Plc | Computer-implemented method, computer software and apparatus for use in a translation system |
| US8433556B2 (en) | 2006-11-02 | 2013-04-30 | University Of Southern California | Semi-supervised training for statistical word alignment |
| US9122674B1 (en) | 2006-12-15 | 2015-09-01 | Language Weaver, Inc. | Use of annotations in statistical machine translation |
| US8468149B1 (en) | 2007-01-26 | 2013-06-18 | Language Weaver, Inc. | Multi-lingual online community |
| US8112402B2 (en) * | 2007-02-26 | 2012-02-07 | Microsoft Corporation | Automatic disambiguation based on a reference resource |
| US8615389B1 (en) | 2007-03-16 | 2013-12-24 | Language Weaver, Inc. | Generation and exploitation of an approximate language model |
| US8831928B2 (en) | 2007-04-04 | 2014-09-09 | Language Weaver, Inc. | Customizable machine translation service |
| US8825466B1 (en) | 2007-06-08 | 2014-09-02 | Language Weaver, Inc. | Modification of annotated bilingual segment pairs in syntax-based machine translation |
| US8280721B2 (en) * | 2007-08-31 | 2012-10-02 | Microsoft Corporation | Efficiently representing word sense probabilities |
| WO2009059297A1 (en) * | 2007-11-01 | 2009-05-07 | Textdigger, Inc. | Method and apparatus for automated tag generation for digital content |
| US8209164B2 (en) * | 2007-11-21 | 2012-06-26 | University Of Washington | Use of lexical translations for facilitating searches |
| US8190423B2 (en) * | 2008-09-05 | 2012-05-29 | Trigent Software Ltd. | Word sense disambiguation using emergent categories |
| US9043209B2 (en) * | 2008-11-28 | 2015-05-26 | Nec Corporation | Language model creation device |
| US9262403B2 (en) | 2009-03-02 | 2016-02-16 | Sdl Plc | Dynamic generation of auto-suggest dictionary for natural language translation |
| GB2468278A (en) * | 2009-03-02 | 2010-09-08 | Sdl Plc | Computer assisted natural language translation outputs selectable target text associated in bilingual corpus with input target text from partial translation |
| CN101901210A (en) * | 2009-05-25 | 2010-12-01 | 日电（中国）有限公司 | Word meaning disambiguating system and method |
| US8990064B2 (en) | 2009-07-28 | 2015-03-24 | Language Weaver, Inc. | Translating documents based on content |
| US8380486B2 (en) | 2009-10-01 | 2013-02-19 | Language Weaver, Inc. | Providing machine-generated translations and corresponding trust levels |
| US10417646B2 (en) | 2010-03-09 | 2019-09-17 | Sdl Inc. | Predicting the cost associated with translating textual content |
| US9128929B2 (en) | 2011-01-14 | 2015-09-08 | Sdl Language Technologies | Systems and methods for automatically estimating a translation time including preparation time in addition to the translation itself |
| US11003838B2 (en) | 2011-04-18 | 2021-05-11 | Sdl Inc. | Systems and methods for monitoring post translation editing |
| US8738375B2 (en) * | 2011-05-09 | 2014-05-27 | At&T Intellectual Property I, L.P. | System and method for optimizing speech recognition and natural language parameters with user feedback |
| US9396725B2 (en) | 2011-05-09 | 2016-07-19 | At&T Intellectual Property I, L.P. | System and method for optimizing speech recognition and natural language parameters with user feedback |
| US8694303B2 (en) | 2011-06-15 | 2014-04-08 | Language Weaver, Inc. | Systems and methods for tuning parameters in statistical machine translation |
| US8886515B2 (en) | 2011-10-19 | 2014-11-11 | Language Weaver, Inc. | Systems and methods for enhancing machine translation post edit review processes |
| US8738364B2 (en) | 2011-12-14 | 2014-05-27 | International Business Machines Corporation | Adaptation of vocabulary levels for enhanced collaboration |
| US20130198268A1 (en) * | 2012-01-30 | 2013-08-01 | David Hyman | Generation of a music playlist based on text content accessed by a user |
| CN103294661A (en) * | 2012-03-01 | 2013-09-11 | 富泰华工业（深圳）有限公司 | Language ambiguity eliminating system and method |
| US8942973B2 (en) | 2012-03-09 | 2015-01-27 | Language Weaver, Inc. | Content page URL translation |
| US10261994B2 (en) | 2012-05-25 | 2019-04-16 | Sdl Inc. | Method and system for automatic management of reputation of translators |
| US9201876B1 (en) * | 2012-05-29 | 2015-12-01 | Google Inc. | Contextual weighting of words in a word grouping |
| US9152622B2 (en) | 2012-11-26 | 2015-10-06 | Language Weaver, Inc. | Personalized machine translation via online adaptation |
| US9208442B2 (en) | 2013-04-26 | 2015-12-08 | Wal-Mart Stores, Inc. | Ontology-based attribute extraction from product descriptions |
| US9213694B2 (en) | 2013-10-10 | 2015-12-15 | Language Weaver, Inc. | Efficient online domain adaptation |
| US9779087B2 (en) * | 2013-12-13 | 2017-10-03 | Google Inc. | Cross-lingual discriminative learning of sequence models with posterior regularization |
| US9436676B1 (en) | 2014-11-25 | 2016-09-06 | Truthful Speaking, Inc. | Written word refinement system and method |
| CN105022805B (en) * | 2015-07-02 | 2018-05-04 | 四川大学 | A kind of sentiment analysis method based on SO-PMI information on commodity comment |
| US10515152B2 (en) * | 2015-08-28 | 2019-12-24 | Freedom Solutions Group, Llc | Mitigation of conflicts between content matchers in automated document analysis |
| CN109726386B (en) * | 2017-10-30 | 2023-05-09 | 中国移动通信有限公司研究院 | Word vector model generation method, device and computer readable storage medium |
| US10635863B2 (en) | 2017-10-30 | 2020-04-28 | Sdl Inc. | Fragment recall and adaptive automated translation |
| US10817676B2 (en) | 2017-12-27 | 2020-10-27 | Sdl Inc. | Intelligent routing services and systems |
| US10430447B2 (en) | 2018-01-31 | 2019-10-01 | International Business Machines Corporation | Predicting intent of a user from anomalous profile data |
| US10741176B2 (en) | 2018-01-31 | 2020-08-11 | International Business Machines Corporation | Customizing responses to users in automated dialogue systems |
| US11449533B2 (en) * | 2018-02-22 | 2022-09-20 | Entigenlogic Llc | Curating knowledge for storage in a knowledge database |
| US11087098B2 (en) * | 2018-09-18 | 2021-08-10 | Sap Se | Computer systems for classifying multilingual text |
| US11256867B2 (en) | 2018-10-09 | 2022-02-22 | Sdl Inc. | Systems and methods of machine learning for digital assets and message creation |
| CN110991196B (en) * | 2019-12-18 | 2021-10-26 | 北京百度网讯科技有限公司 | Translation method and device for polysemous words, electronic equipment and medium |
| US11651156B2 (en) * | 2020-05-07 | 2023-05-16 | Optum Technology, Inc. | Contextual document summarization with semantic intelligence |
Family Cites Families (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPS6140672A (en) * | 1984-07-31 | 1986-02-26 | Hitachi Ltd | Processing system for dissolution of many parts of speech |
| JPS61105671A (en) * | 1984-10-29 | 1986-05-23 | Hitachi Ltd | Natural language processing device |
| JPH083815B2 (en) * | 1985-10-25 | 1996-01-17 | 株式会社日立製作所 | Natural language co-occurrence relation dictionary maintenance method |
| US4930077A (en) * | 1987-04-06 | 1990-05-29 | Fan David P | Information processing expert system for text analysis and predicting public opinion based information available to the public |
| US4868750A (en) * | 1987-10-07 | 1989-09-19 | Houghton Mifflin Company | Collocational grammar system |
| US5146405A (en) * | 1988-02-05 | 1992-09-08 | At&T Bell Laboratories | Methods for part-of-speech determination and usage |
| US4914590A (en) * | 1988-05-18 | 1990-04-03 | Emhart Industries, Inc. | Natural language understanding system |
| NL8900587A (en) * | 1989-03-10 | 1990-10-01 | Bso Buro Voor Systeemontwikkel | METHOD FOR DETERMINING THE SEMANTIC RELATION OF LEXICAL COMPONENTS IN A TEXT |
| US5170349A (en) * | 1989-03-14 | 1992-12-08 | Canon Kabushiki Kaisha | Text processing apparatus using modification relation analysis |
| JPH02242372A (en) * | 1989-03-15 | 1990-09-26 | Toshiba Corp | Sentence generating device |
| JPH02308370A (en) * | 1989-05-24 | 1990-12-21 | Toshiba Corp | Machine translation system |
| US5056021A (en) * | 1989-06-08 | 1991-10-08 | Carolyn Ausborn | Method and apparatus for abstracting concepts from natural language |
| US5243520A (en) * | 1990-08-21 | 1993-09-07 | General Electric Company | Sense discrimination system and method |
| EP0494573A1 (en) * | 1991-01-08 | 1992-07-15 | International Business Machines Corporation | Method for automatically disambiguating the synonymic links in a dictionary for a natural language processing system |
-
1991
- 1991-12-30 US US07/814,850 patent/US5541836A/en not_active Expired - Lifetime
-
1992
- 1992-11-25 CA CA002083733A patent/CA2083733A1/en not_active Abandoned
- 1992-12-04 EP EP92311094A patent/EP0550160A2/en not_active Withdrawn
- 1992-12-24 JP JP4343914A patent/JPH05242138A/en not_active Withdrawn
- 1992-12-30 KR KR1019920027574A patent/KR930014120A/en not_active Application Discontinuation
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105005553A (en) * | 2015-06-19 | 2015-10-28 | 四川大学 | Emotional thesaurus based short text emotional tendency analysis method |
| CN105005553B (en) * | 2015-06-19 | 2017-11-21 | 四川大学 | Short text Sentiment orientation analysis method based on sentiment dictionary |
| CN106021224A (en) * | 2016-05-13 | 2016-10-12 | 中国科学院自动化研究所 | Bilingual discourse annotation method |
| CN106021224B (en) * | 2016-05-13 | 2019-03-15 | 中国科学院自动化研究所 | A kind of bilingual chapter mask method |
Also Published As
| Publication number | Publication date |
|---|---|
| KR930014120A (en) | 1993-07-22 |
| US5541836A (en) | 1996-07-30 |
| EP0550160A3 (en) | 1994-01-12 |
| EP0550160A2 (en) | 1993-07-07 |
| JPH05242138A (en) | 1993-09-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US5541836A (en) | Word disambiguation apparatus and methods | |
| Rayson et al. | The UCREL semantic analysis system | |
| McKeown et al. | Collocations | |
| Gale et al. | Estimating upper and lower bounds on the performance of word-sense disambiguation programs | |
| Chanod et al. | Tagging French--comparing a statistical and a constraint-based method | |
| Black | An experiment in computational discrimination of English word senses | |
| US6167369A (en) | Automatic language identification using both N-gram and word information | |
| Luk | Statistical sense disambiguation with relatively small corpora using dictionary definitions | |
| EP1471440A2 (en) | System and method for word analysis | |
| Megyesi | Improving Brill’s POS tagger for an agglutinative language | |
| Smadja et al. | Translating collocations for use in bilingual lexicons | |
| Vallez et al. | Natural language processing in textual information retrieval and related topics | |
| Burgin et al. | Improving disambiguation in FASIT | |
| Woods | Aggressive morphology for robust lexical coverage | |
| Hodász et al. | MetaMorpho TM: a linguistically enriched translation memory | |
| Verkerk et al. | LASLA and Collatinus: a convergence in lexica | |
| Klang et al. | Linking, searching, and visualizing entities in wikipedia | |
| Liberman | The trend towards statistical models in natural language processing | |
| KR20000036487A (en) | A Database System for Korean-English Translation Using Information Retrieval Techniques | |
| Moens | Using patterns of thematic progression for building a table of contents of a text | |
| Megyesi | Brill’s PoS tagger with extended lexical templates for Hungarian | |
| Neumann et al. | Shallow natural language technology and text mining | |
| Nasukawa | Discourse constraint in computer manuals | |
| KR100404320B1 (en) | Automatic sentence indexing method | |
| JP2003108583A (en) | Retrieving device using language sentence, retrieving system, retrieving method, program and recording medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request | ||
| FZDE | Discontinued |