WO2002039318A1

WO2002039318A1 - User alterable weighting of translations

Info

Publication number: WO2002039318A1
Application number: PCT/US2000/042120
Authority: WO
Inventors: Lewis G. Pringle; Robert W. Swerdlow; Alec Wysoker; Sterling Wright
Original assignee: Logovista Corporation
Priority date: 2000-11-09
Filing date: 2000-11-09
Publication date: 2002-05-16
Also published as: JP2004513458A

Abstract

In one aspect, the invention relates to a method of allowing a user to view and modify a weighting associated with a translation of a source language string. The method includes displaying to the user the weighting associated with the translation of the source language string, the weighting for use by a translation engine in selecting the translation and allowing the usr to modify the weighting associated with the translation. In one embodiment, the method further includes allowing the user to reset the weighting back to a default value subsequent to user modification of the weighting.

Description

USER ALTERABLE WEIGHTING OF TRANSLATIONS

Technical Field

The invention generally relates to automated natural language translation in which source text is translated into another language. More specifically, this invention relates to allowing a user to view and alter weighting values used in automated translation.

Background Information Various schemes for the machine-based translation of natural language have been proposed. Typically, the system used for translation includes a computer that receives input in one language and performs operations on the received input to supply output in another language. This type of translation has been an inexact one, and the resulting output can require significant editing by a skilled operator. The translation operation performed by known systems generally includes a structural conversion operation. The objective of structural conversion is to transform a given parse tree (i.e., a syntactic structure tree) of the source language sentence to the corresponding tree in the target language. Two types of structural conversion have been tried, grammar-rule-based and template-to-template.

In grammar-rule-based structural conversion, the domain of structural conversion is limited to the domain of grammar rules that have been used to obtain the source-language parse tree (i.e., to a set of subnodes that are immediate daughters of a given node). For example, given

VP = NT01 + ΝP (a NerbPhrase consists of a SingleObject

Transitive Verb and a ΝounPhrase, in that order) and

Japanese: 1 + 2 => 2 + 1 (Reverse the order of NT01 and ΝP), each source-language parse tree that involves application of the rule is structurally converted in such a way that the order of the verb and the object is reversed because the verb appears to the right of its object in Japanese. This method is very efficient in that it is easy to find out where the specified conversion applies; it applies exactly at the location where the rule has been used to obtain the source-language parse tree. On the other hand, it can be a weak conversion mechanism in that its domain, as specified above, may be extremely limited, and in that natural language may require conversion rules that straddle over nodes that are not siblings.

In template-to-template structural conversion, structural conversion is specified in terms of input/output (I/O) templates or subtrees. If a given input template matches a given structure tree, that portion of the structure tree that is matched by the template is changed as specified by the corresponding output template. This is a very powerful conversion mechanism, but it can be costly in that it can take a long period of time to find out if a given input template matches any portion of a given structure tree.

It also is known to translate using "experts" that evaluate possible translation options and chose one. Which translation is the best one depends on the scores that the experts give each of the possible translations. The user views the translation selected by the automated translation system after the translation process is completed. If the translation is incorrect the user can correct it. After several corrections, the automated system may learn the preferences of the user, for example as described in U.S. patent number 5,826,220 to Takeda et al.

Summary of the Invention

Automated natural language translation systems according to the invention have many advantages over known machine-based translators. After a system of the invention automatically selects the best possible translation of the input textual information and provides the user with an output (for example, a Japanese language translation of English-language input text), the user can then interface with the system to edit the displayed translation or to obtain alternative translations in an automated fashion. A user of an automated natural language translation system of the invention can be more productive because the system allows the user to retain just the portion of the translation that he or she deems acceptable while causing the remaining portion to be retranslated automatically. Since this selective retranslation operation is precisely directed at portions that require retranslation, users are saved the time and tedium of considering potentially large numbers of incorrect, but highly ranked translations. Furthermore, because the system allows for arbitrary granularity in translation adjustments, more of the final structure of the translation will usually have been generated by the system. The system thus reduces the potential for human (user) error and saves time in edits that may involve structural, accord, and tense changes. The system efficiently gives users the full benefit of its extensive and reliable knowledge of grammar and spelling. The automated natural language translation system's versatile handling of ambiguous sentence boundaries in the source language, and its powerful semantic propagation provide further accuracy and reduced user editing of translations. Stored statistical information also improves the accuracy of translations by tailoring the preferred translation to the specific user site. The system's idiom handling method is advantageous in that it allows sentences that happen to include the sequence of words making up the idiom, without intending the meaning of the idiom, to be correctly translated. The system is efficient but still has versatile functions such as long distance feature matching. The system's structural balance expert and coordinate structure expert effectively distinguish between intended parses and unintended parses. A capitalization expert effectively obtains correct interpretations of capitalized words in sentences, and a capitalized sequence procedure effectively deals with multiple-word proper names, without completely ignoring common noun interpretations.

The invention is generally directed to automated natural language translation system that allows a user to view weighting values that are used by one or more "experts" to select a particular translation among two or more options. The invention also generally relates to allowing a user to change the weightings so that the automated system at least tends to select certain translations preferred by the user. The user can view and or adjust the weightings prior to the system actually performing any translations, and thus it generally is not necessary for the user to perform post-translation error correction one or more times before the automated system "learns" the preferences of the user.

In another aspect, the invention relates to a method of allowing a user to view and modify a weighting associated with a translation of a source language string. The method includes displaying to the user the weighting associated with the translation of the source language string, the weighting for use by a translation engine in selecting the translation and allowing the user to modify the weighting associated with the translation. In one embodiment, the method further includes allowing the user to reset the weighting back to a default value subsequent to user modification of the weighting.

In another embodiment, the displaying step further includes displaying to the user a plurality of weightings, each of the weightings associated with an inflection of the translation, each of the weightings for use by the translation engine in selecting the translation. In another embodiment, the allowing step further includes allowing the user to individually modify each of the one or more weightings associated with the inflections of the translation. In another embodiment, the method further includes displaying in a summary window all of the one or more weightings associate with the inflections on a single representation of a range of possible values. In another embodiment, the source language string and the translation are contained in a translation dictionary. In another embodiment, the method further includes displaying to the user the translation in response to an input from the user. In another embodiment, the input is at least one character in the source language. In another embodiment, the input is at least one character in the target language of the translation. In another embodiment, the method further includes displaying to the user the translation in response to an input from the translation engine.

In another embodiment, the method further includes selecting, with the translation engine, the translation from among a plurality of possible translations of the source language string, each of the plurality of possible translations having associated therewith a weighting, the selection based at least in.part on the weighting. In another embodiment, the method further includes receiving a source language string, determining by at least one expert system a plurality of possible translations, each translation in the plurality equally translating the source language string and selecting the translation that has the highest weighting from the plurality of possible translations.

In another embodiment, the method further includes providing a graphical user interface for displaying to the user the weighting. In another embodiment, the method further includes providing a graphical user interface for displaying to the user at least one source language dictionary entry meeting established criteria and allowing the user to hide and display one or more translations of the at least one source language dictionary entry. In another embodiment, the method further includes adjusting the weighting in response and proportion to the user manipulation of the graphical user interface.

In another embodiment, the graphical user interface represents to the user a current value of the weighting relative to a range of possible values. In another embodiment, the range of possible values is normalized. In another embodiment, the method further includes allowing the user to select the translation from a plurality of possible translations and adjusting, by the translation engine, at least one of the weightings of the plurality of translations in response to the user selection. In another embodiment, the step of adjusting further includes increasing the associated weighting of the selected translation. In another embodiment, the step of adjusting further includes decreasing the associated weighting of the translation replaced by the user selection. In another aspect, the invention relates to a machine translation system for allowing a user to view and modify a weighting associated with a translation of a source language including a dictionary browser configured to provide data used to display to the user the weighting associated with the translation of the source language string, the weighting for use by a translation engine in selecting the translation and configured to allow the user to modify the weighting associated with the translation. The system also includes a translation dictionary in communication with the dictionary browser for receiving and transmitting data, the translation dictionary including at least one translation and the associated weighting, the translation dictionary changing the associated weighting in response to data received from the dictionary browser. The system also includes a user input device in communication with the dictionary browser, the user input device receiving data from the user and transmitting the data to the dictionary browser and a display device in communication with the dictionary browser, the display device receiving data from the dictionary browser and displaying a representation of the data to the user.

In one embodiment, the dictionary browser is further configured to allow the user to reset the weighting back to a default value subsequent to user modification of the weighting. In another embodiment, the dictionary browser is further configured to provide data used to display to the user one or more weightings, each of the weightings associated with an inflection of the source language string, each of the weightings for use by the translation engine in selecting the translation. In another embodiment, the dictionary browser is further configured to allow the user to individually modify each of the one or more weightings associated with the inflections of the translation.

In another embodiment, the system is further configured to provide data to the display device to display in a summary window all of the one or more weightings associate with the inflections on a single representation of a range of possible values. In another embodiment, the dictionary browser is further configured to provide data used to display the translation in response to an input received from the user. In another embodiment, the input is at least one character in the source language. In another embodiment, the input is at least one character in the target language of the translation.

In another embodiment, the dictionary browser is further configured to provide a graphical user interface to display to the user the weighting for the translation. In another embodiment, the dictionary browser is further configured to provide a graphical user interface for displaying to the user at least one source language dictionary entry meeting established criteria and allowing the user to hide and display one or more translations of the at least one source language dictionary entry. In another embodiment, the dictionary browser is further configured to adjust the weighting in response and proportion to the user manipulation of the graphical user interface. In another embodiment, the graphical user interface is configured to represent to the user a current value of the weighting relative to a range of possible values. In another embodiment, the range of possible values is normalized. In another embodiment, the system further includes a translation engine in communication with the dictionary browser, the translation engine selecting the translation based at least in part on the weighting of the translation. In another embodiment, the translation engine is further configured to receive a source language string, configured to determine by at least one expert system a plurality of possible translations, each translation equally translating the source language and configured to select the translation that has the highest weighting from the plurality of possible translations. In another embodiment, the translation engine is further configured to allow the user to select the translation from a plurality of possible translations and the dictionary browser is further configured to adjust at least one of the weightings of the plurality of possible translations in response to the user selection. In another embodiment, the dictionary browser is further configured to increase the associated weighting of the selected translation. In another embodiment, the dictionary browser is further configured to decrease the associated weighting of the translation replaced by the user selection.

In another aspect, the invention relates to an article of manufacture having computer- readable program means for allowing a user to view and modify a weighting of a translation. The article includes computer-readable program means for displaying to the user the weighting associated with the translation of the source language string, the weighting for use by a translation engine in selecting the translation and computer-readable program means for allowing the user to modify the weighting associated with the translation.

The foregoing and other objects, aspects, features, and advantages of the invention will become more apparent from the following description and from the claims. Brief Description of the Drawings In the drawings, like reference characters generally refer to the same parts throughout the different views. Also, the drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the invention. FIG. 1 is a block diagram illustrating a system for performing automated translation of natural language.

FIG. 2 is a data flow diagram illustrating overall functioning of the system of FIG. 1. FIG. 3 is a flow diagram illustrating the operation of the system of FIG. 1. FIG. 4 is a flow diagram illustrating the operation of the end-of-sentence function of the preparser of the system of FIG. 1.

FIG. 5 is a flow diagram illustrating the operation of the parser of the system of FIG. 1. FIG. 6 is a flow diagram illustrating the semantic propagation operations of the system of FIG. 1. FIG. 7 is a flow diagram illustrating the structural conversion operations of the system of

FIG. 1.

FIG. 8 is a flow diagram illustrating the expert evaluator of the system of FIG. 1. FIG. 9 is a diagram of a sample graph used by the system of FIG. 1 for the exemplary phrase "by the bank". FIG. 10 is a block diagram illustrating an embodiment of a system for performing automated translation of natural language according to the invention.

FIG. 11 is a screenshot of an example GUI generated by the dictionary browser of the invention showing the weightings.

FIG. 12 is a screenshot of an example GUI generated by the dictionary browser of the invention showing the modified weightings.

FIG. 13 is a screenshot of an example translation of "By the bank" generated by the automated translation system of the invention.

FIG. 14 is a screenshot of an example GUI generated by the dictionary browser of the invention showing the weightings of the possible translations for "bank." FIG. 15 is a screenshot of an example GUI generated by the dictionary browser of the invention showing the weightings of the possible translations for "by."

FIG. 16 is a screenshot of an example translation of "By the bank" generated by the automated translation system of the invention after the user has altered the weightings. Description A general description of an automated natural language translations system according to the invention is first provided below without reference to any of the drawings. After the general description of the system, reference will be made to the various drawings. An automated natural language translation system can translate from a source natural language to a target natural language. In one embodiment, the system translates from English to Japanese. In another embodiment, the system translates from Japanese to English. In other embodiments, the system translates from/to English to/from, respectively, Spanish, Chinese, German, French, Portuguese, Korean and several other languages that have accessible dictionaries. In general, any language pair is possible. The system comprises means for receiving and storing the source natural language, a translation engine for creating a translation into the target natural language, means for displaying the translation to a user, and means for obtaining for a user and displaying to a user alternative translations. In one embodiment of the system, the translation engine includes a preparser, a parser, a graph maker, an evaluator, a graph scorer, a parse extractor, and a structural converter. The preparser examines the input text and resolves any ambiguities in input sentence boundaries. The preparser then creates and displays the input text in a parse chart seeded with dictionary entries. The parser parses the chart to obtain possible syntactic categories for the input text. The graph maker produces a graph of the possible syntactic interpretations of the input text based on the parse chart. The graph includes nodes and subnodes which are associated with possible interpretations of the input text. The evaluator, which comprises a series of experts, evaluates the graph of the possible interpretations and adds expert weights to the nodes and subnodes of the graph. The graph scorer uses the expert weights to score the subnodes, and the graph scorer then associates the N best scores with each node. The parse extractor assigns a parse tree structure to the preferred interpretation as determined by the graph scorer. The structural converter performs a structural conversion operation on the parse tree structure to obtain a translation in the target language.

In the following three paragraphs, a description is provided of how: (a) the graph scorer combines the expert weights to compute a final weighted score for each subnode; (b) the graph scorer combines the subnode scores to arrive at a final node score; and (c) linguistic information is propagated through the tree of nodes and subnodes.

To compute the final weighted score for each subnode, the graph scorer associates a constant value with each subnode. An analysis of the linguistic information associated with each subnode determines the subnode score. See, for example, FIG. 8 where a series of expert evaluators examine the linguistic information stored at each node and subnode. The graph scorer adds together the individual weighted scores for each expert to obtain a final weighted average for a particular node or subnode. The combination of a plurality of weighted scores into a single weighted average score is a standard problem in computer science. One method that can be used is to multiply each expert result by a constant number (weight) assigned to that expert. The weight assigned to each expert is a matter of design choice. The designer can choose the priority (weight) to assign each expert. The weighted average is the summation of a series of numbers wherein each number is multiplied by a constant. For example,

weighted average = (wι)(xι) + (w₂)(x₂) + ... + (w_n)(x_n)

where the weights, Wi, w₂, ... w_n, are all nonnegative and add up to 1. See, for example, Spiegel, Theory and Problems of Probability and Statistics 76 (McGraw-Hill, Inc. 1975) which discusses the use of weighted averages in the context of statistical expectations.

To combine subnode scores to obtain a final node score, the graph scorer can propagate the subnode scores from the bottom of the graph up to the top of the graph. Given the graph, wherein each node has a set of N scores, it is possible to determine one or more propagation methods. One technique which can be used to propagate the subnode scores is memoization which is a type of dynamic-programming used to solve optimization problems. The solution to optimization problems can involve many possible values (outcomes). The task is to find the optimal value. The algorithm used in optimization solves every subsubproblem just once and saves the outcome, thus avoiding the need to recompute the answer every time the subsubproblem is encountered. For a more detailed explanation of memoization as applied to optimization problems, see, for example, Gormen et al., Introduction to Algorithms 301-314 (McGraw-Hill Book Co. 1990). The method described at pages 301, 302, and 312 of Introduction to Algorithms is one method that can be used for propagating subnode score information through the graph.

In propagating linguistic information through the tree, the semantic propagation part of the system operates to propagate semantic information from smaller constituents to the larger constituents that they comprise. Semantic propagation applies to the four classes of syntactic categories (SEMNP, SEMVP, SEMADJ, and VERB) used in the parsing operation. Before semantic propagation can occur, the linguistic information stored at the nodes must be analyzed. The analysis of the semantic information stored at the nodes is guided by a set of rules that tell from examining the noun-like and verb-like constituents in a grammar rule which selectional restriction slots of the verb-like constituents apply to which noun-like objects. Gerald Gazdar discusses in his text Natural Language Processing In Prolog (Addison- Wesley Publishing Co., 1989) a set of rules which can be used to analyze the semantic information stored at the nodes in a directed acyclic graph similar to that disclosed in the specification. Gazdar discusses the use of feature matching to match information on adjacent nodes. Gazdar states that feature matching involves equations that say that certain features appearing on one node must be identical to the features appearing on another. Most current work assumes a principle that is responsible for equating one class of feature specifications as they appear on the mother category and the daughter which manifests the morphology associated with those features. This daughter is known as the "head" of the phrase. Most phrases only have a single head. Thus, for example, a verb phrase inherits the tense of its verb since the latter is the head of the verb phrase. There is no straightforward way of specifying this principle on a grammar- wide basis with the notational resources that we have used so far, but we can stipulate the effects of the principle on a rule- by-rule basis quite simply if we assume that the relevant features are all to be found on a single branch of the DAG. Let us call the label on this branch head. Then we can write a typical VP rule as follows: VP — > V NP PP

This requires that the value of the head feature on the V and that on the mother VP be identical.

The rules discussed in Gazdar can be easily adapted for each of the syntactic categories discussed herein. The linguistic information assigned to each node using Gazdar's rules can be propagated through the tree using memoization techniques.

Thus, summarizing the previous three paragraphs, the weighted average is one method of determining the subnode score, each subnode score can be propagated through the graph using known memoization techniques as applied to optimization problems, and the strategy discussed in Gazdar's text can be used to analyze the linguistic information stored at each node and this linguistic information can be propagated through the parse tree chart using memoization techniques.

The automated natural language translation system can perform automated re-translation functions after the initial automatic translation. That is, after the system automatically selects the best possible translation of the input textual information and provides the user with an output (preferably a Japanese language translation of the input English text, or a Japanese-to-English translation), the user can then interface with the system to edit the displayed translation or to obtain alternative translations in an automated fashion.

The automated natural language translation system uses a linguistic model which breaks a sentence into substrings. A substring is one or more words which occur in the order specified as part of the sentence. For instance, substrings of "The man is happy" include "The," "The man," "man is happy," "is," and "The man is happy" itself, but not "is man," "man man," and "The is."

Different linguistic models classify substrings in various ways and in different levels of detail. For instance, in "They would like an arrow," "an arrow" is typically classified as a noun phrase (NP). Some models would also classify "an arrow" with syntactic features (for instance, it is a singular noun phrase), and semantic features (it refers to a weapon). If the phrase is ambiguous, there may be two or more ways of classifying it. For instance, "an arrow" can also refer to a symbol with an arrow-like shape. When linguistic models provide a method for resolving ambiguity, they usually do so by combining smaller units into larger units. When evaluating a larger unit, these models consider only a portion of the information contained in the larger unit.

In an exemplary embodiment of the system, the semantic property of "an arrow" (symbol vs. weapon) is used in evaluating the verb phrase "like an arrow" in the sentence "They would like an arrow." In contrast, if the syntax of the phrase "an arrow" were changed as in "He shot it with an arrow," the semantic property of "an arrow" is not used in evaluating the verb phrase "shot it with an arrow."

For any substring of a sentence interpreted in a single way with regard to a specific linguistic model (an interpreted substring), exported properties exist. Exported properties are all properties used to evaluate the combination of an interpreted substring with other units to form larger substrings. An export is an interpreted substring interpreted together with its exported properties. Properties that are contained within the interpreted substring but not exported are called substructures. The parser of the system includes a grammar database. The parser finds all possible interpretations of a sentence using grammatical rules. The grammar database consists of a series of context-free phrase structure rules of the form X = Al A2 ... An. X is composed of, or made from, Al A2 ... An and is referred to as a higher node of lower nodes (subnodes) Al through An. The graph maker of the system graphically represents the many possible interpretations of a sentence. Each node of the graph corresponds to an export of some substring. In one embodiment of the system, a single export is represented by a single node. The graph contains arcs which emanate from the nodes associated with an export. The arcs represent the substructure of the export based on the application of grammar rules. The graph may depict at least two types of arcs for grammar in Chomsky normal form: (1) a unary arc which points to a single different export of the same substring; (2) a binary arc which includes a pair of pointers which points to two exports, the substrings of which when concatenated form the substring of the original export. The graph for grammars not in Chomsky normal form can include an arc having an N-tuple of pointers, pointing to N exports. The graph also includes a single starting export S from which all portions of the graph can be reached by following a series of arcs. The starting export corresponds to the entire sentence.

Multiple arcs emanate from a node if and only if the same export can be composed of one or more exports (the pair of pointers in a binary arc is not considered multiple arcs for this purpose). Multiple arcs point to a node if and only if that export is a component of multiple exports. A node with no arcs projecting from it corresponds to a dictionary entry assigned to the substring.

A plurality of linguistic experts assign a numerical score to a set of exports. The linguistic experts apply the score to each node of the graph. In one embodiment of the system, a scoring array (where each element of the array is a weight to multiply by a particular expert's score) is a fixed length "N" of floating point numbers for any given sentence.

The score is evaluated by a scoring module which may be integrated with the graph- making engine and/or the parser. Scores are computed for all exports that make up a higher export. The score for the higher export is computed as the sum of the exports that make up the higher level export and the scores of any experts that apply to the combination such as a score assigned by the structural balance expert. The order in which nodes are visited and scored is a standard depth-first graph- walking algorithm. In this algorithm, nodes that have been scored are marked and are not scored again. During the scoring process, the scoring module evaluates dictionary entry nodes before evaluating any of the higher unit nodes. Each dictionary entry gives rise to a single score. Multiple scores result where there are multiple ways of making an export, i.e., k ways of making the export result in k possible scores. Multiple scores are handled as follows: (1) For a unary rule, each of the k scores of the lower export is added to the expert values that apply to the unary rule, and the resulting vector of k scores is associated with the parent export. (2) For a binary rule, assume that the left child has g scores and the right child has h scores. Then a total of g times h scores are computed by adding each of the left child's scores to each of the right child's scores, and in addition, adding the expert values that apply to the binary rule. When g times h exceeds N, only the N best scores are kept with the parent node.

(3) When a node's export can be created in multiple ways, at most N scores are added to that node's score list, the best scores being kept. When scoring is complete, the above methods assure that each export has associated with its node a set of g scores (g ranging from 1 to N) which represent the g most likely ways (relative to the linguistic model) of making the export, including all substructure properties which are not represented in the export. In the special case of the root node S, the scoring method gives rise to the g most likely ways of making the sentence.

Each score in each score list described above has an associated pointer. The pointer provides information to indicate which score(s) of the score list of lower export(s) were combined to produce the higher level score. By following the respective pointers, the g most likely interpretations of the sentence can be extracted as unambiguous parse trees.

Further details of the automated natural language translation system will now be disclosed with reference to FIGS. 1 - 9. Various improvements according to the invention are described thereafter with reference to FIGS. 10 - 16. Referring to FIGS. 1 and 2, an automated natural language translation system 10 according to the invention includes an input interface 12, a translation engine 16, storage 18, a user input device 22, a display 20, and an output interface 14. The input interface is constructed to receive a sequence of text in a source language, such as English or Japanese. The input interface may comprise a keyboard, a voice interface, or a digital electronic interface, such as a modem or a serial input. The translation engine performs translation operations on the source text, in conjunction with data in storage. The translation engine may be comprised entirely of hardwired logic circuitry, or it may contain one or more processing units and associated stored instructions. The engine may include the following elements, or parts of them: A preparser 24, a parser 26, a graph maker 28, a parse/translation evaluator 30, a parse extractor 32, a structural converter 34, and a user interface 42, which includes an alternate parse system 37. The structural converter may comprise a grammar rule controlled structural converter 36, a lexicon controlled structural converter 38, and a synthesis rule controlled structural converter 40. The storage 18 may include one or more areas of disk (e.g., hard, floppy, and/or optical) and/or memory (e.g., RAM) storage, or the like. It may store the following elements, in whole or in part: a base dictionary 44, technical dictionaries 46, user-created dictionaries, grammar rules 48, synthesis rules 50, a semantic feature tree 52, structure trees 54, and a graph 56. The storage 18 also is used to store input textual information in a source natural language, output textual information in a target natural language, and all sorts of information used or useful in performing the translation including one or more dictionaries, domain keywords, and grammar rules. The user input interface 22 may comprise a keyboard, a mouse, touchscreen, light pen, or other user input device, and is to be used by the user of the system. The display may be a computer display, printer or other type of display, or it may include other means of communicating information to the user. The output interface 14 communicates a final translation of the source text in the target language, such as Japanese, Spanish or English. The interface may comprise a printer, a display, a voice interface, an electronic interface, such as a modem or serial line, or it may include other means for communicating that text to the end user. In operation of one embodiment of the translation system of the invention, referring to

FIGS. 1, 2 and 3, the preparser 24 first performs a preparsing operation (step 102) on the source text 23. This operation includes the resolution of ambiguities in sentence boundaries in the source text, and results in a parse chart seeded with dictionary entries 25. The parser 26 then parses the chart produced by the preparser (step 104), to obtain a parse chart filled with syntactic possibilities 27. The graph maker 28 produces a graph of possible interpretations 29 (step 106), based on the parse chart resulting from the parsing step. The evaluator 30, which accesses a series of experts 43, evaluates the graph of stored interpretations (step 108), and adds expert weights to the graph 31. The graph scorer 33 scores nodes and associates the N (e.g., 20) best scores with each of them 35. The parse extractor 32 assigns a parse tree structure 39 to this preferred interpretation (step 110). The structural converter 34, which accesses the conversion tables 58, then performs a structural conversion operation (step 112) on the tree to obtain a translation 41 in the target language. The user may interact with the alternate parse system 37 to obtain alternative translations.

Referring to FIG. 4, the system begins the preparsing operation by dividing the input stream into tokens (step 114), which include individual punctuation marks, and groups of letters that form words. The occurrence of whitespace affects the interpretation of characters at this level. For instance, in "x - y" the "-" is a dash, but in "x-y" it is a hyphen.

The preparser then combines the tokens into words (step 116). At this level, it recognizes special constructions (e.g., Internet addresses, telephone numbers, and social security numbers) as single units. The preparser also uses dictionary lookup to find groupings. For example, if "re- enact" is in the dictionary as "reenact" it will become one word in the sentence, but if it is not, then it will remain as three separate "words" (e.g., "re", "-", "enact").

The next preparsing phase involves determining where the sentence ends (step 118). During this operation, the preparser accesses the base dictionary and the technical dictionaries, and any user-created dictionaries, as it follows a sequence of steps for each possible sentence ending point (i.e., after each word of the source text). The preparser need not perform these steps in the particular order presented, and these may be implemented as a series of ordered rules or they may be hard-coded.

Referring to FIG. 5, the preparser interprets and records any nonparsable sequence of characters, such as a series of dashes: " ", as a "sentence" by itself, although not one which will be translated (step 120). The preparser also requires any sequence of two carriage returns in a row to be the end of a sentence (step 122). If the first letter of the next word is a lower case letter, the preparser will not indicate the end of a sentence (step 124). If a sentence started on a new line and is short, the preparser considers it a "sentence" of its own (e.g., a title).

The preparser interprets a period, a question mark, or an exclamation mark as the end of a sentence, except in certain situations involving end parenthesis and end quotes (step 128). In the case of sentences that ends with ." or ?" or the like, the preparser uses virtual punctuation marks after the quote in addition to the punctuation before the quote. Alternatives for the underlying punctuation required for ?" are illustrated in the following examples: The question was "What do you want?".

Did he ask the question "What do you want?"?

Are you concerned about "the other people"?

In English, each of these is likely to end with ?" The virtual punctuation marks added by the preparser indicate that before the quote there is something which can be either a question mark or nothing at all. After the quote there is something that can be either a period or a question mark. The grammatical structure of the rest of the sentence allows later processing stages to select the best choice.

The preparser may also use several further approaches in preparsing a period (steps 130, 132, 134, 136, and 138). Some abbreviations in the dictionary are marked as never beginning sentences and others as never ending sentences (step 130). These rules are always respected. For example, "Ltd" never begins a sentence and "Mr" never ends one. The preparser also will not end a sentence with a single initial followed by a period unless the next word is a common grammatical word (step 132) such as "the", "in", etc. If the word before the period is found in any dictionary, the period will end the sentence (step 134). If the word before the period is not in this dictionary, and it has internal periods (e.g., I.B.M.) and the next word is not in the dictionary in a lowercase form, or the word after that is itself uppercase, then this is not an end of sentence (step 136). In remaining cases the period does mark the end of sentence (step 138).

Referring again to FIGS. 2 and 3, once the sentence boundaries have been defined by the preparser, the parser places the words of the sentence in syntactic categories, and applies grammar rules from the grammar database to them to compute possible syntactic interpretations 25 of the sentence (step 104). These grammar rules 48 can be implemented as a series of computer readable rules that express the grammatical constraints of the language. For the English language, there may be hundreds of such rules, which may apply to hundreds of syntactic categories. To reduce the computational overhead of this operation, the different possible meanings of a word are ignored.

In the next step (step 106), the graph maker employs the dictionary to expand the results of the parser to include the different meanings of words and creates a directed acyclic graph representing all semantic interpretations of the sentence. This graph is generated with the help of a series of semantic propagation procedures, which are described below. These procedures operate on a series of authored grammar rules and, in some cases, access a semantic feature tree for semantic information. The semantic feature tree is a tree structure that includes semantic categories. It is roughly organized from the abstract to the specific, and permits the procedures to determine how semantically related a pair of terms are, both in terms of their separation in the tree and their levels in the tree. For example, "cat" and "dog" are more related than "cat" and "pudding", and hence the former pair would be separated by a smaller distance within the tree. "Animal" and "cat" are examples of words that are stored at different levels in the tree, as "animal" is a more abstract term than "cat."

Referring to FIG. 9, the graph includes nodes 80 and their subnodes 82, 84, 86 linked by pointers 88, 89, 90, 91 in a manner that indicates various types of relationships. A first type of relationship in the graph is one where nodes representing phrases possess pointers to constituent word nodes or sub-phrase nodes. For example, a node 84 representing the phrase "the bank" will be linked by pointers 92, 93 to the constituent words "the" 94, and "bank" 95. A second type of relationship in the graph is where phrase interpretations possess pointers to alternate ways of making the same higher-level constituent from lower-level pieces. For example, a node 80 representing the phrase "by the bank" can have two source interpretation locations 81, 83, which each include pointers 88 & 89, 90 & 91 to their respective constituents. In this example, the different constituents would include different subnodes 84, 86 that each represent different meanings for the phrase "the bank". The structure of the graph is defined by the results of the parsing operation and is constrained by the syntax of the source sentence. The nodes of the graph are associated with storage locations for semantic information, which can be filled in during the process of semantic propagation.

The semantic propagation part of the system operates to propagate semantic information from smaller constituents to the larger constituents they comprise. It applies to four classes of the syntactic categories used in the earlier parsing operation: SEMNP (which includes noun-like objects and prepositional phrases), SEMVP (verb phrase like objects, which usually take subjects), SEMADJ (adjectives) and VERB (lexical verb-like verbs that often take objects). Other syntactic categories are ignored within a rule. The grammar rule author may also override the implicit behavior below by specific markings on rules. These specific instructions are followed first. There are two aspects to the manner in which semantic features are propagated through the system. The first is a set of rules that tell from examining the noun-like and verb-like constituents in a grammar rale, which selectional restriction slots of the verb-like constituents apply to which noun-like objects. For instance, the rule for the verb phrase of the sentence: "I persuaded him to go" is roughly VP = VT11 + NP + VP (where VP is a verb phrase, VT11 is a type of transitive verb, and NP is a noun phrase). One exemplary default rule indicates that when a verb takes objects, selectional restrictions are to be applied to the first NP encountered to the right of the verb. Another rule says that VP restrictions on their subjects should be applied to the first NP found to the left of a VP. Together these rules make sure that "persuade him" and "him go" are both evaluated for their semantic plausibility. As mentioned before, these rules reflect the complex grammar of the English language and there may therefore be quite a few of them. Referring to FIG. 6, the semantic propagation operation includes copying of selectional restrictions from SEMVPs to imperative sentences (step 140). If a SEMNP is being used as a locative expression, its goodness is evaluated against semantic constants defining good locations (step 142). If a rule involves a conjunction of two SEMNPs (detected because of ANDing together of syntactic features), the graph maker ANDs together the semantic features and applies the semantic distance expert (step 144). If, in the course of examining the rules specified for syntactic feature propagation, the graph maker locates a "head" SEMNP which gets propagated to a higher level (e.g., it becomes part of a SEMNP that includes more words), it propagates semantic features as well (step 146). However, if the "head" is a partitive word (e.g., "portion," "part"), it propagates from a SEMNP to the left or right instead. SEMVPs and SEMADJs are propagated in the same way, with the only exception being that SEMVPs and SEMADJs do not have any partitive situations (step 148). Adjectives are part of the SEMVP class for this purpose.

When a SEMVP is made from a rule including VERBs, the graph maker propagates upward the VERB'S subject restriction unless the VP is a passive construction, in which case the VERB'S first object restriction is propagated instead (step 150). In any rule containing SEMVPs, it attempts to apply the selectional restrictions of the SEMVPs to NPs encountered moving leftward from the SEMVP (step 152). In any rule containing SEMADJs, the graph maker attempts to apply the selectional restriction of the SEMADJ first to any SEMNPs encountered moving to the right from the SEMADJ, and if that fails, tries moving to the left (step 154).

For any remaining unused object selectional restrictions of a VERB (that have not been propagated upward because of passives), the graph maker applies them in turn to SEMNPs encountered in order to the right of the VERB (step 156). In all of these rules, a verb selectional restriction is used up as soon as it applies to something. In all rules up to this one, SEMNPs are not used up when something applies to them. Starting at this rule, the SEMNP does get "used up". Finally, if a rale makes a SEMVP, the graph maker determines if there are any SEMVPs or SEMADJs in it that have not yet been used, and if so, propagates them upward (step 158).

The system also performs feature matching of linguistic features. Linguistic features are properties of words and other constituents. Syntactic feature matching is used by the parser, and semantic feature matching is used by the graph maker. But the same techniques are used for both. For instance, "they" has the syntactic feature plural, while "he" has the feature of singular. Feature matching uses marking on grammar rules so that they only apply if the features of the words they are to apply to meet certain conditions. For example, one rule might be:

S =NP{@} + VP{@}

where the @ signs mean that the number features of the NP and VP must match. So while this rule will allow "they are" and "he is", it will not allow "they is" and "he are".

Feature match restrictions are broken into "local" and "long distance". The long distance actions may be computed when the grammar is compiled, rather than when actually processing a sentence. The sequence of long distance operations that must be performed is then encoded in a series of instruction bytes.

The computation of long distance feature operations must start with an n-ary rule (i.e., one that may have more than two inputs on its right). The system then distributes codes to various binary rules so that feature sets end up being propagated between rules in the correct fashion. By breaking the n-ary rales into binary rules, the parsing operations are greatly simplified, but because the system keeps track of feature sets across binary rules, it retains the power of the long distance operations.

The system of the invention also allows multiword "idioms" as part of the dictionary, while retaining representations of the individual words of which they are composed. These two forms may ultimately compete against each other to be the best representation. For instance "black sheep" is found in the dictionary with the meaning of a disfavored person. But in some cases the words "black sheep" may refer to a sheep which is black. Because both of the forms are retained, this non-idiomatic usage may still be chosen as the correct translation.

The idioms may belong to further categorizations. For example, the system may use the following three types: Almighty: United States of America Preferential: long ago Normal: black sheep

Almighty idioms suppress any other possible interpretation of any of the words that make up the sequence. Preferential idioms suppress other constituents of the same general type and that use the very same words. Normal idioms compete on an even footing with other entries.

The resulting graph is to be evaluated by experts (step 108, FIG. 3), which provide scores that express the likelihood of correctness of interpretations in the graph. The system of the invention includes a scoring method that applies to all partial sentences of any length, not just full sentences. An important element in the use of a graph is that a subtree is fully scored and analyzed only once, even though it may appear in a great many sentences. For example, in the phrase "Near the bank there is a bank," the phrase "Near the bank" has at least two meanings, but the best interpretation of that phrase is determined only once. The phrase "there is a bank" similarly has two interpretations, but the best of those two is determined only once. There are therefore four sentence interpretations, but the subphrases are scored just once. Another feature of the graph is that each node is labeled with easily accessible information about the length of that piece of the sentence. This allows the bestN interpretations of any substring of the English sentence to be found without reanalyzing the sentence.

Although, in one implementation, only the N overall best analyses of the sentence are available at any one time (N being a number on the order of 20), the use of a graph allows the system to integrate the result of a user choice about a smaller constituent and give a different N best analyses that respect the user's choice. Because all this is done without reparsing the sentence or rescoring any substrings, it may be done quickly.

Referring to FIG. 8, operation of the expert evaluator 30 is based on various factors that characterize each translation, which are handled by the various experts. The rule probability expert 170 evaluates the average relative frequency of grammar rules used to obtain the initial source language parse tree. The selectional restriction expert 178 evaluates the degree of semantic accord of the given translation. The dictionary entry probability expert 172 evaluates the average relative frequency of particular "parts of speech" of the words in the sentence used to obtain the initial source language parse tree. The statistics expert 174 evaluates the average relative frequency of particular paraphrases chosen for the given translation. In one embodiment, the dictionary entry probability expert 172 and the statistics expert 174 are combined to create a compounded dictionary expert (not shown).

Some of the experts (e.g., the dictionary entry probability expert 172, the statistics expert 174, the compounded dictionary expert (not shown)) perform the evaluation using weightings associated with the dictionary entries of possible translations. The weighting represents the preference that a given entry is used in translation. Weightings correspond to the multiple inflections that a dictionary entry can take. For example, the dictionary entry can have singular and plural inflections. In some languages, the dictionary entry can have masculine, feminine, or neutral inflections. The dictionary entry can also have different inflections based on the part of speech of the dictionary entry (e.g., infinitive, past, past participle, 3^rd person and/or present participle inflections for a verb).

The system automatically determines the English "part of speech" (POS) for various individual English words, English phrases, and groups of English words. The system makes the automatic determination of the POS when translating sentences, and the system usually makes the correct choice. Occasionally, however, the sentence being translated is itself ambiguous. A word or phrase that can be interpreted as more than one POS leads to several distinct but "correct" meanings for the sentence in which the word or phrase appears. It is possible for a user of the system to override the system's automatic POS determination and instead manually set the POS for any word, phrase, or group of words. For example, in the source English sentence "John saw a boy with a telescope", a user of the system can set "a boy with a telescope" as a Noun

Phrase to force the system to interpret the sentence to mean that the boy was carrying a telescope and thus reject the interpretation that John used a telescope to see the boy. A user can address the situation where overriding the system's POS rules yields worse, not better, translation results by applying a few manual POS settings as possible or by applying less restrictive manual POS settings. Noun Phrase is less restrictive than Noun, and Group is the least restrictive POS setting. The following is a list of the various possible POS settings.

PART OF SPEECH (POS)

Noun

Noun Phrase

Verb (transitive or intransitive)

Verb Phrase Adjective Adjectival Phrase Adverb

Adverbial Phrase Preposition Prepositional Phrase Conjunction Group English

The parts of speech "Adjectival Phrase" and "Adverbial Phrase" are useful in the situation where an English sentence may have a different meaning depending on how a particular prepositional phrase is interpreted by the system. For example, the sentence "We need a book on the fourth of July" means "We need a book about the American fourth of July holiday" if "on the fourth of July" has an adjectival interpretation, but the sentence means "On the fourth day of July, we need a book" if the phrase "on the fourth of July" has an adverbial interpretation. If the user believes the system has automatically assigned the incorrect POS to "on the fourth of July", the user can manually set a different POS to "on the fourth of July" in the sentence "We need a book on the fourth of July". If a user does not want the system to translate a particular word, phrase, or group of words from English to Japanese, the user can assign the POS "English" to the desired word(s), ρhrase(s), and/or group(s) of words. It also is possible for a user to remove one or more POS settings, regardless whether the settings were assigned automatically by the system or manually by a user.

The system keeps track of statistical information from translation usage at each customer site at more than one level. For example, the system may maintain statistical counts at the surface form level (how often was "leaving" used as a transitive versus an intransitive verb), and also at the meaning level (did it mean "leave behind" or "depart" from), and this second type is summed over occurrences of "leave", "leaves", "left", and "leaving". The system may also keep statistical counts separately for uses that have occurred within the last several sentences, and uses that have occurred at any time at the customer site. Furthermore, the system may distinguish cases where the user intervened to indicate that a particular word sense should be used, from cases where the system used a particular word sense without any confirmation from the user. The structural balance expert 182 is based on a characteristic of English and many other

European languages pertaining to the lengths of constituents in a given sentence. In some (but not all) constructions, sentences which involve heavy (lengthy) elements to the left of light elements are disliked in these languages. For example: Mary hit Bill with a broom.

Light Heavy {acceptable}

Mary hit with a broom Bill.

Heavy Light {unacceptable}

Mary hit with a broom a dog that tried to bite her.

Heavy Heavier

{acceptable}

Given two parses of a given sentence, if one contains a "Heavy - Light" sequence involving a construction that tends to avoid such a sequence, and if the other parse does not, then it can be assumed that the former does not represent the intended interpretation of the sentence. This expert is an effective way to distinguish between intended parses and unintended parses.

In coordinate structures of the pattern of "A of B and C" it can be difficult to determine whether the intended interpretation is "A of {B and C}" or "A {of B} and C". The coordinate structure expert 180 measures the semantic distance between B and C, and that between A and C to determine which mode of coordination combines two elements that are closer in meaning. This expert accesses the semantic feature tree during its operation. This expert is also an efficient way to distinguish between the intended parses and the unintended parses of a given sentence. Many words in English include potential ambiguities between ordinary-noun and proper- name interpretations. The capitalization expert 176 uses the location of capitalization in a sentence to determine how likely it is that the capitalization is significant. For example, the following sentences:

Brown is my first choice. My first choice is Brown. are different in that while the former is genuinely ambiguous, it is far more likely in the latter that "Brown" is a person name than a color name. This expert takes into consideration factors such as whether a given capitalized word appears at sentence-initial or sentence-noninitial position (as shown above), whether the capitalized spelling is in the dictionary, and whether the lower-case- initial version is in the dictionary. This expert is an effective way to obtain the correct interpretations of capitalized words in sentences.

If a sentence contains a sequence of initial-uppercase words, it can be treated as a proper name or as a sequence of ordinary nouns. The system of the invention employs a capitalized sequence procedure, which favors the former interpretation. Also, if the sequence cannot itself be parsed by normal grammar rules, it can be treated as a single unanalyzed noun phrase to be passed through untranslated. This procedure has proven to be a very effective way of dealing with multiple- word proper names while not completely ignoring the lower-rated common noun interpretations. Referring to FIG. 7, the machine translation system of the invention uses a grammar-rule controlled structural conversion mechanism 162 that has the efficiency of a straightforward grammar-rule-based structural conversion method, but which comes close to the power of the template-to-template structural conversion method. This method relies on the use of grammar rules 160 which can specify non-flat complex substructure. While the following is a rule format used in other translation systems:

Y => Xl + X2 + ... Xn Substructure Specified Y XI X2 ... Xn the system of the invention uses grammar rules of the following format: Y => #Zl(i) #Z2(2) XI + X2 ... + Xi + X(i+1) + ... X(n) Substructure Specified Y

/ I \ #Z1 X(i+1) Xn

/ I \

#S2 Xi X(i+1) / \ XI X2 In this syntax, symbols prefixed with "#" are virtual symbols that are invisible for the purpose of sentence structure parsing, but which are used in building substructures once a given parse is obtained.

Given this type of grammar, it becomes possible to specify multiple structure conversion among any sequences of nodes that are siblings in the substructure. This transforms a grammar- rule-based structure conversion mechanism into one with some of the power of the template-to- template structure conversion mechanism. Although the system of the invention is based on the second type of grammar rales presented above, it automatically compiles the corresponding grammar rales of the first form. It can therefore use grammar rules of the first form to parse sentences, and use grammar rules of the second form to build parse structures.

The structural conversion also includes a dictionary controlled structural conversion operation 166, which accesses dictionaries 161 to operate on the parse tree after it has been operated upon by the grammar-rule controlled structural conversion operation. The synthesis- rule controlled structural conversion operation then applies synthesis rales to the resulting parse tree to provide the target language text 41.

Referring again to FIGS. 1 and 2, after the system has derived a best-ranked translation in the process described above, it is presented to the user via the display 20. The user then has the option of approving or editing the translation by interacting with the alternate parse system 37, via the user input device 22. In the editing operation, the user may constrain portions of the translated sentence that are correctly translated, while requesting retranslation of the remaining portions of the sentence. This operation may be performed rapidly, since the system retains the graph with expert weights 31.

Having described certain details of the automated natural language translation system with reference to FIGS. 1 - 9, the present invention is now described with reference to FIGS. 10 - 16. The present invention relates to allowing a user to view the weightings used by the various experts (e.g., the dictionary entry probability expert 172, the statistics expert 174, the compounded dictionary expert (not shown)) of the automated translation system. Further, the present invention relates to allowing the user to alter the weightings to influence subsequent translations.

Referring to FIG. 10, the automated translation system 200 of the invention includes a dictionary browser 205. The dictionary browser 205 is hardware and/or software code in communication with the storage 18 to obtain and manipulate data in the base dictionary 44, any technical dictionaries 46, any user-created dictionaries (not shown) and/or any other dictionaries, including third party sources, to which the user has access. The dictionary browser 205 is in communication with the user input device 22 for accepting commands and/or data from the user and in communication with the display 20 for displaying dictionary data to the user. The dictionary browser 205 is in communication with the translation engine 16 for allowing the user to call or initiate the dictionary browser 205 during a translation session. In another embodiment, the dictionary browser 205 is part of the software code contained in the translation engine 16. FIG. 11 depicts an exemplary embodiment of a graphical user interface ("GUI") that the dictionary browser 205 generates for the display 20. The GUI includes a dictionary entries frame 225, a selection criteria frame 230 and a weightings display frame 235. The GUI displays to the user in the dictionary entries frame 225 all of the strings (e.g., words, phrases) in a source language that meet the criteria displayed in the selection criteria frame 230 entered by the user, along with each of the possible translations associated with each source language string. In one embodiment, the dictionary entries frame 225 displays the possible translations of a source language string as an indented list (e.g., sub-nodes, children nodes) of the source language string (e.g., source node, parent node). In this embodiment, the user can hide the list of translations by clicking on the minus sign box 228 to the left of the source language string. Similarly, the user can display the list of translations, when they are hidden, by clicking on the plus sign box (not shown) to the left of the source language string.

In FIG. 11, the dictionary entries frame 225 contains all of the entries in all of the dictionaries 23 Oe to which the user has access that have words in the source language 230a that start with 230b the characters "banlc" 230c and are used as any part of speech 23 Od. The GUI allows the user to change each criterion 230a, 230b, 230c, 230d, 230e using drop down menus. For example, the language criterion 230a can be changed to find translations in the target language that begin 230b with the character(s) entered in the text box 230c. The language criterion 230a also can be changed to find either words in the source language or translations in the target language that begin 230b with the character(s) entered in the text box 230c. The position criterion 230b can be changed to find words that end with the character(s) entered in the text box 230c, to find only exact matches, to find rough matches or to list all entries in the selected dictionaries 230e. The text box 230c is where the user enters one or more characters of the word the user is seeking. The POS criterion 230d allows the user to limit the words to specific parts of speech. The selected dictionaries criterion 23 Oe allows the user to pick all or specific dictionaries to which he/she has access. In another embodiment, the dictionary browser 205 can be called from the translation engine 16. In this embodiment, the translation engine 16 passes to the dictionary browser 205 a selected string of characters from an automated translation to be inserted in the text box 230c. The weighting display frame 235 displays the weightings 245 of the different inflections of the highlighted translation 250. As shown in FIG. 11, each of the weightings 245 a, 245b, 245c, 245d, 245e is equal. This represents that the preference that the translation of "depositar" as a transitive verb for the source text banlc is equal among the possible inflections (i.e., infinitive, past, past participle, 3^rd person, present participle) the source text verb can take.

The weighting display frame 235 also includes a summary window 240. The summary window 240 displays the weightings 245a, 245b, 245c, 245d, 245e of each of the inflections of the highlighted dictionary entry 250 on a single bar 241. In this embodiment, all weightings are shown as movable beads (e.g., 240a) on a bar (e.g., 241). The bar 241 represents the inclusive range of possible values within which the weightings 245a, 245b, 245c, 245d, 2145e can be positioned. In one embodiment, the range of possible values is normalized. The normalized range is from —1.0 to +1.0. The dictionary browser 205 normalizes the range because the dictionaries (e.g., the base dictionary 44, any technical dictionaries 46, any user-created dictionaries (not shown) and/or any other dictionaries, including third party sources) that the dictionary browser 205 accesses can have varied ranges (e.g., 0 to 100, -50 to 0) of weighted entries. Normalizing these varied ranges allows the user to see a consistent representation of the weightings (e.g., 245a, 245b, 245c, 245d, 245e) and relative comparison of the weighting of entries among the different dictionaries, regardless of their varied ranges. To change the weighting 245a, 245b, 245c, 245d, 245e, the user moves the bead (e.g., 240a) to the desired value.

In FIG. 11, since all of the weightings 245a, 245b, 245c, 245d, 245e are equal, there is only one bead representing the weightings 245a, 245b, 245c, 245d, 245e in the summary window 240. Therefore, the user moves the bead 240a of the summary window 240 along the bar 241 to change the value of all of the weightings 245a, 245b, 245c, 245d, 245e equally. In one embodiment, the GUI displays to the user the numerical representation of the weightings 245 in a small text box (e.g., a ToolTip) when the user moves the cursor over the corresponding bead. Since the summary window 240 displays a composite of each of the weightings 245, when the cursor is moved over the bead 240a of the summary window 240, each of the inflections (i.e., infinitive, past, past participle, 3 ^rd person, present participle) and their corresponding weightings 245a, 245b, 245c, 245d, 245e are displayed in a small text box (e.g., ToolTip).

In addition to moving the weighting bead 240a to change the weightings equally, the user can change the values of each of the weightings 245a, 245b, 245c, 245d, 245e individually. FIG. 12 depicts an example where the user has set each of the weightings 245 individually to different values. Note however that the two weightings 245b and 245c associated with the past and past participle forms of the verb bank are equal. Since the source language word of these two inflections is the same, the weightings 245b, 245c of the two must also be the same. The summary window 240 is the composite of the weightings 245a, 245b, 245c, 245d, 245e and thus is shown as four beads 240a, 240b, 240c, 240d corresponding to the four different values of the weightings 245. Different weightings might be used where different translations are preferred or more common depending on the form the source text takes. The user can make these adjustments based on the dialect for which the user is translating.

For example, FIG. 12 depicts weightings 245 where the present participle form "banking" has an extremely high preference of being translated using "depositar", because the weighting 245e is nearly +1.0, whereas the infinitive form "banlc" would have a very low preference of being translated using "depositar", because the weighting 245a is nearly -1.0. The vertical line 255 on each of the bars of the weightings 245a, 2245b, 245c, 245d, 245e represents the factory default setting of that weighting. In one embodiment, the user can set all of the values of the weightings 245 back to factory defaults at any time.

As an illustrative example, FIG. 13 depicts a screen shot of the automated translation of the prepositional phrase "by the bank." To translate this phrase, the automated translation system 200 creates a graph as depicted in FIG. 9 and described in the associated text. Table 1 lists the possible translations for each of the constituent words of the prepositional phrase depicted in FIG. 9. The values shown in Table 1 for each of the possible translations are the weightings associated with that translation. The articles do not have a weighting because the article is chosen for agreement based on the gender and the number (i.e., singular or plural) of the noun.

Table 1 FIG. 14 depicts a screenshot where the possible translations for "bank" as a noun are listed in the dictionary entries frame 225. Each of these possible translations has one or more associated weightings. For example, the highlighted translation "banco" 270 has five weightings 260, 261, 262, 263, 264. There are five weightings because there are five different meanings that the translation can have. The meanings are represented to other experts using the features list 275. Other experts compare these features with features of other words and sentences being translated, as described above, to score the possible translations. In this example, all of the five weightings 260, 261, 262, 263, 264 for the highlighted translation "banco" 270 are equal, representing that for each of the five feature lists, the preference that the translation "banco" is used is equivalent. Thus in Table 1, the weighting for the translation "banco" is the same in both columns. If the dialect to which the user is translating does not use "banco" for one of the feature lists, the user can adjust the weighting(s) accordingly (e.g., decrease the weighting associated with that particular feature list not used and/or increase the weighting of all of the other feature lists that are more likely used). FIG. 15 depicts a screenshot where the possible translations for "by" as a preposition are listed in the dictionary entries frame 225. Each of these possible translations has one or more associated weightings. For example, the highlighted translation "por" 280 has six weightings 281, 282, 283, 284, 285, 286. In this example, all of the six weightings 281, 282, 283, 284, 285, 286 for the highlighted translation "por" 280 are equal, representing that for each of the six feature lists, the preference that the translation "por" is used is equivalent. Thus in Table 1, the weighting for the translation "por" has a single value. If the dialect to which the user is translating does not use "por" for one of the feature lists, the user can adjust the weighting(s) accordingly.

Referring back to Table 1, the features of each translation of the noun "bank" help determine the column, side of river or financial institution, the translation belongs. In this example however, since there are no other words or sentences to give the translation context (i.e., aid the other experts in scoring certain translations higher), the automated translation system 200 selects the translation with the highest weighting, regardless of the column the translation is in. If the other experts were able to eliminate a column because of other words or sentences, then the automated translation system 200 selects the translation with the highest value in the remaining column. In other words, with the scoring by the other experts being equal (e.g., one group of translations left to select from where the translations are semantically and grammatically similar), the weighting determines the translation that is selected from the possible translations. As shown in Table 1, the translation "por" for "by" has the highest value and the translation "banco" for "bank" has the highest value. The article "el" is chqsen because the noun "banco" is masculine. Thus, the translation "Por el banco" shown in FIG. 13 is chosen to translate the phrase "By the banlc."

If the user of the automated translation system 200 does not agree with this translation, the user can adjust the weightings to affect the translation. In the GUI shown in FIG. 11, the user slides the bead (e.g., 240a) to adjust the weighting. If the user wants to adjust all of the inflections to an equal value, the user can adjust the value using the bead 240a in the summary window. If the user wants to adjust only one of the weightings 245a, 245b, 245c, 245d, 245e, the user adjusts the bead of the corresponding to a specific inflection weighting only. For example, the user knows that the preferred translation for "by the bank" is "Junto a la banda." The user adjusts each of the weightings for the translations "junto a" and "banda" higher (e.g., moves the bead to the right). In the subsequent translation, as shown in FIG. 16, the translations "junto a" and "banda" are chosen because as described above, with all else being equal, the translations with the highest weightings are chosen and the user has set the weightings of the preferred translations higher than the others. The article "la" is now used because the preferred translation for the noun is now feminine. The value chosen by the user depends on how strong of a preference the user wants the preferred translation to appear. If the user wants the preferred translation regardless of the other experts, the user sets each of the weightings for the preferred translations near the maximum of the range (e.g., normalized value of +1.0). If the user only wants a slight preference above the other translations and wants the other experts to still influence the translations, the user sets each of the weightings to something slightly higher than the other possible translations that are in the same group (e.g., same column).

Even if the user adjustment of the weightings does not quite exactly give the preferred translation, there are some automatic adjustments based on user preferences that will eventually set the weightings to where the user wants them. The automated translation system 200 adjusts the weightings each time the user selects an alternate translation. The automated translation system 200 increases the weighting of the alternate translation that the user selects. In another embodiment, the automated translation system 200 decreases the weighting of the system 200 selected translation the user has replaced. In one embodiment, the value of the increase and/or decrease is based on the location of the value with respect to the range limits. For example, if the value is located in the center of the range (e.g., 0.0 in a range of-1.0 to +1.0), the automated translation system 200 increases the value of the weighting of the preferred translation by a large amount (e.g., 0.5 in a range of _1.0 to +1.0). If the value is located near the end of the range (e.g., 0.88 in a range of-1.0 to +1.0), the automated translation system 200 increases the value of the weighting of the preferred translation by a small amount (e.g., 0.05 in a range of _1.0 to +1.0). The value of the increase and/or decrease can be chosen to optimize the automatic adjustment by the automated translation system 200.

For example, in the "by the bank" translation, the user adjustment of the weightings of "junto a" and "banda" are small increases and on the subsequent translation the automated translation system 200 selects "por el banco" for the translation. Instead of adjusting the weightings again, the user selects the alternative translation "junto a la banda" out of a list of alternative translations. With the selection of the alternative, the automated translation system 200 increases the weightings of "junto a" and "banda" and/or decreases the weightings of "por" and "banco." The amount of increase or decrease can follow any optimized algorithm.

All of the above-described functions and operations may be implemented by a variety of hardwired logic design and/or programming techniques for use with a general purpose computer. The steps as presented in the flowcharts generally need not be applied in the order presented, and combinations of the steps may be combined. Similarly, the functionality of the system may be distributed into programs and data in various ways. Furthermore, it may be advantageous to develop the grammar and other operative rules in one or more high level languages, while supplying them to end users in a compiled format.

Any of the embodiments of the automated natural language translation system described herein, including all of the functionality described herein, can be provided as computer software on a computer-readable medium such as a diskette or an optical compact disc (CD) for execution on a general purpose computer (e.g., an Apple Macintosh, an IBM PC or compatible, a Sun Workstation, etc.). The software can be delivered (to the user, for example) over the Internet or some other network, and not necessarily on a physical device like a CD. Also, the software can be Internet-based in that it is accessed and used by users via the Internet. Variations, modifications, and other implementations of what is described herein will occur to those of ordinary skill in the art without departing from the spirit and the scope of the invention as claimed. Accordingly, the invention is to be defined not by the preceding illustrative description but instead by the spirit and scope of the following claims. What is claimed is:

Claims

Claims 1. A method of allowing a user to view and modify a weighting associated with a translation of a source language string, comprising: displaying to the user the weighting associated with the translation of the source language string, the weighting for use by a translation engine in selecting the translation; and allowing the user to modify the weighting associated with the translation.

2. The method of claim 1 further comprising allowing the user to reset the weighting back to a default value subsequent to user modification of the weighting.

3. The method of claim 1 wherein the displaying step further comprises displaying to the user a plurality of weightings, each of the weightings associated with an inflection of the translation, each of the weightings for use by the translation engine in selecting the translation.

4. The method of claim 3 wherein the allowing step further comprises allowing the user to individually modify each of the one or more weightings associated with the inflections of the translation.

5. The method of claim 3 further comprising displaying in a summary window all of the one or more weightings associate with the inflections on a single representation of a range of possible values.

6. The method of claim 1 wherein the source language string and the translation are contained in a translation dictionary.

7. The method of claim 1 further comprising displaying to the user the translation in response to an input from the user.

8. The method of claim 7 wherein the input is at least one character in the source language.

9. The method of claim 7 wherein the input is at least one character in the target language of the translation.

10. The method of claim 1 further comprising displaying to the user the translation in response to an input from the translation engine.

11. The method of claim 1 further comprising: selecting, with the translation engine, the translation from among a plurality of possible translations of the source language string, each of the plurality of possible translations having associated therewith a weighting, the selection based at least in part on the weighting.

12. The method of claim 1 further comprising: receiving a source language string; determining by at least one expert system a plurality of possible translations, each translation in the plurality equally translating the source language string; and selecting the translation that has the highest weighting from the plurality of possible translations.

13. The method of claim 1 further comprising providing a graphical user interface for displaying to the user the weighting.

14. The method of claim 13 further comprising: providing a graphical user interface for displaying to the user at least one source language dictionary entry meeting established criteria; and allowing the user to hide and display one or more translations of the at least one source language dictionary entry.

15. The method of claim 13 further comprising adjusting the weighting in response and proportion to the user manipulation of the graphical user interface.

16. The method of claim 13 wherein the graphical user interface represents to the user a current value of the weighting relative to a range of possible values.

17. The method of claim 16 wherein the range of possible values is normalized.

18. The method of claim 1 further comprising: allowing the user to select the translation from a plurality of possible translations; and adjusting, by the translation engine, at least one of the weightings of the plurality of translations in response to the user selection.

19. The method of claim 18 wherein the step of adjusting further comprises increasing the associated weighting of the selected translation.

20. The method of claim 18 wherein the step of adjusting further comprises decreasing the associated weighting of the translation replaced by the user selection.

21. A machine translation system for allowing a user to view and modify a weighting associated with a translation of a source language comprising: a dictionary browser configured to provide data used to display to the user the weighting associated with the translation of the source language string, the weighting for use by a translation engine in selecting the translation and configured to allow the user to modify the weighting associated with the translation; a translation dictionary in communication with the dictionary browser for receiving and transmitting data, the translation dictionary comprising at least one translation and the associated weighting, the translation dictionary changing the associated weighting in response to data received from the dictionary browser; a user input device in communication with the dictionary browser, the user input device receiving data from the user and transmitting the data to the dictionary browser; and a display device in communication with the dictionary browser, the display device receiving data from the dictionary browser and displaying a representation of the data to the user.

22. The system of claim 21 wherein the dictionary browser is further configured to allow the user to reset the weighting back to a default value subsequent to user modification of the weighting.

23. The system of claim 21 wherein the dictionary browser is further configured to provide data used to display to the user one or more weightings, each of the weightings associated with an inflection of the source language string, each of the weightings for use by the translation engine in selecting the translation.

24. The system of claim 23 wherein the dictionary browser is further configured to allow the user to individually modify each of the one or more weightings associated with the inflections of the translation.

25. The system of claim 23 further configured to provide data to the display device to display in a summary window all of the one or more weightings associate with the inflections on a single representation of a range of possible values.

26. The system of claim 21 wherein the dictionary browser is further configured to provide data used to display the translation in response to an input received from the user.

27. The system of claim 26 wherein the input is at least one character in the source language.

28. The system of claim 26 wherein the input is at least one character in the target language of the translation.

29. The system of claim 21 wherein the dictionary browser is further configured to provide a graphical user interface to display to the user the weighting for the translation.

30. The system of claim 29 wherein the dictionary browser is further configured to provide a graphical user interface for displaying to the user at least one source language dictionary entry meeting established criteria and allowing the user to hide and display one or more translations of the at least one source language dictionary entry.

31. The system of claim 29 wherein the dictionary browser is further configured to adjust the weighting in response and proportion to the user manipulation of the graphical user interface.

32. The system of claim 29 wherein the graphical user interface is configured to represent to the user a current value of the weighting relative to a range of possible values.

33. The system of claim 32 wherein the range of possible values is normalized.

34. The system of claim 21 further comprising: a translation engine in communication with the dictionary browser, the translation engine selecting the translation based at least in part on the weighting of the translation.

35. The system of claim 34 wherein the translation engine is further configured to receive a source language string, configured to determine by at least one expert system a plurality of possible translations, each translation equally translating the source language and configured to select the translation that has the highest weighting from the plurality of possible translations.

36. The system of claim 34 wherein the translation engine is further configured to allow the user to select the translation from a plurality of possible translations and the dictionary browser is further configured to adjust at least one of the weightings of the plurality of possible translations in response to the user selection.

37. The system of claim 36 wherein the dictionary browser is further configured to increase the associated weighting of the selected translation.

38. The system of claim 36 wherein the dictionary browser is further configured to decrease the associated weighting of the translation replaced by the user selection.

39. An article of manufacture having computer-readable program means for allowing a user to view and modify a weighting of a translation, the article comprising: computer-readable program means for displaying to the user the weighting associated with the translation of the source language string, the weighting for use by a translation engine in selecting the translation; and computer-readable program means for allowing the user to modify the weighting associated with the translation.