WO2007126447A2 - Systems and methods for identifying parallel documents and sentence fragments in multilingual document collections - Google Patents
Systems and methods for identifying parallel documents and sentence fragments in multilingual document collections Download PDFInfo
- Publication number
- WO2007126447A2 WO2007126447A2 PCT/US2007/000853 US2007000853W WO2007126447A2 WO 2007126447 A2 WO2007126447 A2 WO 2007126447A2 US 2007000853 W US2007000853 W US 2007000853W WO 2007126447 A2 WO2007126447 A2 WO 2007126447A2
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- target
- source
- document
- parallel
- sentences
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/40—Processing or translation of natural language
- G06F40/42—Data-driven translation
- G06F40/45—Example-based machine translation; Alignment
Definitions
- the present invention relates generally to statistical machine translation of multilingual documents and more specifically to systems and methods for identifying parallel segments in multilingual document collections.
- Training data comprises parallel segments which are documents or fragments that are literal, or parallel, translations of each other in two languages.
- a language pair refers to the two languages used within the parallel corpora. Examples of language pairs include English-Romanian or English- Arabic.
- these comparable segments may include one or more parallel fragments that can be added to the training data even though the entire segment is not a parallel translation of a comparable segment.
- a quote within a news article may be translated literally even though the rest of the document is merely related to a comparable segment in another language.
- the method for identifying parallel sub-sentential fragments in a bilingual collection comprises translating a source document from a bilingual collection.
- the method further includes querying a target library associated with the bilingual collection using the translated source document, and identifying one or more target documents based on the query.
- a source sentence associated with the source document is aligned to one or more target sentences associated with the one or more target documents.
- the method includes determining whether a source fragment associated with the source sentence comprises a parallel translation of a target fragment associated with the one or more target sentences.
- FIG. 1 depicts an illustration of an environment in which some embodiments may be practiced
- FIG. 2 depicts a block diagram illustrating an exemplary training set generator according to exemplary embodiments
- FIG. 3 depicts a block diagram illustrating an exemplary parallel document engine according to exemplary embodiments
- FIG. 4 depicts a block diagram illustrating an exemplary parallel fragment engine according to exemplary embodiments
- FIG. 5 depicts a flowchart illustrating an exemplary process for automatically generating parallel corpora from multi-lingual document sources according to exemplary embodiments
- FIG. 6 depicts a flowchart of an exemplary process for determining whether documents are parallel based on sentence alignments according to exemplary embodiments
- FIG. 7 depicts a flowchart of an exemplary process for extracting parallel sentence fragments from comparable corpora according to exemplary embodiments.
- FIG. 8 depicts a flowchart of an exemplary process for generating a fine lexicon according to exemplary embodiments.
- a system and method for identifying parallel documents and/or fragments in a bilingual document collection is provided.
- the present method and system can be used with documents posted on the Internet without relying on properties such as page structure or URL. Further, the system and method is able to distinguish between parallel documents and comparable documents.
- the method and system may alternatively or additionally be used to extract parallel fragments from comparable corpora at the sub-sentential level to increase the amount of parallel data for statistical machine translation (SMT).
- SMT statistical machine translation
- FIG. 1 depicts an illustration of an environment 100 in which some embodiments maybe practiced.
- the environment 100 may comprise a document collection server 102 communicatively connected to a client 106 via a network 104.
- the document collection server 102 may comprise a memory 110 configured to store one or more bilingual document collections, one or more parallel documents, one or more parallel fragments, and/or one or more set of annotations indicating parallel data.
- the document collection server 102 may comprise a processor configured to access the memory 110 and process instructions.
- the document collection server 102 may communicate over the network 104 using a communications interface.
- the network 104 may comprise a public network (e.g., the Internet) or a private network.
- the client 106 may include storage, a display, or additional functionality beyond the scope of the embodiments.
- the client 106 may comprise a computational device such as a personal computer.
- the client 106 may include a training set generator 108.
- the training set generator 108 may comprise hardware, software, or firmware and may be configured to identify parallel documents and/or fragments in bilingual collections.
- the training set generator 108 may be configured to access bilingual collections stored in the document collection server 102. Further, the training set generator 108 may transmit data indicating parallel documents or fragments to the document collection server 102.
- FIG. 2 depicts a block diagram illustrating an exemplary training set generator, such as the training set generator 108 of FIG. 1, according to exemplary embodiments.
- the training set generator 108 may identify parallel documents and/or fragments within bilingual document collections.
- the bilingual document collections may comprise documents collected from the Internet or the World Wide Web (WWW) in exemplary embodiments.
- the training set generator 108 may comprise a word translator engine 202, a query engine 204, a document selector engine 206, a parallel document engine 208, and a parallel fragment engine 210. Because sentence alignment requires a large amount of computations, the word translator engine 202, and the query engine 204 may be configured to reduce the number of documents with which sentence alignment is performed.
- the word translator engine 202 is configured to translate each word within a source document to generate a translated source document.
- a source document is a document in the bilingual document collection of which a parallel translation is sought.
- the source document for example, may comprise a news article written in English.
- the training set generator 108 may be configured to determine whether any documents written in Romanian associated with a target library are parallel translations of, or contain parallel translations of fragments of, the source document.
- a target library comprises one or more target documents.
- Target documents may include documents or segments that possibly comprise a parallel translation of the source document.
- Target documents may be obtained from sources such as the Internet or other known libraries and may be classified according to a classifier such as date, URL, or the like.
- the word translator engine 202 translates the source document word-by- word or literally into the language of the target library. To further the example, the news article written in English is translated word-by-word into Romanian, which may be utilized as the language of the target library.
- the query engine 204 is configured to query the target library using the translated source document.
- the query engine 204 selects a subset comprising the documents that are most similar to the translated source document.
- the subset may be limited to the top N documents.
- N may be a constant such as the number twenty (20).
- the document selector engine 206 aligns one or more target sentences in each of the target documents in the subset to one or more sentences in the source document. By aligning the sentences, the training set generator 108 avoids falsely classifying documents as parallel based only on word-level comparisons. Based on the sentence alignments, the document selector engine 206 may discard one or more of the top N documents that, while comprising word-level translations, do not comprise sentence-level translations. For example, the document selector engine 206 may discard documents that do not have a specified number of sentences that can be aligned. In other embodiments, the document selector engine 206 may discard sentence pairs within the aligned documents that do not contain a specified number or percentage of words that are translations of each other.
- the parallel document engine 208 is configured to determine whether a document pair comprising the source document and the target document is a parallel translation. The parallel document engine 208 is discussed further with respect to FIG. 3 and FIG. 6.
- the parallel fragment engine 210 is configured to identify parallel sentence fragments in non-parallel document pairs such as document pairs from news sources on the Internet.
- the parallel fragment engine 210 is discussed further in FIG. 4, FIG. 8, and FIG. 9.
- FIG. 2 shows various engines comprising the training set generator 108, fewer or more engines may comprise the training set generator 108 and still fall within the scope of the various embodiments.
- FIG. 3 depicts a block diagram illustrating an exemplary parallel document engine 208 according to exemplary embodiments.
- the parallel document engine 208 determines whether the target document is the parallel translation of the source document.
- the parallel document engine 208 comprises a sentence analysis module 302 and a document classification module 304.
- the sentence analysis module 302 determines whether aligned sentences based on the sentence alignments generated by the document selector engine 206 are translations of each other independently of the context within the document or segment.
- the sentence analysis module 302 inspects the sentence pairs between the sentences in the source document and the sentences in each of the documents in the subset comprising the documents that are most similar to the translated source document to compute sentence-level links.
- the sentence analysis module 302 distinguishes between parallel and non-parallel sentence pairs based on sentence features that can be extracted from a word-level alignment of the two aligned sentences.
- the sentence features are identified by linking each word in the sentences with its best translation candidate in the paired sentence. This linking process may be referred to as "greedily linking.”
- sp is the percentage of words in a sentence pair that have a translation in the other sentence of the sentence pair.
- Z(sp) is a normalization factor
- sndfy are the feature functions (indexed both by class and by feature).
- a feature function, /f ⁇ ?) is defined as the value of the word overlap of the sentences in the sentence pair sp.
- the resulting model has free parameters ⁇ ,, the feature weights.
- the parameter values that maximize the likelihood of a given training corpus can be computed using known optimization algorithms. Based on the positive and negative classifiers, sentence level links between the source document and the documents within the subset are obtained.
- the document classification module 304 is configured to select the document within the subset that shares the most sentence level links with the source document.
- the document within the subset that shares the most sentence-level links with the source document is referred to as the target document.
- the document classification module 304 determines whether the target document is a parallel translation of the source document by comparing the length of the target document to the length of the source document and determining whether a noisy sentence threshold and a monotone sentence threshold are met. This process is described in greater detail in association with FIG. 6.
- the output of the document classification module 304 may comprise an annotation indicating that the target document is a parallel translation of the source document.
- FIG. 4 depicts a block diagram illustrating an exemplary parallel fragment engine 210 according to exemplary embodiments.
- the parallel fragment engine 210 may be configured to extract parallel sentence fragments from within non- parallel corpora.
- the parallel fragment engine 210 is configured to determine whether a sentence fragment in the source document is a parallel translation of a sentence fragment in the target document. For example, if the document classification module 304 determines that the source document and the target document are non- parallel, that is the target document is not a parallel translation of the source document, the documents are likely to be comparable documents. Comparable documents may also be referred to as "noisy translations."
- the comparable documents may comprise sub-sentential parallel fragments that may be extracted by the parallel fragment engine 210 and added to a training data set.
- the parallel fragment engine 210 may comprise a candidate selection filter 402, a coarse lexicon 404, a probability module 406, a fine lexicon 408, and a fragment detection module 410.
- the candidate selection filter 402 discards sentence pairs in the subset comprising the segments or documents selected by the document selection engine 206.
- the candidate selection filter 402 discards sentence pairs that have very few words that are translations of each other based on the coarse lexicon 404. For example, sentence pairs that comprise three or fewer words that appear in both the source sentence and the target sentence may be discarded. In other embodiments, sentence pairs may be discarded according to a percentage of words appearing in both the source sentence and the target sentence.
- the coarse lexicon 404 comprises a probabilistic lexicon derived from an initial parallel corpus such as a training data set.
- the coarse lexicon 404 may be obtained by running a GIZ A++ implementation of the IBM word alignment models on the initial parallel corpus.
- each source word may be associated with many possible translations. For example, in some embodiments, each source word is associated with, on average, twelve possible translations. Each of these possible translations may be further associated with a probability.
- the coarse lexicon is used to retain most of the existing comparable sentences occurring between the source document and the target document.
- the probability module 406 is configured to greedily link each word in the source sentence to the best translation occurring in the target sentence. Using the fine lexicon 408, the probability module 406 then assigns values to each word alignment indicating the probability that the source word is properly aligned with, or a translation of, the target word. The probability module 406 may filter these values further.
- the fine lexicon 408 may be generated in exemplary embodiments according to the process discussed in connection with FIG. 8.
- the fragment detection module 410 may be configured to detect a parallel sentence fragment based on the values assigned to each word alignment based on the fine lexicon 408. In some embodiments, the fragment detection module 410 may further filter the fragment matches. This process is discussed in greater detail in connection with FIG. 7.
- the output of the parallel fragment engine 210 comprises at least one parallel fragment that may be added to the training set.
- FIG. 5 depicts a flowchart illustrating an exemplary process 500 for automatically generating parallel corpora from bilingual document sources according to exemplary embodiments.
- the process 500 identifies parallel segments and fragments in the bilingual documents according to sentence pairs identified between the documents.
- the process 500 may be used to generate a training set for statistical machine translation (SMT).
- SMT statistical machine translation
- the process 500 comprises accessing a bilingual document collection, translating words in a source document, querying a target library using the translated source document, selecting a subset comprising a target document from the target library, aligning the sentences in the source document with sentences in the target document, and determining whether the source document is a parallel translation of the target document.
- the method may alternatively or additionally include determining whether a fragment in the source document is parallel to a fragment in the target document. If the source document or a fragment thereof is a parallel translation of the target document, the parallel portions may be stored within a training set.
- each word in the source document is independently translated into the language of the target document to generate a translated source document.
- the words may be translated according to a dictionary, for example.
- the translated source document is run as a query against a large library comprising documents written in the target language.
- the source document is paired with a subset comprising the most similar documents written in the target language according to the results of the query.
- the subset may be limited to a constant number, N, of the most similar documents.
- N may be any number up to and including the number of . documents in the target library.
- N may be limited by a computational capacity of a computer such as client 106.
- step 508 parallel sentence pairs between the source document and the documents in the subset are identified.
- the sentence pairs may be identified by the sentence analysis module 302.
- step 510 is performed to determine whether the source document and at least one of the documents in the subset are parallel translations.
- the parallel document engine 208 may perform this determination.
- the process used in step 510 is discussed in greater detail in connection with FIG. 6. If the documents are parallel, the parallel documents are added to the training set in step 512. In other embodiments, step 510 may not be performed and the process 500 proceeds to step 514 without determining whether any documents are parallel translations of each other.
- step 514 is performed to determine whether there are parallel fragments in the paired documents.
- the parallel fragment engine 210 may perform this determination. The process used in step 514 is discussed in greater detail in connection with FIG. 7 and FIG. 8. If there are parallel fragments in the sentences, the parallel fragments are added to the training set in step 516. Ih other embodiments, the process 500 may not perform step 514 and may instead terminate after performing steps 510 and 512.
- FIG. 6 depicts a flowchart of a process 600 for determining whether a source document is a parallel translation of a document in the subset based on sentence alignments, according to exemplary embodiments.
- the process 600 may be performed as step 510 in process 500.
- the process 600 may be performed by the parallel documents engine 208.
- a noisy sentence pair threshold is determined.
- a noisy sentence pair is a sentence pair comprising a first sentence that is at least a rough translation of one or more second sentences.
- the first sentence may include the words, in a first language, "Mary went to the park with her dog," while the second sentence may include the words, in a second language, "Mary went to the park.”
- the noisy sentence threshold may be a percentage of the sentences in the source document.
- the noisy sentence threshold for example, may be thirty percent (30%) of the sentences in the document.
- a monotone sentence pair threshold is determined.
- a monotone sentence pair is a sentence pair comprising words that have a high probability of being a parallel translation of one another.
- the monotone sentence pair may be a percentage of the noisy sentences.
- the monotone sentence pair threshold may be ninety percent (90%) of the noisy sentence pairs.
- the noisy sentence pair threshold and the monotone sentence pair may be determined initially and/or later adjusted according to the accuracy of the identification of the parallel documents.
- step 606 the target document in the subset selected in step 506 having the most sentence pairs aligned with the source document according to step 508 is selected.
- step 606 a series of determinations is made to determine whether the source document is a parallel translation of the target document. If the outcome of any of these determinations is negative, the source document is not determined to be parallel to the target document.
- step 608 comprises determining whether the length of the source document is approximately equal to the length of the target document is determined. Step 608 may be performed by the document classification module 304. This determination may be made according to the number of sentences present in the target document and in the source document. In some embodiments, the lengths of the documents are approximately equal if the length difference is no more than twenty-five percent (25%) of each of their lengths.
- the document classification module 304 may, in exemplary embodiments, determine whether the number of sentence pairs identified between the target document and the source documents satisfies the noisy sentence threshold.
- the monotone sentence pairs between the target document and the source document are identified, in step 612.
- the monotone sentence pairs may be identified from the noisy sentence pairs previously identified.
- step 614 whether the number of identified monotone sentence pairs satisfies the monotone sentence threshold determined in step 604 is determined. This determination may be performed by the document classification module 304, in accordance with exemplary embodiments.
- FIG. 7 depicts a flowchart of a process 700 for extracting parallel sentence fragments from comparable corpora, according to exemplary embodiments.
- the parallel fragment engine 210 may perform process 700.
- Parallel sentence fragments comprise sub-sentential parallel translations found within documents in bilingual collections. For example, corpora may be comparable if the source document and the target document are determined to not be parallel according to the process 600.
- a coarse lexicon and a fine lexicon are generated from an existing parallel training corpus in step 702.
- the coarse lexicon may comprise the coarse lexicon 404 and the fine lexicon may comprise the fine lexicon 410, discussed in FIG. 4.
- the coarse lexicon may be generated by running the GIZA++ implementation of the IBM word alignment models.
- the coarse lexicon is configured to identify most of the existing parallel data.
- the fine lexicon may be generated according to the process described in connection with FIG. 8 according to exemplary embodiments.
- step 704 sentence pairs which have few words that are translations of each other using the coarse lexicon are discarded. A lack of parallel words in a sentence pair according to the coarse lexicon indicates that it is unlikely that there are parallel fragments within the sentence pair.
- each word in the source sentence is greedily linked to the best translation in the target sentence.
- the word linkages are a way to quantify the differences between source sentences that are "mostly translated” and "mostly not translated” in the target sentences.
- the process 700 is able to retain fragments within the sentence that are mostly positive.
- the word linkages may then be assigned a specific numerical value in steps 708 and 710.
- a positive value is assigned to the word linkages that indicate translated words in the sentence pair.
- the positive value corresponds to a positive dependency of the linked words.
- the numerical value is retrieved or otherwise accessed from the fine lexicon generated in step 702 based on the linked words.
- not translated words in the sentence pair are assigned a negative value to indicate that there is little or no probability that the words can, or should be, linked.
- the value assigned to the not translated words is negative one (-1).
- a smoothing filter is applied to the numerical signal generated according to the sentence pair to obtain a filtered signal.
- the smoothing filter may comprise an averaging filter which sets the value at each point to be the average of several values surrounding the value. For example, the surrounding five (5) values may be averaged.
- step 714 fragment matches within the sentence pair are determined. If there is a parallel fragment in the sentence pair, the positive fragments having a corresponding positive filtered signal are retained. If there is no fragment match, or the filtered signal is negative, the sentence pair is discarded.
- step 716 unlikely matching fragments are discarded.
- the application of the smoothing filter may, for example, distort the signal to produce a short fragment that is a false positive. For example, in some embodiments, fragments comprising less than three words may be discarded to avoid false positives. If the fragments that are likely to be a false positive are discarded, the process 700 returns to step 516 comprising adding the parallel fragments to the training set.
- FIG. 8 depicts a flowchart of a process 800 for generating a fine lexicon, such as the fine lexicon 410, according to exemplary embodiments.
- the fine lexicon comprises fewer possible translations ⁇ e.g., an average of five (5) possible translations for each source word) for each source word than the coarse lexicon (e.g., coarse lexicon 404 which may comprise an average of twelve (12) possible translations).
- the fine lexicon is used to identify the fragment pairs that are likely to be parallel translations.
- the fine lexicon comprises a positive numerical probability that the target word is a translation of the source word. Additionally, the fine lexicon also comprises a negative numerical probability that a target word is not a translation of a source word.
- a parallel training corpus is word aligned. Word aligning the parallel training corpus generates annotations in the parallel training corpus indicating that a word in the target language is a possible translation of a word in the source language.
- a Log-Likelihood-Ratio (LLR) score is calculated to measure the likelihood that a source word and a target word are not independent.
- the LLR score is used to estimate the independence of pairs of words which cooccur in a parallel corpus.
- the source word and the target word will cooccur in the source document and the target document if the two words are also linked together in the word aligned parallel training corpus.
- the LLR score may be computed using the equation: s are
- both a positive probability and a negative probability are determined based on classifying the LLR score as indicating a positive association or a negative association.
- the positive probability estimates the probability that the target word translates into the source word.
- the negative probability estimates the probability that the target word does not translate into the source word.
- step 808 normalizing factors for each word in the source document are computed.
- One normalizing factor is based on the summation of the positive LLR scores while a second normalizing factor is based on a summation of the negative LLR scores.
- each term, or value associated with each word alignment is divided by the corresponding normalizing factor calculated in step 808.
- the probability distributions may be recalculated by reversing the source language and the target language. The reversed probability distributions may be calculated using process 800.
- the above-described functions and components can be comprised of instructions that are stored on a storage medium.
- the instructions can be retrieved and executed by a processor.
- Some examples of instructions are software, program code, and firmware.
- Some examples of storage medium are memory devices, tape, disks, integrated circuits, and servers.
- the instructions are operational when executed by the processor to direct the processor to operate in accord with various embodiments. Those skilled in the art are familiar with instructions, processor(s), and storage medium.
Abstract
Systems, computer programs, and methods for identifying parallel documents and/or fragments in a bilingual collection are provided. The method for identifying parallel sub-sentential fragments in a bilingual collection comprises translating a source document from a bilingual collection. The method further includes querying a target library associated with the bilingual collection using the translated source document, and identifying one or more target documents based on the query. Subsequently, a source sentence associated with the source document is aligned to one or more target sentences associated with the one or more target documents. Finally, the method includes determining whether a source fragment associated with the source sentence comprises a parallel translation of a target fragment associated with the one or more target sentences.
Description
Systems and Methods for Identifying Parallel Documents and Sentence Fragments in Multilingual Document Collections
By Dragos Stefan Munteanu and Daniel Marcu
GOVERMVIENT INTERESTS
[0001] The research and development described in this application were supported by the GALE program of the Defense Advanced Research Projects Agency (DARPA), Contract No. HROOl 1-06-C-0022. The U.S. government may have certain rights in the claimed inventions.
BACKGROUND FIELD OF THE INVENTION
[0002] The present invention relates generally to statistical machine translation of multilingual documents and more specifically to systems and methods for identifying parallel segments in multilingual document collections.
DESCRIPTION OF THE RELATED ART
[0003] In the field of statistical machine translation, large collections of training data are required to develop and implement systems and methods for translating documents. Training data comprises parallel segments which are documents or fragments that are literal, or parallel, translations of each other in two languages. Currently, there is a lack of sufficiently large parallel corpora for most language pairs. A language pair refers to the two languages used within the parallel corpora. Examples of language pairs include English-Romanian or English- Arabic.
[0004] Large volumes of material in many languages are produced daily, and in some instances,- this material may comprise translational equivalents. For
example, a news story posted on the World Wide Web (WWW) on an English- language website may be a translation of the same story posted on a Romanian- language website. The ability to identify these translations is important for generating large collections of parallel training data.
[0005] However, because news web pages published on a news website typically have the same structure. As such, structural properties, such as HTML structures, can not be used to identify parallel documents. Further, because web sites in different languages are often organized differently and a connection is not always maintained between translated versions of the same story, UR-Ls of articles may be unreliable. Further, a news website may contain comparable segments of text that relate to the same news story, but the comparable segments or articles should not necessarily be identified as parallel documents. Comparable segments may be referred to as "noisy translations" of the sentences.
[0006] However, these comparable segments may include one or more parallel fragments that can be added to the training data even though the entire segment is not a parallel translation of a comparable segment. For example, a quote within a news article may be translated literally even though the rest of the document is merely related to a comparable segment in another language.
[0007] Current methods perform computations at a word level and do not distinguish parallel translations of documents from comparable documents. As such, these methods result in many false positives where a comparable document may be erroneously classified as a parallel translation.
SUMMARY
[0008] Systems, computer programs, and methods for identifying parallel documents and/or fragments in a bilingual collection are provided. The method for identifying parallel sub-sentential fragments in a bilingual collection comprises translating a source document from a bilingual collection. The method further includes querying a target library associated with the bilingual collection using the translated source document, and identifying one or more target documents based on the query. Subsequently, a source sentence associated with the source document is aligned to one or more target sentences associated with the one or more target documents. Finally, the method includes determining whether a source fragment associated with the source sentence comprises a parallel translation of a target fragment associated with the one or more target sentences.
BRIEF DESCRIPTION OF THE FIGURES
[0009] FIG. 1 depicts an illustration of an environment in which some embodiments may be practiced;
[0010] FIG. 2 depicts a block diagram illustrating an exemplary training set generator according to exemplary embodiments;
[0011] FIG. 3 depicts a block diagram illustrating an exemplary parallel document engine according to exemplary embodiments;
[0012] FIG. 4 depicts a block diagram illustrating an exemplary parallel fragment engine according to exemplary embodiments;
[0013] FIG. 5 depicts a flowchart illustrating an exemplary process for automatically generating parallel corpora from multi-lingual document sources according to exemplary embodiments;
[0014] FIG. 6 depicts a flowchart of an exemplary process for determining whether documents are parallel based on sentence alignments according to exemplary embodiments;
[0015] FIG. 7 depicts a flowchart of an exemplary process for extracting parallel sentence fragments from comparable corpora according to exemplary embodiments; and
[0016] FIG. 8 depicts a flowchart of an exemplary process for generating a fine lexicon according to exemplary embodiments.
DETAILED DESCRIPTION
[0017] A system and method for identifying parallel documents and/or fragments in a bilingual document collection is provided. The present method and system can be used with documents posted on the Internet without relying on properties such as page structure or URL. Further, the system and method is able to distinguish between parallel documents and comparable documents. The method and system may alternatively or additionally be used to extract parallel fragments from comparable corpora at the sub-sentential level to increase the amount of parallel data for statistical machine translation (SMT).
[0018] FIG. 1 depicts an illustration of an environment 100 in which some embodiments maybe practiced. The environment 100 may comprise a document collection server 102 communicatively connected to a client 106 via a network 104. The document collection server 102 may comprise a memory 110 configured to store one or more bilingual document collections, one or more parallel documents, one or more parallel fragments, and/or one or more set of annotations indicating parallel data. The document collection server 102 may comprise a processor configured to access the memory 110 and process instructions. The document collection server 102 may communicate over the network 104 using a communications interface. The network 104 may comprise a public network (e.g., the Internet) or a private network. The client 106 may include storage, a display, or additional functionality beyond the scope of the embodiments.
[0019] The client 106 may comprise a computational device such as a personal computer. The client 106 may include a training set generator 108. The training set generator 108 may comprise hardware, software, or firmware and may be configured to identify parallel documents and/or fragments in bilingual collections.
The training set generator 108 may be configured to access bilingual collections stored in the document collection server 102. Further, the training set generator 108 may transmit data indicating parallel documents or fragments to the document collection server 102.
[0020] FIG. 2 depicts a block diagram illustrating an exemplary training set generator, such as the training set generator 108 of FIG. 1, according to exemplary embodiments. The training set generator 108 may identify parallel documents and/or fragments within bilingual document collections. The bilingual document collections may comprise documents collected from the Internet or the World Wide Web (WWW) in exemplary embodiments. The training set generator 108 may comprise a word translator engine 202, a query engine 204, a document selector engine 206, a parallel document engine 208, and a parallel fragment engine 210. Because sentence alignment requires a large amount of computations, the word translator engine 202, and the query engine 204 may be configured to reduce the number of documents with which sentence alignment is performed.
[0021] The word translator engine 202 is configured to translate each word within a source document to generate a translated source document. A source document is a document in the bilingual document collection of which a parallel translation is sought. The source document, for example, may comprise a news article written in English. For example, the training set generator 108 may be configured to determine whether any documents written in Romanian associated with a target library are parallel translations of, or contain parallel translations of fragments of, the source document.
[0022] A target library comprises one or more target documents. Target documents may include documents or segments that possibly comprise a parallel
translation of the source document. Target documents may be obtained from sources such as the Internet or other known libraries and may be classified according to a classifier such as date, URL, or the like. The word translator engine 202 translates the source document word-by- word or literally into the language of the target library. To further the example, the news article written in English is translated word-by-word into Romanian, which may be utilized as the language of the target library.
[0023] The query engine 204 is configured to query the target library using the translated source document. The query engine 204 selects a subset comprising the documents that are most similar to the translated source document. The subset may be limited to the top N documents. In some embodiments, N may be a constant such as the number twenty (20).
[0024] The document selector engine 206 aligns one or more target sentences in each of the target documents in the subset to one or more sentences in the source document. By aligning the sentences, the training set generator 108 avoids falsely classifying documents as parallel based only on word-level comparisons. Based on the sentence alignments, the document selector engine 206 may discard one or more of the top N documents that, while comprising word-level translations, do not comprise sentence-level translations. For example, the document selector engine 206 may discard documents that do not have a specified number of sentences that can be aligned. In other embodiments, the document selector engine 206 may discard sentence pairs within the aligned documents that do not contain a specified number or percentage of words that are translations of each other. By aligning sentences within the source documents and the target documents, the training set generator 108 can distinguish parallel documents from comparable documents.
[0025] The parallel document engine 208 is configured to determine whether a document pair comprising the source document and the target document is a parallel translation. The parallel document engine 208 is discussed further with respect to FIG. 3 and FIG. 6.
[0026] The parallel fragment engine 210 is configured to identify parallel sentence fragments in non-parallel document pairs such as document pairs from news sources on the Internet. The parallel fragment engine 210 is discussed further in FIG. 4, FIG. 8, and FIG. 9. Although FIG. 2 shows various engines comprising the training set generator 108, fewer or more engines may comprise the training set generator 108 and still fall within the scope of the various embodiments.
[0027] FIG. 3 depicts a block diagram illustrating an exemplary parallel document engine 208 according to exemplary embodiments. The parallel document engine 208 determines whether the target document is the parallel translation of the source document. The parallel document engine 208 comprises a sentence analysis module 302 and a document classification module 304.
[0028] The sentence analysis module 302 determines whether aligned sentences based on the sentence alignments generated by the document selector engine 206 are translations of each other independently of the context within the document or segment. The sentence analysis module 302 inspects the sentence pairs between the sentences in the source document and the sentences in each of the documents in the subset comprising the documents that are most similar to the translated source document to compute sentence-level links. The sentence analysis module 302 distinguishes between parallel and non-parallel sentence pairs based on sentence features that can be extracted from a word-level alignment of the two aligned sentences. The sentence features are identified by linking each word in the sentences
with its best translation candidate in the paired sentence. This linking process may be referred to as "greedily linking."
[0029] Features that can be extracted by greedily linking include, for example, the percentage of words that are aligned, contiguous phrases that are aligned, and contiguous phrases that are not aligned. After the features have been extracted, a Maximum Entropy-based parallel sentence classifier (not shown) may be trained to obtain positive and negative classifiers for the sentences. In exemplary embodiments, the following Maximum Entropy probability equation may be used:
"not parallel"), sp is the percentage of words in a sentence pair that have a translation in the other sentence of the sentence pair. Z(sp) is a normalization factor, sndfy are the feature functions (indexed both by class and by feature). A feature function, /fø?), is defined as the value of the word overlap of the sentences in the sentence pair sp. The resulting model has free parameters λ,, the feature weights. The parameter values that maximize the likelihood of a given training corpus can be computed using known optimization algorithms. Based on the positive and negative classifiers, sentence level links between the source document and the documents within the subset are obtained.
[0030] The document classification module 304 is configured to select the document within the subset that shares the most sentence level links with the source document. The document within the subset that shares the most sentence-level links with the source document is referred to as the target document. The document classification module 304 then determines whether the target document is a parallel translation of the source document by comparing the length of the target document to
the length of the source document and determining whether a noisy sentence threshold and a monotone sentence threshold are met. This process is described in greater detail in association with FIG. 6. The output of the document classification module 304 may comprise an annotation indicating that the target document is a parallel translation of the source document.
[0031] FIG. 4 depicts a block diagram illustrating an exemplary parallel fragment engine 210 according to exemplary embodiments. The parallel fragment engine 210 may be configured to extract parallel sentence fragments from within non- parallel corpora. Thus, the parallel fragment engine 210 is configured to determine whether a sentence fragment in the source document is a parallel translation of a sentence fragment in the target document. For example, if the document classification module 304 determines that the source document and the target document are non- parallel, that is the target document is not a parallel translation of the source document, the documents are likely to be comparable documents. Comparable documents may also be referred to as "noisy translations." The comparable documents may comprise sub-sentential parallel fragments that may be extracted by the parallel fragment engine 210 and added to a training data set. The parallel fragment engine 210 may comprise a candidate selection filter 402, a coarse lexicon 404, a probability module 406, a fine lexicon 408, and a fragment detection module 410.
[0032] The candidate selection filter 402 discards sentence pairs in the subset comprising the segments or documents selected by the document selection engine 206. The candidate selection filter 402 discards sentence pairs that have very few words that are translations of each other based on the coarse lexicon 404. For example, sentence pairs that comprise three or fewer words that appear in both the
source sentence and the target sentence may be discarded. In other embodiments, sentence pairs may be discarded according to a percentage of words appearing in both the source sentence and the target sentence.
[0033] The coarse lexicon 404 comprises a probabilistic lexicon derived from an initial parallel corpus such as a training data set. The coarse lexicon 404 may be obtained by running a GIZ A++ implementation of the IBM word alignment models on the initial parallel corpus. In the coarse lexicon, each source word may be associated with many possible translations. For example, in some embodiments, each source word is associated with, on average, twelve possible translations. Each of these possible translations may be further associated with a probability. The coarse lexicon is used to retain most of the existing comparable sentences occurring between the source document and the target document.
[0034] The probability module 406 is configured to greedily link each word in the source sentence to the best translation occurring in the target sentence. Using the fine lexicon 408, the probability module 406 then assigns values to each word alignment indicating the probability that the source word is properly aligned with, or a translation of, the target word. The probability module 406 may filter these values further. The fine lexicon 408 may be generated in exemplary embodiments according to the process discussed in connection with FIG. 8.
[0035] The fragment detection module 410 may be configured to detect a parallel sentence fragment based on the values assigned to each word alignment based on the fine lexicon 408. In some embodiments, the fragment detection module 410 may further filter the fragment matches. This process is discussed in greater detail in connection with FIG. 7. The output of the parallel fragment engine 210 comprises at least one parallel fragment that may be added to the training set.
i t
[0036] FIG. 5 depicts a flowchart illustrating an exemplary process 500 for automatically generating parallel corpora from bilingual document sources according to exemplary embodiments. The process 500 identifies parallel segments and fragments in the bilingual documents according to sentence pairs identified between the documents. The process 500 may be used to generate a training set for statistical machine translation (SMT). The process 500 comprises accessing a bilingual document collection, translating words in a source document, querying a target library using the translated source document, selecting a subset comprising a target document from the target library, aligning the sentences in the source document with sentences in the target document, and determining whether the source document is a parallel translation of the target document. The method may alternatively or additionally include determining whether a fragment in the source document is parallel to a fragment in the target document. If the source document or a fragment thereof is a parallel translation of the target document, the parallel portions may be stored within a training set.
[0037] At step 502, each word in the source document is independently translated into the language of the target document to generate a translated source document. The words may be translated according to a dictionary, for example.
[0038] At step 504, the translated source document is run as a query against a large library comprising documents written in the target language. In step 506, the source document is paired with a subset comprising the most similar documents written in the target language according to the results of the query. In some embodiments, the subset may be limited to a constant number, N, of the most similar documents. N may be any number up to and including the number of .
documents in the target library. N may be limited by a computational capacity of a computer such as client 106.
[0039] At step 508, parallel sentence pairs between the source document and the documents in the subset are identified. The sentence pairs may be identified by the sentence analysis module 302.
[0040] After the sentence pairs are computed in step 508, step 510 is performed to determine whether the source document and at least one of the documents in the subset are parallel translations. In some embodiments, the parallel document engine 208 may perform this determination. The process used in step 510 is discussed in greater detail in connection with FIG. 6. If the documents are parallel, the parallel documents are added to the training set in step 512. In other embodiments, step 510 may not be performed and the process 500 proceeds to step 514 without determining whether any documents are parallel translations of each other.
[0041] If the documents are not parallel, step 514 is performed to determine whether there are parallel fragments in the paired documents. In some embodiments, the parallel fragment engine 210 may perform this determination. The process used in step 514 is discussed in greater detail in connection with FIG. 7 and FIG. 8. If there are parallel fragments in the sentences, the parallel fragments are added to the training set in step 516. Ih other embodiments, the process 500 may not perform step 514 and may instead terminate after performing steps 510 and 512.
[0042] FIG. 6 depicts a flowchart of a process 600 for determining whether a source document is a parallel translation of a document in the subset based on sentence alignments, according to exemplary embodiments. The process 600 may
be performed as step 510 in process 500. In some embodiments, the process 600 may be performed by the parallel documents engine 208.
[0043] In step 602, a noisy sentence pair threshold is determined. A noisy sentence pair is a sentence pair comprising a first sentence that is at least a rough translation of one or more second sentences. For example, the first sentence may include the words, in a first language, "Mary went to the park with her dog," while the second sentence may include the words, in a second language, "Mary went to the park." In some embodiments, the noisy sentence threshold may be a percentage of the sentences in the source document. The noisy sentence threshold, for example, may be thirty percent (30%) of the sentences in the document.
[0044] At step 604, a monotone sentence pair threshold is determined. A monotone sentence pair is a sentence pair comprising words that have a high probability of being a parallel translation of one another. In exemplary embodiments, the monotone sentence pair may be a percentage of the noisy sentences. For example, the monotone sentence pair threshold may be ninety percent (90%) of the noisy sentence pairs. The noisy sentence pair threshold and the monotone sentence pair may be determined initially and/or later adjusted according to the accuracy of the identification of the parallel documents.
[0045] At step 606, the target document in the subset selected in step 506 having the most sentence pairs aligned with the source document according to step 508 is selected. After step 606, a series of determinations is made to determine whether the source document is a parallel translation of the target document. If the outcome of any of these determinations is negative, the source document is not determined to be parallel to the target document.
[0046] At step 608, comprises determining whether the length of the source document is approximately equal to the length of the target document is determined. Step 608 may be performed by the document classification module 304. This determination may be made according to the number of sentences present in the target document and in the source document. In some embodiments, the lengths of the documents are approximately equal if the length difference is no more than twenty-five percent (25%) of each of their lengths.
[0047] If the target document and the source document are approximately the same length, the document classification module 304 may, in exemplary embodiments, determine whether the number of sentence pairs identified between the target document and the source documents satisfies the noisy sentence threshold.
[0048] If the noisy sentence threshold is satisfied, the monotone sentence pairs between the target document and the source document are identified, in step 612. The monotone sentence pairs may be identified from the noisy sentence pairs previously identified. At step 614, whether the number of identified monotone sentence pairs satisfies the monotone sentence threshold determined in step 604 is determined. This determination may be performed by the document classification module 304, in accordance with exemplary embodiments.
[0049] If the monotone sentence threshold is satisfied, the source document and the target document are identified as parallel and added to the training set, in step 512. The steps performed in process 600 may be reordered or performed • substantially simultaneously and still fall within the scope of exemplary embodiments. Additional determinations known by those skilled in the art may be added to process 600 and still be within the scope of various embodiments.
[0050] FIG. 7 depicts a flowchart of a process 700 for extracting parallel sentence fragments from comparable corpora, according to exemplary embodiments. In exemplary embodiments, the parallel fragment engine 210 may perform process 700. Parallel sentence fragments comprise sub-sentential parallel translations found within documents in bilingual collections. For example, corpora may be comparable if the source document and the target document are determined to not be parallel according to the process 600.
[0051] To extract parallel sentence fragments, a coarse lexicon and a fine lexicon are generated from an existing parallel training corpus in step 702. In exemplary embodiments, the coarse lexicon may comprise the coarse lexicon 404 and the fine lexicon may comprise the fine lexicon 410, discussed in FIG. 4. The coarse lexicon may be generated by running the GIZA++ implementation of the IBM word alignment models. The coarse lexicon is configured to identify most of the existing parallel data. The fine lexicon may be generated according to the process described in connection with FIG. 8 according to exemplary embodiments.
[0052] At step 704, sentence pairs which have few words that are translations of each other using the coarse lexicon are discarded. A lack of parallel words in a sentence pair according to the coarse lexicon indicates that it is unlikely that there are parallel fragments within the sentence pair.
[0053] At step 706, each word in the source sentence is greedily linked to the best translation in the target sentence. The word linkages are a way to quantify the differences between source sentences that are "mostly translated" and "mostly not translated" in the target sentences. By representing the target sentence as a numeric signal where translated words correspond to a positive value and not translated words correspond to a negative value, the process 700 is able to retain fragments within the
sentence that are mostly positive. The word linkages may then be assigned a specific numerical value in steps 708 and 710.
[0054] At step 708, a positive value is assigned to the word linkages that indicate translated words in the sentence pair. The positive value corresponds to a positive dependency of the linked words. The numerical value is retrieved or otherwise accessed from the fine lexicon generated in step 702 based on the linked words.
[0055] At step 710, not translated words in the sentence pair are assigned a negative value to indicate that there is little or no probability that the words can, or should be, linked. In some embodiments, the value assigned to the not translated words is negative one (-1).
[0056] At step 712, a smoothing filter is applied to the numerical signal generated according to the sentence pair to obtain a filtered signal. In some embodiments, the smoothing filter may comprise an averaging filter which sets the value at each point to be the average of several values surrounding the value. For example, the surrounding five (5) values may be averaged.
[0057] At step 714, fragment matches within the sentence pair are determined. If there is a parallel fragment in the sentence pair, the positive fragments having a corresponding positive filtered signal are retained. If there is no fragment match, or the filtered signal is negative, the sentence pair is discarded.
[0058] At step 716, unlikely matching fragments are discarded. The application of the smoothing filter may, for example, distort the signal to produce a short fragment that is a false positive. For example, in some embodiments, fragments comprising less than three words may be discarded to avoid false positives. If the
fragments that are likely to be a false positive are discarded, the process 700 returns to step 516 comprising adding the parallel fragments to the training set.
[0059] FIG. 8 depicts a flowchart of a process 800 for generating a fine lexicon, such as the fine lexicon 410, according to exemplary embodiments. The fine lexicon comprises fewer possible translations {e.g., an average of five (5) possible translations for each source word) for each source word than the coarse lexicon (e.g., coarse lexicon 404 which may comprise an average of twelve (12) possible translations). Thus, the fine lexicon is used to identify the fragment pairs that are likely to be parallel translations. For each possible translation, the fine lexicon comprises a positive numerical probability that the target word is a translation of the source word. Additionally, the fine lexicon also comprises a negative numerical probability that a target word is not a translation of a source word.
[0060] At step 802, a parallel training corpus is word aligned. Word aligning the parallel training corpus generates annotations in the parallel training corpus indicating that a word in the target language is a possible translation of a word in the source language.
[0061] At step 804, a Log-Likelihood-Ratio (LLR) score is calculated to measure the likelihood that a source word and a target word are not independent. The LLR score is used to estimate the independence of pairs of words which cooccur in a parallel corpus. According to exemplary embodiments, the source word and the target word will cooccur in the source document and the target document if the two words are also linked together in the word aligned parallel training corpus. According to some embodiments, the LLR score may be computed using the equation:
s are
corresponding words that are linked together in the respective target and source language sentences of an aligned sentence pair. In this equation, -f and -S are the corresponding words that are not aligned in the respective sentences. The variables f? and s? range over these values, and C(f?, S?) is the observed joint count for the values off? and s?. The probabilities in the formula refer to maximum likelihood estimates.
[0062] At step 806, both a positive probability and a negative probability are determined based on classifying the LLR score as indicating a positive association or a negative association. The positive probability estimates the probability that the target word translates into the source word. The negative probability estimates the probability that the target word does not translate into the source word.
[0063] At step 808, normalizing factors for each word in the source document are computed. One normalizing factor is based on the summation of the positive LLR scores while a second normalizing factor is based on a summation of the negative LLR scores.
[0064] At step 810, each term, or value associated with each word alignment, is divided by the corresponding normalizing factor calculated in step 808. In further embodiments of the fine lexicon, the probability distributions may be recalculated by reversing the source language and the target language. The reversed probability distributions may be calculated using process 800.
[0065] The above-described functions and components can be comprised of instructions that are stored on a storage medium. The instructions can be retrieved and executed by a processor. Some examples of instructions are software, program code, and firmware. Some examples of storage medium are memory devices, tape,
disks, integrated circuits, and servers. The instructions are operational when executed by the processor to direct the processor to operate in accord with various embodiments. Those skilled in the art are familiar with instructions, processor(s), and storage medium.
[0066] While various embodiments have been described above, it should be understood that they have been presented by way of example only, and not limitation. For example, any of the elements associated with the training set generator 108 may employ any of the desired functionality set forth hereinabove. Thus, the breadth and scope of a preferred embodiment should not be limited by any of the above-described exemplary embodiments.
Claims
1. A method for identifying parallel sub-sentential fragments in a bilingual collection, the method comprising: translating a source document from a bilingual collection; querying a target library associated with the bilingual collection using the translated source document; identifying one or more target documents based on the query; aligning a source sentence associated with the source document to one or more target sentences associated with the one or more target documents; and determining whether a source fragment associated with the source sentence comprises a parallel translation of a target fragment associated with the one or more target sentences.
2. The method recited in claim 1 further comprising discarding the one or more target sentences based on a coarse lexicon.
3. The method recited in claims 1 or 2 further comprising assigning a translation probability to words in the aligned source sentence based on a fine lexicon.
4. The method recited in claims 1, 2 or 3 further comprising detecting a parallel fragment based on a threshold associated with a number of essentially continuous words assigned a positive translation probability.
5. The method recited in claims 1, 2, 3, or 4 further comprising determining whether the source document comprises a parallel translation of the target document.
6. A computer program embodied on a computer readable medium having instructions for identifying parallel sub-sentential fragments in a bilingual collection, the instructions comprising the steps: translating a source document from a bilingual collection; querying a target library associated with the bilingual collection using the translated source document; identifying one or more target documents based on the query; aligning a source sentence associated with the source document to one or more target sentences associated with the one or more target documents; and determining whether a source fragment associated with the source sentence comprises a parallel translation of a target fragment associated with the one or more target sentences.
7. The computer program recited in claim 6 further comprising an instruction for discarding the one or more target sentences based on a coarse lexicon.
8. The computer program recited in claims 6 or 7 further comprising an instruction for assigning a translation probability to words in the aligned source sentence based on a fine lexicon.
9. The computer program recited in claims 6, 7, or 8 further comprising an instruction for detecting a parallel fragment based on a threshold associated with a number of essentially continuous words assigned a positive translation probability.
10. The computer program recited in claims 6, 7, 8, or 9 further comprising an instruction for determining whether the source document comprises a parallel translation of the target document.
11. A method for identifying parallel documents in a bilingual collection, the method comprising: translating a source document from a bilingual collection; querying a target library associated with the bilingual collection using the translated source document; identifying one or more target documents based on the query; aligning a source sentence associated with the source document to one or more target sentences associated with the one or more target documents; and determining whether the source document comprises a parallel translation of one of the one or more target documents.
12. The method recited in claim 11 wherein determining whether the source document comprises a parallel translation of the one or more the target documents further comprises selecting one of the one or more target documents according to a number of the source sentences within the source document aligned to the one or more target sentences.
13. The method recited in claims 11 or 12 wherein determining whether the source document comprises a parallel translation of one of the one or more target documents further comprises determining whether the number of the source sentences aligned to the one or more target sentences satisfies a threshold associated with noisy sentences.
14. The method recited in claims 11, 12 or 13 wherein determining whether the source document comprises a parallel translation of the target document further comprises determining whether the number of the source sentences aligned to the one or more target sentences satisfies a threshold associated with monotone sentences.
15. The method recited in claims 11, 12, 13, or 14 further comprising determining whether a source fragment associated with the source sentence comprises a parallel translation of a target fragment associated with one of the one or more target sentences.
16. A system for identifying parallel documents in a bilingual collection, the system comprising: a word translator engine configured to translate a source document from a bilingual collection; a query engine configured to query a target library associated with the bilingual collection using the translated source document and identify one or more target documents based on the query; a document selector engine configured to align a source sentence associated with the source document to one or more target sentences associated with the one or more target documents; and a parallel document engine configured to determine whether the source document comprises a parallel translation of one of the one or more the target documents.
17: The system recited in claim 16 wherein the parallel document engine further comprises a sentence analysis module configured to select the one of the one or more target documents according to a number of the source sentences within the source document aligned to the one or more target sentences.
18. The system recited in claims 16 or 17 wherein the parallel document engine further comprises a document classification module configured to determine whether the number of thee source sentences aligned to the one or more target sentences satisfies a threshold associated with noisy sentences.
19. The system recited in claims 16, 17 or 18 wherein the parallel document engine further comprises a document classification module configured to determine whether the number of the source sentences aligned to the one or more target sentences satisfies a threshold associated with monotone sentences.
20. The system recited in claims 16, 17, 18, or 19 further comprising a parallel fragment engine configured to determine whether a source fragment associated with the source sentence comprises a parallel translation of a target fragment associated with the one or more target sentences.
2δ
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US79013106P | 2006-04-07 | 2006-04-07 | |
US60/790,131 | 2006-04-07 | ||
US11/635,248 | 2006-12-05 | ||
US11/635,248 US8943080B2 (en) | 2006-04-07 | 2006-12-05 | Systems and methods for identifying parallel documents and sentence fragments in multilingual document collections |
Publications (2)
Publication Number | Publication Date |
---|---|
WO2007126447A2 true WO2007126447A2 (en) | 2007-11-08 |
WO2007126447A3 WO2007126447A3 (en) | 2008-04-17 |
Family
ID=38620544
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2007/000853 WO2007126447A2 (en) | 2006-04-07 | 2007-01-12 | Systems and methods for identifying parallel documents and sentence fragments in multilingual document collections |
Country Status (2)
Country | Link |
---|---|
US (1) | US8943080B2 (en) |
WO (1) | WO2007126447A2 (en) |
Families Citing this family (49)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8214196B2 (en) | 2001-07-03 | 2012-07-03 | University Of Southern California | Syntax-based statistical translation model |
WO2004001623A2 (en) | 2002-03-26 | 2003-12-31 | University Of Southern California | Constructing a translation lexicon from comparable, non-parallel corpora |
US8548794B2 (en) | 2003-07-02 | 2013-10-01 | University Of Southern California | Statistical noun phrase translation |
US8296127B2 (en) | 2004-03-23 | 2012-10-23 | University Of Southern California | Discovery of parallel text portions in comparable collections of corpora and training using comparable texts |
US8666725B2 (en) | 2004-04-16 | 2014-03-04 | University Of Southern California | Selection and use of nonstatistical translation components in a statistical machine translation framework |
JP5452868B2 (en) | 2004-10-12 | 2014-03-26 | ユニヴァーシティー オブ サザン カリフォルニア | Training for text-to-text applications that use string-to-tree conversion for training and decoding |
US8676563B2 (en) | 2009-10-01 | 2014-03-18 | Language Weaver, Inc. | Providing human-generated and machine-generated trusted translations |
US8886517B2 (en) | 2005-06-17 | 2014-11-11 | Language Weaver, Inc. | Trust scoring for language translation systems |
US10319252B2 (en) | 2005-11-09 | 2019-06-11 | Sdl Inc. | Language capability assessment and training apparatus and techniques |
US7805289B2 (en) * | 2006-07-10 | 2010-09-28 | Microsoft Corporation | Aligning hierarchal and sequential document trees to identify parallel data |
US8886518B1 (en) | 2006-08-07 | 2014-11-11 | Language Weaver, Inc. | System and method for capitalizing machine translated text |
US8433556B2 (en) | 2006-11-02 | 2013-04-30 | University Of Southern California | Semi-supervised training for statistical word alignment |
US9122674B1 (en) | 2006-12-15 | 2015-09-01 | Language Weaver, Inc. | Use of annotations in statistical machine translation |
US8468149B1 (en) | 2007-01-26 | 2013-06-18 | Language Weaver, Inc. | Multi-lingual online community |
US8615389B1 (en) | 2007-03-16 | 2013-12-24 | Language Weaver, Inc. | Generation and exploitation of an approximate language model |
US8831928B2 (en) | 2007-04-04 | 2014-09-09 | Language Weaver, Inc. | Customizable machine translation service |
US8825466B1 (en) | 2007-06-08 | 2014-09-02 | Language Weaver, Inc. | Modification of annotated bilingual segment pairs in syntax-based machine translation |
TWI457868B (en) * | 2008-03-12 | 2014-10-21 | Univ Nat Kaohsiung 1St Univ Sc | Method for automatically modifying a translation from a machine translation |
US8615388B2 (en) * | 2008-03-28 | 2013-12-24 | Microsoft Corporation | Intra-language statistical machine translation |
US8504354B2 (en) * | 2008-06-02 | 2013-08-06 | Microsoft Corporation | Parallel fragment extraction from noisy parallel corpora |
KR100961717B1 (en) * | 2008-09-16 | 2010-06-10 | 한국전자통신연구원 | Method and apparatus for detecting errors of machine translation using parallel corpus |
US8332205B2 (en) * | 2009-01-09 | 2012-12-11 | Microsoft Corporation | Mining transliterations for out-of-vocabulary query terms |
US9336299B2 (en) * | 2009-04-20 | 2016-05-10 | Microsoft Technology Licensing, Llc | Acquisition of semantic class lexicons for query tagging |
US20100299132A1 (en) * | 2009-05-22 | 2010-11-25 | Microsoft Corporation | Mining phrase pairs from an unstructured resource |
US8990064B2 (en) | 2009-07-28 | 2015-03-24 | Language Weaver, Inc. | Translating documents based on content |
US8380486B2 (en) | 2009-10-01 | 2013-02-19 | Language Weaver, Inc. | Providing machine-generated translations and corresponding trust levels |
US10417646B2 (en) | 2010-03-09 | 2019-09-17 | Sdl Inc. | Predicting the cost associated with translating textual content |
US8560297B2 (en) | 2010-06-07 | 2013-10-15 | Microsoft Corporation | Locating parallel word sequences in electronic documents |
US8271869B2 (en) * | 2010-10-08 | 2012-09-18 | Microsoft Corporation | Identifying language translations for source documents using links |
US9928296B2 (en) | 2010-12-16 | 2018-03-27 | Microsoft Technology Licensing, Llc | Search lexicon expansion |
US11003838B2 (en) | 2011-04-18 | 2021-05-11 | Sdl Inc. | Systems and methods for monitoring post translation editing |
US20120303352A1 (en) * | 2011-05-24 | 2012-11-29 | The Boeing Company | Method and apparatus for assessing a translation |
US8694303B2 (en) | 2011-06-15 | 2014-04-08 | Language Weaver, Inc. | Systems and methods for tuning parameters in statistical machine translation |
US8886515B2 (en) | 2011-10-19 | 2014-11-11 | Language Weaver, Inc. | Systems and methods for enhancing machine translation post edit review processes |
US8942973B2 (en) | 2012-03-09 | 2015-01-27 | Language Weaver, Inc. | Content page URL translation |
US10261994B2 (en) | 2012-05-25 | 2019-04-16 | Sdl Inc. | Method and system for automatic management of reputation of translators |
US9152622B2 (en) | 2012-11-26 | 2015-10-06 | Language Weaver, Inc. | Personalized machine translation via online adaptation |
US10372815B2 (en) | 2013-07-12 | 2019-08-06 | Microsoft Technology Licensing, Llc | Interactive concept editing in computer-human interactive learning |
US9213694B2 (en) | 2013-10-10 | 2015-12-15 | Language Weaver, Inc. | Efficient online domain adaptation |
CN105468609A (en) * | 2014-08-27 | 2016-04-06 | 中兴通讯股份有限公司 | Retelling phrase acquisition method and apparatus as well as question rewriting method and apparatus |
US10452786B2 (en) * | 2014-12-29 | 2019-10-22 | Paypal, Inc. | Use of statistical flow data for machine translations between different languages |
US9934203B2 (en) * | 2015-03-10 | 2018-04-03 | International Business Machines Corporation | Performance detection and enhancement of machine translation |
US9940324B2 (en) | 2015-03-10 | 2018-04-10 | International Business Machines Corporation | Performance detection and enhancement of machine translation |
US10394950B2 (en) * | 2016-08-22 | 2019-08-27 | International Business Machines Corporation | Generation of a grammatically diverse test set for deep question answering systems |
CN107195295B (en) * | 2017-05-04 | 2020-06-23 | 百度在线网络技术(北京)有限公司 | Voice recognition method and device based on Chinese-English mixed dictionary |
CN107301860B (en) * | 2017-05-04 | 2020-06-23 | 百度在线网络技术(北京)有限公司 | Voice recognition method and device based on Chinese-English mixed dictionary |
US11237713B2 (en) * | 2019-01-21 | 2022-02-01 | International Business Machines Corporation | Graphical user interface based feature extraction application for machine learning and cognitive models |
US11580312B2 (en) | 2020-03-16 | 2023-02-14 | Servicenow, Inc. | Machine translation of chat sessions |
US11385916B2 (en) * | 2020-03-16 | 2022-07-12 | Servicenow, Inc. | Dynamic translation of graphical user interfaces |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5907821A (en) * | 1995-11-06 | 1999-05-25 | Hitachi, Ltd. | Method of computer-based automatic extraction of translation pairs of words from a bilingual text |
US20050021517A1 (en) * | 2000-03-22 | 2005-01-27 | Insightful Corporation | Extended functionality for an inverse inference engine based web search |
US20050228643A1 (en) * | 2004-03-23 | 2005-10-13 | Munteanu Dragos S | Discovery of parallel text portions in comparable collections of corpora and training using comparable texts |
Family Cites Families (302)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPS57201958A (en) | 1981-06-05 | 1982-12-10 | Hitachi Ltd | Device and method for interpretation between natural languages |
JPS58201175A (en) | 1982-05-20 | 1983-11-22 | Kokusai Denshin Denwa Co Ltd <Kdd> | Machine translation system |
US4615002A (en) | 1983-03-30 | 1986-09-30 | International Business Machines Corp. | Concurrent multi-lingual use in data processing system |
JPS6140672A (en) | 1984-07-31 | 1986-02-26 | Hitachi Ltd | Processing system for dissolution of many parts of speech |
DE3587009T2 (en) | 1984-09-18 | 1993-05-06 | Sharp Kk | TRANSLATION SYSTEM. |
JPH0664585B2 (en) | 1984-12-25 | 1994-08-22 | 株式会社東芝 | Translation editing device |
JPS61217871A (en) | 1985-03-25 | 1986-09-27 | Toshiba Corp | Translation processor |
US4800522A (en) | 1985-05-14 | 1989-01-24 | Sharp Kabushiki Kaisha | Bilingual translation system capable of memorizing learned words |
DE3616751A1 (en) | 1985-05-20 | 1986-11-20 | Sharp K.K., Osaka | TRANSLATION SYSTEM |
JPS6244877A (en) | 1985-08-22 | 1987-02-26 | Toshiba Corp | Machine translator |
JPH083815B2 (en) | 1985-10-25 | 1996-01-17 | 株式会社日立製作所 | Natural language co-occurrence relation dictionary maintenance method |
JPS63223962A (en) * | 1987-03-13 | 1988-09-19 | Hitachi Ltd | Translating device |
US5146405A (en) | 1988-02-05 | 1992-09-08 | At&T Bell Laboratories | Methods for part-of-speech determination and usage |
JPH0242572A (en) | 1988-08-03 | 1990-02-13 | Hitachi Ltd | Preparation/maintenance method for co-occurrence relation dictionary |
JPH02301869A (en) | 1989-05-17 | 1990-12-13 | Hitachi Ltd | Method for maintaining and supporting natural language processing system |
JPH02308370A (en) | 1989-05-24 | 1990-12-21 | Toshiba Corp | Machine translation system |
US5020112A (en) | 1989-10-31 | 1991-05-28 | At&T Bell Laboratories | Image recognition method using two-dimensional stochastic grammars |
US5218537A (en) | 1989-12-21 | 1993-06-08 | Texas Instruments Incorporated | System and method for using a computer to generate and teach grammar lessons |
JPH03268062A (en) | 1990-03-19 | 1991-11-28 | Fujitsu Ltd | Register for private use word in machine translation electronic mail device |
JP3114181B2 (en) | 1990-03-27 | 2000-12-04 | 株式会社日立製作所 | Interlingual communication translation method and system |
US5369574A (en) | 1990-08-01 | 1994-11-29 | Canon Kabushiki Kaisha | Sentence generating system |
JP3176059B2 (en) | 1990-11-15 | 2001-06-11 | キヤノン株式会社 | Translation equipment |
US5497319A (en) * | 1990-12-31 | 1996-03-05 | Trans-Link International Corp. | Machine translation and telecommunications system |
US5212730A (en) | 1991-07-01 | 1993-05-18 | Texas Instruments Incorporated | Voice recognition of proper names using text-derived recognition models |
US5477451A (en) | 1991-07-25 | 1995-12-19 | International Business Machines Corp. | Method and system for natural language translation |
US5167504A (en) | 1991-09-20 | 1992-12-01 | Mann Harold J | Bilingual dictionary |
US5488725A (en) * | 1991-10-08 | 1996-01-30 | West Publishing Company | System of document representation retrieval by successive iterated probability sampling |
JPH05151260A (en) * | 1991-11-29 | 1993-06-18 | Hitachi Ltd | Method and system for learning translation templet |
US5267156A (en) | 1991-12-05 | 1993-11-30 | International Business Machines Corporation | Method for constructing a knowledge base, knowledge base system, machine translation method and system therefor |
US5541836A (en) | 1991-12-30 | 1996-07-30 | At&T Corp. | Word disambiguation apparatus and methods |
DE69330427T2 (en) | 1992-03-06 | 2002-05-23 | Dragon Systems Inc | VOICE RECOGNITION SYSTEM FOR LANGUAGES WITH COMPOSED WORDS |
US5302132A (en) * | 1992-04-01 | 1994-04-12 | Corder Paul R | Instructional system and method for improving communication skills |
JPH05298360A (en) * | 1992-04-17 | 1993-11-12 | Hitachi Ltd | Method and device for evaluating translated sentence, machine translation system with translated sentence evaluating function and machine translation system evaluating device |
GB9209346D0 (en) | 1992-04-30 | 1992-06-17 | Sharp Kk | Machine translation system |
US6278967B1 (en) | 1992-08-31 | 2001-08-21 | Logovista Corporation | Automated system for generating natural language translations that are domain-specific, grammar rule-based, and/or based on part-of-speech analysis |
US5528491A (en) | 1992-08-31 | 1996-06-18 | Language Engineering Corporation | Apparatus and method for automated natural language translation |
JPH06195373A (en) | 1992-12-24 | 1994-07-15 | Sharp Corp | Machine translation system |
US5495413A (en) | 1992-09-25 | 1996-02-27 | Sharp Kabushiki Kaisha | Translation machine having a function of deriving two or more syntaxes from one original sentence and giving precedence to a selected one of the syntaxes |
GB2272091B (en) | 1992-10-30 | 1996-10-23 | Canon Europa Nv | Apparatus for use in aligning bilingual corpora |
US5675815A (en) | 1992-11-09 | 1997-10-07 | Ricoh Company, Ltd. | Language conversion system and text creating system using such |
JP2745370B2 (en) * | 1993-02-23 | 1998-04-28 | 日本アイ・ビー・エム株式会社 | Machine translation method and machine translation device |
US6186794B1 (en) * | 1993-04-02 | 2001-02-13 | Breakthrough To Literacy, Inc. | Apparatus for interactive adaptive learning by an individual through at least one of a stimuli presentation device and a user perceivable display |
US5432948A (en) | 1993-04-26 | 1995-07-11 | Taligent, Inc. | Object-oriented rule-based text input transliteration system |
GB2279164A (en) * | 1993-06-18 | 1994-12-21 | Canon Res Ct Europe Ltd | Processing a bilingual database. |
US5619709A (en) | 1993-09-20 | 1997-04-08 | Hnc, Inc. | System and method of context vector generation and retrieval |
US5510981A (en) | 1993-10-28 | 1996-04-23 | International Business Machines Corporation | Language translation apparatus and method using context-based translation models |
US6304841B1 (en) | 1993-10-28 | 2001-10-16 | International Business Machines Corporation | Automatic construction of conditional exponential models from elementary features |
US6064819A (en) | 1993-12-08 | 2000-05-16 | Imec | Control flow and memory management optimization |
JPH07210558A (en) | 1994-01-20 | 1995-08-11 | Fujitsu Ltd | Machine translation device |
JP3345763B2 (en) | 1994-03-04 | 2002-11-18 | 日本電信電話株式会社 | Natural language translator |
JP3377290B2 (en) | 1994-04-27 | 2003-02-17 | シャープ株式会社 | Machine translation device with idiom processing function |
US5752052A (en) | 1994-06-24 | 1998-05-12 | Microsoft Corporation | Method and system for bootstrapping statistical processing into a rule-based natural language parser |
US5761689A (en) | 1994-09-01 | 1998-06-02 | Microsoft Corporation | Autocorrecting text typed into a word processing document |
US5850561A (en) * | 1994-09-23 | 1998-12-15 | Lucent Technologies Inc. | Glossary construction tool |
JP3960562B2 (en) | 1994-09-30 | 2007-08-15 | 株式会社東芝 | How to learn machine translation |
JPH08101837A (en) | 1994-09-30 | 1996-04-16 | Toshiba Corp | Translating rule learning method for machine translation system |
JP2855409B2 (en) | 1994-11-17 | 1999-02-10 | 日本アイ・ビー・エム株式会社 | Natural language processing method and system |
GB2295470A (en) | 1994-11-28 | 1996-05-29 | Sharp Kk | Machine translation system |
JP3066274B2 (en) | 1995-01-12 | 2000-07-17 | シャープ株式会社 | Machine translation equipment |
US5987402A (en) | 1995-01-31 | 1999-11-16 | Oki Electric Industry Co., Ltd. | System and method for efficiently retrieving and translating source documents in different languages, and other displaying the translated documents at a client device |
US5715466A (en) | 1995-02-14 | 1998-02-03 | Compuserve Incorporated | System for parallel foreign language communication over a computer network |
US5855015A (en) | 1995-03-20 | 1998-12-29 | Interval Research Corporation | System and method for retrieval of hyperlinked information resources |
CA2170669A1 (en) | 1995-03-24 | 1996-09-25 | Fernando Carlos Neves Pereira | Grapheme-to phoneme conversion with weighted finite-state transducers |
EP0834139A4 (en) | 1995-06-07 | 1998-08-05 | Int Language Engineering Corp | Machine assisted translation tools |
US5903858A (en) | 1995-06-23 | 1999-05-11 | Saraki; Masashi | Translation machine for editing a original text by rewriting the same and translating the rewrote one |
US6073143A (en) | 1995-10-20 | 2000-06-06 | Sanyo Electric Co., Ltd. | Document conversion system including data monitoring means that adds tag information to hyperlink information and translates a document when such tag information is included in a document retrieval request |
US5983169A (en) | 1995-11-13 | 1999-11-09 | Japan Science And Technology Corporation | Method for automated translation of conjunctive phrases in natural languages |
US6993471B1 (en) | 1995-11-13 | 2006-01-31 | America Online, Inc. | Integrated multilingual browser |
US5987404A (en) | 1996-01-29 | 1999-11-16 | International Business Machines Corporation | Statistical natural language understanding using hidden clumpings |
US5779486A (en) | 1996-03-19 | 1998-07-14 | Ho; Chi Fai | Methods and apparatus to assess and enhance a student's understanding in a subject |
SG49804A1 (en) * | 1996-03-20 | 1998-06-15 | Government Of Singapore Repres | Parsing and translating natural language sentences automatically |
JPH09259127A (en) | 1996-03-21 | 1997-10-03 | Sharp Corp | Translation device |
US5909681A (en) | 1996-03-25 | 1999-06-01 | Torrent Systems, Inc. | Computer system and computerized method for partitioning data for parallel processing |
US5870706A (en) | 1996-04-10 | 1999-02-09 | Lucent Technologies, Inc. | Method and apparatus for an improved language recognition system |
US6470306B1 (en) | 1996-04-23 | 2002-10-22 | Logovista Corporation | Automated translation of annotated text based on the determination of locations for inserting annotation tokens and linked ending, end-of-sentence or language tokens |
US6161083A (en) | 1996-05-02 | 2000-12-12 | Sony Corporation | Example-based translation method and system which calculates word similarity degrees, a priori probability, and transformation probability to determine the best example for translation |
US5995922A (en) | 1996-05-02 | 1999-11-30 | Microsoft Corporation | Identifying information related to an input word in an electronic dictionary |
US5848386A (en) | 1996-05-28 | 1998-12-08 | Ricoh Company, Ltd. | Method and system for translating documents using different translation resources for different portions of the documents |
US6233544B1 (en) | 1996-06-14 | 2001-05-15 | At&T Corp | Method and apparatus for language translation |
US5806032A (en) | 1996-06-14 | 1998-09-08 | Lucent Technologies Inc. | Compilation of weighted finite-state transducers from decision trees |
JPH1011447A (en) | 1996-06-21 | 1998-01-16 | Ibm Japan Ltd | Translation method and system based upon pattern |
US5966686A (en) * | 1996-06-28 | 1999-10-12 | Microsoft Corporation | Method and system for computing semantic logical forms from syntax trees |
JPH1074204A (en) * | 1996-06-28 | 1998-03-17 | Toshiba Corp | Machine translation method and text/translation display method |
US5819265A (en) | 1996-07-12 | 1998-10-06 | International Business Machines Corporation | Processing names in a text |
US5836771A (en) | 1996-12-02 | 1998-11-17 | Ho; Chi Fai | Learning method and system based on questioning |
US6498921B1 (en) | 1999-09-01 | 2002-12-24 | Chi Fai Ho | Method and system to answer a natural-language question |
JP3579204B2 (en) | 1997-01-17 | 2004-10-20 | 富士通株式会社 | Document summarizing apparatus and method |
WO1998044433A1 (en) | 1997-03-31 | 1998-10-08 | Sanyo Electric Co., Ltd. | Document preparation method and machine translation device |
US6233545B1 (en) | 1997-05-01 | 2001-05-15 | William E. Datig | Universal machine translator of arbitrary languages utilizing epistemic moments |
US6233546B1 (en) | 1998-11-19 | 2001-05-15 | William E. Datig | Method and system for machine translation using epistemic moments and stored dictionary entries |
US5991710A (en) | 1997-05-20 | 1999-11-23 | International Business Machines Corporation | Statistical translation system with features based on phrases or groups of words |
US6415250B1 (en) | 1997-06-18 | 2002-07-02 | Novell, Inc. | System and method for identifying language using morphologically-based techniques |
US6032111A (en) | 1997-06-23 | 2000-02-29 | At&T Corp. | Method and apparatus for compiling context-dependent rewrite rules and input strings |
DE69837979T2 (en) | 1997-06-27 | 2008-03-06 | International Business Machines Corp. | System for extracting multilingual terminology |
US6356864B1 (en) | 1997-07-25 | 2002-03-12 | University Technology Corporation | Methods for analysis and evaluation of the semantic content of a writing based on vector length |
US6018617A (en) | 1997-07-31 | 2000-01-25 | Advantage Learning Systems, Inc. | Test generating and formatting system |
JPH11143877A (en) | 1997-10-22 | 1999-05-28 | Internatl Business Mach Corp <Ibm> | Compression method, method for compressing entry index data and machine translation system |
JP2001508200A (en) | 1997-11-14 | 2001-06-19 | コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ | Method and system for selective hardware sharing voice processing at multiple levels in a voice-based intercommunication system |
US6161082A (en) | 1997-11-18 | 2000-12-12 | At&T Corp | Network based language translation system |
KR980004126A (en) * | 1997-12-16 | 1998-03-30 | 양승택 | Query Language Conversion Apparatus and Method for Searching Multilingual Web Documents |
JPH11175527A (en) | 1997-12-15 | 1999-07-02 | Fujitsu Ltd | Output controller and output control method |
JP3272288B2 (en) | 1997-12-24 | 2002-04-08 | 日本アイ・ビー・エム株式会社 | Machine translation device and machine translation method |
US6533822B2 (en) | 1998-01-30 | 2003-03-18 | Xerox Corporation | Creating summaries along with indicators, and automatically positioned tabs |
US6031984A (en) | 1998-03-09 | 2000-02-29 | I2 Technologies, Inc. | Method and apparatus for optimizing constraint models |
JP3430007B2 (en) | 1998-03-20 | 2003-07-28 | 富士通株式会社 | Machine translation device and recording medium |
US6077085A (en) | 1998-05-19 | 2000-06-20 | Intellectual Reserve, Inc. | Technology assisted learning |
GB2337611A (en) * | 1998-05-20 | 1999-11-24 | Sharp Kk | Multilingual document retrieval system |
GB2338089A (en) | 1998-06-02 | 1999-12-08 | Sharp Kk | Indexing method |
US7197451B1 (en) | 1998-07-02 | 2007-03-27 | Novell, Inc. | Method and mechanism for the creation, maintenance, and comparison of semantic abstracts |
US6092034A (en) | 1998-07-27 | 2000-07-18 | International Business Machines Corporation | Statistical translation system and method for fast sense disambiguation and translation of large corpora using fertility models and sense models |
US6269351B1 (en) | 1999-03-31 | 2001-07-31 | Dryken Technologies, Inc. | Method and system for training an artificial neural network |
JP3959180B2 (en) | 1998-08-24 | 2007-08-15 | 東芝ソリューション株式会社 | Communication translation device |
US6549918B1 (en) * | 1998-09-21 | 2003-04-15 | Microsoft Corporation | Dynamic information format conversion |
US6285978B1 (en) | 1998-09-24 | 2001-09-04 | International Business Machines Corporation | System and method for estimating accuracy of an automatic natural language translation |
US6473896B1 (en) * | 1998-10-13 | 2002-10-29 | Parasoft, Corp. | Method and system for graphically generating user-defined rules for checking language quality |
JP2000132550A (en) | 1998-10-26 | 2000-05-12 | Matsushita Electric Ind Co Ltd | Chinese generating device for machine translation |
US6182014B1 (en) | 1998-11-20 | 2001-01-30 | Schlumberger Technology Corporation | Method and system for optimizing logistical operations in land seismic surveys |
US6535842B1 (en) | 1998-12-10 | 2003-03-18 | Global Information Research And Technologies, Llc | Automatic bilingual translation memory system |
US6460015B1 (en) | 1998-12-15 | 2002-10-01 | International Business Machines Corporation | Method, system and computer program product for automatic character transliteration in a text string object |
US6275789B1 (en) | 1998-12-18 | 2001-08-14 | Leo Moser | Method and apparatus for performing full bidirectional translation between a source language and a linked alternative language |
US6317708B1 (en) | 1999-01-07 | 2001-11-13 | Justsystem Corporation | Method for producing summaries of text document |
US6223150B1 (en) | 1999-01-29 | 2001-04-24 | Sony Corporation | Method and apparatus for parsing in a spoken language translation system |
WO2000062193A1 (en) | 1999-04-08 | 2000-10-19 | Kent Ridge Digital Labs | System for chinese tokenization and named entity recognition |
US6609087B1 (en) * | 1999-04-28 | 2003-08-19 | Genuity Inc. | Fact recognition system |
US6901361B1 (en) | 1999-07-09 | 2005-05-31 | Digital Esperanto, Inc. | Computerized translator of languages utilizing indexed databases of corresponding information elements |
US6278969B1 (en) * | 1999-08-18 | 2001-08-21 | International Business Machines Corp. | Method and system for improving machine translation accuracy using translation memory |
US7016827B1 (en) | 1999-09-03 | 2006-03-21 | International Business Machines Corporation | Method and system for ensuring robustness in natural language understanding |
US7171348B2 (en) * | 1999-09-10 | 2007-01-30 | Worldlingo.Com Pty Ltd | Communication processing system |
US6601026B2 (en) | 1999-09-17 | 2003-07-29 | Discern Communications, Inc. | Information retrieval by natural language querying |
US20060116865A1 (en) * | 1999-09-17 | 2006-06-01 | Www.Uniscape.Com | E-services translation utilizing machine translation and translation memory |
US6393389B1 (en) | 1999-09-23 | 2002-05-21 | Xerox Corporation | Using ranked translation choices to obtain sequences indicating meaning of multi-token expressions |
JP2001101187A (en) | 1999-09-30 | 2001-04-13 | Sony Corp | Device and method for translation and recording medium |
US6330530B1 (en) | 1999-10-18 | 2001-12-11 | Sony Corporation | Method and system for transforming a source language linguistic structure into a target language linguistic structure based on example linguistic feature structures |
US6529865B1 (en) | 1999-10-18 | 2003-03-04 | Sony Corporation | System and method to compile instructions to manipulate linguistic structures into separate functions |
US7016977B1 (en) | 1999-11-05 | 2006-03-21 | International Business Machines Corporation | Method and system for multilingual web server |
US6848080B1 (en) | 1999-11-05 | 2005-01-25 | Microsoft Corporation | Language input architecture for converting one text form to another text form with tolerance to spelling, typographical, and conversion errors |
US6904402B1 (en) | 1999-11-05 | 2005-06-07 | Microsoft Corporation | System and iterative method for lexicon, segmentation and language model joint optimization |
US6473729B1 (en) | 1999-12-20 | 2002-10-29 | Xerox Corporation | Word phrase translation using a phrase index |
JP2001249922A (en) | 1999-12-28 | 2001-09-14 | Matsushita Electric Ind Co Ltd | Word division system and device |
US6587844B1 (en) | 2000-02-01 | 2003-07-01 | At&T Corp. | System and methods for optimizing networks of weighted unweighted directed graphs |
AUPQ539700A0 (en) | 2000-02-02 | 2000-02-24 | Worldlingo.Com Pty Ltd | Translation ordering system |
US7366714B2 (en) * | 2000-03-23 | 2008-04-29 | Albert Krachman | Method and system for providing electronic discovery on computer databases and archives using statement analysis to detect false statements and recover relevant data |
US6490549B1 (en) | 2000-03-30 | 2002-12-03 | Scansoft, Inc. | Automatic orthographic transformation of a text stream |
US20010029455A1 (en) | 2000-03-31 | 2001-10-11 | Chin Jeffrey J. | Method and apparatus for providing multilingual translation over a network |
US7107204B1 (en) | 2000-04-24 | 2006-09-12 | Microsoft Corporation | Computer-aided writing system and method with cross-language writing wizard |
US20040006560A1 (en) * | 2000-05-01 | 2004-01-08 | Ning-Ping Chan | Method and system for translingual translation of query and search and retrieval of multilingual information on the web |
JP2004501429A (en) | 2000-05-11 | 2004-01-15 | ユニバーシティ・オブ・サザン・カリフォルニア | Machine translation techniques |
US7516154B2 (en) | 2000-06-28 | 2009-04-07 | Qnaturally Systems Inc. | Cross language advertising |
JP3982736B2 (en) | 2000-06-30 | 2007-09-26 | 沖電気工業株式会社 | Translation system |
JP4011268B2 (en) | 2000-07-05 | 2007-11-21 | 株式会社アイアイエス | Multilingual translation system |
US7389234B2 (en) | 2000-07-20 | 2008-06-17 | Microsoft Corporation | Method and apparatus utilizing speech grammar rules written in a markup language |
US6952666B1 (en) | 2000-07-20 | 2005-10-04 | Microsoft Corporation | Ranking parser for a natural language processing system |
US20020078091A1 (en) | 2000-07-25 | 2002-06-20 | Sonny Vu | Automatic summarization of a document |
US20020046262A1 (en) | 2000-08-18 | 2002-04-18 | Joerg Heilig | Data access system and method with proxy and remote processing |
US20030217052A1 (en) | 2000-08-24 | 2003-11-20 | Celebros Ltd. | Search engine method and apparatus |
US20020059566A1 (en) * | 2000-08-29 | 2002-05-16 | Delcambre Lois M. | Uni-level description of computer information and transformation of computer information between representation schemes |
US7085708B2 (en) | 2000-09-23 | 2006-08-01 | Ravenflow, Inc. | Computer system with natural language to machine language translator |
US6782356B1 (en) | 2000-10-03 | 2004-08-24 | Hewlett-Packard Development Company, L.P. | Hierarchical language chunking translation table |
US6983239B1 (en) | 2000-10-25 | 2006-01-03 | International Business Machines Corporation | Method and apparatus for embedding grammars in a natural language understanding (NLU) statistical parser |
ATE297588T1 (en) | 2000-11-14 | 2005-06-15 | Ibm | ADJUSTING PHONETIC CONTEXT TO IMPROVE SPEECH RECOGNITION |
US6885985B2 (en) | 2000-12-18 | 2005-04-26 | Xerox Corporation | Terminology translation for unaligned comparable corpora using category based translation probabilities |
US7054803B2 (en) * | 2000-12-19 | 2006-05-30 | Xerox Corporation | Extracting sentence translations from translated documents |
US20020087313A1 (en) * | 2000-12-29 | 2002-07-04 | Lee Victor Wai Leung | Computer-implemented intelligent speech model partitioning method and system |
US6996518B2 (en) * | 2001-01-03 | 2006-02-07 | International Business Machines Corporation | Method and apparatus for automated measurement of quality for machine translation |
US6990439B2 (en) | 2001-01-10 | 2006-01-24 | Microsoft Corporation | Method and apparatus for performing machine translation using a unified language model and translation model |
JP2002215621A (en) | 2001-01-19 | 2002-08-02 | Nec Corp | Translation server, translation method and program |
US6922809B2 (en) | 2001-01-25 | 2005-07-26 | International Business Machines Corporation | Method and apparatus providing capitalization recovery for text |
US7113903B1 (en) | 2001-01-30 | 2006-09-26 | At&T Corp. | Method and apparatus for providing stochastic finite-state machine translation |
JP2003141018A (en) * | 2001-11-02 | 2003-05-16 | Fujitsu Ltd | Server, relay device, information providing method and program |
AUPR329501A0 (en) | 2001-02-22 | 2001-03-22 | Worldlingo, Inc | Translation information segment |
US7216073B2 (en) | 2001-03-13 | 2007-05-08 | Intelligate, Ltd. | Dynamic natural language understanding |
US7860706B2 (en) | 2001-03-16 | 2010-12-28 | Eli Abir | Knowledge system method and appparatus |
JP3969628B2 (en) * | 2001-03-19 | 2007-09-05 | 富士通株式会社 | Translation support apparatus, method, and translation support program |
JP4574047B2 (en) * | 2001-03-30 | 2010-11-04 | 富士通株式会社 | Machine translation apparatus and program for performing translation using translation example dictionary |
US7107215B2 (en) | 2001-04-16 | 2006-09-12 | Sakhr Software Company | Determining a compact model to transcribe the arabic language acoustically in a well defined basic phonetic study |
US7295962B2 (en) | 2001-05-11 | 2007-11-13 | University Of Southern California | Statistical memory-based translation system |
US20020169592A1 (en) | 2001-05-11 | 2002-11-14 | Aityan Sergey Khachatur | Open environment for real-time multilingual communication |
WO2002093417A1 (en) | 2001-05-17 | 2002-11-21 | University Of Southern California | Statistical method for building a translation memory |
US7177792B2 (en) | 2001-05-31 | 2007-02-13 | University Of Southern California | Integer programming decoder for machine translation |
US7050964B2 (en) * | 2001-06-01 | 2006-05-23 | Microsoft Corporation | Scaleable machine translation system |
US7191115B2 (en) | 2001-06-20 | 2007-03-13 | Microsoft Corporation | Statistical method and apparatus for learning translation relationships among words |
US20020198699A1 (en) | 2001-06-21 | 2002-12-26 | International Business Machines Corporation | Apparatus, system and method for providing open source language translation |
US8214196B2 (en) * | 2001-07-03 | 2012-07-03 | University Of Southern California | Syntax-based statistical translation model |
US6810374B2 (en) | 2001-07-23 | 2004-10-26 | Pilwon Kang | Korean romanization system |
US7024351B2 (en) | 2001-08-21 | 2006-04-04 | Microsoft Corporation | Method and apparatus for robust efficient parsing |
US7146358B1 (en) * | 2001-08-28 | 2006-12-05 | Google Inc. | Systems and methods for using anchor text as parallel corpora for cross-language information retrieval |
JP3452558B2 (en) * | 2001-09-25 | 2003-09-29 | インターナショナル・ビジネス・マシーンズ・コーポレーション | Method, system, and program for associating a dictionary to be translated with a domain dictionary |
WO2003034281A1 (en) * | 2001-10-19 | 2003-04-24 | Intel Zao | Method and apparatus to provide a hierarchical index for a language model data structure |
EP1306775A1 (en) | 2001-10-29 | 2003-05-02 | BRITISH TELECOMMUNICATIONS public limited company | Machine translation |
US7447623B2 (en) | 2001-10-29 | 2008-11-04 | British Telecommunications Public Limited Company | Machine translation |
US20030144832A1 (en) | 2002-01-16 | 2003-07-31 | Harris Henry M. | Machine translation system |
US20030154071A1 (en) * | 2002-02-11 | 2003-08-14 | Shreve Gregory M. | Process for the document management and computer-assisted translation of documents utilizing document corpora constructed by intelligent agents |
US7013262B2 (en) | 2002-02-12 | 2006-03-14 | Sunflare Co., Ltd | System and method for accurate grammar analysis using a learners' model and part-of-speech tagged (POST) parser |
AU2003216329A1 (en) | 2002-02-15 | 2003-09-09 | Mathsoft Engineering And Education, Inc. | Linguistic support for a regognizer of mathematical expressions |
JP2003242136A (en) * | 2002-02-20 | 2003-08-29 | Fuji Xerox Co Ltd | Syntax information tag imparting support system and method therefor |
EP1483687A4 (en) | 2002-03-11 | 2008-08-06 | Univ Southern California | Named entity translation |
JP3959453B2 (en) | 2002-03-14 | 2007-08-15 | 沖電気工業株式会社 | Translation mediation system and translation mediation server |
US7526424B2 (en) | 2002-03-20 | 2009-04-28 | Microsoft Corporation | Sentence realization model for a natural language generation system |
WO2004001623A2 (en) | 2002-03-26 | 2003-12-31 | University Of Southern California | Constructing a translation lexicon from comparable, non-parallel corpora |
AU2003267953A1 (en) | 2002-03-26 | 2003-12-22 | University Of Southern California | Statistical machine translation using a large monlingual corpus |
ES2343786T3 (en) | 2002-03-27 | 2010-08-10 | University Of Southern California | PROBABILITY MODEL OF UNION BASED ON PHRASES FOR STATISTICAL AUTOMATIC TRANSLATION. |
JP2005527894A (en) | 2002-03-28 | 2005-09-15 | ユニバーシティ・オブ・サザン・カリフォルニア | Statistical machine translation |
EP1351158A1 (en) | 2002-03-28 | 2003-10-08 | BRITISH TELECOMMUNICATIONS public limited company | Machine translation |
US7403890B2 (en) * | 2002-05-13 | 2008-07-22 | Roushar Joseph C | Multi-dimensional method and apparatus for automated language interpretation |
US7805302B2 (en) * | 2002-05-20 | 2010-09-28 | Microsoft Corporation | Applying a structured language model to information extraction |
US7031911B2 (en) | 2002-06-28 | 2006-04-18 | Microsoft Corporation | System and method for automatic detection of collocation mistakes in documents |
JP4239505B2 (en) | 2002-07-31 | 2009-03-18 | 日本電気株式会社 | Translation apparatus, translation method, program, and recording medium |
US20040035055A1 (en) * | 2002-08-21 | 2004-02-26 | Tianli Zhu | Sulfur control for fuel processing system for fuel cell power plant |
US7349839B2 (en) * | 2002-08-27 | 2008-03-25 | Microsoft Corporation | Method and apparatus for aligning bilingual corpora |
CN1685341B (en) * | 2002-09-30 | 2014-09-10 | 开拧系统公司 | Blinking annotation callouts highlighting cross language search results |
US7149688B2 (en) | 2002-11-04 | 2006-12-12 | Speechworks International, Inc. | Multi-lingual speech recognition with cross-language context modeling |
GB2395029A (en) * | 2002-11-06 | 2004-05-12 | Alan Wilkinson | Translation of electronically transmitted messages |
US7249012B2 (en) | 2002-11-20 | 2007-07-24 | Microsoft Corporation | Statistical method and apparatus for learning translation relationships among phrases |
WO2004049110A2 (en) * | 2002-11-22 | 2004-06-10 | Transclick, Inc. | Language translation system and method |
US7209875B2 (en) | 2002-12-04 | 2007-04-24 | Microsoft Corporation | System and method for machine learning a confidence metric for machine translation |
US20040111253A1 (en) | 2002-12-10 | 2004-06-10 | International Business Machines Corporation | System and method for rapid development of natural language understanding using active learning |
US20040115597A1 (en) * | 2002-12-11 | 2004-06-17 | Butt Thomas Giles | System and method of interactive learning using adaptive notes |
JP3973549B2 (en) * | 2002-12-19 | 2007-09-12 | 沖電気工業株式会社 | Bilingual dependency structure associating apparatus, method and program, and recording medium recording parallel translation dependency structure associating program |
US7627817B2 (en) * | 2003-02-21 | 2009-12-01 | Motionpoint Corporation | Analyzing web site for translation |
US7356457B2 (en) | 2003-02-28 | 2008-04-08 | Microsoft Corporation | Machine translation using learned word associations without referring to a multi-lingual human authored dictionary of content words |
US7346493B2 (en) | 2003-03-25 | 2008-03-18 | Microsoft Corporation | Linguistically informed statistical models of constituent structure for ordering in sentence realization for a natural language generation system |
US7319949B2 (en) | 2003-05-27 | 2008-01-15 | Microsoft Corporation | Unilingual translator |
US7383542B2 (en) | 2003-06-20 | 2008-06-03 | Microsoft Corporation | Adaptive machine translation service |
US7711545B2 (en) | 2003-07-02 | 2010-05-04 | Language Weaver, Inc. | Empirical methods for splitting compound words with application to machine translation |
US8548794B2 (en) | 2003-07-02 | 2013-10-01 | University Of Southern California | Statistical noun phrase translation |
US7346487B2 (en) | 2003-07-23 | 2008-03-18 | Microsoft Corporation | Method and apparatus for identifying translations |
US7369998B2 (en) * | 2003-08-14 | 2008-05-06 | Voxtec International, Inc. | Context based language translation devices and methods |
US7349845B2 (en) * | 2003-09-03 | 2008-03-25 | International Business Machines Corporation | Method and apparatus for dynamic modification of command weights in a natural language understanding system |
JP3919771B2 (en) * | 2003-09-09 | 2007-05-30 | 株式会社国際電気通信基礎技術研究所 | Machine translation system, control device thereof, and computer program |
KR100542755B1 (en) | 2003-09-15 | 2006-01-20 | 한국전자통신연구원 | Hybrid automatic translation Apparatus and Method by combining Rule-based method and Translation pattern method, and The medium recording the program |
US7283950B2 (en) | 2003-10-06 | 2007-10-16 | Microsoft Corporation | System and method for translating from a source language to at least one target language utilizing a community of contributors |
US7302392B1 (en) | 2003-10-07 | 2007-11-27 | Sprint Spectrum L.P. | Voice browser with weighting of browser-level grammar to enhance usability |
US20050125218A1 (en) | 2003-12-04 | 2005-06-09 | Nitendra Rajput | Language modelling for mixed language expressions |
CN1894688A (en) * | 2003-12-15 | 2007-01-10 | 有限会社言语技术研究所 | Translation determination system, method, and program |
US7587307B2 (en) * | 2003-12-18 | 2009-09-08 | Xerox Corporation | Method and apparatus for evaluating machine translation quality |
US7496497B2 (en) | 2003-12-18 | 2009-02-24 | Taiwan Semiconductor Manufacturing Co., Ltd. | Method and system for selecting web site home page by extracting site language cookie stored in an access device to identify directional information item |
US20050204002A1 (en) | 2004-02-16 | 2005-09-15 | Friend Jeffrey E. | Dynamic online email catalog and trust relationship management system and method |
WO2005089340A2 (en) | 2004-03-15 | 2005-09-29 | University Of Southern California | Training tree transducers |
US7593843B2 (en) | 2004-03-30 | 2009-09-22 | Microsoft Corporation | Statistical language model for logical form using transfer mappings |
US7827025B2 (en) | 2004-04-06 | 2010-11-02 | Microsoft Corporation | Efficient capitalization through user modeling |
US8666725B2 (en) | 2004-04-16 | 2014-03-04 | University Of Southern California | Selection and use of nonstatistical translation components in a statistical machine translation framework |
US7716037B2 (en) | 2004-05-24 | 2010-05-11 | Sri International | Method and apparatus for natural language translation in a finite domain |
GB2415518A (en) | 2004-06-24 | 2005-12-28 | Sharp Kk | Method and apparatus for translation based on a repository of existing translations |
US7620632B2 (en) * | 2004-06-30 | 2009-11-17 | Skyler Technology, Inc. | Method and/or system for performing tree matching |
US7596485B2 (en) * | 2004-06-30 | 2009-09-29 | Microsoft Corporation | Module for creating a language neutral syntax representation using a language particular syntax tree |
US20060015323A1 (en) | 2004-07-13 | 2006-01-19 | Udupa Raghavendra U | Method, apparatus, and computer program for statistical translation decoding |
US7409332B2 (en) | 2004-07-14 | 2008-08-05 | Microsoft Corporation | Method and apparatus for initializing iterative training of translation probabilities |
US20060020448A1 (en) | 2004-07-21 | 2006-01-26 | Microsoft Corporation | Method and apparatus for capitalizing text using maximum entropy |
US20070016401A1 (en) * | 2004-08-12 | 2007-01-18 | Farzad Ehsani | Speech-to-speech translation system with user-modifiable paraphrasing grammars |
US7584103B2 (en) * | 2004-08-20 | 2009-09-01 | Multimodal Technologies, Inc. | Automated extraction of semantic content and generation of a structured document from speech |
US20070269775A1 (en) | 2004-09-14 | 2007-11-22 | Dreams Of Babylon, Inc. | Personalized system and method for teaching a foreign language |
JP5452868B2 (en) | 2004-10-12 | 2014-03-26 | ユニヴァーシティー オブ サザン カリフォルニア | Training for text-to-text applications that use string-to-tree conversion for training and decoding |
US7505894B2 (en) | 2004-11-04 | 2009-03-17 | Microsoft Corporation | Order model for dependency structure |
US7546235B2 (en) | 2004-11-15 | 2009-06-09 | Microsoft Corporation | Unsupervised learning of paraphrase/translation alternations and selective application thereof |
US7680646B2 (en) | 2004-12-21 | 2010-03-16 | Xerox Corporation | Retrieval method for translation memories containing highly structured documents |
US7774192B2 (en) * | 2005-01-03 | 2010-08-10 | Industrial Technology Research Institute | Method for extracting translations from translated texts using punctuation-based sub-sentential alignment |
US7571120B2 (en) * | 2005-01-12 | 2009-08-04 | International Business Machines Corporation | Computer implemented method for estimating future grid job costs by classifying grid jobs and storing results of processing grid job microcosms |
US7672830B2 (en) | 2005-02-22 | 2010-03-02 | Xerox Corporation | Apparatus and methods for aligning words in bilingual sentences |
JP4419871B2 (en) * | 2005-03-02 | 2010-02-24 | 富士ゼロックス株式会社 | Translation request apparatus and program |
WO2006133571A1 (en) | 2005-06-17 | 2006-12-21 | National Research Council Of Canada | Means and method for adapted language translation |
US8676563B2 (en) | 2009-10-01 | 2014-03-18 | Language Weaver, Inc. | Providing human-generated and machine-generated trusted translations |
US7974833B2 (en) | 2005-06-21 | 2011-07-05 | Language Weaver, Inc. | Weighted system of expressing language information using a compact notation |
US7389222B1 (en) | 2005-08-02 | 2008-06-17 | Language Weaver, Inc. | Task parallelization in a text-to-text system |
US7813918B2 (en) * | 2005-08-03 | 2010-10-12 | Language Weaver, Inc. | Identifying documents which form translated pairs, within a document collection |
US7552053B2 (en) * | 2005-08-22 | 2009-06-23 | International Business Machines Corporation | Techniques for aiding speech-to-speech translation |
US7653531B2 (en) * | 2005-08-25 | 2010-01-26 | Multiling Corporation | Translation quality quantifying apparatus and method |
US7624020B2 (en) | 2005-09-09 | 2009-11-24 | Language Weaver, Inc. | Adapter for allowing both online and offline training of a text to text system |
US7558769B2 (en) * | 2005-09-30 | 2009-07-07 | Google Inc. | Identifying clusters of similar reviews and displaying representative reviews from multiple clusters |
US7957953B2 (en) * | 2005-10-03 | 2011-06-07 | Microsoft Corporation | Weighted linear bilingual word alignment model |
US20070083357A1 (en) * | 2005-10-03 | 2007-04-12 | Moore Robert C | Weighted linear model |
US10319252B2 (en) | 2005-11-09 | 2019-06-11 | Sdl Inc. | Language capability assessment and training apparatus and techniques |
US7822596B2 (en) * | 2005-12-05 | 2010-10-26 | Microsoft Corporation | Flexible display translation |
WO2007070558A2 (en) * | 2005-12-12 | 2007-06-21 | Meadan, Inc. | Language translation using a hybrid network of human and machine translators |
GB2433403B (en) | 2005-12-16 | 2009-06-24 | Emil Ltd | A text editing apparatus and method |
US7536295B2 (en) | 2005-12-22 | 2009-05-19 | Xerox Corporation | Machine translation using non-contiguous fragments of text |
US20070168202A1 (en) * | 2006-01-10 | 2007-07-19 | Xpient Solutions, Llc | Restaurant drive-through monitoring system |
US7849144B2 (en) | 2006-01-13 | 2010-12-07 | Cisco Technology, Inc. | Server-initiated language translation of an instant message based on identifying language attributes of sending and receiving users |
US20070180373A1 (en) | 2006-01-30 | 2007-08-02 | Bauman Brian D | Method and system for renderring application text in one or more alternative languages |
US7542893B2 (en) * | 2006-05-10 | 2009-06-02 | Xerox Corporation | Machine translation using elastic chunks |
US20070265826A1 (en) * | 2006-05-10 | 2007-11-15 | Stanley Chen | Systems and methods for fast and memory efficient machine translation using statistical integrated phase lattice |
CN101443759B (en) * | 2006-05-12 | 2010-08-11 | 北京乐图在线科技有限公司 | Multi-lingual information retrieval |
US8886514B2 (en) | 2006-08-18 | 2014-11-11 | National Research Council Of Canada | Means and a method for training a statistical machine translation system utilizing a posterior probability in an N-best translation list |
KR100837358B1 (en) | 2006-08-25 | 2008-06-12 | 한국전자통신연구원 | Domain-Adaptive Portable Machine Translation Device for Translating Closed Captions Using Dynamic Translation Resources and method thereof |
US20080065478A1 (en) * | 2006-09-12 | 2008-03-13 | Microsoft Corporation | Electronic coupon based service for enhancing content |
JP5040256B2 (en) | 2006-10-19 | 2012-10-03 | パナソニック株式会社 | Refrigeration cycle apparatus and control method thereof |
US20080183555A1 (en) | 2007-01-29 | 2008-07-31 | Hunter Walk | Determining and communicating excess advertiser demand information to users, such as publishers participating in, or expected to participate in, an advertising network |
US20080215418A1 (en) | 2007-03-02 | 2008-09-04 | Adready, Inc. | Modification of advertisement campaign elements based on heuristics and real time feedback |
US8615389B1 (en) * | 2007-03-16 | 2013-12-24 | Language Weaver, Inc. | Generation and exploitation of an approximate language model |
US8831928B2 (en) | 2007-04-04 | 2014-09-09 | Language Weaver, Inc. | Customizable machine translation service |
JP2008276517A (en) | 2007-04-27 | 2008-11-13 | Oki Electric Ind Co Ltd | Device and method for evaluating translation and program |
US7877251B2 (en) | 2007-05-07 | 2011-01-25 | Microsoft Corporation | Document translation system |
US20080307481A1 (en) | 2007-06-08 | 2008-12-11 | General Instrument Corporation | Method and System for Managing Content in a Network |
US8060360B2 (en) * | 2007-10-30 | 2011-11-15 | Microsoft Corporation | Word-dependent transition models in HMM based word alignment for statistical machine translation |
US20090119091A1 (en) | 2007-11-01 | 2009-05-07 | Eitan Chaim Sarig | Automated pattern based human assisted computerized translation network systems |
TWI457868B (en) * | 2008-03-12 | 2014-10-21 | Univ Nat Kaohsiung 1St Univ Sc | Method for automatically modifying a translation from a machine translation |
US8910110B2 (en) * | 2008-03-19 | 2014-12-09 | Oracle International Corporation | Application translation cost estimator |
US20100017293A1 (en) | 2008-07-17 | 2010-01-21 | Language Weaver, Inc. | System, method, and computer program for providing multilingual text advertisments |
US8244519B2 (en) * | 2008-12-03 | 2012-08-14 | Xerox Corporation | Dynamic translation memory using statistical machine translation |
US8990064B2 (en) | 2009-07-28 | 2015-03-24 | Language Weaver, Inc. | Translating documents based on content |
US20110066643A1 (en) * | 2009-09-16 | 2011-03-17 | John Cooper | System and method for assembling, verifying, and distibuting financial information |
US8265923B2 (en) * | 2010-05-11 | 2012-09-11 | Xerox Corporation | Statistical machine translation employing efficient parameter training |
US8694303B2 (en) * | 2011-06-15 | 2014-04-08 | Language Weaver, Inc. | Systems and methods for tuning parameters in statistical machine translation |
-
2006
- 2006-12-05 US US11/635,248 patent/US8943080B2/en active Active
-
2007
- 2007-01-12 WO PCT/US2007/000853 patent/WO2007126447A2/en active Application Filing
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5907821A (en) * | 1995-11-06 | 1999-05-25 | Hitachi, Ltd. | Method of computer-based automatic extraction of translation pairs of words from a bilingual text |
US20050021517A1 (en) * | 2000-03-22 | 2005-01-27 | Insightful Corporation | Extended functionality for an inverse inference engine based web search |
US20050228643A1 (en) * | 2004-03-23 | 2005-10-13 | Munteanu Dragos S | Discovery of parallel text portions in comparable collections of corpora and training using comparable texts |
Also Published As
Publication number | Publication date |
---|---|
US20070250306A1 (en) | 2007-10-25 |
WO2007126447A3 (en) | 2008-04-17 |
US8943080B2 (en) | 2015-01-27 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8943080B2 (en) | Systems and methods for identifying parallel documents and sentence fragments in multilingual document collections | |
Ma | Champollion: A Robust Parallel Text Sentence Aligner. | |
Munteanu et al. | Improving machine translation performance by exploiting non-parallel corpora | |
US8296127B2 (en) | Discovery of parallel text portions in comparable collections of corpora and training using comparable texts | |
Uszkoreit et al. | Large scale parallel document mining for machine translation | |
Das et al. | Named entity recognition with word embeddings and wikipedia categories for a low-resource language | |
Antonova et al. | Building a web-based parallel corpus and filtering out machine-translated text | |
Stefanescu et al. | Hybrid parallel sentence mining from comparable corpora | |
WO2005099422A2 (en) | Selection and use of nonstastical translation components in a statistical machine translation framework | |
WO2009035863A2 (en) | Mining bilingual dictionaries from monolingual web pages | |
JPH08500691A (en) | Bilingual database processing method and apparatus | |
JP2002215619A (en) | Translation sentence extracting method from translated document | |
US8204736B2 (en) | Access to multilingual textual resources | |
US8670974B2 (en) | Acquisition of out-of-vocabulary translations by dynamically learning extraction rules | |
Loponen et al. | A dictionary-and corpus-independent statistical lemmatizer for information retrieval in low resource languages | |
Tufiş et al. | Improved lexical alignment by combining multiple reified alignments | |
US7593844B1 (en) | Document translation systems and methods employing translation memories | |
JP3765801B2 (en) | Parallel translation expression extraction apparatus, parallel translation extraction method, and parallel translation extraction program | |
Pinnis et al. | Accurat toolkit for multi-level alignment and information extraction from comparable corpora | |
Zhang et al. | Chinese OOV translation and post-translation query expansion in chinese--english cross-lingual information retrieval | |
Zhao et al. | Extracting paraphrase patterns from bilingual parallel corpora | |
Thompson | Detecting cross-lingual plagiarism using simulated word embeddings | |
Shi et al. | Comparing different units for query translation in Chinese cross-language information retrieval | |
Tufiş et al. | Lexical token alignment: Experiments, results and applications | |
Tufiş et al. | Combined word alignments |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 07709768 Country of ref document: EP Kind code of ref document: A2 |
|
NENP | Non-entry into the national phase in: |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 07709768 Country of ref document: EP Kind code of ref document: A2 |