CA2201189A1 - Compilation of weighted finite-state transducers from decision trees - Google Patents
Compilation of weighted finite-state transducers from decision treesInfo
- Publication number
- CA2201189A1 CA2201189A1 CA002201189A CA2201189A CA2201189A1 CA 2201189 A1 CA2201189 A1 CA 2201189A1 CA 002201189 A CA002201189 A CA 002201189A CA 2201189 A CA2201189 A CA 2201189A CA 2201189 A1 CA2201189 A1 CA 2201189A1
- Authority
- CA
- Canada
- Prior art keywords
- weighted
- decision tree
- generating
- transducer
- finite
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 238000003066 decision tree Methods 0.000 title claims abstract description 69
- 238000000034 method Methods 0.000 claims abstract description 66
- 230000014509 gene expression Effects 0.000 claims description 30
- 239000003550 marker Substances 0.000 claims description 21
- 238000013507 mapping Methods 0.000 claims description 5
- 238000003058 natural language processing Methods 0.000 abstract description 4
- 230000008569 process Effects 0.000 abstract description 4
- 230000006870 function Effects 0.000 description 8
- 101001126084 Homo sapiens Piwi-like protein 2 Proteins 0.000 description 4
- 102100029365 Piwi-like protein 2 Human genes 0.000 description 4
- GYMWQLRSSDFGEQ-ADRAWKNSSA-N [(3e,8r,9s,10r,13s,14s,17r)-13-ethyl-17-ethynyl-3-hydroxyimino-1,2,6,7,8,9,10,11,12,14,15,16-dodecahydrocyclopenta[a]phenanthren-17-yl] acetate;(8r,9s,13s,14s,17r)-17-ethynyl-13-methyl-7,8,9,11,12,14,15,16-octahydro-6h-cyclopenta[a]phenanthrene-3,17-diol Chemical compound OC1=CC=C2[C@H]3CC[C@](C)([C@](CC4)(O)C#C)[C@@H]4[C@@H]3CCC2=C1.O/N=C/1CC[C@@H]2[C@H]3CC[C@](CC)([C@](CC4)(OC(C)=O)C#C)[C@@H]4[C@@H]3CCC2=C\1 GYMWQLRSSDFGEQ-ADRAWKNSSA-N 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 230000007704 transition Effects 0.000 description 4
- 241001313288 Labia Species 0.000 description 2
- 239000000370 acceptor Substances 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000013179 statistical model Methods 0.000 description 2
- XMTQQYYKAHVGBJ-UHFFFAOYSA-N 3-(3,4-DICHLOROPHENYL)-1,1-DIMETHYLUREA Chemical class CN(C)C(=O)NC1=CC=C(Cl)C(Cl)=C1 XMTQQYYKAHVGBJ-UHFFFAOYSA-N 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006698 induction Effects 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000026683 transduction Effects 0.000 description 1
- 238000010361 transduction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/08—Speech classification or search
- G10L15/18—Speech classification or search using natural language modelling
- G10L15/183—Speech classification or search using natural language modelling using context dependencies, e.g. language models
- G10L15/19—Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
- G10L15/193—Formal grammars, e.g. finite state automata, context free grammars or word networks
Abstract
A method for automatically converting a decision tree into one or more weighted finite-state transducers. Specifically, the method in accordance with an illustrative embodiment of the present invention processes one or more terminal (i.e., leaf) nodes of a given decision tree to generate one or more corresponding weighted rewrite rules. Then, these weighted rewrite rules are processed to generate weighted finite-state transducers corresponding to the one or more terminal nodes of the decision tree. In this manner, decision trees may be advantageously compiled into weighted finite-state transducers, and these transducers may then be used directly in various speech and natural language processing systems. The weighted rewrite rules employed herein comprise an extension of conventional rewrite rules, familiar to those skilled in the art.
Description
220~ 18~
COMPILATION OF WEIGHTED FINITE-STATE TRANSDUCERS
FROM DECISION TREES
Field of the Inv~ntinn The present invention relates generally to the field of linguistic modeling as used, for example, in speech recognition and/or speech synthesis systems, and more particularly to finite-state methods for implementing such linguistic models in such systems.
R~ ~ollnd of the Inv~ntioll Much attention has been devoted recently to methods for inferring linguistic models from linguistic data. One powerful inference method that has been used invarious applications are decision trees, and in particular classification and regression trees ("CART"), well known to those of ordinary skill in the art. As is also well known to those skilled in the art, these decision trees can be "grown" from linguistic data by the application of conventional methods, such as are described, for example, in Michael D. Riley, "Some Applications of Tree-based Modelling to Speech and Language," Proceedings of the Speech and Natural Language Workshop, pages 339-352, Cape Cod, Massachusetts, October 1989. Other applications of tree-based modeling to specific problems in speech and natural language processing are 20 tli~cll~sed, for example, in Michael D. Riley, "A Statistical Model for Generating Pronunciation Networks, " Procee-ling~ of the International Confe~ ce on Acoustics, Speech and Signal Processing, pages Sll.l-S11.4, October, 1991, and in David M.
Magerman, "St~ti.ctical Decision-Tree Models for Parsing," 33rd Annual Meeting of the Association for Computational Linguistics, pages 276-283, Morristown, New25 Jersey, April, 1995. Each of these refelel1ces is incorporated by reference as if fully set forth herein.
An increasing amount of attention has also been focused on finite-state methods for implementing linguistic models. These methods include, in particular, the use of finite-state tr~n~ducers and weighted finite-state tr~n~d~1cers. See, e.g., 30 Ronald M. Kaplan and Martin Kay, "Regular Models of Phonological Rule ~2U1 1 89 Systems," Computational Linguistics, 20:331-378, September, 1994, and Fernando Pereira, Michael Riley and Richard Sproat, "Weighted Rational Tr~n~ ctions and their Application to Human Language Processing," ARPA Workshop on Human Language Technology, pages 249-254, March, 1994. Each of these references is 5 also incorporated by reference as if fully set forth herein.
One reason for the interest in the use of finite-state mechanisms is that finite-state machines provide a mathematically well-understood computational framework for representing a wide variety of information, both in natural language processing and in speech processing. Lexicons, phonological rules, Hidden Markov10 Models, and regular grammars, each of which is well known to those of ordinary skill in the art, are all representable as finite-state machines. Moreover, the availability of finite-state operations such as union, intersection and composition means that information from these various sources can be combined in useful and computationally attractive ways. The above-cited papers (among others) provide 15 further justification for the use of finite-state methods for implementing linguistic models.
Despite the recent attention given to both the use of decision trees for inferring linguistic models from linguistic data, and the use of finite-state methods for implementing linguistic models, there have not heretofore been any viable 20 techniques whereby these two strands of research have been combined in such amanner as to leverage the advantages of each. In particular, whereas generating decision trees from linguistic data has proven quite successful, and whereas finite-state based system implementations have been successfully developed, no techniques for automatically deriving finite-state models from decision trees have been 25 developed, and only partial success has been achieved in ~lle~ to directly infer tr~n.~ucers from linguistic data. (See, e.g., Daniel Gildea and Daniel Jurafsky,"Automatic Induction of Finite State Tr~n~llcers for Simple Phonological Rules,"33rd Annual Meeting of the Association for Computational Linguistics, pages 9-15, Morristown, New Jersey, June, 1995, also incorporated by lefele~ce as if fully set 30 forth herein.) Thus, if these two techniques could be "married" in the form of an automated procedure for converting the information in decision trees into weighted - - ~2~ 1 1 8~
finite-state tran.C~lucers~ information inferred from linguistic data and represented in a decision tree could be used directly in a system that represents other information, such as lexicons or grammars, in the form of finite-state machines.
Snmm~ry of the Inv~ntioll The use of decision trees for linguistic modelling and the use of finite-state transducers are advantageously combined by a method in accordance with an illustrative embodiment of the present invention, whereby a decision tree is automatically converted into one or more weighted finite-state transducers.
Specifically, the illustrative method processes one or more terminal (i. e., leaf) nodes of a given decision tree to generate one or more corresponding weighted rewrite rules. Then, these weighted rewrite rules are processed to generate weighted finite-state tr~n~clucers corresponding to the one or more terminal nodes of the decision tree. In this marmer, decision trees may be advantageously compiled into weighted finite-state tr~n~ducers, and these transducers may then be used directly in various 15 speech and natural language processing systems. The weighted rewrite rules employed herein comprise an extension of conventional rewrite rules, f~mili~r tothose skilled in the art.
Rrief Des-~r~ptiol- of the T)r~wi~
Figure 1 shows an illustrative decision tree which may be provided as input 20 to a procedure for generating weighted finite-state transducers from decision trees, such as the illustrative procedure of Figure 3.
Figure 2 shows an illustrative weighted finite-state transducer generated, for example, by the illustrative procedure of Figure 3.
Figure 3 shows a flow diagram of an illustrative procedure for generating 25 weighted finite-state tr~n.cducers from decision trees in accordance with an illustrative embodiment of the present invention.
Det~iled l:)escriptiorl 4 ~ , U i '~
Tree-based mo~e!lin~
Figure 1 shows an illustrative decision tree which may be provided as input to a procedure for generating weighted finite-state tr~n.cducers from decision trees, such as the illustrative procedure of Figure 3. All of the phones in the figure are 5 given in "ARPABET," famili~r to those of ordinary skill in the art. The tree of Figure l was trained, in particular, on the "TIMIT" ~l~t~b~e, also famili~r to those skilled in the art, and models the phonetic realization of the Fnglich phoneme laal in various environments. (See, e.g., "A Statistical Model for Generating Plol,unciation Networks," cited above.) When the tree of Figure l is used for 10 predicting the allophonic form of a particular in~t~nl e of laal, one starts at the root of the tree (i.e., node lOl), and asks questions about the environment in which the laal phoneme is found, traversing a path "down" the tree, dependent on the answers to the questions. In particular, each non-leaf node (e.g., nodes lOl, 102, 103, 105 and 107), dominates two ~l~ught~r nodes. As one traverses a path down the tree, 15 the decision on whether to go left or right depends on the answer to the question being asked at the given node. The following table provides an explanation of the symbols used to label the arcs of the decision tree of Figure 1:
cpn place of articulation of consonant n segments to the right cp-n place of articulation of consonant n segments to the left values: alveolar; bilabial; labiodental; dental; palatal; velar; pharyngeal;
n/a if is a vowel, or t_ere is no such segment.
vpn place of articulation of vowel n segments to the right vp-n place of articulation of vowel n segments to the left values: central-mid-high; back-low; back-mid-low; back-high; front-low;
front-mid-low; front-mid-high; front-high; central-mid-low; back-mid-high;
n/a if is a consonant, or there is no such segment.
Iseg number of preceding segments including the segment of interest within the word 22~1 1&9 rseg number of following segments including the segment of interest within the word values: 1, 2, 3, many str stress assigned to this vowel values: primary, secondary, no (zero) stress n/a if there is no stress mark By way of illustration, consider, for example, that the phoneme laal exists in some particular envi~on,llelll. The first question that is asked (at the root node -- node 101) concerns the number of segments, including the laal phoneme itself,10 that occur to the left of the laal in the word in which laal occurs. In this case, if the laal is initial -- i.e., Iseg is 1, one goes left to node 102; if there is one or more segments to the left in the word, one goes right to node 103. Let us assumefor illustrative purposes that this laal is initial in the word, in which case we go left to node 102. The next question (asked at node 102) concerns the consonantal 15 "place" of articulation of the segment to the right of laal -- if it is alveolar go left to node 104; otherwise, if it is of some other quality, or if the segment to the right of laal is not a consonant, then go right to node 105. Let us now assume that the segment to the right is, for example, Izl, which is alveolar. Thus, we go left to terminal (i.e., leaf) node 104. The in~ ation of "119i308" at node 104 indicates20 that in the training data, 119 out of 308 occurrences of the laal phoneme in this particular environment were realized as the phone [ao]. In other words, we can estimate the probability of laal being realized as [ao] in this environment as 0.385, or, equivalently, we can assign a "weight" (equal to the negative of the logarithm of the probability) of 0.95 thereto. The full set of realizations at node 104 with 25 estirnated non-zero probabilities is, in fact, as follows:
phone probability weight = - log prob.
ao 0.385 0.95 aa 0.289 1.24 q+aa 0.103 2.27 q+ao 0.096 2.34 ah 0.069 2.68 ax 0.058 2.84 Note that a decision tree in general is a complete description, in the sense 10 that for any new data point, there will be some leaf node that corresponds to it.
Thus, for the tree in Figure 1, each new instance of the phoneme laal will be handled by (exactly) one leaf node in the tree, depending upon the ellvilo~ ent in which the laal is found. Also note that each decision tree considered herein has the plopelLy that its predictions specify how to rewrite a symbol (in context) in an input 15 string. In particular, they specify a two-level mapping from a set of input symbols (phonemes) to a set of output symbols (allophones).
Rewrite rule c-mril~ion It is well known to those skilled in the art (and, in particular, finite-state phonology), that systems of rewrite rules of the famili~r form ~ p, where 20 ~ and p are regular expressions, can be represented computationally as finite-state tr~ncducers (FSTs), and that these rewrite rules can be automatically converted to such corresponding FSTs. (Note that ~represents the input to the rewrite rule, ~ represents the output of the rewrite rule, and ~ and p represent the left and right contexts, respectively.) See, e.g., "Regular Models of Phonological Rule 25 Systems," cited above, which presents one particular procedure (hereinafter the Kaplan and Kay, or "KK" procedure) for compiling systems of such rewrite rules into FSTs. In accordance with an illustrative embodiment of the present invention, such a procedure can be advantageously extended to include the compilation of probabilistic or weighted rules into weight~.l finite-state-tr~n.cducers (WFSTs). As 2201 i89 is well known to those skilled in the art, weighted finite-state transducers are defined as transducers such that, in addition to having standard input and output labels, each transition is further labeled with a weight. (See, e. g., "Weighted Rational Transductions and their Application to Human Language Processing," cited above.)Instead of the "KK" procedure, famili~r to those skilled in the art, an improved procedure for compiling rewrite rules into finite-state transducers may be advantageously employed in accordance with one illustrative embodiment of the present invention. Specifically, in contrast to the "KK" algorithm which introduces context labelled brackets widely, only to restrict their occurrence subsequently (see "Regular Models of Phonological Rule Systems," cited above), the improved procedure introduces context symbols just when and where they are needed.
Furthermore, the number of intermediate transducers necessary in the construction of the rules is smaller than in the "KK" procedure, and each of the tran~ducers can be constructed more directly and efficiently from the prirnitive expressions of the 15 rule, ~ and p.
For example, in accordance with the illustrative improved procedure for compiling rewrite rules into finite-state transducers, a transducer which corresponds to the left-to-right obligatory rule ~ p can be obtained by the composition of five transducers:
rOfOreplaceOl~0l2 Specifically, these five tr~ncducers operate as follows:
l. The transducer r introduces in a string a marker " > " before every instance of p. This transformation may be written as follows: ~p ~ ~ > p.
COMPILATION OF WEIGHTED FINITE-STATE TRANSDUCERS
FROM DECISION TREES
Field of the Inv~ntinn The present invention relates generally to the field of linguistic modeling as used, for example, in speech recognition and/or speech synthesis systems, and more particularly to finite-state methods for implementing such linguistic models in such systems.
R~ ~ollnd of the Inv~ntioll Much attention has been devoted recently to methods for inferring linguistic models from linguistic data. One powerful inference method that has been used invarious applications are decision trees, and in particular classification and regression trees ("CART"), well known to those of ordinary skill in the art. As is also well known to those skilled in the art, these decision trees can be "grown" from linguistic data by the application of conventional methods, such as are described, for example, in Michael D. Riley, "Some Applications of Tree-based Modelling to Speech and Language," Proceedings of the Speech and Natural Language Workshop, pages 339-352, Cape Cod, Massachusetts, October 1989. Other applications of tree-based modeling to specific problems in speech and natural language processing are 20 tli~cll~sed, for example, in Michael D. Riley, "A Statistical Model for Generating Pronunciation Networks, " Procee-ling~ of the International Confe~ ce on Acoustics, Speech and Signal Processing, pages Sll.l-S11.4, October, 1991, and in David M.
Magerman, "St~ti.ctical Decision-Tree Models for Parsing," 33rd Annual Meeting of the Association for Computational Linguistics, pages 276-283, Morristown, New25 Jersey, April, 1995. Each of these refelel1ces is incorporated by reference as if fully set forth herein.
An increasing amount of attention has also been focused on finite-state methods for implementing linguistic models. These methods include, in particular, the use of finite-state tr~n~ducers and weighted finite-state tr~n~d~1cers. See, e.g., 30 Ronald M. Kaplan and Martin Kay, "Regular Models of Phonological Rule ~2U1 1 89 Systems," Computational Linguistics, 20:331-378, September, 1994, and Fernando Pereira, Michael Riley and Richard Sproat, "Weighted Rational Tr~n~ ctions and their Application to Human Language Processing," ARPA Workshop on Human Language Technology, pages 249-254, March, 1994. Each of these references is 5 also incorporated by reference as if fully set forth herein.
One reason for the interest in the use of finite-state mechanisms is that finite-state machines provide a mathematically well-understood computational framework for representing a wide variety of information, both in natural language processing and in speech processing. Lexicons, phonological rules, Hidden Markov10 Models, and regular grammars, each of which is well known to those of ordinary skill in the art, are all representable as finite-state machines. Moreover, the availability of finite-state operations such as union, intersection and composition means that information from these various sources can be combined in useful and computationally attractive ways. The above-cited papers (among others) provide 15 further justification for the use of finite-state methods for implementing linguistic models.
Despite the recent attention given to both the use of decision trees for inferring linguistic models from linguistic data, and the use of finite-state methods for implementing linguistic models, there have not heretofore been any viable 20 techniques whereby these two strands of research have been combined in such amanner as to leverage the advantages of each. In particular, whereas generating decision trees from linguistic data has proven quite successful, and whereas finite-state based system implementations have been successfully developed, no techniques for automatically deriving finite-state models from decision trees have been 25 developed, and only partial success has been achieved in ~lle~ to directly infer tr~n.~ucers from linguistic data. (See, e.g., Daniel Gildea and Daniel Jurafsky,"Automatic Induction of Finite State Tr~n~llcers for Simple Phonological Rules,"33rd Annual Meeting of the Association for Computational Linguistics, pages 9-15, Morristown, New Jersey, June, 1995, also incorporated by lefele~ce as if fully set 30 forth herein.) Thus, if these two techniques could be "married" in the form of an automated procedure for converting the information in decision trees into weighted - - ~2~ 1 1 8~
finite-state tran.C~lucers~ information inferred from linguistic data and represented in a decision tree could be used directly in a system that represents other information, such as lexicons or grammars, in the form of finite-state machines.
Snmm~ry of the Inv~ntioll The use of decision trees for linguistic modelling and the use of finite-state transducers are advantageously combined by a method in accordance with an illustrative embodiment of the present invention, whereby a decision tree is automatically converted into one or more weighted finite-state transducers.
Specifically, the illustrative method processes one or more terminal (i. e., leaf) nodes of a given decision tree to generate one or more corresponding weighted rewrite rules. Then, these weighted rewrite rules are processed to generate weighted finite-state tr~n~clucers corresponding to the one or more terminal nodes of the decision tree. In this marmer, decision trees may be advantageously compiled into weighted finite-state tr~n~ducers, and these transducers may then be used directly in various 15 speech and natural language processing systems. The weighted rewrite rules employed herein comprise an extension of conventional rewrite rules, f~mili~r tothose skilled in the art.
Rrief Des-~r~ptiol- of the T)r~wi~
Figure 1 shows an illustrative decision tree which may be provided as input 20 to a procedure for generating weighted finite-state transducers from decision trees, such as the illustrative procedure of Figure 3.
Figure 2 shows an illustrative weighted finite-state transducer generated, for example, by the illustrative procedure of Figure 3.
Figure 3 shows a flow diagram of an illustrative procedure for generating 25 weighted finite-state tr~n.cducers from decision trees in accordance with an illustrative embodiment of the present invention.
Det~iled l:)escriptiorl 4 ~ , U i '~
Tree-based mo~e!lin~
Figure 1 shows an illustrative decision tree which may be provided as input to a procedure for generating weighted finite-state tr~n.cducers from decision trees, such as the illustrative procedure of Figure 3. All of the phones in the figure are 5 given in "ARPABET," famili~r to those of ordinary skill in the art. The tree of Figure l was trained, in particular, on the "TIMIT" ~l~t~b~e, also famili~r to those skilled in the art, and models the phonetic realization of the Fnglich phoneme laal in various environments. (See, e.g., "A Statistical Model for Generating Plol,unciation Networks," cited above.) When the tree of Figure l is used for 10 predicting the allophonic form of a particular in~t~nl e of laal, one starts at the root of the tree (i.e., node lOl), and asks questions about the environment in which the laal phoneme is found, traversing a path "down" the tree, dependent on the answers to the questions. In particular, each non-leaf node (e.g., nodes lOl, 102, 103, 105 and 107), dominates two ~l~ught~r nodes. As one traverses a path down the tree, 15 the decision on whether to go left or right depends on the answer to the question being asked at the given node. The following table provides an explanation of the symbols used to label the arcs of the decision tree of Figure 1:
cpn place of articulation of consonant n segments to the right cp-n place of articulation of consonant n segments to the left values: alveolar; bilabial; labiodental; dental; palatal; velar; pharyngeal;
n/a if is a vowel, or t_ere is no such segment.
vpn place of articulation of vowel n segments to the right vp-n place of articulation of vowel n segments to the left values: central-mid-high; back-low; back-mid-low; back-high; front-low;
front-mid-low; front-mid-high; front-high; central-mid-low; back-mid-high;
n/a if is a consonant, or there is no such segment.
Iseg number of preceding segments including the segment of interest within the word 22~1 1&9 rseg number of following segments including the segment of interest within the word values: 1, 2, 3, many str stress assigned to this vowel values: primary, secondary, no (zero) stress n/a if there is no stress mark By way of illustration, consider, for example, that the phoneme laal exists in some particular envi~on,llelll. The first question that is asked (at the root node -- node 101) concerns the number of segments, including the laal phoneme itself,10 that occur to the left of the laal in the word in which laal occurs. In this case, if the laal is initial -- i.e., Iseg is 1, one goes left to node 102; if there is one or more segments to the left in the word, one goes right to node 103. Let us assumefor illustrative purposes that this laal is initial in the word, in which case we go left to node 102. The next question (asked at node 102) concerns the consonantal 15 "place" of articulation of the segment to the right of laal -- if it is alveolar go left to node 104; otherwise, if it is of some other quality, or if the segment to the right of laal is not a consonant, then go right to node 105. Let us now assume that the segment to the right is, for example, Izl, which is alveolar. Thus, we go left to terminal (i.e., leaf) node 104. The in~ ation of "119i308" at node 104 indicates20 that in the training data, 119 out of 308 occurrences of the laal phoneme in this particular environment were realized as the phone [ao]. In other words, we can estimate the probability of laal being realized as [ao] in this environment as 0.385, or, equivalently, we can assign a "weight" (equal to the negative of the logarithm of the probability) of 0.95 thereto. The full set of realizations at node 104 with 25 estirnated non-zero probabilities is, in fact, as follows:
phone probability weight = - log prob.
ao 0.385 0.95 aa 0.289 1.24 q+aa 0.103 2.27 q+ao 0.096 2.34 ah 0.069 2.68 ax 0.058 2.84 Note that a decision tree in general is a complete description, in the sense 10 that for any new data point, there will be some leaf node that corresponds to it.
Thus, for the tree in Figure 1, each new instance of the phoneme laal will be handled by (exactly) one leaf node in the tree, depending upon the ellvilo~ ent in which the laal is found. Also note that each decision tree considered herein has the plopelLy that its predictions specify how to rewrite a symbol (in context) in an input 15 string. In particular, they specify a two-level mapping from a set of input symbols (phonemes) to a set of output symbols (allophones).
Rewrite rule c-mril~ion It is well known to those skilled in the art (and, in particular, finite-state phonology), that systems of rewrite rules of the famili~r form ~ p, where 20 ~ and p are regular expressions, can be represented computationally as finite-state tr~ncducers (FSTs), and that these rewrite rules can be automatically converted to such corresponding FSTs. (Note that ~represents the input to the rewrite rule, ~ represents the output of the rewrite rule, and ~ and p represent the left and right contexts, respectively.) See, e.g., "Regular Models of Phonological Rule 25 Systems," cited above, which presents one particular procedure (hereinafter the Kaplan and Kay, or "KK" procedure) for compiling systems of such rewrite rules into FSTs. In accordance with an illustrative embodiment of the present invention, such a procedure can be advantageously extended to include the compilation of probabilistic or weighted rules into weight~.l finite-state-tr~n.cducers (WFSTs). As 2201 i89 is well known to those skilled in the art, weighted finite-state transducers are defined as transducers such that, in addition to having standard input and output labels, each transition is further labeled with a weight. (See, e. g., "Weighted Rational Transductions and their Application to Human Language Processing," cited above.)Instead of the "KK" procedure, famili~r to those skilled in the art, an improved procedure for compiling rewrite rules into finite-state transducers may be advantageously employed in accordance with one illustrative embodiment of the present invention. Specifically, in contrast to the "KK" algorithm which introduces context labelled brackets widely, only to restrict their occurrence subsequently (see "Regular Models of Phonological Rule Systems," cited above), the improved procedure introduces context symbols just when and where they are needed.
Furthermore, the number of intermediate transducers necessary in the construction of the rules is smaller than in the "KK" procedure, and each of the tran~ducers can be constructed more directly and efficiently from the prirnitive expressions of the 15 rule, ~ and p.
For example, in accordance with the illustrative improved procedure for compiling rewrite rules into finite-state transducers, a transducer which corresponds to the left-to-right obligatory rule ~ p can be obtained by the composition of five transducers:
rOfOreplaceOl~0l2 Specifically, these five tr~ncducers operate as follows:
l. The transducer r introduces in a string a marker " > " before every instance of p. This transformation may be written as follows: ~p ~ ~ > p.
2. The tr~n~lucerf introduces markers " < ~ " and " < 2" before each instance 25 of ~thatisfollowedbya">"marker: (~U{>}) ~> ~(~){>}) {<l~< 2}
> . In other words, this tr~n~ducer marks just those ~ that occur before p.
> . In other words, this tr~n~ducer marks just those ~ that occur before p.
3 . The replacement tr~n~ducer replace replaces ~ with ~ in the context <, >, simult~nPously deleting the " > " marker in all positions. Since " > ", " C ,", and " <2" need to be ignored when determining an occurrence of ~, there are loops 220~ 1~J
over the transitions > :E, < ,:~, <2:~ at all states of ~, or equivalently of the states of the cross product tr~nc~ cer ~ x ~.
over the transitions > :E, < ,:~, <2:~ at all states of ~, or equivalently of the states of the cross product tr~nc~ cer ~ x ~.
4. The tr~n.cdllcer ll admits only those strings in which occurrences of a " <, " marker are preceded by ~ and deletes the " <, " at such occurrences: ~ ~ <, 5 ~ ~
5. The transducer 12 admits only those strings in which occurrences of a " <2" marker are n~ preceded by ~ and deletes the " < 2" at such occurrences: ~
~ <2 ~ ~
The above procedure is described in further detail in the Appendix attached 10 hereto.
Illu~lla~ive extended rewrite rule compilation procedure In accordance with an illustrative embodiment of the present invention, a rewrite rule to finite-state tr~ncducer compilation procedure which has been extended to handle weighted rules is employed. With such an extended procedure, we can 15 allow ~ in the rule ~ p to represent a wei~ht~l regular expression. That is, the tr~nc~llcer corresponding to such a weighted rewrite rule (e.g., the transducer generated by the illustrative extended rule compilation procedure described herein) will replace ~ with ~ h~vir~ th~ a~ roy~i~t~ wei~htc in the context ~_p.
Specifically, rewrite rules can be advantageously generalized by letting ~be 2 0 a rational power series, f~mili~r to those skilled in the art (and, in particular, formal language theory). The result of the application of a generalized rule to a string is ~en a set of weighted strings which can be represented by a weighted automaton.
The rational power series considered herein are functions which map ~ to ~+ ~J
{~, which can be described by regular expressions over the alphabet (~+ ~J {o~) 25x ~ For example, S = (4a)(2b) (3b) is a rational power series. It defines a function in the following way -- it associates a non-null number only with the strings recognized by the regular expression ab-b. This number may be obtained by adding the coefficients involved in the recognition of the string. For example, the value which may be associated with abbb is (S,abbb) = 4 + 2 + 2 + 3 = 11.
3 0In general, such extended regular expressions can be re~--n~l~nt . Some strings 2201 18~
g can be matched in different ways with distinct coefficients . The value associated with those strings may then be the mi~ of all possible results. S' = (2a)(3b)(4b) + (Sa)(3b$) matches abb, for example, with the different weights 2 + 3 + 4 = 9 and 5 + 3 + 3 = 11. The miniml-m of the two is the value advantageously 5 associated with abb -- that is, (S',abb) = 9. Non-negative numbers in the definition of these power series are often interpreted as the negative logarithm of probabilities. This explains the above choice of the operations -- addition of the weights along the string recognition, and minimi7~tion, since we are interested in the result which has the highest probability. Note that based on the terminology of 10 the theory of languages, f~mili~r to those skilled in the art, the functions we consider here are power series defined on the tropical semiring (~+ ~) {a~, min, +, oo, 0).
By way of example, consider the following rule, which states that an abstract nasal, denoted N, is obligatorily rewritten as m in the context of a following labial:
N ~ m I [+labia~.
15 Now suppose that this is only probabili.ctir~lly true, and that while ninety percent of the time N does indeed become m in this environment, about ten percent of the time in real speech it becomes n instead. Converting from probabilities to weights, one would say that N becomes m with weight a = -log(0.9), and n with weight ,B =
-log(0.1), in the given environment. One could represent this by the following 20 weighted obligatory rule:
N ~ am + ~ l [ + labia~J .
The result of the application of a weighted transducer to a string, or more generally to an automaton, is a weighted automaton. The corresponding operation is similar to that in the unweighted case. However, the weight of the transducer and 25 those of the string or automaton need to be combined too, here added, during composition. The composition operation has been generalized to the weighted caseby introducing this combination of weights. The procedure described above can then 2~0 1 ~ t~
be advantageously used to compile weighted rewrite rules into weighted finite-state tr~n~ ers.
Figure 2 shows an illustrative weighted finite-state tr~n~ cer which may, for example, have been generated by the illustrative extended rewrite rule procedure5 described above (and which may, therefore, have been generated by a decision tree compilation procedure in accordance with the illustrative embodiment of the present invention described below). Figure 2 shows, in particular, a weighted transducerrepresenting the weighted obligatory rule shown above (i.e., N ~ arn + ~n /_[+labiall). As used in the figure, the symbol "~" denotes all letters different 10 from b, m, n, p and N.
The above-described extenslon of rewrite rule to finite-state transducer compilation procedures so as to enable the h~n(lling of weighted rewrite rules is described in further detail in the Appendix attached hereto.
An illu~ live decision tree to WFST compil~tion l~locellule Two assumptions on the nature of a given decision tree advantageously enables efficient compilation into a WFST in accordance with an illustrative embodiment of the present invention. First, we advantageously assume that the predictions at the leaf nodes specify how to rewrite a particular symbol in an input string. And, second, we advantageously presume that the decisions at each node are 20 stateable as regular expressions over the input string. In accordance with the illustrative tree compilation procedure, each leaf node will be considered to represent a single weighted rewrite rule.
Specifically, the regular expressions for each branch describe one aspect of the left context ~, the right context p, or both. The left and right contexts for the 25 rule consist of the intersections of the partial descriptions of these contexts defined for each branch traversed between the root and the given leaf node. The input ~ is predefined for the entire tree, whereas the output ~y is defined as the union of the set of outputs, along with their weights, that are associated with the given leaf node.
The weighted rule belonging to the leaf node can then be compiled into a transducer 30 using the illustrative weighted rule compilation procedure described above. Then, 220~ ~8~
the transducer for the entire tree can be derived by the intersection of the entire set of tr~n~ cers generated from the individual leaf nodes. Note that while regular relations are not generally closed under intersection, the subset of same-length (or more strictly speaking length-preserving) relations is, in fact, closed.
By way of an illustrative example, refer to the sample decision tree shown in Figure 1. Since this tree models the phonetic realization of laal, we can immediately set ~ to be aa for the whole tree. Next, consider again the traversal of the tree from root node 101 to leaf node 104, as described above. The first decision concerns the number of segments to the left of the laal in the word, either none for the left branch, or one or more for the right branch. Assuming that a symbol, a, represents a single segment, a symbol, #, represents a word boundary,and allowing for the possibility of intervening optional stress marks (') which do not count as segments, these two possibilities can be represented by the following regular expressions for ~
For the left branch, ~ = #Opt('), and for the right branch, (#Opt( ' ) aOpt( ' )) ~) (#Opt( ' ) aOpt( ' ) aOpt( ' )) ~J
(#opt(~)aopt(~)aopt(~)(aopt(~))+) = (#Opt(')a)+opt(l).
(Note that the notation used herein is essentially that of Kaplan and Kay, f~mili~r to those skilled in the art, and defined in "Regular Models of Phonological Rule Systems," cited above. Note also that, as per convention, superscript " + " denotes one or more instances of an expression.) At this node there is no decision based on the righth~n-l context, so the righ~h~n~ context is free. We can represent this by setting p at this node to be ~, 25 where ~(conventionally) lepresenl~ the entire alphabet. (Note that the alphabet is defined to be an alphabet of all correspondence pairs that were determined empirically to be possible.) Next, the decision at the left ~ ghter of the root node (i.e., node 102) 22() I I ~
concerns whether or not the segment to the right is an alveolar. Assuming we have defined classes of segments alv, blab, and so forth (represented as unions of segments) we can represent the regular expression for p as follows:
For the left branch, p = Opt(')(alv), and for the right branch, 5 p = (Opt(')(blab)) U(Opt(')(labd)) U(Opt(')(den)) U(Opt(')~al)) U(Opt(')(vel)) U (Opt(')~Dha)) U (Opt(')(n/a)) In this case, ~ is unrestricted, so we can set ~ at this node to be ~.
We can then derive the ~ and p expressions for the rule at leaf node 104 by intersecting together the expressions for these contexts deflned for each branch10 traversed on the way to the leaf. In particular, for leaf node 104, ~ = #Opt(') = #Opt('), and p = ~ r~ Opt(')(alv) = Opt(')(alv). (Strictly speaking, since the ~s and ps at each branch may define expressions of different lengths, it is advantageous to left-pad each ~with ~, and to right-pad each pwith ~.) The rule input ~ has already been given as aa. The output ~ is defined as the union of all 15 of the possible expressions (at the leaf node in question) that aa could become, with their associated weights (e. g., negative log probabilities), which are represented here as subscripted floating-point numbers:
~J = aoO 95 U aal 24 U q+aa2 27 U q+ao2 34 ~) ah2.68 U aX2.84 Thus, the entire weighted rule for leaf node 104 may be written as:
aa ~ (aoO.95 ~J aal.24 U q+aa227 U q+ao234 U ah268 U aX284) /
#Opt(')_Opt(')(alv) And by a similar construction, the rule for node 106, for example, may be written as:
2~01 l&q aa ~ (aa040 U aol.~l) / (#(opt(~)a)+opt(~ ((cmh) U (bl) U (bml) U
(bh)))_~
In this manner, each node may be advantageously used to generate a rule which states that a mapping occurs between the input symbol ~ and the weighted 5 expression ~ in the condition described by ~_p. In cases where ~ finds itself in a context that is nQ~ subsumed by ~_p, the rule behaves exactly as a two-level surface coercion rule, familiar to those of ordinary skill in the art -- it freely allows ~ to correspond to any ~ as specified by the alphabet of pairs. These ~:~correspondences may, however, be constrained by other rules derived from the tree, 10 as described below.
The ul~elplelation of the full tree is that it represents the conjunction of allsuch mappings. That is, for rules 1, 2 . . . n, ~ corresponds to Yll given condition ~_Pl ~ corresponds to ~12 given condition ~P2 . . . ~ corresponds to ~JJn given condition ~Pn. But this conjunction is simply the intersection of the15 entire set of transducers defined for the leaves of the tree. Observe now that the ~:~ correspondences that were left free by the rule of one leaf node, may be constrained by intersection with the other leaf nodes. Since, as noted above, the tree is a complete description, it follows that for any leaf node i, and for any context ~_p nQ~ subsumed by ~p" there is some leaf node j such that ~_p subsumes 20 ~_p. Thus, the tr~n~ cers compiled for the rules at nodes 104 and 106, for example, are intersected together, along with the rules for all the other leaf nodes.
As noted above, and as known to those skilled in the art, regular relations -- the algebraic countel~art of FSTs -- are not in general closed under intersection. However, the subset of s~me-le~th regular relations i~ closed under 25 intersection, since they can be thought of as finite-state acceptors expressed over pairs of symbols. Note, therefore, that one can define intersection for transducers analogously with intersection for acceptors -- given two machines G, and G2, with transition functions ~, and ~2, one can define the transition function of G, ~, as follows: for an input-output pair (i,o), ~((ql,q2),(i,o)) = (q~,q2) if and only if 30 ~(ql,(i,o)) = q l and ~2(q2,(i,o)) = q2. This point can be extended somewhat to - 14 - 220 1 i 8q include relations that involve bounded deletions or insertions. This is precisely the ~ el~re~tion n~cess~ry for systems of two-level rules, where a single transducerexpressing the entire system may be constructed via intersection of the tr~n.cdllcers expressing the individual rules. Indeed, the decision tree represents neither more 5 nor less than a set of weighted two-level rules. Each of the symbols in the expressions for ~ and p actually represent (sets of) pairs of symbols. Thus, alv, for example, represents all lexical alveolars paired with all of their possible surface realizations. And just as each tree represents a system of weighted two-level rules, so a set of trees -- for example, a set in which each tree deals with the realization 10 of a particular phone -- represents a system of weighted two-level rules, where each two-level rule is compiled from each of the individual trees.
The above description of the illustrative tree compilation procedure may be summarized in more formal terms as follows. Define a function Compile, which, when given a weighted rewrite rule as input, returns a WFST coll.~ulmg that rule15 as its output. This function may, for example, comprise the illustrative extended rewrite rule compilation procedure as described above (and in the Appendix attached hereto). Alternatively, this function may comprise the "KK" procedure, well known to those skilled in the art, also extended as described above (and in the Appendix attached hereto) so as to generate a WFST from a weighted rewrite rule. In any 20 event, the WFST for a single leaf L is thus defined as follows, where ~ is the input symbol for the entire tree, ~IJL is the output expression defined at L, PL represents the path traversed from the root node to L, p is an individual branch on that path, and ~ and pp are the expressions for ~ and p defined at p:
RuleL = Compile (d;~T~ L/ n 1p n pp) The transducer for an entire tree T is defined as:
RllleT = n RuleL
L~T
25 And, finally, the transducer for a forest F of trees is:
Figure 3 shows a flow diagram of the above-described illustrative procedure -15- 22Uli8~
~leF = n RllleT
r~F
for generating one or more weighted finite-state tran~-lucers from one or more decision trees in accordance with an illustrative embodiment of the present invention.
The input provided to the procedure comprises:
1. a forest F of one or more decision trees T;
2. a regular-expression characterization defining a property of either the left context, LC or the right context, RC, or both, of each decision at each node N of each tree T in F;
3. a notation of the input symbol S being tr~n~dllced by each tree T in F;
and 4. the alphabet of all possible input/output symbol pairs allowed by all trees T in F.
The output produced by the procedure comprises one or more transducers representing the forest F of trees.
Specifically, for each tree T in the forest F (selected in step 301), and for each leaf node L in the given tree T (selected in step 302), a weighted finite-state transducer is generated based on the leaf node L (steps 303-306). In particular, the path P from the root node of tree T to the selected leaf node L is determined in step 303. Then, in step 304, for each node N on path P, the regular expression at node N is intersected with the previous nodes on the path to build either the left context (LC) or the right context (RC) as defined for the given node (N). When the leaf node (L) is reached, the illustrative procedure of Figure 3 generates the weighted disjunction D of output symbols defned for the input symbol S for tree T (step 305).
And finally, step 306 compiles the weighted rewrite rule S ~D / LC_RC to generate a tran~ducer corresponding to leaf node L. The operation of step 306 may, 25 for example, be implemented with use of one of the illustrative weighted rewrite rule compilation procedures described above. (For example, either the "KK" procedure,extended to handle the compilation of weighted rewrite rules, or the extended improved procedure, as described above and in the Appendix attached hereto, may 2201 18'~
illustratively be used to implement step 306 of the illustrative procedure of Figure 3.) After the transducer corresponding to a given leaf node has been generated, the illustrative procedure of Figure 3 determines in decision box 307 whether a 5 transducer has been generated for every leaf node in tree T. If not (i. e., if there are more leaf nodes in tree T to process), flow returns to step 302 to select another leaf node from which a tr~n.cducer will be generated. When transducers have been generated for all leaf nodes in tree T, step 308 intersects together all of the transducers so generated, resulting in a single transducer for tree T as a whole. In 10 other illustrative embodiments of the present invention, step 308 may be avoided, thereby resulting in a plurality of trancdllcers (i. e., one for each leaf node) for tree T, rather than a single combined automaton.
After the generation of the tr~ncdllcer(s) for the given tree is complete, decision box 309 determines whether transducers have been generated for every tree 15 in forest F. If not (i.e., if there are more trees in forest F to process), flow returns to step 301 to select another tree from which one or more trancducers will be generated. When tr~ncdllcers have been generated for all trees in forest F, step 310 intersects together all of the transducers so generated, resulting in a single transducer for forest F as a whole. Note that in other illustrative embodiments of the present 20 invention, step 310 also may be avoided, thereby reslllting in a plurality oftransducers (i.e., one or more for each tree) for forest F, rather than a singlecombined automaton.
Although a number of specific embodiments of this invention have been shown and described herein, it is to be understood that these embodiments are 25 merely illustrative of the many possible specific arrangements which can be devised in application of the principles of the invention. Numerous and varied other arrangements can be devised in accordance with these principles by those of ordinary skill in the art without departing from the spirit and scope of the invention.
5. The transducer 12 admits only those strings in which occurrences of a " <2" marker are n~ preceded by ~ and deletes the " < 2" at such occurrences: ~
~ <2 ~ ~
The above procedure is described in further detail in the Appendix attached 10 hereto.
Illu~lla~ive extended rewrite rule compilation procedure In accordance with an illustrative embodiment of the present invention, a rewrite rule to finite-state tr~ncducer compilation procedure which has been extended to handle weighted rules is employed. With such an extended procedure, we can 15 allow ~ in the rule ~ p to represent a wei~ht~l regular expression. That is, the tr~nc~llcer corresponding to such a weighted rewrite rule (e.g., the transducer generated by the illustrative extended rule compilation procedure described herein) will replace ~ with ~ h~vir~ th~ a~ roy~i~t~ wei~htc in the context ~_p.
Specifically, rewrite rules can be advantageously generalized by letting ~be 2 0 a rational power series, f~mili~r to those skilled in the art (and, in particular, formal language theory). The result of the application of a generalized rule to a string is ~en a set of weighted strings which can be represented by a weighted automaton.
The rational power series considered herein are functions which map ~ to ~+ ~J
{~, which can be described by regular expressions over the alphabet (~+ ~J {o~) 25x ~ For example, S = (4a)(2b) (3b) is a rational power series. It defines a function in the following way -- it associates a non-null number only with the strings recognized by the regular expression ab-b. This number may be obtained by adding the coefficients involved in the recognition of the string. For example, the value which may be associated with abbb is (S,abbb) = 4 + 2 + 2 + 3 = 11.
3 0In general, such extended regular expressions can be re~--n~l~nt . Some strings 2201 18~
g can be matched in different ways with distinct coefficients . The value associated with those strings may then be the mi~ of all possible results. S' = (2a)(3b)(4b) + (Sa)(3b$) matches abb, for example, with the different weights 2 + 3 + 4 = 9 and 5 + 3 + 3 = 11. The miniml-m of the two is the value advantageously 5 associated with abb -- that is, (S',abb) = 9. Non-negative numbers in the definition of these power series are often interpreted as the negative logarithm of probabilities. This explains the above choice of the operations -- addition of the weights along the string recognition, and minimi7~tion, since we are interested in the result which has the highest probability. Note that based on the terminology of 10 the theory of languages, f~mili~r to those skilled in the art, the functions we consider here are power series defined on the tropical semiring (~+ ~) {a~, min, +, oo, 0).
By way of example, consider the following rule, which states that an abstract nasal, denoted N, is obligatorily rewritten as m in the context of a following labial:
N ~ m I [+labia~.
15 Now suppose that this is only probabili.ctir~lly true, and that while ninety percent of the time N does indeed become m in this environment, about ten percent of the time in real speech it becomes n instead. Converting from probabilities to weights, one would say that N becomes m with weight a = -log(0.9), and n with weight ,B =
-log(0.1), in the given environment. One could represent this by the following 20 weighted obligatory rule:
N ~ am + ~ l [ + labia~J .
The result of the application of a weighted transducer to a string, or more generally to an automaton, is a weighted automaton. The corresponding operation is similar to that in the unweighted case. However, the weight of the transducer and 25 those of the string or automaton need to be combined too, here added, during composition. The composition operation has been generalized to the weighted caseby introducing this combination of weights. The procedure described above can then 2~0 1 ~ t~
be advantageously used to compile weighted rewrite rules into weighted finite-state tr~n~ ers.
Figure 2 shows an illustrative weighted finite-state tr~n~ cer which may, for example, have been generated by the illustrative extended rewrite rule procedure5 described above (and which may, therefore, have been generated by a decision tree compilation procedure in accordance with the illustrative embodiment of the present invention described below). Figure 2 shows, in particular, a weighted transducerrepresenting the weighted obligatory rule shown above (i.e., N ~ arn + ~n /_[+labiall). As used in the figure, the symbol "~" denotes all letters different 10 from b, m, n, p and N.
The above-described extenslon of rewrite rule to finite-state transducer compilation procedures so as to enable the h~n(lling of weighted rewrite rules is described in further detail in the Appendix attached hereto.
An illu~ live decision tree to WFST compil~tion l~locellule Two assumptions on the nature of a given decision tree advantageously enables efficient compilation into a WFST in accordance with an illustrative embodiment of the present invention. First, we advantageously assume that the predictions at the leaf nodes specify how to rewrite a particular symbol in an input string. And, second, we advantageously presume that the decisions at each node are 20 stateable as regular expressions over the input string. In accordance with the illustrative tree compilation procedure, each leaf node will be considered to represent a single weighted rewrite rule.
Specifically, the regular expressions for each branch describe one aspect of the left context ~, the right context p, or both. The left and right contexts for the 25 rule consist of the intersections of the partial descriptions of these contexts defined for each branch traversed between the root and the given leaf node. The input ~ is predefined for the entire tree, whereas the output ~y is defined as the union of the set of outputs, along with their weights, that are associated with the given leaf node.
The weighted rule belonging to the leaf node can then be compiled into a transducer 30 using the illustrative weighted rule compilation procedure described above. Then, 220~ ~8~
the transducer for the entire tree can be derived by the intersection of the entire set of tr~n~ cers generated from the individual leaf nodes. Note that while regular relations are not generally closed under intersection, the subset of same-length (or more strictly speaking length-preserving) relations is, in fact, closed.
By way of an illustrative example, refer to the sample decision tree shown in Figure 1. Since this tree models the phonetic realization of laal, we can immediately set ~ to be aa for the whole tree. Next, consider again the traversal of the tree from root node 101 to leaf node 104, as described above. The first decision concerns the number of segments to the left of the laal in the word, either none for the left branch, or one or more for the right branch. Assuming that a symbol, a, represents a single segment, a symbol, #, represents a word boundary,and allowing for the possibility of intervening optional stress marks (') which do not count as segments, these two possibilities can be represented by the following regular expressions for ~
For the left branch, ~ = #Opt('), and for the right branch, (#Opt( ' ) aOpt( ' )) ~) (#Opt( ' ) aOpt( ' ) aOpt( ' )) ~J
(#opt(~)aopt(~)aopt(~)(aopt(~))+) = (#Opt(')a)+opt(l).
(Note that the notation used herein is essentially that of Kaplan and Kay, f~mili~r to those skilled in the art, and defined in "Regular Models of Phonological Rule Systems," cited above. Note also that, as per convention, superscript " + " denotes one or more instances of an expression.) At this node there is no decision based on the righth~n-l context, so the righ~h~n~ context is free. We can represent this by setting p at this node to be ~, 25 where ~(conventionally) lepresenl~ the entire alphabet. (Note that the alphabet is defined to be an alphabet of all correspondence pairs that were determined empirically to be possible.) Next, the decision at the left ~ ghter of the root node (i.e., node 102) 22() I I ~
concerns whether or not the segment to the right is an alveolar. Assuming we have defined classes of segments alv, blab, and so forth (represented as unions of segments) we can represent the regular expression for p as follows:
For the left branch, p = Opt(')(alv), and for the right branch, 5 p = (Opt(')(blab)) U(Opt(')(labd)) U(Opt(')(den)) U(Opt(')~al)) U(Opt(')(vel)) U (Opt(')~Dha)) U (Opt(')(n/a)) In this case, ~ is unrestricted, so we can set ~ at this node to be ~.
We can then derive the ~ and p expressions for the rule at leaf node 104 by intersecting together the expressions for these contexts deflned for each branch10 traversed on the way to the leaf. In particular, for leaf node 104, ~ = #Opt(') = #Opt('), and p = ~ r~ Opt(')(alv) = Opt(')(alv). (Strictly speaking, since the ~s and ps at each branch may define expressions of different lengths, it is advantageous to left-pad each ~with ~, and to right-pad each pwith ~.) The rule input ~ has already been given as aa. The output ~ is defined as the union of all 15 of the possible expressions (at the leaf node in question) that aa could become, with their associated weights (e. g., negative log probabilities), which are represented here as subscripted floating-point numbers:
~J = aoO 95 U aal 24 U q+aa2 27 U q+ao2 34 ~) ah2.68 U aX2.84 Thus, the entire weighted rule for leaf node 104 may be written as:
aa ~ (aoO.95 ~J aal.24 U q+aa227 U q+ao234 U ah268 U aX284) /
#Opt(')_Opt(')(alv) And by a similar construction, the rule for node 106, for example, may be written as:
2~01 l&q aa ~ (aa040 U aol.~l) / (#(opt(~)a)+opt(~ ((cmh) U (bl) U (bml) U
(bh)))_~
In this manner, each node may be advantageously used to generate a rule which states that a mapping occurs between the input symbol ~ and the weighted 5 expression ~ in the condition described by ~_p. In cases where ~ finds itself in a context that is nQ~ subsumed by ~_p, the rule behaves exactly as a two-level surface coercion rule, familiar to those of ordinary skill in the art -- it freely allows ~ to correspond to any ~ as specified by the alphabet of pairs. These ~:~correspondences may, however, be constrained by other rules derived from the tree, 10 as described below.
The ul~elplelation of the full tree is that it represents the conjunction of allsuch mappings. That is, for rules 1, 2 . . . n, ~ corresponds to Yll given condition ~_Pl ~ corresponds to ~12 given condition ~P2 . . . ~ corresponds to ~JJn given condition ~Pn. But this conjunction is simply the intersection of the15 entire set of transducers defined for the leaves of the tree. Observe now that the ~:~ correspondences that were left free by the rule of one leaf node, may be constrained by intersection with the other leaf nodes. Since, as noted above, the tree is a complete description, it follows that for any leaf node i, and for any context ~_p nQ~ subsumed by ~p" there is some leaf node j such that ~_p subsumes 20 ~_p. Thus, the tr~n~ cers compiled for the rules at nodes 104 and 106, for example, are intersected together, along with the rules for all the other leaf nodes.
As noted above, and as known to those skilled in the art, regular relations -- the algebraic countel~art of FSTs -- are not in general closed under intersection. However, the subset of s~me-le~th regular relations i~ closed under 25 intersection, since they can be thought of as finite-state acceptors expressed over pairs of symbols. Note, therefore, that one can define intersection for transducers analogously with intersection for acceptors -- given two machines G, and G2, with transition functions ~, and ~2, one can define the transition function of G, ~, as follows: for an input-output pair (i,o), ~((ql,q2),(i,o)) = (q~,q2) if and only if 30 ~(ql,(i,o)) = q l and ~2(q2,(i,o)) = q2. This point can be extended somewhat to - 14 - 220 1 i 8q include relations that involve bounded deletions or insertions. This is precisely the ~ el~re~tion n~cess~ry for systems of two-level rules, where a single transducerexpressing the entire system may be constructed via intersection of the tr~n.cdllcers expressing the individual rules. Indeed, the decision tree represents neither more 5 nor less than a set of weighted two-level rules. Each of the symbols in the expressions for ~ and p actually represent (sets of) pairs of symbols. Thus, alv, for example, represents all lexical alveolars paired with all of their possible surface realizations. And just as each tree represents a system of weighted two-level rules, so a set of trees -- for example, a set in which each tree deals with the realization 10 of a particular phone -- represents a system of weighted two-level rules, where each two-level rule is compiled from each of the individual trees.
The above description of the illustrative tree compilation procedure may be summarized in more formal terms as follows. Define a function Compile, which, when given a weighted rewrite rule as input, returns a WFST coll.~ulmg that rule15 as its output. This function may, for example, comprise the illustrative extended rewrite rule compilation procedure as described above (and in the Appendix attached hereto). Alternatively, this function may comprise the "KK" procedure, well known to those skilled in the art, also extended as described above (and in the Appendix attached hereto) so as to generate a WFST from a weighted rewrite rule. In any 20 event, the WFST for a single leaf L is thus defined as follows, where ~ is the input symbol for the entire tree, ~IJL is the output expression defined at L, PL represents the path traversed from the root node to L, p is an individual branch on that path, and ~ and pp are the expressions for ~ and p defined at p:
RuleL = Compile (d;~T~ L/ n 1p n pp) The transducer for an entire tree T is defined as:
RllleT = n RuleL
L~T
25 And, finally, the transducer for a forest F of trees is:
Figure 3 shows a flow diagram of the above-described illustrative procedure -15- 22Uli8~
~leF = n RllleT
r~F
for generating one or more weighted finite-state tran~-lucers from one or more decision trees in accordance with an illustrative embodiment of the present invention.
The input provided to the procedure comprises:
1. a forest F of one or more decision trees T;
2. a regular-expression characterization defining a property of either the left context, LC or the right context, RC, or both, of each decision at each node N of each tree T in F;
3. a notation of the input symbol S being tr~n~dllced by each tree T in F;
and 4. the alphabet of all possible input/output symbol pairs allowed by all trees T in F.
The output produced by the procedure comprises one or more transducers representing the forest F of trees.
Specifically, for each tree T in the forest F (selected in step 301), and for each leaf node L in the given tree T (selected in step 302), a weighted finite-state transducer is generated based on the leaf node L (steps 303-306). In particular, the path P from the root node of tree T to the selected leaf node L is determined in step 303. Then, in step 304, for each node N on path P, the regular expression at node N is intersected with the previous nodes on the path to build either the left context (LC) or the right context (RC) as defined for the given node (N). When the leaf node (L) is reached, the illustrative procedure of Figure 3 generates the weighted disjunction D of output symbols defned for the input symbol S for tree T (step 305).
And finally, step 306 compiles the weighted rewrite rule S ~D / LC_RC to generate a tran~ducer corresponding to leaf node L. The operation of step 306 may, 25 for example, be implemented with use of one of the illustrative weighted rewrite rule compilation procedures described above. (For example, either the "KK" procedure,extended to handle the compilation of weighted rewrite rules, or the extended improved procedure, as described above and in the Appendix attached hereto, may 2201 18'~
illustratively be used to implement step 306 of the illustrative procedure of Figure 3.) After the transducer corresponding to a given leaf node has been generated, the illustrative procedure of Figure 3 determines in decision box 307 whether a 5 transducer has been generated for every leaf node in tree T. If not (i. e., if there are more leaf nodes in tree T to process), flow returns to step 302 to select another leaf node from which a tr~n.cducer will be generated. When transducers have been generated for all leaf nodes in tree T, step 308 intersects together all of the transducers so generated, resulting in a single transducer for tree T as a whole. In 10 other illustrative embodiments of the present invention, step 308 may be avoided, thereby resulting in a plurality of trancdllcers (i. e., one for each leaf node) for tree T, rather than a single combined automaton.
After the generation of the tr~ncdllcer(s) for the given tree is complete, decision box 309 determines whether transducers have been generated for every tree 15 in forest F. If not (i.e., if there are more trees in forest F to process), flow returns to step 301 to select another tree from which one or more trancducers will be generated. When tr~ncdllcers have been generated for all trees in forest F, step 310 intersects together all of the transducers so generated, resulting in a single transducer for forest F as a whole. Note that in other illustrative embodiments of the present 20 invention, step 310 also may be avoided, thereby reslllting in a plurality oftransducers (i.e., one or more for each tree) for forest F, rather than a singlecombined automaton.
Although a number of specific embodiments of this invention have been shown and described herein, it is to be understood that these embodiments are 25 merely illustrative of the many possible specific arrangements which can be devised in application of the principles of the invention. Numerous and varied other arrangements can be devised in accordance with these principles by those of ordinary skill in the art without departing from the spirit and scope of the invention.
Claims (20)
1. An automated method for generating a finite-state representation of linguistic data based on one or more decision tree models of said linguistic data, each of the one or more decision tree models comprising one or more terminal nodes thereof, the method comprising the steps of:
generating one or more weighted rewrite rules based on one or more of the terminal nodes of the one or more decision tree models; and generating one or more weighted finite-state transducers based on one or more of the one or more weighted rewrite rules.
generating one or more weighted rewrite rules based on one or more of the terminal nodes of the one or more decision tree models; and generating one or more weighted finite-state transducers based on one or more of the one or more weighted rewrite rules.
2. The method of claim 1 wherein the decision tree models comprise classification and regression trees.
3. The method of claim 2 wherein the decision tree models represent a phoneticrealization of one or more phonemes by specifying a mapping of the phonemes intoone or more allophones.
4. The method of claim 3 wherein one or more of the decision trees has been trained based on a linguistic database.
5. The method of claim 1 wherein the weighted rewrite rules comprise rules having a form .PHI. -->~/.lambda.__~, where .PHI., .lambda., and ~ represent regular expressions, represents a weighted regular expression, and where the form .PHI. -->~/.lambda.__~
represents that an occurrence of .PHI. in an input string is to be replaced by ~ whenever the occurrence of .PHI. is preceded by .lambda. and succeeded by ~.
represents that an occurrence of .PHI. in an input string is to be replaced by ~ whenever the occurrence of .PHI. is preceded by .lambda. and succeeded by ~.
6. The method of claim 5 wherein the weighted regular expression represented by ~ comprises a rational power series.
7. The method of claim 5 wherein the step of generating the weighted finite-state transducers comprises generating a weighted finite-state transducer for a corresponding weighted rewrite rule having the form .PHI. -->~/.lambda.__~ by generating a composition of a first transducer, a second transducer, a third transducer, a fourth transducer and a fifth transducer, wherein:
the first transducer introduces a marker ">" before every occurrence of ~;
the second transducer introduces markers "<1" and "<2" before each occurrence of .PHI. that is followed by a ">" marker;
the third transducer replaces .PHI. with ~ when .PHI. is preceded by a "<1" marker and succeeded by a ">" marker, and deletes each of the ">" markers;
the fourth transducer admits only those input strings in which occurrences of a "<1" marker are preceded by .lambda., and deletes the "<1" marker at each suchoccurrence; and the fifth transducer admits only those input strings in which occurrences of a "<2" marker are not preceded by .lambda., and deletes the "<2" marker at each such occurrence.
the first transducer introduces a marker ">" before every occurrence of ~;
the second transducer introduces markers "<1" and "<2" before each occurrence of .PHI. that is followed by a ">" marker;
the third transducer replaces .PHI. with ~ when .PHI. is preceded by a "<1" marker and succeeded by a ">" marker, and deletes each of the ">" markers;
the fourth transducer admits only those input strings in which occurrences of a "<1" marker are preceded by .lambda., and deletes the "<1" marker at each suchoccurrence; and the fifth transducer admits only those input strings in which occurrences of a "<2" marker are not preceded by .lambda., and deletes the "<2" marker at each such occurrence.
8. The method of claim 1 wherein a given one of the decision tree models represents an input symbol, S, the given decision tree model further comprising a root node and a plurality of branches labelled with corresponding regular expressions, and wherein the step of generating the weighted rewrite rules comprises:
selecting one of the terminal nodes of the given decision tree model;
determining a path from the root node of the given decision tree model to the selected terminal node thereof, the path comprising one or more of the branches of the given decision tree model;
intersecting one or more of the regular expressions which label the branches of the given decision tree model comprised in the determined path, generating a left context, LC, and a right context, RC, therefrom;
generating a weighted disjunction, D, of output symbols based on the selected terminal node; and generating a weighted rewrite rule having a form S --> D / LC_RC, representing that an occurrence of the input symbol, S, in an input string is to be replaced by the weighted disjunction, D, whenever the occurrence of S is preceded by the left context, LC, and succeeded by the right context, RC.
selecting one of the terminal nodes of the given decision tree model;
determining a path from the root node of the given decision tree model to the selected terminal node thereof, the path comprising one or more of the branches of the given decision tree model;
intersecting one or more of the regular expressions which label the branches of the given decision tree model comprised in the determined path, generating a left context, LC, and a right context, RC, therefrom;
generating a weighted disjunction, D, of output symbols based on the selected terminal node; and generating a weighted rewrite rule having a form S --> D / LC_RC, representing that an occurrence of the input symbol, S, in an input string is to be replaced by the weighted disjunction, D, whenever the occurrence of S is preceded by the left context, LC, and succeeded by the right context, RC.
9. The method of claim 1 wherein a given one of the decision tree models comprises a plurality of terminal nodes, wherein the step of generating the weighted rewrite rules comprises generating a plurality of weighted rewrite rules corresponding to the plurality of terminal nodes of the given decision tree model, and wherein the step of generating the weighted finite-state transducers comprises:
generating a plurality of weighted finite-state transducers corresponding to the plurality of generated weighted rewrite rules; and intersecting the plurality of weighted finite-state transducers to produce a weighted finite-state transducer corresponding to the given decision tree model.
generating a plurality of weighted finite-state transducers corresponding to the plurality of generated weighted rewrite rules; and intersecting the plurality of weighted finite-state transducers to produce a weighted finite-state transducer corresponding to the given decision tree model.
10. The method of claim 9 wherein the step of generating the weighted finite-state transducers further comprises intersecting the weighted finite-state transducer corresponding to the given decision tree model with one or more other weighted finite-state transducers corresponding to one or more other decision tree models to produce a weighted finite-state transducer corresponding to a plurality of decision tree models.
11. An apparatus for automatically generating a finite-state representation oflinguistic data based on one or more decision tree models of said linguistic data, each of the one or more decision tree models comprising one or more terminal nodes thereof, the apparatus comprising:
means for generating one or more weighted rewrite rules based on one or more of the terminal nodes of the one or more decision tree models; and means for generating one or more weighted finite-state transducers based on one or more of the one or more weighted rewrite rules.
means for generating one or more weighted rewrite rules based on one or more of the terminal nodes of the one or more decision tree models; and means for generating one or more weighted finite-state transducers based on one or more of the one or more weighted rewrite rules.
12. The apparatus of claim 11 wherein the decision tree models comprise classification and regression trees.
13. The apparatus of claim 12 wherein the decision tree models represent a phonetic realization of one or more phonemes by specifying a mapping of the phonemes into one or more allophones.
14. The apparatus of claim 13 wherein one or more of the decision trees has been trained based on a linguistic database.
15. The apparatus of claim 11 wherein the weighted rewrite rules comprise rules having a form .PHI. -->~/.lambda.__~, where .PHI., .lambda., and ~ represent regular expressions, represents a weighted regular expression, and where the form .PHI. -->~/.lambda.__~
represents that an occurrence of .PHI. in an input string is to be replaced by ~ whenever the occurrence of .PHI. is preceded by .lambda. and succeeded by ~.
represents that an occurrence of .PHI. in an input string is to be replaced by ~ whenever the occurrence of .PHI. is preceded by .lambda. and succeeded by ~.
16. The apparatus of claim 15 wherein the weighted regular expression represented by ~ comprises a rational power series.
17. The apparatus of claim 15 wherein the means for generating the weighted finite-state transducers comprises means for generating a weighted finite-state transducer for a corresponding weighted rewrite rule having the form .PHI. -->~/.lambda.__~
by generating a composition of a first transducer, a second transducer, a third transducer, a fourth transducer and a fifth transducer, wherein:
the first transducer introduces a marker ">" before every occurrence of ~;
the second transducer introduces markers "<1" and "<2" before each occurrence of .PHI. that is followed by a ">" marker;
the third transducer replaces .PHI. with ~ when .PHI. is preceded by a "<1" marker and succeeded by a ">" marker, and deletes each of the ">" markers;
the fourth transducer admits only those input strings in which occurrences of a "<1" marker are preceded by .lambda., and deletes the "<1" marker at each suchoccurrence; and the fifth transducer admits only those input strings in which occurrences of a "<2" marker are not preceded by .lambda., and deletes the "<2" marker at each such occurrence.
by generating a composition of a first transducer, a second transducer, a third transducer, a fourth transducer and a fifth transducer, wherein:
the first transducer introduces a marker ">" before every occurrence of ~;
the second transducer introduces markers "<1" and "<2" before each occurrence of .PHI. that is followed by a ">" marker;
the third transducer replaces .PHI. with ~ when .PHI. is preceded by a "<1" marker and succeeded by a ">" marker, and deletes each of the ">" markers;
the fourth transducer admits only those input strings in which occurrences of a "<1" marker are preceded by .lambda., and deletes the "<1" marker at each suchoccurrence; and the fifth transducer admits only those input strings in which occurrences of a "<2" marker are not preceded by .lambda., and deletes the "<2" marker at each such occurrence.
18. The apparatus of claim 11 wherein a given one of the decision tree models represents an input symbol, S, the given decision tree model further comprising a root node and a plurality of branches labelled with corresponding regular expressions, and wherein the means for generating the weighted rewrite rules comprises:
means for selecting one of the terminal nodes of the given decision tree model;
means for determining a path from the root node of the given decision tree model to the selected terminal node thereof, the path comprising one or more of the branches of the given decision tree model;
means for intersecting one or more of the regular expressions which label the branches of the given decision tree model comprised in the determined path, generating a left context, LC, and a right context, RC, therefrom;
means for generating a weighted disjunction, D, of output symbols based on the selected terminal node; and means for generating a weighted rewrite rule having a form S --> D /
LC_RC, representing that an occurrence of the input symbol, S, in an input string is to be replaced by the weighted disjunction, D, whenever the occurrence of S is preceded by the left context, LC, and succeeded by the right context, RC.
means for selecting one of the terminal nodes of the given decision tree model;
means for determining a path from the root node of the given decision tree model to the selected terminal node thereof, the path comprising one or more of the branches of the given decision tree model;
means for intersecting one or more of the regular expressions which label the branches of the given decision tree model comprised in the determined path, generating a left context, LC, and a right context, RC, therefrom;
means for generating a weighted disjunction, D, of output symbols based on the selected terminal node; and means for generating a weighted rewrite rule having a form S --> D /
LC_RC, representing that an occurrence of the input symbol, S, in an input string is to be replaced by the weighted disjunction, D, whenever the occurrence of S is preceded by the left context, LC, and succeeded by the right context, RC.
19. The apparatus of claim 11 wherein a given one of the decision tree models comprises a plurality of terminal nodes, wherein the means for generating the weighted rewrite rules comprises means for generating a plurality of weighted rewrite rules corresponding to the plurality of terminal nodes of the given decision tree model, and wherein the means for generating the weighted finite-state transducers comprises:
means for generating a plurality of weighted finite-state transducers corresponding to the plurality of generated weighted rewrite rules; and means for intersecting the plurality of weighted finite-state transducers to produce a weighted finite-state transducer corresponding to the given decision tree model.
means for generating a plurality of weighted finite-state transducers corresponding to the plurality of generated weighted rewrite rules; and means for intersecting the plurality of weighted finite-state transducers to produce a weighted finite-state transducer corresponding to the given decision tree model.
20. The apparatus of claim 19 wherein the means for generating the weighted finite-state transducers further comprises means for intersecting the weighted finite-state transducer corresponding to the given decision tree model with one or more other weighted finite-state transducers corresponding to one or more other decision tree models to produce a weighted finite-state transducer corresponding to a plurality of decision tree models.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US663,767 | 1996-06-14 | ||
| US08/663,767 US5806032A (en) | 1996-06-14 | 1996-06-14 | Compilation of weighted finite-state transducers from decision trees |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CA2201189A1 true CA2201189A1 (en) | 1997-12-14 |
Family
ID=24663187
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002201189A Abandoned CA2201189A1 (en) | 1996-06-14 | 1997-03-27 | Compilation of weighted finite-state transducers from decision trees |
Country Status (4)
| Country | Link |
|---|---|
| US (1) | US5806032A (en) |
| EP (1) | EP0813185A3 (en) |
| JP (1) | JPH1063290A (en) |
| CA (1) | CA2201189A1 (en) |
Families Citing this family (65)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6032111A (en) * | 1997-06-23 | 2000-02-29 | At&T Corp. | Method and apparatus for compiling context-dependent rewrite rules and input strings |
| US6816830B1 (en) * | 1997-07-04 | 2004-11-09 | Xerox Corporation | Finite state data structures with paths representing paired strings of tags and tag combinations |
| US6249761B1 (en) * | 1997-09-30 | 2001-06-19 | At&T Corp. | Assigning and processing states and arcs of a speech recognition model in parallel processors |
| US6373985B1 (en) * | 1998-08-12 | 2002-04-16 | Lucent Technologies, Inc. | E-mail signature block analysis |
| DE19849855C1 (en) * | 1998-10-29 | 2000-04-27 | Ibm | Method for using a computer system to generate a text expression automatically while retaining meaning determines a statistical model on a number of preset pairs of word meanings and associated expressions. |
| US6411930B1 (en) * | 1998-11-18 | 2002-06-25 | Lucent Technologies Inc. | Discriminative gaussian mixture models for speaker verification |
| US6587844B1 (en) * | 2000-02-01 | 2003-07-01 | At&T Corp. | System and methods for optimizing networks of weighted unweighted directed graphs |
| GB2375210B (en) * | 2001-04-30 | 2005-03-23 | Vox Generation Ltd | Grammar coverage tool for spoken language interface |
| US20020174088A1 (en) * | 2001-05-07 | 2002-11-21 | Tongwei Liu | Segmenting information records with missing values using multiple partition trees |
| US7428526B2 (en) * | 2001-05-29 | 2008-09-23 | Claritas, Inc. | Household level segmentation method and system |
| US8214196B2 (en) | 2001-07-03 | 2012-07-03 | University Of Southern California | Syntax-based statistical translation model |
| US7027988B1 (en) | 2001-07-20 | 2006-04-11 | At&T Corp. | System and method of ε removal of weighted automata and transducers |
| US8290774B1 (en) | 2001-07-20 | 2012-10-16 | At&T Intellectual Property Ii, L.P. | System and method of epsilon removal of weighted automata and transducers |
| AU2003269808A1 (en) | 2002-03-26 | 2004-01-06 | University Of Southern California | Constructing a translation lexicon from comparable, non-parallel corpora |
| US8234115B2 (en) * | 2002-03-29 | 2012-07-31 | At&T Intellectual Property Ii, L.P. | Systems and methods for determining the N-best strings |
| US20040117188A1 (en) * | 2002-07-03 | 2004-06-17 | Daniel Kiecza | Speech based personal information manager |
| US20040006628A1 (en) * | 2002-07-03 | 2004-01-08 | Scott Shepard | Systems and methods for providing real-time alerting |
| US20040083090A1 (en) * | 2002-10-17 | 2004-04-29 | Daniel Kiecza | Manager for integrating language technology components |
| US7139688B2 (en) * | 2003-06-20 | 2006-11-21 | International Business Machines Corporation | Method and apparatus for classifying unmarked string substructures using Markov Models |
| US8548794B2 (en) | 2003-07-02 | 2013-10-01 | University Of Southern California | Statistical noun phrase translation |
| US7617091B2 (en) * | 2003-11-14 | 2009-11-10 | Xerox Corporation | Method and apparatus for processing natural language using tape-intersection |
| US7698125B2 (en) * | 2004-03-15 | 2010-04-13 | Language Weaver, Inc. | Training tree transducers for probabilistic operations |
| US8296127B2 (en) | 2004-03-23 | 2012-10-23 | University Of Southern California | Discovery of parallel text portions in comparable collections of corpora and training using comparable texts |
| US8666725B2 (en) | 2004-04-16 | 2014-03-04 | University Of Southern California | Selection and use of nonstatistical translation components in a statistical machine translation framework |
| US20050240462A1 (en) * | 2004-04-21 | 2005-10-27 | Inman Kenneth L | Method and apparatus for population segmentation |
| WO2006036150A1 (en) | 2004-09-28 | 2006-04-06 | Nielsen Media Research, Inc | Data classification methods and apparatus for use with data fusion |
| JP5452868B2 (en) | 2004-10-12 | 2014-03-26 | ユニヴァーシティー オブ サザン カリフォルニア | Training for text-to-text applications that use string-to-tree conversion for training and decoding |
| CA2525729A1 (en) * | 2004-11-08 | 2006-05-08 | At&T Corp. | System and method for compiling rules created by machine learning program |
| US8676563B2 (en) | 2009-10-01 | 2014-03-18 | Language Weaver, Inc. | Providing human-generated and machine-generated trusted translations |
| US8886517B2 (en) | 2005-06-17 | 2014-11-11 | Language Weaver, Inc. | Trust scoring for language translation systems |
| US10319252B2 (en) | 2005-11-09 | 2019-06-11 | Sdl Inc. | Language capability assessment and training apparatus and techniques |
| KR100735559B1 (en) * | 2005-11-18 | 2007-07-04 | 삼성전자주식회사 | Apparatus and method for constructing language model |
| US7835911B2 (en) * | 2005-12-30 | 2010-11-16 | Nuance Communications, Inc. | Method and system for automatically building natural language understanding models |
| US20070206759A1 (en) * | 2006-03-01 | 2007-09-06 | Boyanovsky Robert M | Systems, methods, and apparatus to record conference call activity |
| US8943080B2 (en) | 2006-04-07 | 2015-01-27 | University Of Southern California | Systems and methods for identifying parallel documents and sentence fragments in multilingual document collections |
| US7502771B2 (en) * | 2006-05-11 | 2009-03-10 | Schlumberger Technology Corporation | Method, system and apparatus for generating decision trees integrated with petro-technical workflows |
| US8886518B1 (en) | 2006-08-07 | 2014-11-11 | Language Weaver, Inc. | System and method for capitalizing machine translated text |
| US7624075B2 (en) * | 2006-09-15 | 2009-11-24 | Microsoft Corporation | Transformation of modular finite state transducers |
| US7627541B2 (en) * | 2006-09-15 | 2009-12-01 | Microsoft Corporation | Transformation of modular finite state transducers |
| US8433556B2 (en) | 2006-11-02 | 2013-04-30 | University Of Southern California | Semi-supervised training for statistical word alignment |
| US9122674B1 (en) | 2006-12-15 | 2015-09-01 | Language Weaver, Inc. | Use of annotations in statistical machine translation |
| US8468149B1 (en) | 2007-01-26 | 2013-06-18 | Language Weaver, Inc. | Multi-lingual online community |
| US9325749B2 (en) * | 2007-01-31 | 2016-04-26 | At&T Intellectual Property I, Lp | Methods and apparatus to manage conference call activity with internet protocol (IP) networks |
| US8615389B1 (en) | 2007-03-16 | 2013-12-24 | Language Weaver, Inc. | Generation and exploitation of an approximate language model |
| US8831928B2 (en) | 2007-04-04 | 2014-09-09 | Language Weaver, Inc. | Customizable machine translation service |
| US8340086B2 (en) | 2007-04-19 | 2012-12-25 | At&T Intellectual Property I, Lp | Methods and apparatus to protect and audit communication line status |
| US8825466B1 (en) | 2007-06-08 | 2014-09-02 | Language Weaver, Inc. | Modification of annotated bilingual segment pairs in syntax-based machine translation |
| US8266168B2 (en) | 2008-04-24 | 2012-09-11 | Lexisnexis Risk & Information Analytics Group Inc. | Database systems and methods for linking records and entity representations with sufficiently high confidence |
| US8311806B2 (en) | 2008-06-06 | 2012-11-13 | Apple Inc. | Data detection in a sequence of tokens using decision tree reductions |
| US8738360B2 (en) | 2008-06-06 | 2014-05-27 | Apple Inc. | Data detection of a character sequence having multiple possible data types |
| US8990064B2 (en) | 2009-07-28 | 2015-03-24 | Language Weaver, Inc. | Translating documents based on content |
| US8380486B2 (en) | 2009-10-01 | 2013-02-19 | Language Weaver, Inc. | Providing machine-generated translations and corresponding trust levels |
| US10417646B2 (en) | 2010-03-09 | 2019-09-17 | Sdl Inc. | Predicting the cost associated with translating textual content |
| JP5232191B2 (en) * | 2010-03-19 | 2013-07-10 | 株式会社東芝 | Information processing apparatus, information processing method, and program |
| US8543374B2 (en) * | 2010-08-12 | 2013-09-24 | Xerox Corporation | Translation system combining hierarchical and phrase-based models |
| US11003838B2 (en) | 2011-04-18 | 2021-05-11 | Sdl Inc. | Systems and methods for monitoring post translation editing |
| US8694303B2 (en) | 2011-06-15 | 2014-04-08 | Language Weaver, Inc. | Systems and methods for tuning parameters in statistical machine translation |
| US8886515B2 (en) | 2011-10-19 | 2014-11-11 | Language Weaver, Inc. | Systems and methods for enhancing machine translation post edit review processes |
| US8942973B2 (en) | 2012-03-09 | 2015-01-27 | Language Weaver, Inc. | Content page URL translation |
| US10261994B2 (en) | 2012-05-25 | 2019-04-16 | Sdl Inc. | Method and system for automatic management of reputation of translators |
| US9152622B2 (en) | 2012-11-26 | 2015-10-06 | Language Weaver, Inc. | Personalized machine translation via online adaptation |
| US9594744B2 (en) * | 2012-11-28 | 2017-03-14 | Google Inc. | Speech transcription including written text |
| US9213694B2 (en) | 2013-10-10 | 2015-12-15 | Language Weaver, Inc. | Efficient online domain adaptation |
| JP2018013590A (en) * | 2016-07-20 | 2018-01-25 | 株式会社東芝 | Generation device, recognition system, finite state transducer generation method and data |
| CN108735201B (en) * | 2018-06-29 | 2020-11-17 | 广州视源电子科技股份有限公司 | Continuous speech recognition method, device, equipment and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2170669A1 (en) * | 1995-03-24 | 1996-09-25 | Fernando Carlos Neves Pereira | Grapheme-to phoneme conversion with weighted finite-state transducers |
-
1996
- 1996-06-14 US US08/663,767 patent/US5806032A/en not_active Expired - Lifetime
-
1997
- 1997-03-27 CA CA002201189A patent/CA2201189A1/en not_active Abandoned
- 1997-06-03 EP EP97303743A patent/EP0813185A3/en not_active Withdrawn
- 1997-06-16 JP JP9192981A patent/JPH1063290A/en active Pending
Also Published As
| Publication number | Publication date |
|---|---|
| US5806032A (en) | 1998-09-08 |
| EP0813185A2 (en) | 1997-12-17 |
| JPH1063290A (en) | 1998-03-06 |
| EP0813185A3 (en) | 1998-11-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA2201189A1 (en) | Compilation of weighted finite-state transducers from decision trees | |
| US6501833B2 (en) | Method and apparatus for dynamic adaptation of a large vocabulary speech recognition system and for use of constraints from a database in a large vocabulary speech recognition system | |
| Lu et al. | Combining a vector space representation of linguistic context with a deep neural network for text-to-speech synthesis | |
| US5761637A (en) | Dialogue-sound processing apparatus and method | |
| Humphries et al. | Using accent-specific pronunciation modelling for robust speech recognition | |
| US20020087313A1 (en) | Computer-implemented intelligent speech model partitioning method and system | |
| KR950034052A (en) | Speech Recognition System and Speech Recognition Method | |
| Riley et al. | Transducer composition for context-dependent network expansion. | |
| Sproat et al. | Compilation of weighted finite-state transducers from decision trees | |
| Chen | Identification of contextual factors for pronunciation networks | |
| Elderkin | The significance and origin of the use of pitch in Sandawe | |
| Schmid et al. | Automatically generated word pronunciations from phoneme classifier output | |
| Cheng et al. | A generalized dynamic composition algorithm of weighted finite state transducers for large vocabulary speech recognition | |
| Ström | Continuous speech recognition in the WAXHOLM dialogue system | |
| GB2391993A (en) | System for generating the grammar of a spoken dialogue system | |
| Eide | Automatic modeling of pronunciation variations. | |
| Horne et al. | Referent tracking in restricted texts using a lemmatized lexicon: implications for generation of intonation. | |
| JP3265864B2 (en) | Voice recognition device | |
| Tao et al. | Program synthesis with generative pre-trained transformers and grammar-guided genetic programming grammar | |
| Huang et al. | Internet-accessible speech recognition technology | |
| Cremelie et al. | In search of pronunciation rules | |
| CN110955768A (en) | Question-answering system answer generating method based on syntactic analysis | |
| Prieto et al. | Continuous speech understanding based on automatic learning of acoustic and semantic models. | |
| Lüngen et al. | Morphology and speech technology | |
| Brenner et al. | Word recognition in continuous speech using a phonological based two-network matching parser and a synthesis based prediction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request | ||
| FZDE | Discontinued |