US20120143858A1 - Method And Means For Data Searching And Language Translation - Google Patents
Method And Means For Data Searching And Language Translation Download PDFInfo
- Publication number
- US20120143858A1 US20120143858A1 US13/390,526 US201013390526A US2012143858A1 US 20120143858 A1 US20120143858 A1 US 20120143858A1 US 201013390526 A US201013390526 A US 201013390526A US 2012143858 A1 US2012143858 A1 US 2012143858A1
- Authority
- US
- United States
- Prior art keywords
- search
- context
- synonym
- translation
- expanded
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013519 translation Methods 0.000 title claims abstract description 79
- 238000000034 method Methods 0.000 title claims abstract description 47
- 230000014616 translation Effects 0.000 description 60
- 241000220225 Malus Species 0.000 description 10
- 230000000694 effects Effects 0.000 description 8
- 241000251468 Actinopterygii Species 0.000 description 6
- 235000019688 fish Nutrition 0.000 description 6
- 230000006399 behavior Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 4
- 238000000605 extraction Methods 0.000 description 3
- 230000010006 flight Effects 0.000 description 3
- 238000003909 pattern recognition Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000002950 deficient Effects 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 238000012384 transportation and delivery Methods 0.000 description 2
- UKUVVAMSXXBMRX-UHFFFAOYSA-N 2,4,5-trithia-1,3-diarsabicyclo[1.1.1]pentane Chemical compound S1[As]2S[As]1S2 UKUVVAMSXXBMRX-UHFFFAOYSA-N 0.000 description 1
- 241000972773 Aulopiformes Species 0.000 description 1
- 241000442132 Lactarius lactarius Species 0.000 description 1
- 241000581835 Monodora junodii Species 0.000 description 1
- 235000019892 Stellar Nutrition 0.000 description 1
- 235000009754 Vitis X bourquina Nutrition 0.000 description 1
- 235000012333 Vitis X labruscana Nutrition 0.000 description 1
- 240000006365 Vitis vinifera Species 0.000 description 1
- 235000014787 Vitis vinifera Nutrition 0.000 description 1
- 235000021016 apples Nutrition 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- RGNPBRKPHBKNKX-UHFFFAOYSA-N hexaflumuron Chemical compound C1=C(Cl)C(OC(F)(F)C(F)F)=C(Cl)C=C1NC(=O)NC(=O)C1=C(F)C=CC=C1F RGNPBRKPHBKNKX-UHFFFAOYSA-N 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 235000019515 salmon Nutrition 0.000 description 1
- 241000894007 species Species 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 230000002195 synergetic effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 235000020097 white wine Nutrition 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9537—Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2452—Query translation
- G06F16/24522—Translation of natural language queries to structured queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2457—Query processing with adaptation to user needs
- G06F16/24578—Query processing with adaptation to user needs using ranking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/3332—Query translation
- G06F16/3338—Query expansion
Definitions
- the invention relates to data searching and language translation.
- the invention relates to searching documents from the Internet or databases.
- the invention also relates to translating words in documents, WebPages, images or speech from one language to the next.
- search engines are known in the prior art, as well as machine translation software.
- One document that describes the situation in the prior art is WO 2009/002864 A2, which is cited here as reference.
- a search query is inputted to a computer, for example of the form “How to ship a box”.
- the terms in the query are synonym expanded.
- the word ship can have synonyms e.g., “boat” and “send”.
- the expanded synonym set is limited by removing the expanded synonyms that are contextually irrelevant. That is, a synonym of a term in the search query is selected based on a context of occurrence of the term in the received search query in accordance with this prior art. For example in the prior art, search results related to fishing trawlers are probably not relevant to shipping a box and thus these search results are discarded.
- the invention under study is directed towards a system and a method for effectively improving computerised search queries from data and computerised language translations from one language to the next.
- a further object of the invention is to solve the above problem and produce search results and translations of improved quality.
- “synonym” of a word A is construed as a word B that has similar meaning with word A, but is a different word. Spelling and typing errors in a word are also regarded to change the word into a synonym in this application. I.e. “aple” could be a synonym to “apple” in accordance with the invention.
- “Synonym expansion” is construed as the expansion of the original search and/or translation query into word sets having a similar meaning. For example a query “Big Aple” could be synonym expanded to:
- Natural language means a past, present or future language spoken and/or written such as French, German, Esperanto, English or Chinese.
- the search query is expanded with search term synonyms.
- Search results are retrieved based on the expanded query.
- valid synonyms are not excluded because they merely appear to be irrelevant. For example in the case of the query: “How to ship a box”, a trawler, albeit a fishing ship, would not be excluded. Now this will probably lead to a larger set of search results.
- the search results related to fishing trawlers are now still in the synonym expanded search result set.
- the synonym expanded search results are then ranked based on their contextual relevance.
- the contextual information that is used to perform the contextual ranking on the synonym expanded search results can be derived from various sources: global known parameters, data available from the computer of the user, recent browsing history of web pages, a statistical analysis of how users in the past have accessed sites listed in the synonym expanded search query, data open in an application on the computer of the user and any other information that characterises the context in which the query is launched. For example, if the user has a browser window open with words “Cod” and/or “Salmon”, both generally known fish, the contextual relevance of the “fishing trawler” is very strong, albeit it is not the closest synonym to ship. Therefore the search results with the fishing trawlers will not only be included in the search results, but will also rank high in the final list of search results that is shown to the user.
- the search query “how to ship a box” comes from a computer known to reside in the Arabian Desert in, say Saudi Arabia.
- the same synonym expanded search result set is retrieved.
- the search result that deals with fishing trawlers that also deliver boxes of fish is in the retrieved results it will not rank high based on its contextual relevance.
- the websites of the Saudi Arabian Post Office come up quite high based on their contextual relevance, as well as any courier service that has a branch there.
- Another embodiment of the invention deploys the same aforementioned inventive technique in translating documents from one language to the next.
- a word to be translated into another language is analogous to a search term in some embodiments of the invention.
- the sentences to be translated into English are for example in Finnish: “Matti Oath kayda kylassa. Saavuttuaan Liisa tarjosi kahvit tuvassa.”
- Synonym expanded translations may be of the following type:
- search result list with validly expanded synonyms that is broad is desirable.
- This quite extended list of synonym expanded search results will now be subject to context based ranking.
- the context based ranking can be based on general statistical trends observed from other documents in the same language, words surrounding the text to be translated, data available from the computer of the user, recent browsing history of WebPages, and any other information. Now, if the context somehow reveals that for example the events take place in Helsinki, the translations 1-2 are ranked higher, because generally Finnish speakers would not refer to Helsinki as a village as it is the biggest city and the capital city in the country.
- the translation 4 would rank contextually higher because “hisydä kylvesssä” implies a social nature to the visit, and if Matti and Liisa do not know each other, it is more contextually relevant to assume that they meet in a village and have coffee that is merely offered by Liisa (e.g. in a house that is a café) rather than on Matti's personal visit in the living room of Liisa.
- the invention can be used to produce the best matching translation. As long as outright wrong decisions in the synonym expansion are avoided, and there is enough contextual information available the outcome will be a considerably improved translation.
- Some or all of the aforementioned advantages of the invention are accrued with a browser and a search engine that translates the foreign pages using the synonym expansion-context ranking technique and displays the most relevant search results that are also retrieved with the synonym expansion-context ranking technique.
- the invention therefore guarantees the widest searches in scope, and the most logically relevant search results are displayed first.
- a memory unit in accordance with the invention comprises software capable of executing the computer implemented search method and/or operating the search arrangement described above.
- a memory unit in accordance with the invention comprises software capable of executing the computer implemented translation method and/or operating the translation arrangement described above.
- a memory unit in accordance with the invention comprises software, characterised in that, the software is arranged to read data from a dedicated part of a file system to extract contextual data for search and/or translation.
- a search engine software in accordance with the invention is characterised in that, the search engine is arranged to rank synonym expanded search results based on context data derived from user's Internet browser, operating system, file system, database software and/or applications.
- Operating system software in accordance with the invention is characterised in that, the operating system is arranged to rank synonym expanded search results based on context data derived from user's Internet browser, operating system, file system, database software and/or applications.
- a software application in accordance with the invention is characterised in that, the software application is arranged to rank synonym expanded search results based on context data derived from user's Internet browser, operating system, file system, database software and/or applications.
- Database software in accordance with the invention is characterised in that, the database software is arranged to rank synonym expanded search results based on context data derived from user's Internet browser, operating system, file system, database software and/or applications.
- the best mode of the invention is considered to be an Internet search engine that delivers better search results.
- FIG. 1 demonstrates an embodiment of the inventive search method.
- FIG. 2 demonstrates an embodiment 20 of the arrangement used to perform the searches and/or translations in accordance with the invention.
- FIG. 3 demonstrates an embodiment 30 of the translation method in accordance with the invention.
- FIG. 4 demonstrates an embodiment 40 of the translation method using audiovisual input in accordance with the invention.
- FIG. 1 shows an optimum computerised search method as a flow diagram in accordance with the invention.
- the user inputs a search query.
- the input can take place by typing text for example with a keyboard or other computer peripheral, by speech when preferably speech recognition is conducted by the computer to extract the query terms, and/or with an image when preferably pattern recognition is conducted by the computer to extract the query terms.
- the user can type:“how to ship a box” by a keyboard, or he can say “how to ship a box” and the speech recognition recognises the words and uses them as search terms, and/or he can take an image with a camera or a mobile phone, of a ship with a box on board and the pattern recognition software recognises the ship and the box and extracts these as search terms, and/or the user takes an image of a sign reading “how to ship a box” and the computer uses pattern and/or character recognition, such as Object Character Recognition (OCR) to identify the characters and thus the words in the sign in accordance with the invention.
- OCR Object Character Recognition
- these input methods can be used in a mix: the user can for example write “how to” and take an image of the box on the ship and the written words “how to” and the pattern recognised words “ship” and “box” are combined to form the search query in accordance with the invention.
- speech and text and image and speech or in fact all three forms of input, can be used in a mix in accordance with the invention.
- the computer will arrange the terms in the order in which they arrive or are recognised, or the user can specify the order of the search terms, and/or the computer can suggest entire search queries to the user based on the recognised search terms in accordance with the invention.
- the search query is expanded with search term synonyms. “how to” could be recognised as a phrase and expanded with a synonym “in what way”, the “ship” could be expanded with synonyms such as “boat” and “send”, the “box” could be expanded with a synonym “square” and “parcel”.
- the synonym expansion will expand obvious writing versions of different words: for example “centre” (UK English) would be expanded to “center” (US English) in accordance with the invention.
- the inventive computing system can correct spelling errors such as “boks”, “sip”, “hou” that might have been caused by phonetic miss-spellings in the synonym expansion, or separately before the synonym expansion.
- inventive computing system can correct spelling errors based on typographical errors, for example in a QWERTY keyboard potential errors could be “boz”, “shio”, “hoq” in accordance with the invention.
- computer system can expand different states of a noun, and/or correct words from different writing forms to standard forms.
- the Finnish language is notorious for having many states for a noun.
- the synonym expansion will also expand the search terms to different linguistic states.
- a search term “mélla” would be expanded to also include “ismessa”, “lahti” and possibly other states of the noun in accordance with the invention.
- some terms from the search query can be omitted based on context of occurrence of a search term in order to limit the search terms in accordance with practical limitations, such as available computing power.
- the search results are retrieved based on the expanded query.
- the more expanded the query the greater the number of search results.
- the expanded search query is divided into several different search queries that are executed by the same or different computers in parallel or in series, and the search results of each separate query are combined to form the overall search results for the expanded search query.
- the search results are ranked contextually. It is important in the invention that the ranking is based on the synonym expanded results set.
- the synonym expansion increases the reach of the search and delivers more search results, some of which may be or are probably even undesirable to the user.
- the combined synergistic effect of the contextual ranking imposed on the synonym expanded search result set is important in accordance with the invention.
- a search result that would have been omitted by contextual rules used in the synonym expansion will still be in the search result set in accordance with the invention, all though it is likely to be proverbially buried underneath hundreds if not millions of search results (depending on how expansive the synonym expansion is) that also match the expanded search query.
- Contextual data i.e. data for defining the context in which the synonym expanded search results are ranked may be deduced from previous browsing behaviour of the user, location of the user, time, date, nationality of the user, data on the computer of the user, general population browsing behaviour, general statistics and frequency analysis of words, and data that is open in an application on the computer of the user.
- Using previous web pages of the user to define the context is preferable if the user has a consistent browsing behaviour, but many people don't. In the case of a person that checks the company intranet, and then reads a newspaper online, looks up an auction site or classifieds it might take a long browsing history to determine meaningful context data.
- One preferable embodiment of the invention is the use of an open application on the computer to define the context data used in ranking. For example if the user has a word processor open with a special term, say “NGC 2547”, it will immediately suggest that a search term “star” actually means a stellar object in space rather than people in Hollywood or Israel. In this particular search the documents associated with celebrities will be ranked to the last places in the context ranked synonym expanded search result list.
- context based ranking can be performed by several different methods and means in accordance with the invention.
- context of occurrence we mean the context in which at least one search term occurs, i.e. the context of occurrence of at least one search term.
- the context of occurrence of these search terms comprise data in other applications on the computer of the user when he made the search query, and any data that is associated to the context in which the search terms now occur.
- the context of occurrence thus means the context data on the user's computer, and/or to which the user's computer has access to as the search terms occur, i.e. are provided to a search engine and/or program.
- the system picks words or numbers from the context, such as open applications, emails, messages, temporary folder, Desktop, calendar, clock, GPS chip, operating system, previously browsed WebPages and/or the like.
- the picking of the words, numbers or other limiting factors to define the context can be done randomly, or by some algorithm, or even by the user in some embodiments, in accordance with the invention.
- the system now has the search terms that were synonym expanded, typically a huge number of search results, and terms that define the context.
- search results can be rearranged into an order where the document that is closest in similarity to the selected context terms is placed first.
- the received search results are simply ranked in the order of common hits with the context terms.
- the search query “How to ship a box” the user has a webpage open that is titled “Shipping fish fast to sushi restaurants”, and suppose for example that the system knows we are in Vigo, Spain, probably the largest fishing port in the world.
- the context terms “fish”, “fast”, “sushi” and “restaurant” are selected in accordance with the invention.
- the document that tells about a fishing trawler that sends sushi filets by direct delivery in boxes of ice after it docks in Vigo is probably proverbially buried on the millionth seed or similar in the search results.
- this document will now feature the highest frequency of the contextual terms, and will be rearranged to first place in contextual ranking.
- context based ranking of the invention is running a new search on the original search results based only on acquired context words.
- search terms and/or context terms may have different weights, that may be assigned by the user, by the system, be updated from the network, or be pre-programmed into the computer in accordance with the invention.
- context terms may be functional, rather than only plain words or numbers, for example context terms can be acquired in functional form such as the date, time, location, and i.e. context terms can be functional or conditional beyond their plain meaning.
- contextual ranking is performed based on context ontologies.
- ontology (computer science) a rigorous and exhaustive organization of some knowledge domain that is usually hierarchical and contains all the relevant entities and their relations source: Wordnet).
- the word apple might have a conditional ontology, e.g. meaning New York City when having the prefix “Big” and being in capital letters “Big Apple”.
- Ontology of a word can be typically stored in a file that is read by at least one search engine program.
- a search engine or the like can have data files for all words in all common languages that define ontologies of those words on one computer or distributed over a network of computers.
- those files or a part of those files could be stored in the subscriber terminal, like the terminal computer the user is using, such as a PC or Apple Mac computer.
- variable weights for different parts may exist within a word ontology.
- “Granny Smith” could have a higher relative weight in the “apple” ontology than “Calville Blanc” in some embodiments of the invention, because a priori one could assume more users searching for apples will be searching for Granny Smiths, because it is better known.
- the choice can of course be made the other way around providing a higher relative weight for “Calville Blanc” in some embodiments of the invention.
- the further the connection is from the actual word itself the lower the relative weight.
- words that were chosen to define the context may have ontologies, and in fact in some embodiments those words with well defined ontologies are typically chosen as context words over words that do not have an ontology, or have an ontology that is poorly defined or less used or the like.
- search results are searched with the ontologies of the context terms and those search result documents that are most similar to the ontologies of the context terms are prioritized in the ranking, i.e. ranked before less relevant search result documents.
- some or all selected context terms may be synonym expanded too.
- the retrieved search results are then searched again with the synonym expanded context terms, and the search results that are most similar or have the largest number of hits of synonym expanded context terms are ranked first in the contextual ranking in some embodiments.
- search terms, context terms and/or ontologies may have different weights, which may be assigned by the user, by the system, be updated from the network, or be pre-programmed into the computer in accordance with the invention.
- context terms and/or ontologies may be functional, rather than only plain words or numbers, for example context terms can be acquired in functional form such as the date, time, location, i.e. context terms can be functional beyond their plain meaning.
- context term For example consider the context term “time: 12:00”. If the time is +/ ⁇ 15 min from 12:00 in human terms time: 12:00 will very strongly equate with “Now”. Quite clearly in this situation “now” would be in the ontology of the context term time: 12:00, but only functionally and/or conditionally so, if it were 8.00 pm, “now” might not be in the ontology of the context term.
- a user or the user terminal may have several contexts that have different ontologies such as for example Email context, Browsing context, and/or Publication context.
- an email context will contain other information besides content, such as the time, sender and/or receiver, their identity. Many times the actual content in the email is not remembered by people who were corresponding; it is rather the sender and the time when it was sent and/or received, its importance classification or other parameters that define the context ontology of the email context.
- Browsing context can be read from the browser cache in some embodiments of the invention. For example a webpage that was accessed at a certain time should be interpreted in the context of other WebPages that were accessed at the same time, or with the same or similar query string.
- the browser cache when read in this way can be used to determine the browsing context ontology in accordance with the invention.
- Publication context on the other hand relates to what the user or user terminal decides to publish. Many users receive a lot of crap that is uninteresting to them, but people rarely post a webpage or send electronic mail or FTP files about issues that are completely uninteresting to themselves, those issues may of course be uninteresting to others.
- the data that the user or the user terminal publishes on the Internet or to a select group in the form of messaging can be used to define the ontology of the publication context.
- a user can assign a weight to particular regions of the ontology, or to entire ontologies. For example in the email context ontology all sent emails or terms or data that appear in sent emails could be given a user assigned relative weight of 100, when terms and/or data in received emails could be given a weight of 1.
- the user could set the aforementioned relative weights himself or they could be pre-programmed or set from an Internet and/or network server by a third party.
- the weights of some elements in the ontology could be a function of publication frequency for example.
- the different context and context ontologies may also have different weights, which can be user assigned, assigned from a network server and/or pre-programmed.
- An attorney that mainly uses email for his self expression will have an email context ontology that will have a higher relevant weight to browsing or publication ontology.
- a media marketing person will by contrast have a publication context ontology outweighing email context ontology for example, as the most important things on her mind are the things she publishes by way of WebPages or press releases for example.
- a certain context could have one or a collection of context terms that would be the same all the time. If the user is a travel agent selling British Airways flights “British Airways” and “Eurotraveller” are pretty constant context terms, because nearly all business is associated with booking Eurotraveller economy seats to customers. If the ontology of these constant context terms would be constant all the time, this would in effect mean that the ontology of the context, i.e. email context in this case would also be constant.
- a number of contexts are being up to date as the user uses the computer system, and the context with the best fit to user activity is preferably chosen all the time, the context terms that best reflect user activity are preferably chosen all the time, and the ontology of at least one context term and/or context is preferably up to date to reflect the user activity.
- contexts may be combined in some embodiments to yield the context definition with different relative weights.
- a user having an email message open and browsing the Internet could be in a 70% email context 30% browsing context in accordance with the invention.
- different contexts may be used in combination in accordance with the invention.
- context term within a context ontology may have different relative weights in accordance with the invention. For example frequency of a term could add weight to a context term. If the user has an email message open that recites “flight” 5 times, the context term “flight” should have a higher relative weight than some less recited context term, on its own as an independent context term without a ontology, or in a context term ontology.
- Context based ranking based on context ontologies can be visualised with the assistance of RDF (Resource Description Framework)-metadata graphs.
- RDF Resource Description Framework
- FIG. 1 shows a context ontology which specifies context metadata from emails, files, web pages and publications with relations among them. The relations are described in “Activity Based Metadata for Semantic Desktop Search”—Proceedings of the 2 nd European Semantic Web Conference, Heraklion, Greece, May 2005 which is also cited here as reference and incorporated into this application.
- the context ontology elements and the relations between them can be used to search the synonym expanded search results, and prioritise these search results by means of contextual ranking.
- the relations and metadata within the ontology are typically derived from user activity in some embodiments, for example from when, to whom, and about what subject the user is emailing messages and with what attachment files.
- those synonym expanded search results that rate highest in similarity to the context ontology that contains the metadata and/or relations are placed with the best rank in the contextual ranking in accordance with the invention.
- Context-Aware Semantic Association Ranking by Aleman-Meza, Halaschek, Arpinar and Sheth describes techniques for context based ranking in detail, and is included here as reference to this application. It is in accordance with the invention that the contextual ranking is based on semantic association ranking.
- context ontology is defined as explained earlier.
- the synonym expanded search results are compared against the context ontology by seeing which search results are most closely semantically associated with the context and/or context ontology. Semantic association between a context and a synonym expanded search result is measured by overall path weight.
- the synonym expanded search results are contextually ranked so, that those synonym expanded search results that are most closely semantically associated (and/or have the highest overall path weight) with the specified context and/or context ontology are ranked first.
- the semantic association and/or the overall path length can be composed from any permutation and/or combination of the aforementioned components. For example user defined weights could be left out in some embodiments of the invention.
- contextual ranking of phase 130 may be performed with different techniques. It is in accordance with the invention that these techniques can be combined and/or permuted in accordance with the invention.
- the contextually ranked synonym expanded search results are displayed to the user in the order of contextual relevance.
- the said list is displayed in the monitor and the user can move to different positions in the list and select any search result for more detailed viewing.
- some portion of the final search results might be discarded, for example the contextually least relevant search results. Discarding search results now is not as dangerous as based on a contextual analysis of a search term earlier.
- the search result to be discarded has been considered vis-a-vis its objective content of the substantive meaning of the search terms as well as its context in comparison to the context of the query as a whole or in part in some embodiments.
- FIG. 2 shows an exemplary arrangement for conducting inventive searches.
- the embodiment 20 refers more particularly to Internet searches, but the invention can of course be used with any data searches from any number of documents or databases.
- the user will use the computer 210 with the keyboard 220 and/or a peripheral like a mouse 200 and read and look at results from the screen 230 .
- the inventive search method can be operated “off-line” i.e. internally in the computer and the search results are shown on the screen 230 .
- the search software needed to carry out the searches with the inventive method resides on the server 240 only and the documents on computer 210 need to be searched with the inventive search method.
- the server 240 searches the hard drives, memory, files or databases on computer 210 .
- part of the inventive system can be distributed to several servers, several client computers, and/or so that some parts of the system are distributed to at least one client computer, and some parts that are different or the same are distributed to a server computer.
- the most typical use embodiment of the invention is normal Internet search.
- a search query is forwarded from the computer of the user to a server of an Internet Search Engine.
- Exemplary prior art search engines are currently known to operate by various names such as GoogleTM, BingTM, YahooTM, LycosTM and the like.
- the server 240 receives the search query and conducts the inventive Internet search.
- the server 240 is typically arranged to search the Internet or databases but it may also search any files, documents, and/or data that it has access to in accordance with the invention. It is in accordance with the invention that there are more than one server 240 that share the processing of the search query, or the server 240 may relay the search query to another server in accordance with the invention.
- the computer 210 , screen 230 , keyboard 220 and/or mouse 200 can be replaced by a mobile phone, laptop or any portable computing device, or in fact any device that can receive a search query, irrespective of whether the computational resources reside on that device and/or the server 240 .
- the search queries can be inputted by speech or images, in which case the computer could be replaced merely by a microphone, or a normal telephone wireless or wire line to receive the search query.
- image input the computer features, or is replaced by a camera used to take the images.
- it would be preferable to receive digital images as input for the inventive search but naturally film images can be digitised and used by the inventive computerised search in accordance with the invention.
- All communications connections between any computers or devices whether client computers, dumb terminals or server devices can be arranged as wireless connections, such as radio/microwave/optical, for example GSM, CDMA, WIFI, free space optics (FSO), and/or fixed communications connections such as data cable, POTS, and/or optical network e.g. SONET/SDH/Ethernet.
- wireless connections such as radio/microwave/optical, for example GSM, CDMA, WIFI, free space optics (FSO), and/or fixed communications connections such as data cable, POTS, and/or optical network e.g. SONET/SDH/Ethernet.
- the contextual data needed to perform the contextual ranking of the synonym expanded search results can be obtained in a variety of ways: the server 240 may observe the previous searches made by the user, the Internet and/or database and/or document browsing behaviour of the user and deduce the context based on the occurrence of words in that document in some embodiments.
- the user may specify a context by text entry or multiple choice to the computer in some embodiments.
- the data on the user's computer can be used to define context data in a multitude of ways: It is possible to use all data on the computer 210 to define the context but this requires a great deal of data processing.
- the user has a dedicated folder that contains documents that relate to the area of interest to the user. The documents only in this folder are read to define the context data in some embodiments.
- the user could have a scanned fishing permit in his context folder on the computer. Whilst he would input “how to ship a box” the context would be “fishing” and the documents related to fishing trawlers shipping boxes of fish would rank high in the final search results.
- the contextual ranking based on a dedicated context folder on the computer of the user can be used without synonym expansion in accordance with the invention.
- inventive search method can be used also to search data stored in an application, for example an email application in some embodiments of the invention.
- the contextual ranking based on a dedicated context folder on the computer of the user can be used without synonym expansion to search for data in applications in accordance with the invention in some embodiments.
- the contextual ranking based on a dedicated context folder on the computer of the user or in the network is inventive on its own right and may be used to rank search results, but also synonyms in accordance with the invention.
- the synonyms of the synonym expansion could be ranked by the contextual ranking based on contents in a dedicated context folder, with only the highest ranking synonyms being accepted to at least one search query in accordance with the invention.
- any contextual ranking methods explained in association with phase 130 may be used in or by the arrangement 20 .
- any features, phases or parts of the arrangement 20 can be freely permuted and combined with embodiments 10 , 30 , 40 in accordance with the invention.
- FIG. 3 shows an embodiment of the inventive search method deployed to translation from one language to another language.
- Language translation is analogous to search in the sense that a corresponding word in another language is being searched when a translation is performed.
- a user, software, Internet browser, or search engine software enters a webpage with text in a foreign language.
- the process of entering a webpage of a foreign language can be made invisible to the user in some embodiments, i.e. the webpage entered is not shown to the user requesting it in this embodiment.
- phase 310 text is recognised by the inventive computation system from the webpage that was entered in phase 300 .
- text is recognised in phase 310 in any language, even if it has a different character set to European Alphabets used here, such as Chinese, Japanese, Korean, Thai, Cyrillic, Greek, Indonesian, any Indian alphabet such as Urdu or Hindi or the like.
- languages that are not used anymore such as Latin or Egyptian hieroglyphs can be recognised in some embodiments in accordance with the invention.
- textual data is not stored in the form of text, but rather the text is in an image, which image is on the webpage. It is in accordance with the invention to recognise the text from the image data by character recognition, for example OCR (Object Character Recognition), or similar recognition algorithm.
- character recognition for example OCR (Object Character Recognition), or similar recognition algorithm.
- the text in the foreign language is translated to synonym expanded text sets in the preferred language of the user.
- the webpage would have contained a sentence “How to ship a box” and the user would use Finnish as his preferred language, be it native language or best language or any language of choice
- the translated synonym expanded sets could include for example:
- the inventive translation system defines a text set as the text between commas, a comma and a full stop, between full stops, or a block of words of some length, e.g. 10 words.
- the synonym expanded text set results are ranked based on contextual relevance. If there is only one text set on the webpage, the contextually most relevant text set is shown to the user as the translation in his preferred or native language. In some embodiments the translation is shown in a different application and/or Internet browser window on the computer of the user. In some embodiments the Internet browser shows the translated text in the same place on the webpage, i.e. thereby producing a ‘virtual webpage’ in a different language that otherwise looks the same as the original webpage, except that it is in a different language.
- the contextual data needed to perform the contextual ranking can be obtained similarly to other embodiments explained in this application.
- phase 340 the combinations of translated text set results are ranked based on their contextual relevance.
- the contextual relevance can be estimated for example based on how well the different translated text sets fit together, i.e. does the translation make sense in combination?
- the contextual ranking of translations can be performed by any methods discussed in association with phase 130 .
- a sentence is translated into several synonym expanded translation results.
- the synonym expanded translation results will then be compared in similarity to the other sentences in the text.
- a paragraph of manmade text will feature antecedents and the like that cause the same or very similar terms to be carried out all through the text.
- the translation result that is contextually most similar to at least one surrounding sentence, or the words in a translation result that are most similar to surrounding words will be chosen as the translation in preferred embodiments of the invention.
- a translated word can be chosen based on its contextual similarity to only at least one other single word.
- search terms and/or context terms may have different weights, that may be assigned by the user, by the system, be updated from the network, or be pre-programmed into the computer in accordance with the invention.
- context terms may be functional, rather than only plain words or numbers, for example context terms can be acquired in functional form such as the date, time, location, i.e. context terms can be functional or conditional beyond their plain meaning.
- translated words can be compared against context words, context word ontologies and/or context ontologies in accordance with the invention.
- translated word ontologies may be compared against context words, context word ontologies, and/or context ontologies.
- translated word groups and/or sentences can be compared against context words, context word ontologies, and/or context ontologies in accordance with the invention.
- the translation result that has the greatest similarity with the context words, context word ontologies, and/or context ontologies will be ranked the highest in the contextual ranking and chosen as the translation in accordance with the invention.
- the translation result that has the highest semantic path weight to at least one context word and/or context words is typically selected as the translation by the contextual ranking in some embodiments of the invention.
- different grammar rules could be implemented by path weights: If the sentence has a subject followed by a verb which appears to be the predicate of the sentence, it is likely that at least one noun that follows will be the object of the verb. The nouns that are in the state of an object will thus carry higher path weights in that sentence.
- grammar rules can be deployed by the other contextual ranking techniques in accordance with the invention also.
- basic grammar rules are arranged to be programmed into the ontology of words.
- Contextual relevance can also be based on data in the context folder.
- a Finnish user looking for fishing gear from typically Swedish shops that manufactures most of the desired fishing equipment can place for example Finnish Fishing Permit documentation into the context folder, as an image, text or document file, for example.
- the Internet Browser and/or the Search program will sense that the overall context now is fishing by analyzing the data in the context folder.
- the Finnish user starts to browse the web pages in Swedish explaining product descriptions of lures, nets, rods etc. in Swedish the internet browser will translate the Swedish into Finnish and the Finnish customer can read the translated ‘virtual’ webpage from his browser.
- the combination of text sets with the highest contextual rank is displayed to the user as the translation in phase 350 on the webpage, in a separate window, and/or on the webpage in the same place where the original text was, i.e. producing a virtual webpage in the translated language.
- an Internet browser and/or search engine that translates web pages to a defined language, and shows the foreign web pages in the desired language is in accordance with the invention.
- the invention can be used to translate any document from one language to the next; the use of the invention is not restricted to web browsing.
- FIG. 4 displays the search-translation embodiment 40 in accordance with the invention, which is better suited to different categories of input.
- a user speaks at least one word in a baseline language, which typically is his preferred language, native language or the language he knows best.
- the baseline language can be any language in accordance with the invention.
- speech recognition identifies the spoken words and converts them to text.
- speech can be supplemented and/or replaced by image data, from which (image data) text is extracted by pattern and/or character recognition.
- audiovisual data can be distilled into text. This text can also be translated to another language in accordance with the invention. For example a distilled text in natural language English can be translated to French natural language in accordance with the invention.
- the text extracted from audiovisual data is synonym expanded in the desired target language as explained elsewhere in this application.
- Different text sets are expanded with different synonym expansions.
- the text that is derived by speech recognition could be sorted separately, based on what voice speaks the words and other factors.
- the computer system can identify who does the speaking.
- the speech recognised words with a certain voice can be recognised in blocks of 10 words or more or less, or for example the word groups in between pauses can be recognised as distinct text sets in accordance with the invention.
- Pattern recognised words from image data can be identified as individual words or word set that can be synonym expanded as a text set for example. It is also in accordance with the invention to combine words extracted from different types of audiovisual data.
- a recognised pattern/character will have synonyms in accordance with the invention, possibly many synonyms.
- a pattern shaped like a “C” could be the letter “C” in the European alphabet but it could also be a half part of a stadium or a velodrome in a picture. “half velodrome”, “half stadium” and “C” would be synonyms in this case.
- This is the translation from character based language to a natural language in accordance with the invention. It is in accordance with the invention that the translation from audiovisual data may not use contextual ranking or synonym expansion in some embodiments as it is inventive in its own right.
- phase 430 the translated text sets, or those sets that were originally in the baseline language and did not need to be translated, are ranked based on contextual relevance.
- the combinations of translated text sets are then ranked based on contextual relevance in phase 440 .
- the combination with the highest contextual rank is reproduced to the user as the translation and/or extraction of text.
- the reproduction can take place by reproducing the words as voice from a loudspeaker, or as images and/or footage on a screen and/or video that combines the two in the baseline language of the user in phase 450 .
- the translation feature between natural languages e.g. (translating English traffic signs into English text then speech).
- Useful embodiment being for example a car camera and a computer that reads out traffic signs to the driver as they are videoed and/or photographed.
- the user is possible and in accordance with the invention for the user to operate only the text extraction feature, i.e. translating patterns, characters, image, voice, into textual words.
- the extracted textual words that correspond to the data are searched by the synonym expansion—contextual ranking inventive technique.
- the invention therefore makes it possible to extract a textual narrative from audiovisual data in the Internet or in the surrounding world around us that is video recorded, and provides for the textual translation of that data.
- One simple embodiment of the invention could be realised for example into a portable device that translates English speech to a French loudspeaker broadcast using the synonym expand—context rank technique.
- the invention has been explained above with reference to the aforementioned embodiments and several commercial and industrial advantages have been demonstrated.
- the methods and arrangements of the invention allow people to find relevant documents, based on the inventive synonym expansion-contextual ranking on that synonym expanded word set -technique.
- the same technique provides for more accurate translations from one natural language to another language from the Internet or documents, or from a combination of image language and speech to a natural language, either in textual or spoken form.
Abstract
-
- receiving a search query including at least one search term, deriving at least one synonym for at least one search term, expanding the received search query with the at least one synonym, searching at least one document using the expanded search query, retrieving the search results obtained with the expanded query, ranking the search results based on context of occurrence of at least one search term. The best mode of the invention is considered to be an Internet search engine that delivers better search results.
Description
- The invention relates to data searching and language translation. In particular, the invention relates to searching documents from the Internet or databases. Even further, the invention also relates to translating words in documents, WebPages, images or speech from one language to the next.
- Searching from the Internet which spans the globe is a common experience for most office workers of today. Quite often though the search experience is frustrating, you either cannot find the documents you were looking for, or the documents that might be relevant are in a language you cannot understand. This greatly hinders the productivity of the world economy.
- Internet search engines are known in the prior art, as well as machine translation software. One document that describes the situation in the prior art is WO 2009/002864 A2, which is cited here as reference. In this prior art a search query is inputted to a computer, for example of the form “How to ship a box”. In this prior art the terms in the query are synonym expanded. For example the word ship can have synonyms e.g., “boat” and “send”. The expanded synonym set is limited by removing the expanded synonyms that are contextually irrelevant. That is, a synonym of a term in the search query is selected based on a context of occurrence of the term in the received search query in accordance with this prior art. For example in the prior art, search results related to fishing trawlers are probably not relevant to shipping a box and thus these search results are discarded.
- The invention under study is directed towards a system and a method for effectively improving computerised search queries from data and computerised language translations from one language to the next.
- The prior art has serious disadvantages. The inventor now asks: What if the person in the introduction is looking to ship a box of fish? Search results related to fishing trawlers might not be so irrelevant after all.
- The prior art methods for searching and translating data are deficient at least in the perspective that they can make an incorrect isolated decision in the synonym expansion that leads to wrong results in the overall search execution, as the search cannot recover to the desired search results.
- A further object of the invention is to solve the above problem and produce search results and translations of improved quality.
- In this application “synonym” of a word A is construed as a word B that has similar meaning with word A, but is a different word. Spelling and typing errors in a word are also regarded to change the word into a synonym in this application. I.e. “aple” could be a synonym to “apple” in accordance with the invention.
- In this application “context” is construed as the overall and entire logical, physical, and/or philosophical surroundings in which the word is presented or might/should be understood.
- “Synonym expansion” is construed as the expansion of the original search and/or translation query into word sets having a similar meaning. For example a query “Big Aple” could be synonym expanded to:
- “Big Aple”
- “large apple”
- “Big Apple”
- “New York City”
- “NYC”
- “Natural language” means a past, present or future language spoken and/or written such as French, German, Esperanto, English or Chinese.
- In one aspect of the invention the search query is expanded with search term synonyms. Search results are retrieved based on the expanded query. However, valid synonyms are not excluded because they merely appear to be irrelevant. For example in the case of the query: “How to ship a box”, a trawler, albeit a fishing ship, would not be excluded. Now this will probably lead to a larger set of search results. The search results related to fishing trawlers are now still in the synonym expanded search result set.
- In the invention the synonym expanded search results are then ranked based on their contextual relevance. The contextual information that is used to perform the contextual ranking on the synonym expanded search results can be derived from various sources: global known parameters, data available from the computer of the user, recent browsing history of web pages, a statistical analysis of how users in the past have accessed sites listed in the synonym expanded search query, data open in an application on the computer of the user and any other information that characterises the context in which the query is launched. For example, if the user has a browser window open with words “Cod” and/or “Salmon”, both generally known fish, the contextual relevance of the “fishing trawler” is very strong, albeit it is not the closest synonym to ship. Therefore the search results with the fishing trawlers will not only be included in the search results, but will also rank high in the final list of search results that is shown to the user.
- In another use scenario of the invention, the search query “how to ship a box” comes from a computer known to reside in the Arabian Desert in, say Saudi Arabia. The same synonym expanded search result set is retrieved. Now, assume the user just visited FedEx™ website for 15 seconds. Even though the search result that deals with fishing trawlers that also deliver boxes of fish is in the retrieved results it will not rank high based on its contextual relevance. The websites of the Saudi Arabian Post Office come up quite high based on their contextual relevance, as well as any courier service that has a branch there.
- Again, ranking is changed in favour of the fishing trawler search result that deals with boxes and deliveries, if the user has a document on Sauvignon Blanc (the famous grape for white wine compatible with white fish) open on his desktop.
- In the invention no search results are lost by a deficient contextual decision that performs a wrong decision on an isolated search term. This is a significant advantage of the invention.
- Another embodiment of the invention deploys the same aforementioned inventive technique in translating documents from one language to the next. A word to be translated into another language is analogous to a search term in some embodiments of the invention. Let us assume the sentences to be translated into English are for example in Finnish: “Matti Oath kayda kylassa. Saavuttuaan Liisa tarjosi kahvit tuvassa.” Synonym expanded translations may be of the following type:
-
- 1) “Matti decided to visit. On arrival Liisa offered coffee in the living room”
- 2) “Matti decided to drop by. On arrival Liisa offered coffee in the house.”
- 3) “Matti decided to visit a village. On arrival Liisa offered coffee in the living room”
- 4) “Matti determined to drop by in a village. On arrival Liisa treated to coffee in a house”
- And so on.
- Now the list of search results will be very long as different permutations and combinations of different expanded synonyms combined in a different way amount to a sizable list of search results. In accordance with the invention, a search result list with validly expanded synonyms that is broad is desirable. This quite extended list of synonym expanded search results will now be subject to context based ranking.
- The context based ranking can be based on general statistical trends observed from other documents in the same language, words surrounding the text to be translated, data available from the computer of the user, recent browsing history of WebPages, and any other information. Now, if the context somehow reveals that for example the events take place in Helsinki, the translations 1-2 are ranked higher, because generally Finnish speakers would not refer to Helsinki as a village as it is the biggest city and the capital city in the country. Likewise if the context reveals that Matti is unknown to Liisa, the translation 4 would rank contextually higher because “käydä kylässä” implies a social nature to the visit, and if Matti and Liisa do not know each other, it is more contextually relevant to assume that they meet in a village and have coffee that is merely offered by Liisa (e.g. in a house that is a café) rather than on Matti's personal visit in the living room of Liisa.
- Thus in addition to determining the best matching document to a search, the invention can be used to produce the best matching translation. As long as outright wrong decisions in the synonym expansion are avoided, and there is enough contextual information available the outcome will be a considerably improved translation.
- Some or all of the aforementioned advantages of the invention are accrued with a browser and a search engine that translates the foreign pages using the synonym expansion-context ranking technique and displays the most relevant search results that are also retrieved with the synonym expansion-context ranking technique. The invention therefore guarantees the widest searches in scope, and the most logically relevant search results are displayed first.
- A computer implemented method in accordance with the invention comprises at least one computer and is characterised by the following steps:
-
- receiving a search query comprising at least one search term,
- deriving at least one synonym for at least one search term,
- expanding the received search query with the at least one synonym,
- searching at least one document using the said expanded search query,
- retrieving the search results obtained with the said expanded query,
- ranking the said synonym expanded search results based on context of occurrence of at least one search term.
- An arrangement in accordance with the invention comprises at least one computer and is characterised in that:
-
- a search query comprising at least one search term is arranged to be received,
- at least one synonym for at least one search term is arranged to be derived,
- the received search query is arranged to be expanded with the at least one synonym,
- at least one document is arranged to be searched using the said expanded search query,
- search results obtained with the said expanded query are arranged to be retrieved,
- the said synonym expanded search results are arranged to be ranked based on context of occurrence of at least one search term.
- A memory unit in accordance with the invention comprises software capable of executing the computer implemented search method and/or operating the search arrangement described above.
- A computer implemented method in accordance with the invention for translating text from one language to another language comprising at least one computer is characterised by the following steps:
-
- receiving a translation query comprising at least one term to be translated,
- deriving at least one synonym for at least one translation term,
- expanding the received translation query with the at least one synonym,
- retrieving the translation results obtained with the said expanded translation query,
- ranking the said synonym expanded translation results based on context of occurrence of at least one translation term.
- A computer implemented arrangement in accordance with the invention for translating text from one language to another language comprising at least one computer is characterised in that:
-
- at least one translation query comprising at least one term to be translated is arranged to be received,
- at least one synonym for at least one translation term is arranged to be derived,
- the received translation query is arranged to be expanded with the at least one synonym,
- the translation results obtained with the said expanded translation query are arranged to be retrieved,
- the said synonym expanded translation results are arranged to be ranked based on context of occurrence of at least one translation term.
- A memory unit in accordance with the invention comprises software capable of executing the computer implemented translation method and/or operating the translation arrangement described above.
- A memory unit in accordance with the invention comprises software, characterised in that, the software is arranged to read data from a dedicated part of a file system to extract contextual data for search and/or translation.
- A search engine software in accordance with the invention is characterised in that, the search engine is arranged to rank synonym expanded search results based on context data derived from user's Internet browser, operating system, file system, database software and/or applications.
- Operating system software in accordance with the invention is characterised in that, the operating system is arranged to rank synonym expanded search results based on context data derived from user's Internet browser, operating system, file system, database software and/or applications.
- A software application in accordance with the invention is characterised in that, the software application is arranged to rank synonym expanded search results based on context data derived from user's Internet browser, operating system, file system, database software and/or applications.
- Database software in accordance with the invention is characterised in that, the database software is arranged to rank synonym expanded search results based on context data derived from user's Internet browser, operating system, file system, database software and/or applications.
- In addition and with reference to the aforementioned advantage accruing embodiments, the best mode of the invention is considered to be an Internet search engine that delivers better search results.
- In the following the invention will be described in greater detail with reference to exemplary embodiments in accordance with the accompanying drawings, in which
-
FIG. 1 demonstrates an embodiment of the inventive search method. -
FIG. 2 demonstrates anembodiment 20 of the arrangement used to perform the searches and/or translations in accordance with the invention. -
FIG. 3 demonstrates anembodiment 30 of the translation method in accordance with the invention. -
FIG. 4 demonstrates anembodiment 40 of the translation method using audiovisual input in accordance with the invention. - Some of the embodiments are described in the dependent claims.
-
FIG. 1 shows an optimum computerised search method as a flow diagram in accordance with the invention. Inphase 100 the user inputs a search query. The input can take place by typing text for example with a keyboard or other computer peripheral, by speech when preferably speech recognition is conducted by the computer to extract the query terms, and/or with an image when preferably pattern recognition is conducted by the computer to extract the query terms. For example the user can type:“how to ship a box” by a keyboard, or he can say “how to ship a box” and the speech recognition recognises the words and uses them as search terms, and/or he can take an image with a camera or a mobile phone, of a ship with a box on board and the pattern recognition software recognises the ship and the box and extracts these as search terms, and/or the user takes an image of a sign reading “how to ship a box” and the computer uses pattern and/or character recognition, such as Object Character Recognition (OCR) to identify the characters and thus the words in the sign in accordance with the invention. It is also in accordance with the invention that these input methods can be used in a mix: the user can for example write “how to” and take an image of the box on the ship and the written words “how to” and the pattern recognised words “ship” and “box” are combined to form the search query in accordance with the invention. Likewise speech and text and image and speech, or in fact all three forms of input, can be used in a mix in accordance with the invention. The computer will arrange the terms in the order in which they arrive or are recognised, or the user can specify the order of the search terms, and/or the computer can suggest entire search queries to the user based on the recognised search terms in accordance with the invention. - In
phase 110 the search query is expanded with search term synonyms. “how to” could be recognised as a phrase and expanded with a synonym “in what way”, the “ship” could be expanded with synonyms such as “boat” and “send”, the “box” could be expanded with a synonym “square” and “parcel”. In addition to words that have a synonymous meaning, the synonym expansion will expand obvious writing versions of different words: for example “centre” (UK English) would be expanded to “center” (US English) in accordance with the invention. The inventive computing system can correct spelling errors such as “boks”, “sip”, “hou” that might have been caused by phonetic miss-spellings in the synonym expansion, or separately before the synonym expansion. In addition the inventive computing system can correct spelling errors based on typographical errors, for example in a QWERTY keyboard potential errors could be “boz”, “shio”, “hoq” in accordance with the invention. Likewise the computer system can expand different states of a noun, and/or correct words from different writing forms to standard forms. For example the Finnish language is notorious for having many states for a noun. A bay=lahti will be written landella=“on the bay” or “landessa=in the bay”. In some embodiments of the invention the synonym expansion will also expand the search terms to different linguistic states. In the example above a search term “landella” would be expanded to also include “landessa”, “lahti” and possibly other states of the noun in accordance with the invention. - In some embodiments of the invention some terms from the search query can be omitted based on context of occurrence of a search term in order to limit the search terms in accordance with practical limitations, such as available computing power.
- In
phase 120 the search results are retrieved based on the expanded query. The more expanded the query, the greater the number of search results. In some embodiments of the invention the expanded search query is divided into several different search queries that are executed by the same or different computers in parallel or in series, and the search results of each separate query are combined to form the overall search results for the expanded search query. In some embodiments of the invention it is especially preferable that redundancies are eliminated from the combined list of search results. - In
phase 130 the search results are ranked contextually. It is important in the invention that the ranking is based on the synonym expanded results set. The synonym expansion increases the reach of the search and delivers more search results, some of which may be or are probably even undesirable to the user. The combined synergistic effect of the contextual ranking imposed on the synonym expanded search result set is important in accordance with the invention. A search result that would have been omitted by contextual rules used in the synonym expansion will still be in the search result set in accordance with the invention, all though it is likely to be proverbially buried underneath hundreds if not millions of search results (depending on how expansive the synonym expansion is) that also match the expanded search query. Now, when all these results are ranked contextually, a search result that might have been on the millionth seed number in the synonym expanded search result will pop up to the first place, provided it has the best contextual match. - Contextual data, i.e. data for defining the context in which the synonym expanded search results are ranked may be deduced from previous browsing behaviour of the user, location of the user, time, date, nationality of the user, data on the computer of the user, general population browsing behaviour, general statistics and frequency analysis of words, and data that is open in an application on the computer of the user.
- Using previous web pages of the user to define the context is preferable if the user has a consistent browsing behaviour, but many people don't. In the case of a person that checks the company intranet, and then reads a newspaper online, looks up an auction site or classifieds it might take a long browsing history to determine meaningful context data. One preferable embodiment of the invention is the use of an open application on the computer to define the context data used in ranking. For example if the user has a word processor open with a special term, say “NGC 2547”, it will immediately suggest that a search term “star” actually means a stellar object in space rather than people in Hollywood or Bollywood. In this particular search the documents associated with celebrities will be ranked to the last places in the context ranked synonym expanded search result list. Documents dealing with stars in space will rank higher, documents dealing with stars that have a New General Catalogue (NGC) number will rank higher still, whereas articles on stars in cluster NGC 2547 will rank higher still. An article on a Hollywood actress claimed as a rising star, but with an explicit interest in New General Catalogue numbers would of course be dragged higher on the list than other celebrity articles, because that article would also have contextual merit from the point of view of the search.
- In more detail, the context based ranking can be performed by several different methods and means in accordance with the invention. By context of occurrence we mean the context in which at least one search term occurs, i.e. the context of occurrence of at least one search term. For example when a user types a search query:“flights to London” in a search engine or program, the context of occurrence of these search terms comprise data in other applications on the computer of the user when he made the search query, and any data that is associated to the context in which the search terms now occur. If the user has an email message open with “visit to London 1-3rd March”, the “flights to London” search terms occur in a context of the person going to London between 1-3rd March, and therefore any last minute 3-day 2-night packages for 1-3rd of March will be ranked contextually higher. The context of occurrence thus means the context data on the user's computer, and/or to which the user's computer has access to as the search terms occur, i.e. are provided to a search engine and/or program.
- In one embodiment of the invention, the system picks words or numbers from the context, such as open applications, emails, messages, temporary folder, Desktop, calendar, clock, GPS chip, operating system, previously browsed WebPages and/or the like. The picking of the words, numbers or other limiting factors to define the context can be done randomly, or by some algorithm, or even by the user in some embodiments, in accordance with the invention. The system now has the search terms that were synonym expanded, typically a huge number of search results, and terms that define the context.
- Now the search results can be rearranged into an order where the document that is closest in similarity to the selected context terms is placed first.
- In one simple embodiment of the invention the received search results are simply ranked in the order of common hits with the context terms. Suppose that in our earlier example with the search query:“How to ship a box” the user has a webpage open that is titled “Shipping fish fast to sushi restaurants”, and suppose for example that the system knows we are in Vigo, Spain, probably the largest fishing port in the world. The context terms “fish”, “fast”, “sushi” and “restaurant” are selected in accordance with the invention. Now the document that tells about a fishing trawler that sends sushi filets by direct delivery in boxes of ice after it docks in Vigo is probably proverbially buried on the millionth seed or similar in the search results. However, this document will now feature the highest frequency of the contextual terms, and will be rearranged to first place in contextual ranking. In its simplest embodiment context based ranking of the invention is running a new search on the original search results based only on acquired context words.
- It should be understood that different search terms and/or context terms may have different weights, that may be assigned by the user, by the system, be updated from the network, or be pre-programmed into the computer in accordance with the invention.
- It should also be understood that the context terms may be functional, rather than only plain words or numbers, for example context terms can be acquired in functional form such as the date, time, location, and i.e. context terms can be functional or conditional beyond their plain meaning.
- In one embodiment contextual ranking is performed based on context ontologies. By ontology we mean: ontology ((computer science) a rigorous and exhaustive organization of some knowledge domain that is usually hierarchical and contains all the relevant entities and their relations source: Wordnet).
- A word may have an ontology in accordance with the invention, for example the word “apple” will have an ontology of it being an example of a fruit in general as well as being the genus to species varieties like “Calville Blanc”=the French culinary apple, or “Granny Smith (=the Australian green apple with a strong taste) or Red Delicious (=the bright red apple that has a broad consumer appeal especially in the US), and the popular personal computer from California. In addition to the above the word apple might have a conditional ontology, e.g. meaning New York City when having the prefix “Big” and being in capital letters “Big Apple”.
- Ontology of a word can be typically stored in a file that is read by at least one search engine program. A search engine or the like can have data files for all words in all common languages that define ontologies of those words on one computer or distributed over a network of computers. Alternatively, those files or a part of those files could be stored in the subscriber terminal, like the terminal computer the user is using, such as a PC or Apple Mac computer.
- It should be noted that in accordance with the invention variable weights for different parts may exist within a word ontology. With reflection to earlier example, “Granny Smith” could have a higher relative weight in the “apple” ontology than “Calville Blanc” in some embodiments of the invention, because a priori one could assume more users searching for apples will be searching for Granny Smiths, because it is better known. The choice can of course be made the other way around providing a higher relative weight for “Calville Blanc” in some embodiments of the invention. As a general rule, the further the connection is from the actual word itself, the lower the relative weight. For the word “apple” the relative weight in the ontology for the “apple” itself will be 100% in some embodiments, whereas for “Calville Blanc” and “Granny Smith” it could be 1% and 2% respectively or vice versa in accordance with the invention.
- As said the words that were chosen to define the context may have ontologies, and in fact in some embodiments those words with well defined ontologies are typically chosen as context words over words that do not have an ontology, or have an ontology that is poorly defined or less used or the like.
- In some embodiments the search results are searched with the ontologies of the context terms and those search result documents that are most similar to the ontologies of the context terms are prioritized in the ranking, i.e. ranked before less relevant search result documents.
- In some embodiments of the invention some or all selected context terms may be synonym expanded too. The retrieved search results are then searched again with the synonym expanded context terms, and the search results that are most similar or have the largest number of hits of synonym expanded context terms are ranked first in the contextual ranking in some embodiments.
- It should be understood that different search terms, context terms and/or ontologies may have different weights, which may be assigned by the user, by the system, be updated from the network, or be pre-programmed into the computer in accordance with the invention.
- It should also be understood that the context terms and/or ontologies may be functional, rather than only plain words or numbers, for example context terms can be acquired in functional form such as the date, time, location, i.e. context terms can be functional beyond their plain meaning. For example consider the context term “time: 12:00”. If the time is +/−15 min from 12:00 in human terms time: 12:00 will very strongly equate with “Now”. Quite clearly in this situation “now” would be in the ontology of the context term time: 12:00, but only functionally and/or conditionally so, if it were 8.00 pm, “now” might not be in the ontology of the context term.
- In more elaborate embodiments, a user or the user terminal may have several contexts that have different ontologies such as for example Email context, Browsing context, and/or Publication context.
- For example an email context will contain other information besides content, such as the time, sender and/or receiver, their identity. Many times the actual content in the email is not remembered by people who were corresponding; it is rather the sender and the time when it was sent and/or received, its importance classification or other parameters that define the context ontology of the email context.
- Browsing context can be read from the browser cache in some embodiments of the invention. For example a webpage that was accessed at a certain time should be interpreted in the context of other WebPages that were accessed at the same time, or with the same or similar query string. The browser cache when read in this way can be used to determine the browsing context ontology in accordance with the invention.
- Publication context on the other hand relates to what the user or user terminal decides to publish. Many users receive a lot of crap that is uninteresting to them, but people rarely post a webpage or send electronic mail or FTP files about issues that are completely uninteresting to themselves, those issues may of course be uninteresting to others. The data that the user or the user terminal publishes on the Internet or to a select group in the form of messaging can be used to define the ontology of the publication context.
- When the context ontology is specified a user can assign a weight to particular regions of the ontology, or to entire ontologies. For example in the email context ontology all sent emails or terms or data that appear in sent emails could be given a user assigned relative weight of 100, when terms and/or data in received emails could be given a weight of 1. The user could set the aforementioned relative weights himself or they could be pre-programmed or set from an Internet and/or network server by a third party. Quite clearly in the publication context the weights of some elements in the ontology could be a function of publication frequency for example. A webpage that the user is updating all the time is likely to have a weight of 87 if it was updated 87 times in comparison to a webpage that was only updated once (=relative weight 1) for example, in some embodiments of the invention.
- The different context and context ontologies may also have different weights, which can be user assigned, assigned from a network server and/or pre-programmed. An attorney that mainly uses email for his self expression will have an email context ontology that will have a higher relevant weight to browsing or publication ontology. A media marketing person will by contrast have a publication context ontology outweighing email context ontology for example, as the most important things on her mind are the things she publishes by way of WebPages or press releases for example.
- In its simplest embodiment, a certain context, take email context for example, could have one or a collection of context terms that would be the same all the time. If the user is a travel agent selling British Airways flights “British Airways” and “Eurotraveller” are pretty constant context terms, because nearly all business is associated with booking Eurotraveller economy seats to customers. If the ontology of these constant context terms would be constant all the time, this would in effect mean that the ontology of the context, i.e. email context in this case would also be constant.
- Of course having the same context terms or context ontology is not desirable for versatile users, which account for the majority of all users. In order to adapt to the changing wishes of users, the different contexts need to change too, and choose those context terms that appear to reflect the current user activity and intentions the best. If the user for example has an email program open where he is writing an email to be sent, it is a good bet for the system to acquire at least some context terms from the email the user is currently writing.
- The aforementioned of course leads to the embodiment of dynamical ontology derivation for a context, when a context has different context terms at different times or different situations, which is in accordance with the invention. In some embodiments of the invention a number of contexts are being up to date as the user uses the computer system, and the context with the best fit to user activity is preferably chosen all the time, the context terms that best reflect user activity are preferably chosen all the time, and the ontology of at least one context term and/or context is preferably up to date to reflect the user activity.
- It should be noted that different contexts may be combined in some embodiments to yield the context definition with different relative weights. For example a user having an email message open and browsing the Internet could be in a 70
% email context 30% browsing context in accordance with the invention. In this way different contexts may be used in combination in accordance with the invention. - It should be noted that different context terms within a context ontology may have different relative weights in accordance with the invention. For example frequency of a term could add weight to a context term. If the user has an email message open that recites “flight” 5 times, the context term “flight” should have a higher relative weight than some less recited context term, on its own as an independent context term without a ontology, or in a context term ontology.
- In summary different relative weights can be applied to search terms, context terms, contexts, search term ontologies, context term ontologies, context ontologies and/or individual elements within any ontology and in this way any criteria for contextual ranking can be formed in accordance with the invention in any embodiments of the invention.
- Context based ranking based on context ontologies can be visualised with the assistance of RDF (Resource Description Framework)-metadata graphs. One example of this is in
FIG. 1 of “Desktop Search—How Contextual Information Influences Search Results and Rankings” by Paiu and Nejdl which is cited here and incorporated as reference to this application. This RDF-metadata graph shows a context ontology which specifies context metadata from emails, files, web pages and publications with relations among them. The relations are described in “Activity Based Metadata for Semantic Desktop Search”—Proceedings of the 2nd European Semantic Web Conference, Heraklion, Greece, May 2005 which is also cited here as reference and incorporated into this application. The context ontology elements and the relations between them can be used to search the synonym expanded search results, and prioritise these search results by means of contextual ranking. The relations and metadata within the ontology are typically derived from user activity in some embodiments, for example from when, to whom, and about what subject the user is emailing messages and with what attachment files. Typically those synonym expanded search results that rate highest in similarity to the context ontology that contains the metadata and/or relations are placed with the best rank in the contextual ranking in accordance with the invention. - Another embodiment with which to visualise context based ranking is with authority transfer annotations using schema graphs which is also explained in “Desktop Search—How Contextual Information Influences Search Results and Rankings” by Paiu and Nejdl. Some metadata is more important than other metadata, for example if the sender field contains the boss of the employee, this would be more authorative metadata than the sender fields of the endless spam-emails most of us receive. In
FIG. 2 of the aforementioned document it is shown how authority is transferred within the context ontology, for example the attachment that was attached to an email that the boss sent is more authorative than if the file would have arrived by spam email. The more authorative an element in the context ontology is, the higher relative weight it will have in some embodiments of the invention. When the synonym expanded search results are ranked contextually by ranking them against the context ontology, those synonym expanded search results that have similarities to more important and/or authorative parts of the context ontology will be ranked first in accordance with the invention. - Furthermore, the document “Context-Aware Semantic Association Ranking by Aleman-Meza, Halaschek, Arpinar and Sheth describes techniques for context based ranking in detail, and is included here as reference to this application. It is in accordance with the invention that the contextual ranking is based on semantic association ranking. In this embodiment context ontology is defined as explained earlier. Now, the synonym expanded search results are compared against the context ontology by seeing which search results are most closely semantically associated with the context and/or context ontology. Semantic association between a context and a synonym expanded search result is measured by overall path weight. The overall path weigh Wp=+User defined weight+Context weight+Subsumption weight (class specialization in a path)+Trust weight (of the path between context & search result), as explained in Context-Aware Semantic Association Ranking by Aleman-Meza, Halaschek, Arpinar and Sheth. It is in accordance with the invention that the synonym expanded search results are contextually ranked so, that those synonym expanded search results that are most closely semantically associated (and/or have the highest overall path weight) with the specified context and/or context ontology are ranked first. Quite clearly it is in accordance with the invention that the semantic association and/or the overall path length can be composed from any permutation and/or combination of the aforementioned components. For example user defined weights could be left out in some embodiments of the invention.
- As explained before contextual ranking of
phase 130 may be performed with different techniques. It is in accordance with the invention that these techniques can be combined and/or permuted in accordance with the invention. - In
phase 140 the contextually ranked synonym expanded search results are displayed to the user in the order of contextual relevance. Typically the said list is displayed in the monitor and the user can move to different positions in the list and select any search result for more detailed viewing. In some embodiments some portion of the final search results might be discarded, for example the contextually least relevant search results. Discarding search results now is not as dangerous as based on a contextual analysis of a search term earlier. The search result to be discarded has been considered vis-a-vis its objective content of the substantive meaning of the search terms as well as its context in comparison to the context of the query as a whole or in part in some embodiments. - In fact it could be stated that as long as the synonym expansion is expansive enough and the user provides enough information to define the context the invention will find the best match to the query of the user in the searched documents. It is another matter of course, if the desired document is available, but by connecting the invention to the Internet users will find their desired information better than ever before.
- It should be noted that any features, phases or parts of the
method 10 can be freely permuted and combined withembodiments -
FIG. 2 shows an exemplary arrangement for conducting inventive searches. Theembodiment 20 refers more particularly to Internet searches, but the invention can of course be used with any data searches from any number of documents or databases. The user will use thecomputer 210 with thekeyboard 220 and/or a peripheral like amouse 200 and read and look at results from thescreen 230. If the search query is directed to at least one database or document on thecomputer 210 the inventive search method can be operated “off-line” i.e. internally in the computer and the search results are shown on thescreen 230. In some embodiments the search software needed to carry out the searches with the inventive method resides on theserver 240 only and the documents oncomputer 210 need to be searched with the inventive search method. In this embodiment theserver 240 searches the hard drives, memory, files or databases oncomputer 210. - In many embodiments part of the inventive system can be distributed to several servers, several client computers, and/or so that some parts of the system are distributed to at least one client computer, and some parts that are different or the same are distributed to a server computer.
- The most typical use embodiment of the invention is normal Internet search. A search query is forwarded from the computer of the user to a server of an Internet Search Engine. Exemplary prior art search engines are currently known to operate by various names such as Google™, Bing™, Yahoo™, Lycos™ and the like. The
server 240 receives the search query and conducts the inventive Internet search. In some embodiments, especially those where the synonym expansion leads to a very large number of synonym expanded search queries, it is preferable for the server to execute the search queries in parallel or in series as separate search queries and collect the total list of the search results from the searches with the synonym expanded search queries. In this embodiment it is preferable to eliminate redundancies from the total list, (i.e. the sum list of synonym expanded search results) prior to the contextual ranking, as the redundancy elimination leaves less search results to be ranked and saves processing resources. - The
server 240 is typically arranged to search the Internet or databases but it may also search any files, documents, and/or data that it has access to in accordance with the invention. It is in accordance with the invention that there are more than oneserver 240 that share the processing of the search query, or theserver 240 may relay the search query to another server in accordance with the invention. - It is in accordance with the invention that the
computer 210,screen 230,keyboard 220 and/ormouse 200 can be replaced by a mobile phone, laptop or any portable computing device, or in fact any device that can receive a search query, irrespective of whether the computational resources reside on that device and/or theserver 240. It is in accordance with the invention that the search queries can be inputted by speech or images, in which case the computer could be replaced merely by a microphone, or a normal telephone wireless or wire line to receive the search query. In the case of image input the computer features, or is replaced by a camera used to take the images. In some embodiments it would be preferable to receive digital images as input for the inventive search, but naturally film images can be digitised and used by the inventive computerised search in accordance with the invention. - All communications connections between any computers or devices whether client computers, dumb terminals or server devices can be arranged as wireless connections, such as radio/microwave/optical, for example GSM, CDMA, WIFI, free space optics (FSO), and/or fixed communications connections such as data cable, POTS, and/or optical network e.g. SONET/SDH/Ethernet. Quite clearly the inventive method can be executed in an arrangement that comprises only one computer, several computers or a computer network.
- The contextual data needed to perform the contextual ranking of the synonym expanded search results can be obtained in a variety of ways: the
server 240 may observe the previous searches made by the user, the Internet and/or database and/or document browsing behaviour of the user and deduce the context based on the occurrence of words in that document in some embodiments. - The user may specify a context by text entry or multiple choice to the computer in some embodiments. There may be data on the
computer 210 of the user that is used to define the context data in some embodiments. The data on the user's computer can be used to define context data in a multitude of ways: It is possible to use all data on thecomputer 210 to define the context but this requires a great deal of data processing. In some embodiments of the invention the user has a dedicated folder that contains documents that relate to the area of interest to the user. The documents only in this folder are read to define the context data in some embodiments. For example, the user could have a scanned fishing permit in his context folder on the computer. Whilst he would input “how to ship a box” the context would be “fishing” and the documents related to fishing trawlers shipping boxes of fish would rank high in the final search results. - Indeed in some embodiments the contextual ranking based on a dedicated context folder on the computer of the user can be used without synonym expansion in accordance with the invention.
- It should be noted that the inventive search method can be used also to search data stored in an application, for example an email application in some embodiments of the invention. The contextual ranking based on a dedicated context folder on the computer of the user can be used without synonym expansion to search for data in applications in accordance with the invention in some embodiments. The contextual ranking based on a dedicated context folder on the computer of the user or in the network is inventive on its own right and may be used to rank search results, but also synonyms in accordance with the invention. For example the synonyms of the synonym expansion could be ranked by the contextual ranking based on contents in a dedicated context folder, with only the highest ranking synonyms being accepted to at least one search query in accordance with the invention.
- Quite clearly any contextual ranking methods explained in association with
phase 130 may be used in or by thearrangement 20. - It should be noted that any features, phases or parts of the
arrangement 20 can be freely permuted and combined withembodiments -
FIG. 3 shows an embodiment of the inventive search method deployed to translation from one language to another language. Language translation is analogous to search in the sense that a corresponding word in another language is being searched when a translation is performed. Now, in phase 300 a user, software, Internet browser, or search engine software enters a webpage with text in a foreign language. The process of entering a webpage of a foreign language can be made invisible to the user in some embodiments, i.e. the webpage entered is not shown to the user requesting it in this embodiment. - In
phase 310 text is recognised by the inventive computation system from the webpage that was entered inphase 300. Preferably text is recognised inphase 310 in any language, even if it has a different character set to European Alphabets used here, such as Chinese, Japanese, Korean, Thai, Cyrillic, Greek, Indonesian, any Indian alphabet such as Urdu or Hindi or the like. Also languages that are not used anymore, such as Latin or Egyptian hieroglyphs can be recognised in some embodiments in accordance with the invention. - On some web pages textual data is not stored in the form of text, but rather the text is in an image, which image is on the webpage. It is in accordance with the invention to recognise the text from the image data by character recognition, for example OCR (Object Character Recognition), or similar recognition algorithm.
- In
phase 320 the text in the foreign language is translated to synonym expanded text sets in the preferred language of the user. For example if the webpage would have contained a sentence “How to ship a box” and the user would use Finnish as his preferred language, be it native language or best language or any language of choice the translated synonym expanded sets could include for example: - “Kuinka randata laatikko laivalla”,
- “Kuinka lähettää laatikko”,
- “Mina keinoin lähettää paketti”,
- and so on.
- Now, in some embodiments of the invention the inventive translation system defines a text set as the text between commas, a comma and a full stop, between full stops, or a block of words of some length, e.g. 10 words.
- In
phase 330 the synonym expanded text set results are ranked based on contextual relevance. If there is only one text set on the webpage, the contextually most relevant text set is shown to the user as the translation in his preferred or native language. In some embodiments the translation is shown in a different application and/or Internet browser window on the computer of the user. In some embodiments the Internet browser shows the translated text in the same place on the webpage, i.e. thereby producing a ‘virtual webpage’ in a different language that otherwise looks the same as the original webpage, except that it is in a different language. The contextual data needed to perform the contextual ranking can be obtained similarly to other embodiments explained in this application. - If there is more than one text set on the webpage, in
phase 340 the combinations of translated text set results are ranked based on their contextual relevance. The contextual relevance can be estimated for example based on how well the different translated text sets fit together, i.e. does the translation make sense in combination? - The contextual ranking of translations can be performed by any methods discussed in association with
phase 130. For example a sentence is translated into several synonym expanded translation results. The synonym expanded translation results will then be compared in similarity to the other sentences in the text. Typically a paragraph of manmade text will feature antecedents and the like that cause the same or very similar terms to be carried out all through the text. The translation result that is contextually most similar to at least one surrounding sentence, or the words in a translation result that are most similar to surrounding words, will be chosen as the translation in preferred embodiments of the invention. In its simplest embodiment a translated word can be chosen based on its contextual similarity to only at least one other single word. - Quite clearly translation results can be contextually ranked by any of the methods explained in association with
phase 130 in accordance with the invention. As the ranking has been achieved, typically the contextually highest ranking translation will be offered to the user as the final translation in accordance with the invention. - It should be understood that different search terms and/or context terms may have different weights, that may be assigned by the user, by the system, be updated from the network, or be pre-programmed into the computer in accordance with the invention.
- It should also be understood that the context terms may be functional, rather than only plain words or numbers, for example context terms can be acquired in functional form such as the date, time, location, i.e. context terms can be functional or conditional beyond their plain meaning.
- Similarly to as explained in
phase 130 translated words can be compared against context words, context word ontologies and/or context ontologies in accordance with the invention. In some embodiments of the invention translated word ontologies may be compared against context words, context word ontologies, and/or context ontologies. Similarly to embodiments explained inphase 130 translated word groups and/or sentences can be compared against context words, context word ontologies, and/or context ontologies in accordance with the invention. Typically the translation result that has the greatest similarity with the context words, context word ontologies, and/or context ontologies will be ranked the highest in the contextual ranking and chosen as the translation in accordance with the invention. - It is also possible that semantic associations are used similarly as explained in
phase 130. The translation result that has the highest semantic path weight to at least one context word and/or context words is typically selected as the translation by the contextual ranking in some embodiments of the invention. For example, different grammar rules could be implemented by path weights: If the sentence has a subject followed by a verb which appears to be the predicate of the sentence, it is likely that at least one noun that follows will be the object of the verb. The nouns that are in the state of an object will thus carry higher path weights in that sentence. Quite clearly grammar rules can be deployed by the other contextual ranking techniques in accordance with the invention also. In one embodiment of the invention basic grammar rules are arranged to be programmed into the ontology of words. - Contextual relevance can also be based on data in the context folder. I.e. a Finnish user looking for fishing gear from typically Swedish shops that manufactures most of the desired fishing equipment can place for example Finnish Fishing Permit documentation into the context folder, as an image, text or document file, for example. The Internet Browser and/or the Search program will sense that the overall context now is fishing by analyzing the data in the context folder. As the Finnish user starts to browse the web pages in Swedish explaining product descriptions of lures, nets, rods etc. in Swedish the internet browser will translate the Swedish into Finnish and the Finnish customer can read the translated ‘virtual’ webpage from his browser.
- The combination of text sets with the highest contextual rank is displayed to the user as the translation in
phase 350 on the webpage, in a separate window, and/or on the webpage in the same place where the original text was, i.e. producing a virtual webpage in the translated language. - Therefore an Internet browser and/or search engine that translates web pages to a defined language, and shows the foreign web pages in the desired language is in accordance with the invention.
- It is also in accordance with the invention that as the inventive translation software works on the web pages, the total body of web pages in any preferred language will increase. Some of these web pages may be indexed or even stored by search engines in accordance with the invention. These translated web pages can now of course be subjected to searches, normal searches or searches with the inventive search methods discussed earlier in accordance with the invention.
- Quite clearly the invention can be used to translate any document from one language to the next; the use of the invention is not restricted to web browsing.
- It should be noted that any features, phases or parts of the
method 30 can be freely permuted and combined withembodiments -
FIG. 4 displays the search-translation embodiment 40 in accordance with the invention, which is better suited to different categories of input. In phase 400 a user speaks at least one word in a baseline language, which typically is his preferred language, native language or the language he knows best. In fact, the baseline language can be any language in accordance with the invention. Inphase 410 speech recognition identifies the spoken words and converts them to text. In some embodiments speech can be supplemented and/or replaced by image data, from which (image data) text is extracted by pattern and/or character recognition. In some embodiments image and speech data can be also recognised in combination, for example a video camera will record both voice and images, and speech and/or sound recognition can be used to identify words from the audio and pattern recognition and/or character recognition can be used to recognise text in image format, words that have image equivalent, (image of a ball identified as=“ball”). In the aforementioned ways, and other ways, audiovisual data can be distilled into text. This text can also be translated to another language in accordance with the invention. For example a distilled text in natural language English can be translated to French natural language in accordance with the invention. - In
phase 420 the text extracted from audiovisual data is synonym expanded in the desired target language as explained elsewhere in this application. Different text sets are expanded with different synonym expansions. Now, as the input data arrives from a variety of sources, all the text can be synonym expanded as one text set in accordance with the invention, but sometimes it is more preferable to divide the input text into text sets. The text that is derived by speech recognition could be sorted separately, based on what voice speaks the words and other factors. In exemplary embodiments of the invention the computer system can identify who does the speaking. The speech recognised words with a certain voice can be recognised in blocks of 10 words or more or less, or for example the word groups in between pauses can be recognised as distinct text sets in accordance with the invention. Pattern recognised words from image data can be identified as individual words or word set that can be synonym expanded as a text set for example. It is also in accordance with the invention to combine words extracted from different types of audiovisual data. - In fact, the textual extraction of words from audiovisual data by pattern/character recognition is an inventive embodiment in its own right, without the synonym expansion or contextual ranking in accordance with the invention.
- However, it is possible and in accordance with the invention that a recognised pattern/character will have synonyms in accordance with the invention, possibly many synonyms. For example a pattern shaped like a “C” could be the letter “C” in the European alphabet but it could also be a half part of a stadium or a velodrome in a picture. “half velodrome”, “half stadium” and “C” would be synonyms in this case. This is the translation from character based language to a natural language in accordance with the invention. It is in accordance with the invention that the translation from audiovisual data may not use contextual ranking or synonym expansion in some embodiments as it is inventive in its own right. It is also possible in accordance with the invention that only synonym expansion or contextual ranking are used individually in accordance with the invention. However, in some embodiments it is preferable to use both synonym expansion and contextual ranking, and in some embodiments even more preferably so that the contextual ranking will only be conducted to the synonym expanded translation text set candidate.
- In
phase 430 the translated text sets, or those sets that were originally in the baseline language and did not need to be translated, are ranked based on contextual relevance. - The combinations of translated text sets are then ranked based on contextual relevance in
phase 440. - The combination with the highest contextual rank is reproduced to the user as the translation and/or extraction of text. The reproduction can take place by reproducing the words as voice from a loudspeaker, or as images and/or footage on a screen and/or video that combines the two in the baseline language of the user in
phase 450. - In some embodiments of the invention it is possible not to use the translation feature between natural languages e.g. (translating English traffic signs into English text then speech). Useful embodiment being for example a car camera and a computer that reads out traffic signs to the driver as they are videoed and/or photographed. It is possible and in accordance with the invention for the user to operate only the text extraction feature, i.e. translating patterns, characters, image, voice, into textual words. In some embodiments of the invention the extracted textual words that correspond to the data are searched by the synonym expansion—contextual ranking inventive technique.
- The invention therefore makes it possible to extract a textual narrative from audiovisual data in the Internet or in the surrounding world around us that is video recorded, and provides for the textual translation of that data.
- One simple embodiment of the invention could be realised for example into a portable device that translates English speech to a French loudspeaker broadcast using the synonym expand—context rank technique.
- However, the ability to extract data from different media types greatly enhances especially the context based ranking. Say for example the user is with a mobile phone in a Finnish harbour. He just took a picture from the harbour. The pattern recognition will recognise words “sea” and “ship” from the image, which was incidentally saved to the context folder (that acts as the source of context data) of the mobile device. “How to ship a box” will now be translated into Finnish “Kuinka randata laatikko laivalla”, instead of the other alternatives, because the context is clear, we are trying to get a box onto an actual ship to some destination, rather than looking for the post office.
- It is also in accordance with the invention to use a text document in the context folder of a language speech converter in accordance with the invention.
- It should be noted that any features, phases or parts of the
method 40 can be freely permuted and combined withembodiments - The invention has been explained above with reference to the aforementioned embodiments and several commercial and industrial advantages have been demonstrated. The methods and arrangements of the invention allow people to find relevant documents, based on the inventive synonym expansion-contextual ranking on that synonym expanded word set -technique. The same technique provides for more accurate translations from one natural language to another language from the Internet or documents, or from a combination of image language and speech to a natural language, either in textual or spoken form.
- The invention has been explained above with reference to the aforementioned embodiments. However, it is clear that the invention is not only restricted to these embodiments, but comprises all possible embodiments within the spirit and scope of the inventive thought and the following patent claims.
- WO 2009/002864 A2, Reizler, Stefan, Vasserman, Alexander, I. “Machine translation for query expansion”.
- “Desktop Search—How Contextual Information Influences Search Results and Rankings” Raluca Paiu and Wolfgang Nejdl.
- “Context-Aware Semantic Association Ranking” Aleman-Meza Boanerges, Halaschek Chris, Arpinar I. Budak and Sheth Amit.
- “Activity Based Metadata for Semantic Desktop Search”—Proceedings of the 2nd European Semantic Web Conference, Heraklion, Greece, May 2005, P. Chirita, R. Gavriloaie, S. Ghita, W. Nejdl and R. Paiu.
Claims (14)
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP09168388.8 | 2009-08-21 | ||
EP09168388A EP2341450A1 (en) | 2009-08-21 | 2009-08-21 | Method and means for data searching and language translation |
PCT/EP2010/061611 WO2011020742A1 (en) | 2009-08-21 | 2010-08-10 | Method and means for data searching and language translation |
Related Parent Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/758,258 Continuation US20110047149A1 (en) | 2009-08-21 | 2010-04-12 | Method and means for data searching and language translation |
PCT/EP2010/061611 A-371-Of-International WO2011020742A1 (en) | 2009-08-21 | 2010-08-10 | Method and means for data searching and language translation |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/008,537 Division US9953092B2 (en) | 2009-08-21 | 2016-01-28 | Method and means for data searching and language translation |
Publications (1)
Publication Number | Publication Date |
---|---|
US20120143858A1 true US20120143858A1 (en) | 2012-06-07 |
Family
ID=41361190
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/758,258 Abandoned US20110047149A1 (en) | 2009-08-21 | 2010-04-12 | Method and means for data searching and language translation |
US13/390,526 Pending US20120143858A1 (en) | 2009-08-21 | 2010-08-10 | Method And Means For Data Searching And Language Translation |
US15/008,537 Active US9953092B2 (en) | 2009-08-21 | 2016-01-28 | Method and means for data searching and language translation |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/758,258 Abandoned US20110047149A1 (en) | 2009-08-21 | 2010-04-12 | Method and means for data searching and language translation |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/008,537 Active US9953092B2 (en) | 2009-08-21 | 2016-01-28 | Method and means for data searching and language translation |
Country Status (7)
Country | Link |
---|---|
US (3) | US20110047149A1 (en) |
EP (3) | EP2629211A1 (en) |
JP (1) | JP2013502631A (en) |
CN (1) | CN102483757A (en) |
AU (1) | AU2010285122A1 (en) |
CA (1) | CA2770563A1 (en) |
WO (1) | WO2011020742A1 (en) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20120296635A1 (en) * | 2011-05-19 | 2012-11-22 | Microsoft Corporation | User-modifiable word lattice display for editing documents and search queries |
US20130080162A1 (en) * | 2011-09-23 | 2013-03-28 | Microsoft Corporation | User Query History Expansion for Improving Language Model Adaptation |
WO2015094871A3 (en) * | 2013-12-18 | 2015-10-22 | Microsoft Technology Licensing, Llc. | Intent-based user experience |
US9953092B2 (en) | 2009-08-21 | 2018-04-24 | Mikko Vaananen | Method and means for data searching and language translation |
Families Citing this family (188)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8645137B2 (en) | 2000-03-16 | 2014-02-04 | Apple Inc. | Fast, language-independent method for user authentication by voice |
US8677377B2 (en) | 2005-09-08 | 2014-03-18 | Apple Inc. | Method and apparatus for building an intelligent automated assistant |
US9318108B2 (en) | 2010-01-18 | 2016-04-19 | Apple Inc. | Intelligent automated assistant |
US8977255B2 (en) | 2007-04-03 | 2015-03-10 | Apple Inc. | Method and system for operating a multi-function portable electronic device using voice-activation |
US10002189B2 (en) | 2007-12-20 | 2018-06-19 | Apple Inc. | Method and apparatus for searching using an active ontology |
US9330720B2 (en) | 2008-01-03 | 2016-05-03 | Apple Inc. | Methods and apparatus for altering audio output signals |
US8996376B2 (en) | 2008-04-05 | 2015-03-31 | Apple Inc. | Intelligent text-to-speech conversion |
US8464150B2 (en) * | 2008-06-07 | 2013-06-11 | Apple Inc. | Automatic language identification for dynamic text processing |
US20100030549A1 (en) | 2008-07-31 | 2010-02-04 | Lee Michael M | Mobile device having human language translation capability with positional feedback |
US8676904B2 (en) | 2008-10-02 | 2014-03-18 | Apple Inc. | Electronic devices with voice command and contextual data processing capabilities |
US10241644B2 (en) | 2011-06-03 | 2019-03-26 | Apple Inc. | Actionable reminder entries |
US20120311585A1 (en) | 2011-06-03 | 2012-12-06 | Apple Inc. | Organizing task items that represent tasks to perform |
US10241752B2 (en) | 2011-09-30 | 2019-03-26 | Apple Inc. | Interface for a virtual digital assistant |
US9431006B2 (en) | 2009-07-02 | 2016-08-30 | Apple Inc. | Methods and apparatuses for automatic speech recognition |
US10002198B2 (en) | 2009-10-28 | 2018-06-19 | Verizon Patent And Licensing Inc. | Mobile taxi dispatch system |
US10276170B2 (en) | 2010-01-18 | 2019-04-30 | Apple Inc. | Intelligent automated assistant |
US8150859B2 (en) | 2010-02-05 | 2012-04-03 | Microsoft Corporation | Semantic table of contents for search results |
US8903794B2 (en) | 2010-02-05 | 2014-12-02 | Microsoft Corporation | Generating and presenting lateral concepts |
US8983989B2 (en) | 2010-02-05 | 2015-03-17 | Microsoft Technology Licensing, Llc | Contextual queries |
US8682667B2 (en) | 2010-02-25 | 2014-03-25 | Apple Inc. | User profiling for selecting user specific voice input processing information |
US20110302149A1 (en) * | 2010-06-07 | 2011-12-08 | Microsoft Corporation | Identifying dominant concepts across multiple sources |
US10061850B1 (en) * | 2010-07-27 | 2018-08-28 | Google Llc | Using recent queries for inserting relevant search results for navigational queries |
US20120047247A1 (en) * | 2010-08-18 | 2012-02-23 | Openwave Systems Inc. | System and method for allowing data traffic search |
US9710429B1 (en) * | 2010-11-12 | 2017-07-18 | Google Inc. | Providing text resources updated with translation input from multiple users |
WO2012103191A2 (en) * | 2011-01-26 | 2012-08-02 | Veveo, Inc. | Method of and system for error correction in multiple input modality search engines |
US8527483B2 (en) | 2011-02-04 | 2013-09-03 | Mikko VÄÄNÄNEN | Method and means for browsing by walking |
CN102650986A (en) * | 2011-02-27 | 2012-08-29 | 孙星明 | Synonym expansion method and device both used for text duplication detection |
US9262612B2 (en) | 2011-03-21 | 2016-02-16 | Apple Inc. | Device access using voice authentication |
US9183199B2 (en) * | 2011-03-25 | 2015-11-10 | Ming-Yuan Wu | Communication device for multiple language translation system |
US9842168B2 (en) * | 2011-03-31 | 2017-12-12 | Microsoft Technology Licensing, Llc | Task driven user intents |
CN102737021B (en) * | 2011-03-31 | 2014-10-22 | 北京百度网讯科技有限公司 | Search engine and realization method thereof |
US9760566B2 (en) | 2011-03-31 | 2017-09-12 | Microsoft Technology Licensing, Llc | Augmented conversational understanding agent to identify conversation context between two humans and taking an agent action thereof |
US10642934B2 (en) | 2011-03-31 | 2020-05-05 | Microsoft Technology Licensing, Llc | Augmented conversational understanding architecture |
US9183294B2 (en) * | 2011-04-08 | 2015-11-10 | Siemens Aktiengesellschaft | Meta-data approach to querying multiple biomedical ontologies |
US9064006B2 (en) | 2012-08-23 | 2015-06-23 | Microsoft Technology Licensing, Llc | Translating natural language utterances to keyword search queries |
US10057736B2 (en) | 2011-06-03 | 2018-08-21 | Apple Inc. | Active transport based notifications |
US8386477B1 (en) * | 2011-11-30 | 2013-02-26 | Google Inc. | System and method for determining user language intent |
KR20130082617A (en) * | 2011-12-12 | 2013-07-22 | 에스케이플래닛 주식회사 | Apparatus and method for market extension searching for apllication |
US20130218876A1 (en) * | 2012-02-22 | 2013-08-22 | Nokia Corporation | Method and apparatus for enhancing context intelligence in random index based system |
US10134385B2 (en) | 2012-03-02 | 2018-11-20 | Apple Inc. | Systems and methods for name pronunciation |
US9092504B2 (en) | 2012-04-09 | 2015-07-28 | Vivek Ventures, LLC | Clustered information processing and searching with structured-unstructured database bridge |
US9280610B2 (en) | 2012-05-14 | 2016-03-08 | Apple Inc. | Crowd sourcing information to fulfill user requests |
US10417037B2 (en) | 2012-05-15 | 2019-09-17 | Apple Inc. | Systems and methods for integrating third party services with a digital assistant |
US9721563B2 (en) | 2012-06-08 | 2017-08-01 | Apple Inc. | Name recognition system |
US9218333B2 (en) * | 2012-08-31 | 2015-12-22 | Microsoft Technology Licensing, Llc | Context sensitive auto-correction |
US10127271B2 (en) * | 2012-09-06 | 2018-11-13 | Koninklijke Philips N.V. | Generating a query |
US9547647B2 (en) | 2012-09-19 | 2017-01-17 | Apple Inc. | Voice-based media searching |
GB201300031D0 (en) * | 2013-01-02 | 2013-02-13 | Canonical Ltd | Ubuntu UX innovations |
BR112015018905B1 (en) | 2013-02-07 | 2022-02-22 | Apple Inc | Voice activation feature operation method, computer readable storage media and electronic device |
US10652394B2 (en) | 2013-03-14 | 2020-05-12 | Apple Inc. | System and method for processing voicemail |
US10748529B1 (en) | 2013-03-15 | 2020-08-18 | Apple Inc. | Voice activated device for use with a voice-based digital assistant |
WO2014146032A2 (en) * | 2013-03-15 | 2014-09-18 | Suarez Sergio David Jr | System for method for data sweeping using keywords |
WO2014197334A2 (en) | 2013-06-07 | 2014-12-11 | Apple Inc. | System and method for user-specified pronunciation of words for speech synthesis and recognition |
US9582608B2 (en) | 2013-06-07 | 2017-02-28 | Apple Inc. | Unified ranking with entropy-weighted information for phrase-based semantic auto-completion |
WO2014197335A1 (en) | 2013-06-08 | 2014-12-11 | Apple Inc. | Interpreting and acting upon commands that involve sharing information with remote devices |
CN105264524B (en) | 2013-06-09 | 2019-08-02 | 苹果公司 | For realizing the equipment, method and graphic user interface of the session continuity of two or more examples across digital assistants |
US10176167B2 (en) | 2013-06-09 | 2019-01-08 | Apple Inc. | System and method for inferring user intent from speech inputs |
KR101491843B1 (en) * | 2013-11-13 | 2015-02-11 | 네이버 주식회사 | Conversation based search system and search method |
US10296160B2 (en) | 2013-12-06 | 2019-05-21 | Apple Inc. | Method for extracting salient dialog usage from live data |
CN103744841A (en) * | 2013-12-23 | 2014-04-23 | 武汉传神信息技术有限公司 | Information fragment translating method and system |
US9633004B2 (en) | 2014-05-30 | 2017-04-25 | Apple Inc. | Better resolution when referencing to concepts |
US10170123B2 (en) | 2014-05-30 | 2019-01-01 | Apple Inc. | Intelligent assistant for home automation |
EP3149728B1 (en) | 2014-05-30 | 2019-01-16 | Apple Inc. | Multi-command single utterance input method |
US9430463B2 (en) | 2014-05-30 | 2016-08-30 | Apple Inc. | Exemplar-based natural language processing |
US9715875B2 (en) | 2014-05-30 | 2017-07-25 | Apple Inc. | Reducing the need for manual start/end-pointing and trigger phrases |
US9338493B2 (en) | 2014-06-30 | 2016-05-10 | Apple Inc. | Intelligent automated assistant for TV user interactions |
US9818400B2 (en) | 2014-09-11 | 2017-11-14 | Apple Inc. | Method and apparatus for discovering trending terms in speech requests |
US10127911B2 (en) | 2014-09-30 | 2018-11-13 | Apple Inc. | Speaker identification and unsupervised speaker adaptation techniques |
US9668121B2 (en) | 2014-09-30 | 2017-05-30 | Apple Inc. | Social reminders |
US10074360B2 (en) | 2014-09-30 | 2018-09-11 | Apple Inc. | Providing an indication of the suitability of speech recognition |
US10152299B2 (en) | 2015-03-06 | 2018-12-11 | Apple Inc. | Reducing response latency of intelligent automated assistants |
US10567477B2 (en) | 2015-03-08 | 2020-02-18 | Apple Inc. | Virtual assistant continuity |
US9886953B2 (en) | 2015-03-08 | 2018-02-06 | Apple Inc. | Virtual assistant activation |
US9721566B2 (en) | 2015-03-08 | 2017-08-01 | Apple Inc. | Competing devices responding to voice triggers |
US10460227B2 (en) | 2015-05-15 | 2019-10-29 | Apple Inc. | Virtual assistant in a communication session |
US10200824B2 (en) | 2015-05-27 | 2019-02-05 | Apple Inc. | Systems and methods for proactively identifying and surfacing relevant content on a touch-sensitive device |
US10083688B2 (en) | 2015-05-27 | 2018-09-25 | Apple Inc. | Device voice control for selecting a displayed affordance |
US9578173B2 (en) | 2015-06-05 | 2017-02-21 | Apple Inc. | Virtual assistant aided communication with 3rd party service in a communication session |
US11025565B2 (en) | 2015-06-07 | 2021-06-01 | Apple Inc. | Personalized prediction of responses for instant messaging |
US20160371342A1 (en) * | 2015-06-19 | 2016-12-22 | Lenovo (Singapore) Pte. Ltd. | Adapting search queries using a specific result |
US20160378747A1 (en) | 2015-06-29 | 2016-12-29 | Apple Inc. | Virtual assistant for media playback |
US10747498B2 (en) | 2015-09-08 | 2020-08-18 | Apple Inc. | Zero latency digital assistant |
US10740384B2 (en) | 2015-09-08 | 2020-08-11 | Apple Inc. | Intelligent automated assistant for media search and playback |
US10671428B2 (en) | 2015-09-08 | 2020-06-02 | Apple Inc. | Distributed personal assistant |
US10331312B2 (en) | 2015-09-08 | 2019-06-25 | Apple Inc. | Intelligent automated assistant in a media environment |
KR102407630B1 (en) * | 2015-09-08 | 2022-06-10 | 삼성전자주식회사 | Server, user terminal and a method for controlling thereof |
US9864598B2 (en) | 2015-09-18 | 2018-01-09 | ReactiveCore LLC | System and method for providing supplemental functionalities to a computer program |
US11157260B2 (en) | 2015-09-18 | 2021-10-26 | ReactiveCore LLC | Efficient information storage and retrieval using subgraphs |
US9372684B1 (en) * | 2015-09-18 | 2016-06-21 | ReactiveCore LLC | System and method for providing supplemental functionalities to a computer program via an ontology instance |
US11010550B2 (en) | 2015-09-29 | 2021-05-18 | Apple Inc. | Unified language modeling framework for word prediction, auto-completion and auto-correction |
US10366158B2 (en) | 2015-09-29 | 2019-07-30 | Apple Inc. | Efficient word encoding for recurrent neural network language models |
US11587559B2 (en) | 2015-09-30 | 2023-02-21 | Apple Inc. | Intelligent device identification |
US10691473B2 (en) | 2015-11-06 | 2020-06-23 | Apple Inc. | Intelligent automated assistant in a messaging environment |
US10956666B2 (en) | 2015-11-09 | 2021-03-23 | Apple Inc. | Unconventional virtual assistant interactions |
US10049668B2 (en) | 2015-12-02 | 2018-08-14 | Apple Inc. | Applying neural network language models to weighted finite state transducers for automatic speech recognition |
US10223066B2 (en) | 2015-12-23 | 2019-03-05 | Apple Inc. | Proactive assistance based on dialog communication between devices |
US10446143B2 (en) | 2016-03-14 | 2019-10-15 | Apple Inc. | Identification of voice inputs providing credentials |
US9934775B2 (en) | 2016-05-26 | 2018-04-03 | Apple Inc. | Unit-selection text-to-speech synthesis based on predicted concatenation parameters |
US9972304B2 (en) | 2016-06-03 | 2018-05-15 | Apple Inc. | Privacy preserving distributed evaluation framework for embedded personalized systems |
US11227589B2 (en) | 2016-06-06 | 2022-01-18 | Apple Inc. | Intelligent list reading |
US10249300B2 (en) | 2016-06-06 | 2019-04-02 | Apple Inc. | Intelligent list reading |
US10049663B2 (en) | 2016-06-08 | 2018-08-14 | Apple, Inc. | Intelligent automated assistant for media exploration |
DK179588B1 (en) | 2016-06-09 | 2019-02-22 | Apple Inc. | Intelligent automated assistant in a home environment |
US10509862B2 (en) | 2016-06-10 | 2019-12-17 | Apple Inc. | Dynamic phrase expansion of language input |
US10067938B2 (en) | 2016-06-10 | 2018-09-04 | Apple Inc. | Multilingual word prediction |
US10586535B2 (en) | 2016-06-10 | 2020-03-10 | Apple Inc. | Intelligent digital assistant in a multi-tasking environment |
US10490187B2 (en) | 2016-06-10 | 2019-11-26 | Apple Inc. | Digital assistant providing automated status report |
US10192552B2 (en) | 2016-06-10 | 2019-01-29 | Apple Inc. | Digital assistant providing whispered speech |
DK179343B1 (en) | 2016-06-11 | 2018-05-14 | Apple Inc | Intelligent task discovery |
DK179049B1 (en) | 2016-06-11 | 2017-09-18 | Apple Inc | Data driven natural language event detection and classification |
DK201670540A1 (en) | 2016-06-11 | 2018-01-08 | Apple Inc | Application integration with a digital assistant |
DK179415B1 (en) | 2016-06-11 | 2018-06-14 | Apple Inc | Intelligent device arbitration and control |
CN106202286B (en) * | 2016-06-30 | 2019-11-01 | 北京奇虎科技有限公司 | The searching method and device of entity word are provided based on natural search result |
US10474753B2 (en) | 2016-09-07 | 2019-11-12 | Apple Inc. | Language identification using recurrent neural networks |
US10043516B2 (en) | 2016-09-23 | 2018-08-07 | Apple Inc. | Intelligent automated assistant |
US10297254B2 (en) | 2016-10-03 | 2019-05-21 | Google Llc | Task initiation using long-tail voice commands by weighting strength of association of the tasks and their respective commands based on user feedback |
US10534645B2 (en) * | 2016-11-23 | 2020-01-14 | Wipro Limited | Method and system for executing processes in a virtual storage area network |
US11281993B2 (en) | 2016-12-05 | 2022-03-22 | Apple Inc. | Model and ensemble compression for metric learning |
TWI645303B (en) * | 2016-12-21 | 2018-12-21 | 財團法人工業技術研究院 | Method for verifying string, method for expanding string and method for training verification model |
US10593346B2 (en) | 2016-12-22 | 2020-03-17 | Apple Inc. | Rank-reduced token representation for automatic speech recognition |
US11138208B2 (en) | 2016-12-30 | 2021-10-05 | Microsoft Technology Licensing, Llc | Contextual insight system |
US11204787B2 (en) | 2017-01-09 | 2021-12-21 | Apple Inc. | Application integration with a digital assistant |
US9965798B1 (en) | 2017-01-31 | 2018-05-08 | Mikko Vaananen | Self-shopping refrigerator |
US10417266B2 (en) | 2017-05-09 | 2019-09-17 | Apple Inc. | Context-aware ranking of intelligent response suggestions |
DK201770383A1 (en) | 2017-05-09 | 2018-12-14 | Apple Inc. | User interface for correcting recognition errors |
US10395654B2 (en) | 2017-05-11 | 2019-08-27 | Apple Inc. | Text normalization based on a data-driven learning network |
US10726832B2 (en) | 2017-05-11 | 2020-07-28 | Apple Inc. | Maintaining privacy of personal information |
DK201770439A1 (en) | 2017-05-11 | 2018-12-13 | Apple Inc. | Offline personal assistant |
US11301477B2 (en) | 2017-05-12 | 2022-04-12 | Apple Inc. | Feedback analysis of a digital assistant |
DK179496B1 (en) | 2017-05-12 | 2019-01-15 | Apple Inc. | USER-SPECIFIC Acoustic Models |
DK201770429A1 (en) | 2017-05-12 | 2018-12-14 | Apple Inc. | Low-latency intelligent automated assistant |
DK179745B1 (en) | 2017-05-12 | 2019-05-01 | Apple Inc. | SYNCHRONIZATION AND TASK DELEGATION OF A DIGITAL ASSISTANT |
DK201770432A1 (en) | 2017-05-15 | 2018-12-21 | Apple Inc. | Hierarchical belief states for digital assistants |
DK201770431A1 (en) | 2017-05-15 | 2018-12-20 | Apple Inc. | Optimizing dialogue policy decisions for digital assistants using implicit feedback |
US10403278B2 (en) | 2017-05-16 | 2019-09-03 | Apple Inc. | Methods and systems for phonetic matching in digital assistant services |
US10311144B2 (en) | 2017-05-16 | 2019-06-04 | Apple Inc. | Emoji word sense disambiguation |
DK179549B1 (en) | 2017-05-16 | 2019-02-12 | Apple Inc. | Far-field extension for digital assistant services |
US20180336892A1 (en) | 2017-05-16 | 2018-11-22 | Apple Inc. | Detecting a trigger of a digital assistant |
US10303715B2 (en) | 2017-05-16 | 2019-05-28 | Apple Inc. | Intelligent automated assistant for media exploration |
US10657328B2 (en) | 2017-06-02 | 2020-05-19 | Apple Inc. | Multi-task recurrent neural network architecture for efficient morphology handling in neural language modeling |
CN109213777A (en) * | 2017-06-29 | 2019-01-15 | 杭州九阳小家电有限公司 | A kind of voice-based recipe processing method and system |
US20190034555A1 (en) * | 2017-07-31 | 2019-01-31 | Splunk Inc. | Translating a natural language request to a domain specific language request based on multiple interpretation algorithms |
US10901811B2 (en) | 2017-07-31 | 2021-01-26 | Splunk Inc. | Creating alerts associated with a data storage system based on natural language requests |
US11494395B2 (en) | 2017-07-31 | 2022-11-08 | Splunk Inc. | Creating dashboards for viewing data in a data storage system based on natural language requests |
US10445429B2 (en) | 2017-09-21 | 2019-10-15 | Apple Inc. | Natural language understanding using vocabularies with compressed serialized tries |
US10755051B2 (en) | 2017-09-29 | 2020-08-25 | Apple Inc. | Rule-based natural language processing |
JP7232478B2 (en) * | 2017-10-17 | 2023-03-03 | フォト バトラー インコーポレイテッド | Context-based image selection |
US10636424B2 (en) | 2017-11-30 | 2020-04-28 | Apple Inc. | Multi-turn canned dialog |
US10733982B2 (en) | 2018-01-08 | 2020-08-04 | Apple Inc. | Multi-directional dialog |
US10733375B2 (en) | 2018-01-31 | 2020-08-04 | Apple Inc. | Knowledge-based framework for improving natural language understanding |
US10789959B2 (en) | 2018-03-02 | 2020-09-29 | Apple Inc. | Training speaker recognition models for digital assistants |
US10592604B2 (en) | 2018-03-12 | 2020-03-17 | Apple Inc. | Inverse text normalization for automatic speech recognition |
US10818288B2 (en) | 2018-03-26 | 2020-10-27 | Apple Inc. | Natural assistant interaction |
US10909331B2 (en) | 2018-03-30 | 2021-02-02 | Apple Inc. | Implicit identification of translation payload with neural machine translation |
US11145294B2 (en) | 2018-05-07 | 2021-10-12 | Apple Inc. | Intelligent automated assistant for delivering content from user experiences |
US10928918B2 (en) | 2018-05-07 | 2021-02-23 | Apple Inc. | Raise to speak |
US10984780B2 (en) | 2018-05-21 | 2021-04-20 | Apple Inc. | Global semantic word embeddings using bi-directional recurrent neural networks |
US10892996B2 (en) | 2018-06-01 | 2021-01-12 | Apple Inc. | Variable latency device coordination |
DK201870355A1 (en) | 2018-06-01 | 2019-12-16 | Apple Inc. | Virtual assistant operation in multi-device environments |
US11386266B2 (en) | 2018-06-01 | 2022-07-12 | Apple Inc. | Text correction |
DK179822B1 (en) | 2018-06-01 | 2019-07-12 | Apple Inc. | Voice interaction at a primary device to access call functionality of a companion device |
DK180639B1 (en) | 2018-06-01 | 2021-11-04 | Apple Inc | DISABILITY OF ATTENTION-ATTENTIVE VIRTUAL ASSISTANT |
US10496705B1 (en) | 2018-06-03 | 2019-12-03 | Apple Inc. | Accelerated task performance |
CN108932218B (en) * | 2018-06-29 | 2022-09-30 | 北京百度网讯科技有限公司 | Instance extension method, device, equipment and medium |
CN108920473B (en) * | 2018-07-04 | 2022-08-09 | 中译语通科技股份有限公司 | Data enhancement machine translation method based on same-class word and synonym replacement |
US20200043479A1 (en) * | 2018-08-02 | 2020-02-06 | Soundhound, Inc. | Visually presenting information relevant to a natural language conversation |
US11010561B2 (en) | 2018-09-27 | 2021-05-18 | Apple Inc. | Sentiment prediction from textual data |
US11170166B2 (en) | 2018-09-28 | 2021-11-09 | Apple Inc. | Neural typographical error modeling via generative adversarial networks |
US11462215B2 (en) | 2018-09-28 | 2022-10-04 | Apple Inc. | Multi-modal inputs for voice commands |
US10839159B2 (en) | 2018-09-28 | 2020-11-17 | Apple Inc. | Named entity normalization in a spoken dialog system |
US11475898B2 (en) | 2018-10-26 | 2022-10-18 | Apple Inc. | Low-latency multi-speaker speech recognition |
US11638059B2 (en) | 2019-01-04 | 2023-04-25 | Apple Inc. | Content playback on multiple devices |
US11348573B2 (en) | 2019-03-18 | 2022-05-31 | Apple Inc. | Multimodality in digital assistant systems |
US11315082B2 (en) | 2019-04-17 | 2022-04-26 | Mikko Vaananen | Mobile secretary meeting scheduler |
DK201970509A1 (en) | 2019-05-06 | 2021-01-15 | Apple Inc | Spoken notifications |
US11475884B2 (en) | 2019-05-06 | 2022-10-18 | Apple Inc. | Reducing digital assistant latency when a language is incorrectly determined |
US11423908B2 (en) | 2019-05-06 | 2022-08-23 | Apple Inc. | Interpreting spoken requests |
US11307752B2 (en) | 2019-05-06 | 2022-04-19 | Apple Inc. | User configurable task triggers |
US11140099B2 (en) | 2019-05-21 | 2021-10-05 | Apple Inc. | Providing message response suggestions |
DK180129B1 (en) | 2019-05-31 | 2020-06-02 | Apple Inc. | User activity shortcut suggestions |
US11289073B2 (en) | 2019-05-31 | 2022-03-29 | Apple Inc. | Device text to speech |
US11496600B2 (en) | 2019-05-31 | 2022-11-08 | Apple Inc. | Remote execution of machine-learned models |
DK201970511A1 (en) | 2019-05-31 | 2021-02-15 | Apple Inc | Voice identification in digital assistant systems |
US11360641B2 (en) | 2019-06-01 | 2022-06-14 | Apple Inc. | Increasing the relevance of new available information |
US11488406B2 (en) | 2019-09-25 | 2022-11-01 | Apple Inc. | Text detection using global geometry estimators |
US11043220B1 (en) | 2020-05-11 | 2021-06-22 | Apple Inc. | Digital assistant hardware abstraction |
US11755276B2 (en) | 2020-05-12 | 2023-09-12 | Apple Inc. | Reducing description length based on confidence |
CN112256822A (en) * | 2020-10-21 | 2021-01-22 | 平安科技(深圳)有限公司 | Text search method and device, computer equipment and storage medium |
Citations (47)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5579471A (en) * | 1992-11-09 | 1996-11-26 | International Business Machines Corporation | Image query system and method |
US20020007384A1 (en) * | 1998-02-03 | 2002-01-17 | Akira Ushioda | Apparatus and method for retrieving data from a document database |
US20030120478A1 (en) * | 2001-12-21 | 2003-06-26 | Robert Palmquist | Network-based translation system |
US20030164819A1 (en) * | 2002-03-04 | 2003-09-04 | Alex Waibel | Portable object identification and translation system |
US20030229622A1 (en) * | 2002-04-12 | 2003-12-11 | Targit A/S | Method of processing multi-lingual queries |
US20040003001A1 (en) * | 2002-04-03 | 2004-01-01 | Fuji Photo Film Co., Ltd. | Similar image search system |
US20040260535A1 (en) * | 2003-06-05 | 2004-12-23 | International Business Machines Corporation | System and method for automatic natural language translation of embedded text regions in images during information transfer |
US20050036042A1 (en) * | 2003-08-13 | 2005-02-17 | Haas William Robert | Systems and methods for associating images |
US20050073436A1 (en) * | 2003-08-22 | 2005-04-07 | Negreiro Manuel I. | Method and system for alerting a patrol officer of a wanted vehicle |
US20050086046A1 (en) * | 1999-11-12 | 2005-04-21 | Bennett Ian M. | System & method for natural language processing of sentence based queries |
US20050289124A1 (en) * | 2004-06-29 | 2005-12-29 | Matthias Kaiser | Systems and methods for processing natural language queries |
US20060015337A1 (en) * | 2004-04-02 | 2006-01-19 | Kurzweil Raymond C | Cooperative processing for portable reading machine |
US20060017752A1 (en) * | 2004-04-02 | 2006-01-26 | Kurzweil Raymond C | Image resizing for optical character recognition in portable reading machine |
US20060047649A1 (en) * | 2003-12-29 | 2006-03-02 | Ping Liang | Internet and computer information retrieval and mining with intelligent conceptual filtering, visualization and automation |
US20060085477A1 (en) * | 2004-10-01 | 2006-04-20 | Ricoh Company, Ltd. | Techniques for retrieving documents using an image capture device |
US20060114516A1 (en) * | 2004-11-29 | 2006-06-01 | Rothschild Leigh M | System and method for embedding symbology in digital images and using the symbology to organize and control the digital images |
US20060176516A1 (en) * | 2004-11-29 | 2006-08-10 | Rothschild Trust Holdings, Llc | System and method for embedding and retrieving information in digital images and using the information to copyright the digital images |
US20060184476A1 (en) * | 2001-02-28 | 2006-08-17 | Voice-Insight | Natural language query system for accessing an information system |
US20060230022A1 (en) * | 2005-03-29 | 2006-10-12 | Bailey David R | Integration of multiple query revision models |
US20060240862A1 (en) * | 2004-02-20 | 2006-10-26 | Hartmut Neven | Mobile image-based information retrieval system |
US20070172155A1 (en) * | 2006-01-21 | 2007-07-26 | Elizabeth Guckenberger | Photo Automatic Linking System and method for accessing, linking, and visualizing "key-face" and/or multiple similar facial images along with associated electronic data via a facial image recognition search engine |
US20070214131A1 (en) * | 2006-03-13 | 2007-09-13 | Microsoft Corporation | Re-ranking search results based on query log |
US20080009268A1 (en) * | 2005-09-14 | 2008-01-10 | Jorey Ramer | Authorized mobile content search results |
US20080082497A1 (en) * | 2006-09-29 | 2008-04-03 | Leblang Jonathan A | Method and system for identifying and displaying images in response to search queries |
US20080162471A1 (en) * | 2005-01-24 | 2008-07-03 | Bernard David E | Multimodal natural language query system for processing and analyzing voice and proximity-based queries |
US20080208774A1 (en) * | 2007-02-23 | 2008-08-28 | Sungkyunkwan University Foundation For Corporate Collaboration | Ontology system for context-aware, method thereof, and recording medium storing the same |
US20080240572A1 (en) * | 2007-03-26 | 2008-10-02 | Seiko Epson Corporation | Image Search Apparatus and Image Search Method |
US20080267504A1 (en) * | 2007-04-24 | 2008-10-30 | Nokia Corporation | Method, device and computer program product for integrating code-based and optical character recognition technologies into a mobile visual search |
US20090049132A1 (en) * | 2007-08-15 | 2009-02-19 | Moshe Livne Gutovski | Device, system, and method of routing electronic mail |
US20090059014A1 (en) * | 2004-11-29 | 2009-03-05 | Rothschild Trust Holdings, Llc | Device and method for embedding and retrieving information in digital images |
US20090063129A1 (en) * | 2007-08-29 | 2009-03-05 | Inventec Appliances Corp. | Method and system for instantly translating text within image |
US20090125497A1 (en) * | 2006-05-12 | 2009-05-14 | Eij Group Llc | System and method for multi-lingual information retrieval |
US20090132506A1 (en) * | 2007-11-20 | 2009-05-21 | International Business Machines Corporation | Methods and apparatus for integration of visual and natural language query interfaces for context-sensitive data exploration |
US20090144233A1 (en) * | 2007-11-29 | 2009-06-04 | Grigsby Travis M | System and method for automotive image capture and retrieval |
US20090182622A1 (en) * | 2008-01-15 | 2009-07-16 | Agarwal Amit D | Enhancing and storing data for recall and use |
US7565139B2 (en) * | 2004-02-20 | 2009-07-21 | Google Inc. | Image-based search engine for mobile phones with camera |
US7593605B2 (en) * | 2004-02-15 | 2009-09-22 | Exbiblio B.V. | Data capture from rendered documents using handheld device |
US7636714B1 (en) * | 2005-03-31 | 2009-12-22 | Google Inc. | Determining query term synonyms within query context |
US20100082585A1 (en) * | 2008-09-23 | 2010-04-01 | Disney Enterprises, Inc. | System and method for visual search in a video media player |
US20100173269A1 (en) * | 2009-01-07 | 2010-07-08 | Manika Puri | Food recognition using visual analysis and speech recognition |
US20110055203A1 (en) * | 2009-08-31 | 2011-03-03 | Microsoft Corporation | Visual search and three-dimensional results |
US20110092251A1 (en) * | 2004-08-31 | 2011-04-21 | Gopalakrishnan Kumar C | Providing Search Results from Visual Imagery |
US20110125735A1 (en) * | 2009-08-07 | 2011-05-26 | David Petrou | Architecture for responding to a visual query |
US20110311140A1 (en) * | 2010-06-18 | 2011-12-22 | Google Inc. | Selecting Representative Images for Establishments |
US20120084323A1 (en) * | 2010-10-02 | 2012-04-05 | Microsoft Corporation | Geographic text search using image-mined data |
US20120128239A1 (en) * | 2010-11-18 | 2012-05-24 | Ebay Inc. | Image quality assessment to merchandise an item |
US20120267504A1 (en) * | 2009-06-16 | 2012-10-25 | Zen Technologies Limited | Motion platform system |
Family Cites Families (56)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5758257A (en) * | 1994-11-29 | 1998-05-26 | Herz; Frederick | System and method for scheduling broadcast of and access to video programs and other data using customer profiles |
US6026388A (en) * | 1995-08-16 | 2000-02-15 | Textwise, Llc | User interface and other enhancements for natural language information retrieval system and method |
DE19842176A1 (en) * | 1998-09-15 | 2000-03-16 | Bosch Gmbh Robert | Method to identify traffic signs in surroundings of vehicle and for vehicle navigation, involves including satellite-based navigation system and unit to identify vehicle signs, which are used to update data from each other |
US6266442B1 (en) * | 1998-10-23 | 2001-07-24 | Facet Technology Corp. | Method and apparatus for identifying objects depicted in a videostream |
US6314398B1 (en) | 1999-03-01 | 2001-11-06 | Matsushita Electric Industrial Co., Ltd. | Apparatus and method using speech understanding for automatic channel selection in interactive television |
US6643620B1 (en) | 1999-03-15 | 2003-11-04 | Matsushita Electric Industrial Co., Ltd. | Voice activated controller for recording and retrieving audio/video programs |
US6598039B1 (en) | 1999-06-08 | 2003-07-22 | Albert-Inc. S.A. | Natural language interface for searching database |
US6446064B1 (en) | 1999-06-08 | 2002-09-03 | Albert Holding Sa | System and method for enhancing e-commerce using natural language interface for searching database |
US6813395B1 (en) | 1999-07-14 | 2004-11-02 | Fuji Photo Film Co., Ltd. | Image searching method and image processing method |
US7450734B2 (en) | 2000-01-13 | 2008-11-11 | Digimarc Corporation | Digital asset management, targeted searching and desktop searching using digital watermarks |
EP1170677B1 (en) * | 2000-07-04 | 2009-03-18 | International Business Machines Corporation | Method and system of weighted context feedback for result improvement in information retrieval |
AU2001288469A1 (en) * | 2000-08-28 | 2002-03-13 | Emotion, Inc. | Method and apparatus for digital media management, retrieval, and collaboration |
US6748375B1 (en) | 2000-09-07 | 2004-06-08 | Microsoft Corporation | System and method for content retrieval |
US6885869B2 (en) * | 2001-01-26 | 2005-04-26 | Ericsson Inc. | Method for mating a mobile terminal with a cordless phone system |
US7860706B2 (en) * | 2001-03-16 | 2010-12-28 | Eli Abir | Knowledge system method and appparatus |
EP1402408A1 (en) * | 2001-07-04 | 2004-03-31 | Cogisum Intermedia AG | Category based, extensible and interactive system for document retrieval |
US7668718B2 (en) | 2001-07-17 | 2010-02-23 | Custom Speech Usa, Inc. | Synchronized pattern recognition source data processed by manual or automatic means for creation of shared speaker-dependent speech user profile |
US6778979B2 (en) * | 2001-08-13 | 2004-08-17 | Xerox Corporation | System for automatically generating queries |
US7421660B2 (en) * | 2003-02-04 | 2008-09-02 | Cataphora, Inc. | Method and apparatus to visually present discussions for data mining purposes |
US20040209515A1 (en) | 2003-04-03 | 2004-10-21 | Caveney Jack E. | High density patch panel |
US7509313B2 (en) * | 2003-08-21 | 2009-03-24 | Idilia Inc. | System and method for processing a query |
US20050149498A1 (en) * | 2003-12-31 | 2005-07-07 | Stephen Lawrence | Methods and systems for improving a search ranking using article information |
US7707218B2 (en) * | 2004-04-16 | 2010-04-27 | Mobot, Inc. | Mobile query system and method based on visual cues |
KR100462542B1 (en) * | 2004-05-27 | 2004-12-17 | 엔에이치엔(주) | contents search system for providing confidential contents through network and method thereof |
JP5372369B2 (en) | 2004-06-22 | 2013-12-18 | ディジマーク コーポレイション | Digital asset management, targeted search, and desktop search using digital watermark |
US8953908B2 (en) | 2004-06-22 | 2015-02-10 | Digimarc Corporation | Metadata management and generation using perceptual features |
US9058388B2 (en) | 2004-06-22 | 2015-06-16 | Digimarc Corporation | Internet and database searching with handheld devices |
US7293017B2 (en) | 2004-07-01 | 2007-11-06 | Microsoft Corporation | Presentation-level content filtering for a search result |
JP4346084B2 (en) | 2004-07-06 | 2009-10-14 | 東洋ゴム工業株式会社 | Mold for tire molding |
US7853574B2 (en) * | 2004-08-26 | 2010-12-14 | International Business Machines Corporation | Method of generating a context-inferenced search query and of sorting a result of the query |
GB2419208A (en) | 2004-10-18 | 2006-04-19 | Qinetiq Ltd | Optical correlation employing an optical bit delay |
US20120113273A1 (en) | 2004-11-29 | 2012-05-10 | Ariel Inventions Llc | System, Method, and Devices for Searching for a Digital Image over a Communication Network |
US7765231B2 (en) * | 2005-04-08 | 2010-07-27 | Rathus Spencer A | System and method for accessing electronic data via an image search engine |
US7945099B2 (en) * | 2005-05-09 | 2011-05-17 | Like.Com | System and method for use of images with recognition analysis |
US7760917B2 (en) * | 2005-05-09 | 2010-07-20 | Like.Com | Computer-implemented method for performing similarity searches |
US7809722B2 (en) | 2005-05-09 | 2010-10-05 | Like.Com | System and method for enabling search and retrieval from image files based on recognized information |
US20070133947A1 (en) | 2005-10-28 | 2007-06-14 | William Armitage | Systems and methods for image search |
US20070185862A1 (en) * | 2006-01-31 | 2007-08-09 | Intellext, Inc. | Methods and apparatus for determining if a search query should be issued |
US20070233806A1 (en) * | 2006-03-29 | 2007-10-04 | Mehrzad Asadi | Method and system for conducting an internet search using a mobile radio terminal |
US7854347B2 (en) | 2006-06-02 | 2010-12-21 | Hudson Wang | Manual gas inflator |
WO2008014499A2 (en) * | 2006-07-27 | 2008-01-31 | Nervana Inc. | Information nervous system |
US8041730B1 (en) * | 2006-10-24 | 2011-10-18 | Google Inc. | Using geographic data to identify correlated geographic synonyms |
JP2008204416A (en) * | 2007-02-19 | 2008-09-04 | Yasuhiro Yamazaki | Advanced information acquisition system |
US8861898B2 (en) * | 2007-03-16 | 2014-10-14 | Sony Corporation | Content image search |
US20080268876A1 (en) * | 2007-04-24 | 2008-10-30 | Natasha Gelfand | Method, Device, Mobile Terminal, and Computer Program Product for a Point of Interest Based Scheme for Improving Mobile Visual Searching Functionalities |
IL183391A (en) * | 2007-05-24 | 2011-06-30 | Peretz Shoval | Ontology-content-based filtering method for personalized newspapers |
US20080306924A1 (en) * | 2007-06-05 | 2008-12-11 | Paolini Michael A | Mobile telephone object lookup and identification |
US9002869B2 (en) * | 2007-06-22 | 2015-04-07 | Google Inc. | Machine translation for query expansion |
KR101434394B1 (en) * | 2007-08-31 | 2014-08-27 | 삼성전자주식회사 | Method for searching of contact list and Mobile Terminal |
EP2201484A4 (en) | 2007-09-21 | 2010-09-22 | Google Inc | Cross-language search |
CA2726037A1 (en) | 2008-06-06 | 2009-12-10 | Thomson Licensing | System and method for similarity search of images |
US20100030803A1 (en) * | 2008-07-30 | 2010-02-04 | Erik Rothenberg | Method for generating business intelligence |
US20100070488A1 (en) * | 2008-09-12 | 2010-03-18 | Nortel Networks Limited | Ranking search results based on affinity criteria |
US20100161641A1 (en) * | 2008-12-22 | 2010-06-24 | NBC Universal, Inc., a New York Corporation | System and method for computerized searching with a community perspective |
EP2629211A1 (en) | 2009-08-21 | 2013-08-21 | Mikko Kalervo Väänänen | Method and means for data searching and language translation |
US20120323967A1 (en) | 2011-06-14 | 2012-12-20 | Microsoft Corporation | Spelling Using a Fuzzy Pattern Search |
-
2009
- 2009-08-21 EP EP12197388.7A patent/EP2629211A1/en not_active Ceased
- 2009-08-21 EP EP09168388A patent/EP2341450A1/en not_active Withdrawn
-
2010
- 2010-04-12 US US12/758,258 patent/US20110047149A1/en not_active Abandoned
- 2010-08-10 EP EP10742816A patent/EP2467792A1/en not_active Ceased
- 2010-08-10 WO PCT/EP2010/061611 patent/WO2011020742A1/en active Application Filing
- 2010-08-10 JP JP2012525129A patent/JP2013502631A/en active Pending
- 2010-08-10 US US13/390,526 patent/US20120143858A1/en active Pending
- 2010-08-10 AU AU2010285122A patent/AU2010285122A1/en not_active Abandoned
- 2010-08-10 CA CA2770563A patent/CA2770563A1/en not_active Abandoned
- 2010-08-10 CN CN2010800369232A patent/CN102483757A/en active Pending
-
2016
- 2016-01-28 US US15/008,537 patent/US9953092B2/en active Active
Patent Citations (56)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5579471A (en) * | 1992-11-09 | 1996-11-26 | International Business Machines Corporation | Image query system and method |
US20020007384A1 (en) * | 1998-02-03 | 2002-01-17 | Akira Ushioda | Apparatus and method for retrieving data from a document database |
US20100005081A1 (en) * | 1999-11-12 | 2010-01-07 | Bennett Ian M | Systems for natural language processing of sentence based queries |
US20050086046A1 (en) * | 1999-11-12 | 2005-04-21 | Bennett Ian M. | System & method for natural language processing of sentence based queries |
US20060184476A1 (en) * | 2001-02-28 | 2006-08-17 | Voice-Insight | Natural language query system for accessing an information system |
US20030120478A1 (en) * | 2001-12-21 | 2003-06-26 | Robert Palmquist | Network-based translation system |
US20030164819A1 (en) * | 2002-03-04 | 2003-09-04 | Alex Waibel | Portable object identification and translation system |
US20040003001A1 (en) * | 2002-04-03 | 2004-01-01 | Fuji Photo Film Co., Ltd. | Similar image search system |
US20030229622A1 (en) * | 2002-04-12 | 2003-12-11 | Targit A/S | Method of processing multi-lingual queries |
US20040260535A1 (en) * | 2003-06-05 | 2004-12-23 | International Business Machines Corporation | System and method for automatic natural language translation of embedded text regions in images during information transfer |
US20050036042A1 (en) * | 2003-08-13 | 2005-02-17 | Haas William Robert | Systems and methods for associating images |
US20050073436A1 (en) * | 2003-08-22 | 2005-04-07 | Negreiro Manuel I. | Method and system for alerting a patrol officer of a wanted vehicle |
US20060047649A1 (en) * | 2003-12-29 | 2006-03-02 | Ping Liang | Internet and computer information retrieval and mining with intelligent conceptual filtering, visualization and automation |
US7707039B2 (en) * | 2004-02-15 | 2010-04-27 | Exbiblio B.V. | Automatic modification of web pages |
US7593605B2 (en) * | 2004-02-15 | 2009-09-22 | Exbiblio B.V. | Data capture from rendered documents using handheld device |
US8447144B2 (en) * | 2004-02-15 | 2013-05-21 | Google Inc. | Data capture from rendered documents using handheld device |
US20100260373A1 (en) * | 2004-02-20 | 2010-10-14 | Google Inc. | Mobile image-based information retrieval system |
US7565139B2 (en) * | 2004-02-20 | 2009-07-21 | Google Inc. | Image-based search engine for mobile phones with camera |
US20060240862A1 (en) * | 2004-02-20 | 2006-10-26 | Hartmut Neven | Mobile image-based information retrieval system |
US7751805B2 (en) * | 2004-02-20 | 2010-07-06 | Google Inc. | Mobile image-based information retrieval system |
US20060017752A1 (en) * | 2004-04-02 | 2006-01-26 | Kurzweil Raymond C | Image resizing for optical character recognition in portable reading machine |
US20060015337A1 (en) * | 2004-04-02 | 2006-01-19 | Kurzweil Raymond C | Cooperative processing for portable reading machine |
US20050289124A1 (en) * | 2004-06-29 | 2005-12-29 | Matthias Kaiser | Systems and methods for processing natural language queries |
US20110092251A1 (en) * | 2004-08-31 | 2011-04-21 | Gopalakrishnan Kumar C | Providing Search Results from Visual Imagery |
US20060085477A1 (en) * | 2004-10-01 | 2006-04-20 | Ricoh Company, Ltd. | Techniques for retrieving documents using an image capture device |
US20110218018A1 (en) * | 2004-10-01 | 2011-09-08 | Ricoh Company, Ltd. | Techniques for Retrieving Documents Using an Image Capture Device |
US20060176516A1 (en) * | 2004-11-29 | 2006-08-10 | Rothschild Trust Holdings, Llc | System and method for embedding and retrieving information in digital images and using the information to copyright the digital images |
US20060114516A1 (en) * | 2004-11-29 | 2006-06-01 | Rothschild Leigh M | System and method for embedding symbology in digital images and using the symbology to organize and control the digital images |
US20090059014A1 (en) * | 2004-11-29 | 2009-03-05 | Rothschild Trust Holdings, Llc | Device and method for embedding and retrieving information in digital images |
US7995118B2 (en) * | 2004-11-29 | 2011-08-09 | Rothschild Trust Holdings, Llc | Device and method for embedding and retrieving information in digital images |
US20080162471A1 (en) * | 2005-01-24 | 2008-07-03 | Bernard David E | Multimodal natural language query system for processing and analyzing voice and proximity-based queries |
US20060230022A1 (en) * | 2005-03-29 | 2006-10-12 | Bailey David R | Integration of multiple query revision models |
US7636714B1 (en) * | 2005-03-31 | 2009-12-22 | Google Inc. | Determining query term synonyms within query context |
US20080009268A1 (en) * | 2005-09-14 | 2008-01-10 | Jorey Ramer | Authorized mobile content search results |
US20070172155A1 (en) * | 2006-01-21 | 2007-07-26 | Elizabeth Guckenberger | Photo Automatic Linking System and method for accessing, linking, and visualizing "key-face" and/or multiple similar facial images along with associated electronic data via a facial image recognition search engine |
US20070214131A1 (en) * | 2006-03-13 | 2007-09-13 | Microsoft Corporation | Re-ranking search results based on query log |
US20090125497A1 (en) * | 2006-05-12 | 2009-05-14 | Eij Group Llc | System and method for multi-lingual information retrieval |
US20080082497A1 (en) * | 2006-09-29 | 2008-04-03 | Leblang Jonathan A | Method and system for identifying and displaying images in response to search queries |
US8631012B2 (en) * | 2006-09-29 | 2014-01-14 | A9.Com, Inc. | Method and system for identifying and displaying images in response to search queries |
US20080208774A1 (en) * | 2007-02-23 | 2008-08-28 | Sungkyunkwan University Foundation For Corporate Collaboration | Ontology system for context-aware, method thereof, and recording medium storing the same |
US20080240572A1 (en) * | 2007-03-26 | 2008-10-02 | Seiko Epson Corporation | Image Search Apparatus and Image Search Method |
US20080267504A1 (en) * | 2007-04-24 | 2008-10-30 | Nokia Corporation | Method, device and computer program product for integrating code-based and optical character recognition technologies into a mobile visual search |
US20090049132A1 (en) * | 2007-08-15 | 2009-02-19 | Moshe Livne Gutovski | Device, system, and method of routing electronic mail |
US20090063129A1 (en) * | 2007-08-29 | 2009-03-05 | Inventec Appliances Corp. | Method and system for instantly translating text within image |
US20090132506A1 (en) * | 2007-11-20 | 2009-05-21 | International Business Machines Corporation | Methods and apparatus for integration of visual and natural language query interfaces for context-sensitive data exploration |
US20090144233A1 (en) * | 2007-11-29 | 2009-06-04 | Grigsby Travis M | System and method for automotive image capture and retrieval |
US7961080B2 (en) * | 2007-11-29 | 2011-06-14 | International Business Machines Corporation | System and method for automotive image capture and retrieval |
US20090182622A1 (en) * | 2008-01-15 | 2009-07-16 | Agarwal Amit D | Enhancing and storing data for recall and use |
US20100082585A1 (en) * | 2008-09-23 | 2010-04-01 | Disney Enterprises, Inc. | System and method for visual search in a video media player |
US20100173269A1 (en) * | 2009-01-07 | 2010-07-08 | Manika Puri | Food recognition using visual analysis and speech recognition |
US20120267504A1 (en) * | 2009-06-16 | 2012-10-25 | Zen Technologies Limited | Motion platform system |
US20110125735A1 (en) * | 2009-08-07 | 2011-05-26 | David Petrou | Architecture for responding to a visual query |
US20110055203A1 (en) * | 2009-08-31 | 2011-03-03 | Microsoft Corporation | Visual search and three-dimensional results |
US20110311140A1 (en) * | 2010-06-18 | 2011-12-22 | Google Inc. | Selecting Representative Images for Establishments |
US20120084323A1 (en) * | 2010-10-02 | 2012-04-05 | Microsoft Corporation | Geographic text search using image-mined data |
US20120128239A1 (en) * | 2010-11-18 | 2012-05-24 | Ebay Inc. | Image quality assessment to merchandise an item |
Non-Patent Citations (8)
Title |
---|
Arias et al, "Context-Based Personilization for Mobile Web Search", 2008. * |
Bradley J. Rhodes, "Margin Notes: Building a Contextually Aware Associative Memory", 2000. * |
Budzik et al, "User Interactions with Everyday Applications as Context for Just-in-time Information Access", 2000. * |
Carbonell et al, "Context-Based Machine Translation", 2006. * |
Leake et al, "Towards Context-Based Search Engine Selection", 2001. * |
Ratprasartporn et al, "Context-based literature digital collection search", 2008. * |
Sieg et al, "Ontological User Profiles for Representing Context in Web Search", 2007. * |
Soules et al, "Connections: Using Context to Enhance File Search", ACM, 2005. * |
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9953092B2 (en) | 2009-08-21 | 2018-04-24 | Mikko Vaananen | Method and means for data searching and language translation |
US20120296635A1 (en) * | 2011-05-19 | 2012-11-22 | Microsoft Corporation | User-modifiable word lattice display for editing documents and search queries |
US8972240B2 (en) * | 2011-05-19 | 2015-03-03 | Microsoft Corporation | User-modifiable word lattice display for editing documents and search queries |
US20130080162A1 (en) * | 2011-09-23 | 2013-03-28 | Microsoft Corporation | User Query History Expansion for Improving Language Model Adaptation |
US9129606B2 (en) * | 2011-09-23 | 2015-09-08 | Microsoft Technology Licensing, Llc | User query history expansion for improving language model adaptation |
US20150325237A1 (en) * | 2011-09-23 | 2015-11-12 | Microsoft Technology Licensing, Llc | User query history expansion for improving language model adaptation |
US9299342B2 (en) * | 2011-09-23 | 2016-03-29 | Microsoft Technology Licensing, Llc | User query history expansion for improving language model adaptation |
WO2015094871A3 (en) * | 2013-12-18 | 2015-10-22 | Microsoft Technology Licensing, Llc. | Intent-based user experience |
Also Published As
Publication number | Publication date |
---|---|
CN102483757A (en) | 2012-05-30 |
US20160154810A1 (en) | 2016-06-02 |
EP2629211A1 (en) | 2013-08-21 |
CA2770563A1 (en) | 2011-02-24 |
WO2011020742A1 (en) | 2011-02-24 |
EP2467792A1 (en) | 2012-06-27 |
US9953092B2 (en) | 2018-04-24 |
AU2010285122A1 (en) | 2012-03-08 |
JP2013502631A (en) | 2013-01-24 |
US20110047149A1 (en) | 2011-02-24 |
EP2341450A1 (en) | 2011-07-06 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9953092B2 (en) | Method and means for data searching and language translation | |
CN110543574B (en) | Knowledge graph construction method, device, equipment and medium | |
CN105183761B (en) | Sensitive word replacing method and device | |
US9721008B1 (en) | Recipe generation utilizing natural language processing | |
US8856119B2 (en) | Holistic disambiguation for entity name spotting | |
US10552539B2 (en) | Dynamic highlighting of text in electronic documents | |
US20150347920A1 (en) | Search system and corresponding method | |
US20230177360A1 (en) | Surfacing unique facts for entities | |
US20100287162A1 (en) | method and system for text summarization and summary based query answering | |
WO2015149533A1 (en) | Method and device for word segmentation processing on basis of webpage content classification | |
US20070175674A1 (en) | Systems and methods for ranking terms found in a data product | |
WO2022052817A1 (en) | Search processing method and apparatus, and terminal and storage medium | |
Mondal et al. | How to choose title and keywords for manuscript according to medical subject headings | |
JP5224453B2 (en) | Geographic feature information extraction method and system | |
US20110072045A1 (en) | Creating Vertical Search Engines for Individual Search Queries | |
JP2009140263A (en) | Term co-occurrence degree extractor | |
US20150052140A1 (en) | Information processing apparatus, information processing method, and program | |
EP3905061A1 (en) | Artificial intelligence for content discovery | |
Sahmoudi et al. | Towards a linguistic patterns for arabic keyphrases extraction | |
US10579660B2 (en) | System and method for augmenting search results | |
Bakar et al. | TikTok Video Cluster Analysis Based on Trending Topic | |
JP7139271B2 (en) | Information processing device, information processing method, and program | |
JP7323484B2 (en) | Information processing device, information processing method, and program | |
JP2004287696A (en) | Retrieval system and retrieval program | |
Paul et al. | Bangla news summarization |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STCV | Information on status: appeal procedure |
Free format text: NOTICE OF APPEAL FILED |
|
STCV | Information on status: appeal procedure |
Free format text: APPEAL BRIEF (OR SUPPLEMENTAL BRIEF) ENTERED AND FORWARDED TO EXAMINER |
|
STCV | Information on status: appeal procedure |
Free format text: APPEAL BRIEF (OR SUPPLEMENTAL BRIEF) ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STCV | Information on status: appeal procedure |
Free format text: NOTICE OF APPEAL FILED |
|
STCV | Information on status: appeal procedure |
Free format text: APPEAL BRIEF (OR SUPPLEMENTAL BRIEF) ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STCV | Information on status: appeal procedure |
Free format text: NOTICE OF APPEAL FILED |
|
STCV | Information on status: appeal procedure |
Free format text: NOTICE OF APPEAL FILED |
|
STCV | Information on status: appeal procedure |
Free format text: APPEAL BRIEF (OR SUPPLEMENTAL BRIEF) ENTERED AND FORWARDED TO EXAMINER |
|
STCV | Information on status: appeal procedure |
Free format text: ON APPEAL -- AWAITING DECISION BY THE BOARD OF APPEALS |
|
STCV | Information on status: appeal procedure |
Free format text: BOARD OF APPEALS DECISION RENDERED |