US20120047025A1 - Query stem advertising - Google Patents

Query stem advertising Download PDF

Info

Publication number
US20120047025A1
US20120047025A1 US13/213,909 US201113213909A US2012047025A1 US 20120047025 A1 US20120047025 A1 US 20120047025A1 US 201113213909 A US201113213909 A US 201113213909A US 2012047025 A1 US2012047025 A1 US 2012047025A1
Authority
US
United States
Prior art keywords
query
stem
search
bid
advertisement
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US13/213,909
Inventor
Trevor Strohman
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Google LLC
Original Assignee
Google LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Google LLC filed Critical Google LLC
Priority to US13/213,909 priority Critical patent/US20120047025A1/en
Assigned to GOOGLE INC. reassignment GOOGLE INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: STROHMAN, TREVOR
Assigned to GOOGLE INC. reassignment GOOGLE INC. CORRECTIVE ASSIGNMENT TO CORRECT THE ATTORNEY DOCKET NUMBER OF 16113-540001 PREVIOUSLY RECORDED ON REEL 027084 FRAME 0033. ASSIGNOR(S) HEREBY CONFIRMS THE ATTORNEY DOCKET NUMBER AS 16113-2540001. Assignors: STROHMAN, TREVOR
Publication of US20120047025A1 publication Critical patent/US20120047025A1/en
Assigned to GOOGLE LLC reassignment GOOGLE LLC CHANGE OF NAME (SEE DOCUMENT FOR DETAILS). Assignors: GOOGLE INC.
Abandoned legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33Querying
    • G06F16/332Query formulation
    • G06F16/3322Query formulation using system suggestions
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33Querying
    • G06F16/338Presentation of query results
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/903Querying
    • G06F16/9032Query formulation
    • G06F16/90324Query formulation using system suggestions
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/95Retrieval from the web
    • G06F16/953Querying, e.g. by the use of web search engines
    • G06F16/9535Search customisation based on user profiles and personalisation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/02Marketing; Price estimation or determination; Fundraising
    • G06Q30/0241Advertisements
    • G06Q30/0273Determination of fees for advertising
    • G06Q30/0275Auctions
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33Querying
    • G06F16/3331Query processing
    • G06F16/3332Query translation
    • G06F16/3338Query expansion

Definitions

  • This specification describes technologies relating to providing content responsive to query stems.
  • the Internet provides access to a wide variety of resources, for example, video files, image files, audio files, or Web pages, including content for particular subjects, book articles, or news articles.
  • a search system can select one or more resources in response to receiving a search query.
  • a search query is data that a user submits to a search engine to satisfy the user's informational needs.
  • the search queries are usually in the form of text, e.g., one or more query terms.
  • the search system selects and scores resources based on their relevance to the search query and on their importance relative to other resources to provide search results that link to the selected resources.
  • the search results are typically ordered according to the scores and presented according to this order.
  • an advertising system may provide advertisements that are responsive to the query.
  • the search query may be used in keyword targeting to identify advertisements that are responsive to the query. These advertisements are provided with the search results.
  • Search systems can provide search suggestions to users to help users satisfy their informational needs.
  • the term “query suggestion” is a suggested data for a query that can be used to refine a search or refine a search strategy.
  • Some search systems provide query suggestions in the form of a list of query suggestions as the user is typing a query. The user can select one of the query suggestions from the list without having to type the entire query suggestion.
  • a client device typically sends suggestion requests to a search engine with each keystroke, and the search engine provides the query suggestions with prefixes that match the entered characters. Once received, the client device displays these suggestions for user selection.
  • Some search systems also provide search results as the user is entering a query. However, while a user is entering a query, the query is not complete. Accordingly, some keyword advertising targeting processes cannot rely on the incomplete query to identify advertisements.
  • This specification describes technologies relating to providing search query suggestions and search results related to the search query suggestions.
  • one aspect of the subject matter described in this specification can be embodied in methods that include the actions defining word stems, each word stem being one or more characters ordered in a sequence and to which one or more characters can be attached to the end of the sequence to form a word; receiving stem bids for the words stems, each stem bid being a bid for a corresponding word stem and corresponding to a price an advertiser pays for display of an advertisement targeted to the corresponding word stem, and wherein the targeting to the corresponding word stem is independent of keyword targeting; receiving a query stem from a client device, the query stem being a query input from a client device and being one or more characters ordered in an input sequence that defines an order in which the one or more characters were input as the query input; in response to receiving the query stem: identifying word stems that match the query stem, providing the corresponding stem bids of the matching word stems as bids to an advertisement auction for advertisement slots for displaying advertisements, and receiving selected advertisements that are determined to have won an advertisement slot in the auction; and providing the selected advertisements for display in
  • Advertisements can be targeted on query stems, allowing advertisers with objectives such as building brand recognition, to serve advertisements on otherwise ambiguous query inputs.
  • the advertisements can be charged based on the duration which the advertisements were displayed, thereby allowed advertisers to better manage their advertising ROI.
  • FIG. 1 is a block diagram of an example environment in which a search engine provides search services.
  • FIG. 2 is a timing diagram of an example process for determining if a timer expires and search results should be provided to a client.
  • FIG. 3 is a timing diagram of an example process in which a confirmation of a query suggestion results in a prediction criterion being met.
  • FIG. 4 is a timing diagram of a process for monitoring multiple timers.
  • FIGS. 5A-5I depict a web search page displaying query suggestions and search results responsive one of the query suggestions.
  • FIG. 6 is a flow diagram of a process for providing search results responsive to a query suggestion to a client device.
  • FIG. 7 is a flow diagram of a process for determining if a prediction criterion is met.
  • FIG. 8 is a flow diagram of a process for indicating a query suggestion for which search results are responsive.
  • FIG. 9 is a flow diagram of a process for displaying second search results responsive to a second query suggestion.
  • FIG. 10 depicts a web search page displaying query suggestions for a query stem, search results responsive to one of the query suggestions, and advertisements responsive to the query stem.
  • FIG. 11 is a graph of illustrating an amount based on a relationship between a stem bid and a display duration.
  • FIG. 12 is a flow diagram of a process for selecting advertisements responsive to a query stem.
  • FIG. 13 is a block diagram illustrating the selection of resources for a query input in response to a prediction criteria being met and based on resource metrics.
  • This specification describes technologies relating to providing query suggestions in response to a query suggestion request and providing search results for at least one of the suggestions if a prediction criterion is met.
  • search results are provided to a client device associated with the query suggestion request and presented in a search interface.
  • the search results are provided to the client device without receiving a search result request (e.g., without a user selecting one of the query suggestions or causing the client device to send a search request to the search engine).
  • the prediction criterion can be a probability, a timeout or some other signal of likely user interest. For example, a determination can be made that the probability a user will select one of the query suggestions is above a threshold value. Search results associated with the query suggestion with the probability above the threshold value can be provided to the client device. Alternatively, a determination can be made that a timeout occurred and search results associated with a highest ranked query suggestion (e.g., a most frequent query suggestion or a query suggestion with the highest probability of being selected) can be provided to the client device. Alternatively, detection of a language boundary (e.g., a syllable, a word, or a bi-gram) can cause a prediction criterion to be met.
  • a language boundary e.g., a syllable, a word, or a bi-gram
  • Various user interface techniques are used to create a user experience in which the search results appear and disappear in a smooth and unobtrusive manner.
  • FIG. 1 is a block diagram of an example environment 100 in which a search engine 110 provides search services.
  • a computer network 102 such as a local area network (LAN), wide area network (WAN), the Internet, or a combination thereof, connects publisher web sites 104 , client devices 106 , and the search engine 110 .
  • the online environment 100 may include many thousands of publisher web sites 104 and client devices 106 .
  • a web site 104 is a one or more web page resources 105 associated with a domain name, and each web site is hosted by one or more servers.
  • An example web site is a collection of web pages formatted in hypertext markup language (HTML) that can contain text, graphic images, multimedia content, and programming elements, such as scripts.
  • HTML hypertext markup language
  • Each web site 104 is maintained by a publisher, e.g., an entity that manages and/or owns the web site.
  • a resource is any data that can be provided by the publisher 104 over the network 102 and that is associated with a resource address.
  • Resources include HTML pages, word processing documents, and portable document format (PDF) documents, images, video, and feed sources, to name just a few.
  • PDF portable document format
  • the resources can include content, such as words, phrases, pictures, and so on, and may include embedded information (such as meta information and hyperlinks) and/or embedded instructions (such as JavaScript scripts).
  • a client device 106 is an electronic device that is under the control of a user and is capable of requesting and receiving resources over the network 102 .
  • Example client devices 106 include personal computers, mobile communication devices, and other devices that can send and receive data over the network 102 .
  • a client device 106 typically includes a user application, such as a web browser, to facilitate the sending and receiving of data over the network 102 .
  • the web browser can enable a user to display and interact with text, images, videos, music and other information typically located on a web page at a website on the world wide web or a local area network.
  • the search engine 110 identifies the resources by crawling the publisher web sites 104 and indexing the resources provided by the publisher web sites 104 .
  • the indexed and, optionally, cached copies of the resources are stored in an indexed cache 112 .
  • the client devices 106 submit search queries 109 to the search engine 110 .
  • the search queries 109 are submitted in the form of a search request that includes the search request and, optionally, a unique identifier that identifies the client device 106 that submits the request.
  • the unique identifier can be data from a cookie stored at the client device, or a user account identifier if the user maintains an account with the search engine 110 , or some other identifier that identifies the client device 106 or the user using the client device.
  • the search engine 110 uses the indexed cache 112 to identify resources that are relevant to the queries.
  • the search engine 110 identifies the resources in the form of search results 111 and returns the search results to the client devices 106 in search results page resource.
  • a search result is data generated by the search engine 110 that identifies a resource that satisfies a particular search query, and includes a resource locator for the resource.
  • An example search result can include a web page title, a snippet of text extracted from the web page, and the URL of the web page.
  • the search results are ranked based on scores related to the resources identified by the search results, such as information retrieval (“IR”) scores, and optionally a separate ranking of each resource relative to other resources (e.g., an authority score).
  • IR scores are computed from dot products of feature vectors corresponding to a query and a resource, and the ranking of the search results is based on relevance scores that are a combination of the IR scores and authority scores.
  • the search results are ordered according to these scores and provided to the client device according to the order.
  • the client devices 106 receive the search results pages and render the pages for presentation to users. In response to the user selecting a search result at a client device 106 , the client device 106 requests the resource identified by the resource locator included in the selected search result.
  • the publisher of the web site 104 hosting the resource receives the request for the resource from the client device 106 and provides the resource to the requesting client device 106 .
  • the queries 109 submitted from client devices 106 are stored in query logs 114 .
  • Click data for the queries and the web pages referenced by the search results are stored in click logs 116 .
  • the query logs 114 and the click logs 116 define search history data 117 that include data from and related to previous search requests associated with unique identifiers.
  • the click logs define actions taken responsive to search results provided by the search engine 110 .
  • the query logs 114 and click logs 116 can be used to map queries submitted by the client devices to web pages that were identified in search results and the actions taken by users (i.e., that data are associated with the identifiers from the search requests so that a search history for each identifier can be accessed).
  • the click logs 116 and query logs 114 can thus be used by the search engine to determine the sequence of queries submitted by the client devices, the actions taken in response to the queries, and how often the queries are submitted.
  • An advertisement management system 130 facilitates the provisioning of advertisements with the resources 105 .
  • the advertisement management system 130 allows advertisers to define targeting rules that take into account attributes of the particular user to provide targeted advertisements for the users.
  • Example targeting rules include keyword targeting, in which advertiser provide bids for keywords that are present in either search queries or webpage content. Advertisements that are associated with keywords having bids that result in an advertisement slot being awarded in response to an auction are selected for displaying in the advertisement slots.
  • a resource 105 from the syndication publisher 104 includes instructions that cause the client device to request advertisements from the advertisement management system 130 .
  • the request includes a publisher identifier and, optionally, keyword identifiers related to the content of the resource.
  • the advertisement management system 130 provides targeted advertisements to the particular user.
  • the client device 106 When a user selects an advertisement, the client device 106 generates a request for a landing page of the advertisement, which is typically a webpage of the advertiser.
  • the client device renders the search results page and sends a request to the advertisement management system 130 , along with one or more keywords related to the query that the user provide to the search engine 110 .
  • advertisements can be based on keywords related to query suggestions, and the advertisements can be provided with the search results that are provided in response to a prediction criterion being met.
  • advertisements can also be provided based on a current query input that does not constitute a completed query input, e.g., advertisements can be provided based on a single character input, or on a current input that forms a stem for many different words.
  • advertiser submit stem bids for word stems.
  • a query input is a query stem that matches a word stem
  • advertisements that are associated with query stems having stem bids that result in an advertisement slot being awarded in response to an auction are selected for displaying in advertisement slots.
  • the advertisement management system 130 includes a data storage system that stores campaign data 132 and performance data 134 .
  • the campaign data 132 stores advertisements, targeting information, and budgeting information for advertisers.
  • the performance data 134 stores data indicating the performance of the advertisements that are served. Such performance data can include, for example, click through rates for advertisements, the number of impressions for advertisements, and the number of conversions for advertisements. Other performance data can also be stored.
  • the performance data 134 can be partitioned based on queries that are actually entered by users (e.g., queries that a user types and then selects a “search” command) and on queries that are suggested. As the performance of advertisements may vary based on the query type (e.g., actually entered queries and suggested queries), the advertisements that are selected for a query consisting of particular words can vary on whether the query is actually entered or suggested.
  • the campaign data 132 and the performance data 134 are used as input parameters to an advertisement auction.
  • the advertisement management system 130 in response to each request for advertisements, conducts an auction to select advertisements that are provided in response to the request.
  • the advertisements are ranked according to a score that, in some implementations, is proportional to a value based on an advertisement bid and one or more parameters specified in the performance data 134 .
  • the search engine 110 In addition to providing search results in response to queries, the search engine 110 , by use of a query suggestion system 118 , provides query suggestions 113 to a client device.
  • the query suggestions 113 can be provided by the search engine 110 in response to a query suggestion request from the client device.
  • the query suggestion system 118 accesses the search history data 117 and determines a list of query suggestions based on search queries that have the query characters as a stem (or, alternatively or in addition, queries that are related by topic or co-occurrence).
  • the query suggestions are based on search history data 117 associated with the unique identifier of the client device that submitted the query suggestion request.
  • the query suggestions can be suggestions based on frequency co-occurrence and query stemming of queries stored in query logs 114 , and then filtered according to the particular search history data 117 associated with the unique identifier.
  • a client device requests a search resource from the search engine 110 .
  • the search engine 110 provides the requested search resource and interface instructions to the client device.
  • the search resource and interface instructions cause the client device to generate a search interface 120 that includes a query input field 122 .
  • the client device can launch a web browser and request a web, an image, or a video search page from the search engine 110 .
  • the client device can request a search page that searches a combination of text, images, and/or videos.
  • the search engine 110 can provide HTML and scripts that cause the client device to generate the search interface 120 with the query input field 122 .
  • the query input field 122 can receive query characters from a user, e.g., keystroke inputs, and provides each input to the search engine 110 in the form of a query suggestion request.
  • the query suggestion system 118 identifies and ranks query suggestions according to an order from highest rank to a lowest rank, and provides the client device with the query suggestions 113 .
  • the query suggestion system 118 can access the search history data 117 and determine query suggestions based on search queries that have the query characters “ba” as a query stem, e.g., “bank,” “banksy,” “Bankrate,” “ball,” etc.
  • the query suggestions are ranked based on the probability of the query suggestion being selected by a user.
  • the query suggestion with the highest rank is the shortest word with the highest probability of being selected by the user.
  • the query suggestions are ranked or re-ranked so natural extensions are shown and grouped together.
  • the client device receives the query suggestions 113 and renders the first query suggestions according to an order.
  • the first query suggestions can be presented in a query suggestion box 124 a according to an order from a highest rank to a lowest rank.
  • the client device can present the first query suggestions in the query suggestion box 124 a in the search interface 120 .
  • the query suggestions can be presented in the query suggestion box 124 a such that the first query suggestion (e.g., “bank”) has the highest probability of being selected by a user, the second query suggestion (e.g., “banksy”) has the second highest probability, and so on.
  • the search engine 110 determines if a prediction criterion is met.
  • the prediction criterion is independent of a user selection on the client device of a query suggestion provided in the first query suggestions, and is independent of user selection on the client device to generate a search request, e.g., selection of a search button or the user pressing enter.
  • search results responsive to one of the query suggestions are sent to the client device and displayed.
  • the search engine 110 can determine that the prediction criterion is met when a timer initialized in response to the query suggestion request expires after a predefined time period and no additional query suggestion requests are received during the predefined time period. For example, the search engine 110 can receive the first query suggestion request and initialize and start a timer that expires after one second. The search engine 110 can monitor the timer and determine that the prediction criterion is met if the timer expires after two seconds and before another query suggestion is received. If the timer expires before another query suggestion is sent, then search results responsive to one of the query suggestions are sent to the client device and displayed. Alternatively, if the relevant search results are already present on the client device (e.g., cached), then the search results stored in the cache are displayed.
  • probability data indicates, for each query suggestion, a respective probability of being selected by the user.
  • the prediction criterion can be based on this probability, or can be combined with a time-based prediction criterion.
  • the search engine 110 can identify the query suggestion with the highest respective probability relative to the probabilities of the other query suggestions provided to the client device, and then determine that the prediction criterion is met when the highest respective probability of the query suggestion exceeds a threshold probability that is greater than zero.
  • the search engine 110 can determine that the query suggestion “bank” has a higher probability of being selected than the query suggestions “banksy,” “Bankrate,” and “ball.”
  • the search engine 110 compares the probability of the query suggestion “bank,” e.g., five percent, with a threshold probability, e.g., three percent, and determines that search results associated with the query suggestion “bank” should be sent to the client device and displayed, or, alternatively, displayed from the cache if they are already present on the client device.
  • the query suggestions can also be ordered so that natural extensions are shown and grouped together.
  • the shortest query suggestion in a group can be presented first, even though the shortest query suggestion may not have the highest probability of being selected by a user. For example, “Bankrate” can have a probability of five percent and “bank” can have a probability of four percent of being selected by the user.
  • the search engine 110 can compare the probability of the query suggestion “bank” to the threshold probability and determine that search results associated with the query suggestion “bank” are to be sent to the client device.
  • the prediction criterion can also be based on a language model. For example, a prediction criterion can be met if the search engine 110 detects a language boundary in the query input associated with the first query suggestion request.
  • the language boundary can be a syllable, a word, or a bi-gram, to name a few examples.
  • FIG. 1 The providing and displaying of search results in response to prediction criteria being met is illustrated in FIG. 1 in the four time-sequenced representations 120 - 1 - 120 - 4 of the search resource 120 .
  • the user has input the partial query “ba”, and a search criterion has been met. Accordingly, the search results SR 11 , SR 12 , SR 13 . . . are provided and displayed, as indicated in the representation 120 - 2 .
  • a “partial query” is a query input that can be completed to form completed query of one or more words.
  • a partial query can thus be one or more keyboard inputs that define a prefix or stem of a word, or can also be an entire word or words.
  • the letter “a” is a partial query, is the letter “a” is the stem for the words “aardvark,” “Appalachia,” etc.
  • the partial query itself can also be a word, such as the letter “a” is a word.
  • the word “New” is also a partial query, as it can be combined with “York”, “London”, etc., to form other queries, and the partial query “New York” can be combined with “Mets” or “Yankees” to form other queries.
  • a partial query is a word stem of one or more characters ordered in a sequence and to which one or more characters can be attached to the end of the sequence to form a word
  • the partial query is also referred to as a query stem.
  • a query stem does not constitute a complete word, or constitutes only a word of few characters, e.g., “a” or “an”.
  • Query stems can be targeted separately from keywords, as will be discussed in more detail below.
  • the client device When search results are to be displayed in response to a prediction criterion being met, the client device, executing the instructions, generates an indication in the search interface that indicates the query suggestion for which the results are responsive.
  • the indication can be an automatic completion of a query input in the query input field 122 indicating the query suggestion for which the search results are responsive.
  • the automatic completion includes differentiated text 126 a appended to the query input in the query input field 122 .
  • the differentiated text in the query input field 122 can be presented with a gray scale background.
  • the differentiated text 126 a has a color background.
  • the font of the differentiated text 126 a can have a different font than the query input, e.g., bold, italics, font type, etc.
  • the cursor in the text input field remains at the last actual input, i.e., after the letter “a” in the stem “ba”, so that user may resume or continue typing to add to the stem “ba”.
  • the query suggestion “bank” can be highlighted in the query suggestion box 124 a , e.g., with a box or a background color.
  • the client device can display the first search results in substantial synchronization with the indication in the search interface 120 that indicates the first query suggestion for which the first search results are responsive. For example, the client device can display the search results “SR 11 ,” “SR 12 ,” “SR 13 ,” etc., in the search interface 120 at the same time that the differentiated text 126 a is displayed in the search interface 120 . In another example, the search results are presented after the differentiated text is shown, e.g., after a short delay (e.g., one second or less).
  • the interface instructions can cause the client device to generate a query suggestion request in response to each character input in the query input field 122 .
  • the query input field 122 can receive the character “r” to form a query input of “bar.”
  • the scripts can receive the query input of “bar” and cause the client device to provide “bar” as a subsequent query suggestion request to the search engine 110 (or, alternatively, only the letter “r” if the search engine 110 stores a current state of the input of the client device). Because the partial query input “bar” does not match the predicted query completion “bank”, the client device removes the search results SR 11 , SR 12 , SR 13 , etc., from view.
  • the search engine 110 receives the query suggestion request and provides second query suggestions responsive to the request from the client device.
  • the query suggestion system 118 can determine the second query suggestions based on the search history data 117 and the query input of “bar”, and the client device renders the second query suggestions according to the order of the query suggestions.
  • the client device presents the query suggestions in a query suggestion box 124 b in the order “hart,” “barbie,” “barney.”
  • the search engine 110 determine if a prediction criterion for the second query suggestions is met. Assuming the prediction criterion is met, the search engine 110 provides another set of search results to the client device responsive to a second query suggestion from the second query suggestions. For example, the search engine 110 can determine that a timer associated with the second query suggestions expired after a predefined time period and before another query suggestion request was received from the client device.
  • the client device receives the search results and generates an indication in the search interface 120 that indicates the second query suggestion.
  • the client device can present differentiated text 126 b appended to the query input in the query input field 122 , and displays the second search results “SR 21 ,” “SR 22 ,” “SR 23 ,” etc., in substantial synchronization with generating the indication in the interface that indicates the second query suggestion.
  • FIG. 2 is a timing diagram 200 of an example process for determining if a timer expires and search results should be provided to a client.
  • the timing diagram 200 depicts a process associated with a client 202 and a search service 204 as time elapses downward along a vertical axis t.
  • the timer can be initiated in response to receiving a request for query suggestions, or can be initiated in response to the search service 204 providing the suggestions. In the examples that follow, the timer is initiated when the query suggestions are provided. Time periods relative to other events can also be used to define a prediction criterion, e.g., a time period after search suggestions have been presented but have not receive user interaction, or a time period between consecutive query suggestion requests, to name a few examples.
  • the client 202 provides a first query suggestion request R b to the search service 204 in response to receiving a query input character “b”. For example, the client 202 receives the query input character “b” from a query input field in a search interface.
  • the search service 204 identifies first query suggestions QS b responsive to the first query suggestion request R b , and at time t 1 provides the suggestions to the client 202 . Additionally, the search service 204 determines that the query suggestion “bank” is the highest ranked query suggestion in the first query suggestions QS b and identifies search results SR bank for the query “bank.”
  • the search service 204 initializes and starts a timer T 1 associated with search results SR bank .
  • the timer T 1 is set to expire after a predefined time period, e.g., at time t 3 , and the search service 204 monitors the timer T 1 for an expiration timeout.
  • the search service 204 provides first query suggestions QS b to the client 202 and the client 202 renders the first query suggestions QS b in the search interface. If the timer T 1 expires and the search results SR bank are still responsive to the most recent query suggestion request, the search service 204 will provide the search results SR bank to the client 202 .
  • the client 202 provides a second query suggestion request R br to the search service 204 .
  • the query input field receives a second character “r” as input from a user and the combined query input of “br” is used for the second query suggestion request R br .
  • the timer T 1 expires and the search service 204 determines that the search results SR bank are not responsive to the second query suggestion request R br .
  • the second character in “bank” is not “r” so the user was not typing “bank” into the query input field.
  • the timer T 1 is disabled at time t 2 when the search service 204 determines that the search results SR bank are not responsive to the second query suggestion request R br . In either case, this results in the search service 204 not providing the search results SR bank .
  • the search service 204 provides second query suggestions QS br responsive to the second query suggestion request R br .
  • the search service 204 determines that the query suggestion “brink” is the highest ranked query suggestion in the second query suggestions QS br and identifies search results SR brink .
  • the search service 204 initializes and starts a timer T 2 associated with search results SR brink , where the timer will expire at time t 5 .
  • the client 202 renders the second query suggestions QS br in the search interface after they are received.
  • the search service 204 determines that the timer T 2 has expired and no query suggestion requests have been received since time t 4 , when T 2 was initialized, and no other inputs have been received from the client device (e.g., an explicit command to perform a search for a particular query).
  • the search service 204 provides the second search results SR brink to the client 202 and the client 202 renders the second search results SR brink in the search interface along with an indication of the query suggestion “brink” associated with the search results SR brink .
  • FIG. 3 is a timing diagram of an example process in which a confirmation of a query suggestion results in a prediction criterion being met.
  • the timing diagram 300 depicts a process associated with a client 302 and a search service 304 as time elapses downward along a vertical axis t.
  • the search service 304 receives second query suggestions QS ba .
  • the second query suggestions QS ba includes the query suggestion “bank” with the highest probability of being selected by a user.
  • the second search suggestion request i.e., “ba”
  • the search service 304 interprets this as meeting a prediction criterion (i.e., receiving a confirmation of a query for pending search results for that query).
  • the search results SR bank are sent with the query suggestions QS ba before the timer T 1 expires.
  • FIG. 4 is a timing diagram 400 of a process for monitoring multiple timers.
  • the timing diagram 400 depicts a process associated with a client 402 and a search service 404 as time elapses downward along a vertical axis t.
  • the search service 404 is able to process two or more sets of pending search results simultaneously and decide which set to send to the client device 402 .
  • the client 402 receives the character “b” entered in a query input field and input by a user.
  • the client 402 provides a first query suggestion request R b to the search service 404 .
  • the search service 404 sends first query suggestions QS b , which includes the highest ranked query suggestion “bank” in response to the first query suggestion request R b .
  • the search service 404 identifies first search results SR bank responsive to the query suggestion “bank.”
  • the search service 404 initializes and starts a timer T 1 associated with the first search results SR bank .
  • the timer T 1 is initialized with a first duration that expires at time t 5 .
  • the search service 404 provides the first query suggestions QS b to the client 402 and the client 402 renders the first query suggestions QS b in a search interface that includes the query input field.
  • the client 402 provides a second query suggestion request R ba to the search service 404 in response to the user entering “a” in the query input field.
  • the second query suggestion request R ba is received less than a threshold time period after the first query suggestion request R b .
  • the query input field is receiving query input characters from a user who is a fast typist.
  • the search service 404 sends second query suggestions QS ba , which again has “bank” as the highest ranked query suggestion in response to the second query suggestion request R ba .
  • the query suggestions for the input “ba” also have a very highly rated query suggestion “bart.”
  • the search service 404 starts a second timer T 2 for search results SR bart . If no additional suggestion requests are received, then at the expiration of the timer T 1 the search results SR bank are provided. However, if the user enters the letter “r”, then the search results for SR bank are purged, the timer T 2 continues, however, if no additional requests are received, and when the timer T 2 expires, the search results SR bart are provided.
  • the process of queuing multiple sets of search results can be extended for subsequent inputs.
  • the search service 404 may identify the next query suggestion with the highest rank in the third query suggestions QS bar , for example “barn” and third search results SR barn associated with the query suggestion “barn.”
  • the search service 404 can initialize and start a timer T 3 (not shown) associated with the third search results SR barn .
  • the search results SR bart and SR barn and, optionally, their respective timers, are then handled in a manner similar to the way the search results SR bank and SR bart and the timers T 1 and T 2 were handled.
  • the client 402 if the time period between query suggestion requests remains short (e.g., less than a threshold time period T z apart), the client 402 does not present search results in the search interface, regardless of whether the query suggestion request confirms a set of search results SR. This is because fast typing is interpreted as a signal that the user has a specific intent to enter a particular query, and thus does not need search results in the form of suggestions.
  • the search service 404 can extend the timer T 1 to time t y so that search results are not sent to the client 402 at time t 4 . As long as the user continues to type quickly, the timer T 1 , or any other timers for any other search results SR that may be pending, can continue to be extended.
  • the time periods associated with consecutive query suggestion requests can be customized to a user.
  • the search service 404 can determine that a first client used by a first user provides query suggestion requests faster than the standard predetermined time period, e.g., the user may be a fast typist compared to many other users.
  • the standard time period can be, for example, a half a second, and the search service can determine that the first client is providing query suggestion requests every third of a second.
  • the search service can use the customized time period (e.g., a third of a second) to determine when to provide search results to the first client.
  • the search service 404 can identify a second client based on a client identifier.
  • the search service 404 can use client search history information associated with the second client to determine a customized time period. This threshold time can be reduced to a minimum amount, e.g., an amount that is greater than the threshold time period T z .
  • the example time periods above are illustrative and not exhaustive, and other time periods can also be used.
  • the search service 404 determines that consecutive query suggestion requests are received less than a threshold time period apart, the search service 404 does not check to see if prediction criterion is met until after a time period greater than the threshold time period has passed after receiving the last query suggestion request.
  • a pause between query suggestion requests e.g., greater than the threshold time period
  • the search service 404 provides the search results SR to the client device automatically with the query suggestions, and the timers are processed locally on the client device. Accordingly, the search service 404 need not monitor the state of the client device, as the client device 402 is monitoring its own state with local timers.
  • the search results that are provided are not immediately rendered, and include metadata indicating the query suggestion to which they are responsive.
  • the client 402 searches the local cache for the relevant search results.
  • the search service 404 does not provide search results to the client device 402 when the search results have been sent to the client device 402 previously.
  • the search service 404 can store data (e.g., that expires after the end of the session) indicating the search results that have been provided to the client device 402 and the length of the cache timer associated with the results. If the search service 404 determines that a prediction criterion is met for search results that were already sent to the client device 402 (e.g., during the same session), the search service 404 does not provide the search results to the client. Examples of such data indicating the search results that have been sent include a query suggestion for which search results were sent or the search results for the query suggestion, or other data that identifies the query suggestions and search results that were previously sent to the client device.
  • the query suggestions and/or the search results can also be set to expire from the local cache after a predetermined amount of time.
  • the expiration time can be based on the time sensitivity of query suggestions.
  • topical queries e.g., news, pop culture
  • non-topical queries e.g., history, medicine, etc.
  • query suggestions related to news can be set to expire after one hour
  • query suggestions related to roman history can be set to expire after one day (or longer).
  • prediction criterion can also be used, e.g., the probability of a query suggestion being selected being in excess of a threshold, or the current query input meeting a language boundary criterion, such as a word, a syllable, or an n-gram.
  • the search service 404 (or the client 402 ) may implement any one of the above prediction criterion either alone or in combination with each other.
  • the search service 404 can determine that the search service 404 is unable to provide search results for all query suggestion requests received or all prediction criteria met, e.g., based on limited resources.
  • the search service 404 can invoke serving delay criteria in order to determine when to provide search results. For example, the search service 404 can prioritize query suggestion requests based on timers, the time period between consecutive query suggestion requests, and/or probabilities that a user will select a query suggestion request. Similarly, the search service 404 can determine that there is extra bandwidth or resources that are available, and the search service 404 can remove or reduce the serving delay criteria.
  • the search service 404 can invoke different delay criteria for different clients based on the number of query suggestion requests received from respective clients, the number of prediction criteria met for respective clients, based on the time interval between consecutive query suggestion requests for a respective client, and/or based on the time interval between the determination of consecutive prediction criteria being met for a respective client.
  • the search service 404 can adjust the duration of timers associated with search results. For example, if the search service 404 determines that it is unable to provide search results to some clients based on the current prediction criteria, the search service 404 can increase a global time period for all timers that determine when search results are served to a client.
  • the search service 404 can provide search results to a client every n times a prediction criterion is met or a query suggestion request is received. For example, the search service 404 can determine that there are not available resources to provide search results to the client for every query suggestion request but there are enough resources to send search results to the client for every other query suggestion request from all clients.
  • the search service 404 can increase the probability threshold associated with the probability of a user selecting a query suggestion so that the search service 404 provides search results for fewer query suggestions based on the probability threshold being met. For example, the search service 404 may determine that it cannot serve search results for all pending search suggestions in which the prediction criterion is met, and thus may increase the probability threshold to a value that reduces the likelihood of the prediction criterion being met to a level for which all requests can again be served for which the prediction criterion is met.
  • the duration of a timer associated with search results is based on the probability associated with a query suggestion the search results are responsive to. For example, search results responsive to a query suggestion with a high probability of being selected by a user can be associated with a timer having a short duration, e.g., zero seconds, 100 milliseconds, or some other short duration, and search results responsive to a query suggestion with a low probability of being selected by a user can be associated with timer having a longer duration, e.g., a half a second.
  • the duration is inversely proportional to the probability according to a function, such as a logarithmic function.
  • the search service 404 throttles the provision of search results to the client 402 based on the bandwidth or data connection capabilities of the connection with the client 402 .
  • the higher the bandwidth or better the data connection capability e.g., lag time
  • the search service 404 can determine that a client has a high bandwidth connection and thus provides the client with search results in response to each query suggestion request received.
  • the search service 404 determines that a client has a low bandwidth connection and provides the client with search results after a timer with a long duration expires.
  • FIGS. 5A-5I depict a web search page 500 displaying query suggestions and search results responsive one of the query suggestions.
  • the web search page 500 can be provided by the search engine or the search service to the client device or client, described above, in response to a request for a search resource (e.g., a search page).
  • a search resource e.g., a search page
  • the web search page 500 includes a search interface 502 .
  • the search interface 502 includes a query input field 504 and can display search results responsive to a query suggestion associated with a search query in the query input field 504 .
  • the web search page 500 can display web, image, video, product, or document search results, to name a few examples.
  • FIG. 5A shows a query suggestion box 506 a with query suggestions QS It .
  • the query suggestions “Italy,” “Italy map,” “Italian,” and “Italian History” are query suggestions responsive to the query input “It” in the query input field. For example, a user has entered the text “It” into the query input field but is unsure about the specific spelling of the word “Italy” or the topic that they are searching for (e.g., “Italy” or “Italian”).
  • the search engine can provide query suggestions QS It in the query suggestion box 506 a in order to help the user choose a search query.
  • the query suggestions QS It can be ranked according to an order. For example, the query suggestions can be ranked based on the probability that the user will select the respective query suggestion.
  • the ranking of the query suggestions according to probability is indicated by the numerical values 508 a adjacent to the query suggestions. These values are illustrative only and are not shown in the actual user interface. In the example ranking, “Italy” has the highest probability and a rank of one, “Italian” has the second probability and a highest rank of two, “Italy map” has the third probability and a highest rank of three, and “Italian History” has the lowest probability and a rank of four.
  • the search engine can re-rank the query suggestions QS It so natural query extensions are presented together in a group. For example, according to the re-ranking, “Italy” has the highest rank, “Italy map” (with the same first term) has the second rank, “Italian” has the third rank, and “Italian History” has the fourth rank.
  • FIG. 5B shows the progression from FIG. 5A in response to a prediction criterion being met.
  • the search results are received at the client and the differentiated text “aly” 510 a is presented in the query input field 504 .
  • the input query combined with the differentiated text “aly” 510 a form one of the query suggestions (“Italy”) presented in the query suggestion box 506 a , e.g., the query suggestion with the highest rank.
  • Search results 514 a - c are presented in the search interface 502 .
  • the search results 514 a - c are responsive to the query suggestion “Italy” 512 a .
  • the search engine receives a query suggestion request R It from the client device rendering the web search page 500 (e.g., a laptop displaying the web search page 500 ).
  • the search engine identifies and provides query suggestions QS It to the client device and the client device presents the query suggestions QS It in the query suggestion box 506 a according to a rank.
  • the search engine determines that a prediction criterion is met and sends the search results SR Italy associated with the query suggestion “Italy” 512 a to the web browser presenting the web search page 500 .
  • the web search page 500 presents the differentiated text “aly” 510 a and the search results 514 a - c in the search interface 502 .
  • the client can render a proper subset of the search results.
  • the search service provides the search results SR Italy to the client, and the client only renders the search result 514 a without rendering the search results 514 b - c .
  • the remaining search results may be rendered.
  • FIG. 5C shows the progression from FIG. 5B where the user enters the character “h” in the query input field 504 so that the query input presented is “Ith.”
  • the partial query “Ith” is not a stem (or otherwise part of) the query suggestions Q It .
  • the search results 514 a - c are not responsive to the query suggestions QS Ith .
  • the client device presents the query suggestions QS Ith in the query suggestion box 506 b
  • the client device also fades out the search results 514 a - c , as indicated by the dashed outlines of the search results 514 a - c .
  • the fading effect can be implemented by a script or other programming instructions.
  • search results 514 a - c are immediately removed from the search interface 502 ; they can be grayed out (e.g., the search results are changed to gray or another color and become partially transparent); or they can fade from view.
  • the search results 514 a - c when the search results 514 a - c are no longer responsive to the query input, the search results remain in the search interface 502 in grayed-out form, but the search interface no longer allows user interaction (e.g., selection) of the search results 514 a - c in grayed out form.
  • the user can interact (e.g., mouse over, select, etc.) with the search results, and various actions can occur as a result. For example, mousing over a grayed out search result will cause it to appear in original form, and the user may select the search result to request the underlying resource it references.
  • all of the search results may return to their original, selectable form, and the user query in the query input may revert to the query suggestion for which the search results are relevant. For example, in FIG. 5C , if the user were to mouse over search result 514 a , the search results 514 a - c would return to their normal form, and the query input would revert to “Italy”.
  • the query suggestions QS Ith include the suggestions “Ithaca College,” “Ithaca,” and “Ithaca NY”, and are re-ranked as shown.
  • the search engine can rank the query suggestions QS Ith according to the length of the query suggestion or so that natural extensions are shown and grouped together.
  • FIG. 5D shows the progression from FIG. 5C when a prediction criterion is met for the query suggestion “Ithaca” 512 b and search results are presented on the search interface 502 .
  • the client device receives search results SR Ithaca that are responsive to the query suggestion “Ithaca” 512 b .
  • the client device can present the differentiated text “aca” 510 b in the query input field 504 .
  • the differentiated text “aca” 510 b are the characters from the query suggestion “Ithaca” 512 b that are not already present in the query input field 504 .
  • search results are removed or added in response to query suggestion requests.
  • search results can be show in the absence of any additional user input.
  • One example way is by use of a presentation timeout, which occurs when a user takes no action for a specified period of time, e.g., 30 seconds, after search results are shown.
  • FIG. 5E shows the progression from FIG. 5D after a presentation timeout.
  • a presentation timeout can occur when search results are presented in the search interface 502 for a predetermined time period and no user interaction within the search interface 502 is detected. For example, the search results 516 a - b were presented in the search interface 502 and user interaction with the search results 516 a - b during the predetermined time period was not detected.
  • the search interface 502 can automatically select a different query suggestion or re-rank the query suggestions, and notify the search engine of the selection.
  • the query suggestion “Ithaca NY” 512 c can be switched with the query suggestion “Ithaca” so that “Ithaca NY” 512 c is the highest ranked query suggestion in the query suggestion box 506 c .
  • the search engine provides search results (e.g., search results 518 a - c ) for the newly selected query suggestion (e.g., “Ithaca NY”).
  • the search interface 502 instead of selecting a new query suggestion, requests a new set of search results (e.g., a next page of search results, such as results 11 - 20 if results 1 - 10 are displayed) in response to the presentation timeout.
  • FIG. 5E also shows another example of how search results can be transitioned out of the search interface.
  • the search results 516 a - c fade out and move to the edge of the search interface 502 while the search results 518 a - c fade in and appear in the search interface 502 .
  • the search results 518 a - c scroll onto the search interface 502 while the search results 516 a - c scroll off the search interface 502 .
  • a presentation timeout does not occur if the web search page 500 is minimized or no longer in focus (i.e., no longer the active view port). For example, a user can receive an email and switch to an email client. As the web search page 500 is no longer in focus, the timer for the presentation timeout is suspended. When the search page 500 returns to being the active view port, the timer resumes its countdown.
  • the user interface features and search service features described above do not constitute and exhaustive set of examples. Other user interface features and search service features can also be implements. Additional examples of user interface features and search service features are described in the following sections.
  • selection of the tab key allows a user to add or complete a single word in the query input field 504 at a time. For example, if the user pressed the tab key a first time in FIG. 5D , the portion of the differentiated text “aca” would be added to the query input “Ith” changing the query input to “Ithaca” and the updated differentiated text to “_NY.” Pressing the tab key a second time would change the query input to “Ithaca NY” and there would no longer be any differentiated text. Alternatively, the user may select from the query suggestions 512 b.
  • the client detects an unexpected “space” character (e.g. two consecutive spaces after a complete word or space that creates an unexpected word break in a word)
  • the current input is provided as a query.
  • the input “goog” may results in the suggestions of “google maps” or “google translate.”
  • the next query input is a space character, creating an input of “goog”, the input “goog” is used as a query for search results.
  • the client device will revert to the query input field when a user resumes typing.
  • the search results that are provided may scroll off the web page. If the user of the client devices uses a mouse to scroll down and review search results, and then resumes typing the query input, the client device will scroll up to query input field to resume modifying the current query input.
  • the query input field includes a command (e.g., an “x” button) on a side. Selection of the command clears the query input, and can additional send a notification to the search service to kill all pending searches and timers related to the query suggestions.
  • a command e.g., an “x” button
  • the status field can be displayed adjacent (e.g., on top of) the query input field.
  • the status field can indicate the predicted query suggestion for which the current search results being shown. As the user continues to type, and the search results change, the status indicator can also change when a different query suggestion is used for the search results.
  • selecting an enter command enters the predicted completion query as the search query.
  • the user selects an alternate key, such as a backspace, escape, or enters an extra (or unexpected) space character.
  • search results are not automatically displayed. Instead, one or more indicators are used to indicate when search results are ready to be displayed. For example, for each search query suggestion, a small round circle may be displayed and the circle is red when there are no search results to display for the suggestions, and the circle changes to green when search results are available to display. For example, circles 508 b in FIG. 5E , which represent rank and are not actually displayed, can instead be color coded red and green and actually be displayed. The user may use an arrow key or tab key, or mouse over, to change focus to certain suggestions, and when the focus is on a particular selection with a green indicator, the search results are displayed.
  • a progress bar that moves along as the user types, behind the text of the query input field, is displayed, and catches up with the cursor position in the query input text when the search results are received.
  • Other indicators such as bars in sequence (similar to a signal strength indicator), or the query input field being highlighted (e.g., the border changing to a blue color), can be used.
  • selecting a right arrow key causes a top-ranked query suggestion to be entered into the query input field.
  • the user may then resume typing by adding to the suggestion. For example, a user may desire to enter the query “Mississippi blues festival.” Instead of typing in Mississippi, the user may type “Miss ⁇ blues festival.”
  • a partial query input, or a query stem can be provided as a query to the search engine 110 .
  • the partial query input can be provided to the search engine 110 as an additional query input in addition to the selected query suggestion.
  • the search engine 110 may select the query suggestion “amazon rain forest” for identifying search results.
  • the search engine may also conduct a search on the partial query input “ama”, and thus the search results that are shown in response to a prediction criteria being met include search results responsive to the query suggestion “amazon rain forest” and search results responsive to the query input “ama.”
  • the respective search results for the query suggestion and the query input can be distinguished as described above with respect to FIG. 5F , e.g., color coded, etc.
  • the search engine 110 will only process the partial query input if a confidence measure or quality measure associated with the query suggestions are below respective thresholds. For example, for the partial query input “klf”, the search engine 110 may determine that none of the query suggestions meet a confidence measure or equality measure. Accordingly, the search engine conducts a search only on the query “klf.”
  • An example confidence measure is a frequency of occurrence of the query suggestion, a probability that the query suggestion will be selected by the user, or some other measure indicative that the query suggestion is a likely completion of the query input.
  • An example quality measure for query suggestion is a value indicating the quality of the underlying resources referenced by search results responsive to the query suggestion.
  • a query suggestion only identifies resources that are of little importance, e.g., resources that ranked low in an authority graph, the query suggestion can have a very low quality measure.
  • the query suggestion identifies resources that are of high importance, e.g., resources are ranked very high in authority graph, the query suggestion can have a very high quality measure.
  • suggestions that are personalized based on the user's history, along with the search results that are provided can be protected (e.g., encrypted). Such protection can be used to prevent replay attacks and other exploits that may compromise the user's privacy.
  • data for cookie exchanges personalized suggestions and search results can be provided over a secure socket layer (SSL).
  • SSL secure socket layer
  • an on-screen virtual keyboard can be displayed (e.g., such as in the case of a mobile device user interface), and the virtual keyboard color code certain keys that correspond to likely completions by letter.
  • indicators are used to indicate the probability of completion for each query suggestion, e.g., a bar shown to the right of the suggestion.
  • the user can visualize a confidence level shown for each suggestion.
  • the user may use an arrow key or tab key, or mouse over, to change focus to certain suggestions.
  • search results for that suggestion are shown. If the user does not select a suggestion, the user may resume typing to change focus back to the current query input.
  • query suggestions are continually formed from a last word or term of a query.
  • the last word or term of a query is used as a basis for a new query suggestion, and thus the number of terms in the query suggestions can remain unbounded.
  • a query suggestion can include a command that, when selected, causes the display of a first (e.g., most highly ranked) search result for the query. For example, in FIG. 5F , mousing over (or otherwise selecting) one of the circles next to one of the search results 522 b will cause one of the corresponding search results 524 a , 524 b or 524 c to appear. Alternatively, selecting the circle will cause the client device to request the underlying resource referenced by the corresponding search result. For example, selecting the circle next to the search result “Ithaca NY” causes the client device to request the resource referenced by the search result 524 a.
  • a first e.g., most highly ranked
  • the search engine does not provide query suggestions to the client device, and additional query suggestions are not displayed, until a character is again added to the query input field 504 .
  • the search engine does not provide query suggestions to the client device and query suggestions are not displayed until the search engine determines that a character has not been deleted from the query input field 504 for a predetermined amount of time. For example, if the search interface 502 determines that changes to the query input are the removal of characters, the client device does not send a query suggestion request to the search engine until three seconds after the last character has been deleted. As with the presentation timer, the client device uses an edit timer in order to determine when to send another query suggestion request after character deletion or backspace.
  • search results for two or more of the query suggestions can be presented in the search interface.
  • the search result 518 a is responsive to “Ithaca”
  • the search result 518 b is responsive to “Ithaca NY”
  • the search results 518 c is responsive to “Ithaca College.”
  • Indication of the query suggestion associated with the respective search results can be presented to the side of the search results.
  • the search interface 502 can include a box to the left of each search result, where the box contains the text of the query suggestion associated with the search result.
  • multiple query suggestions can have a high priority or the same priority.
  • FIG. 5G shows two query suggestions 512 d - e and search results 520 a - c responsive to the two query suggestions 512 d - e .
  • both of the two query suggestions 512 d - e have a high probability of being selected by a user.
  • the background of the query suggestions (or the text of the query suggestions) are color coded in a query suggestion box 506 d and the background of the search results 520 a - c (or text of the search results) are color coded to match the query suggestion the results are responsive to.
  • the search results 520 a - b are responsive to the query suggestion “Ithaca College” 512 e and the search result 520 c is responsive to the query suggestion “Ithaca” 512 d.
  • the search results that are provided can be based on the top predicted query, or on several of the top predicted queries, e.g., grouped as in FIG. 5G , or in other implementations, based on the top results (e.g., the top 1 or 2 results) for each predicted query given the user's typed prefix.
  • pressing a particular key such as a page down key, or placing the mouse in the scroll bar to scroll down, causes the client device to page through search results that are shown for a query suggestion while still maintaining the current cursor position in the query input.
  • the user may resume typing at any time to continue to develop the query.
  • search results that are shown for suggestions do not disappear when the user continues to type. Instead, new search results may “scroll in” to the search page, e.g., from the top, and old search results may “scroll out” of the search page, e.g., to the bottom.
  • Search results that are in the processing of scrolling downward and which are not search results that would be found for a current suggestion or query input are marked with an indication that indicates that the search results are “stale”, i.e., not relevant to the current query suggestion.
  • An example of this interface is shown in FIG. 5H . Search results 526 are moving in a downward direction, as indicated by the arrow 527 .
  • the search results 526 a and 526 b are relevant to the query suggestion “Ithaca NY”, while the search results 526 c and 526 d are relevant to the query suggestion “Italy.” Accordingly, an indication, such as the search results 526 c and 526 d becoming transparent, or some other indicator, is generated to notify the user that the search results are stale.
  • the number of search results that are shown for a query suggestion vary, depending on whether the query suggestion is a navigational query or an informational query.
  • a navigational query is a query that usually has only one satisfactory result, as indicated by a very high relevance score, by traffic patters, or a combination of relevance and traffic patterns.
  • An informational query is a query that has many search results for which several or more the most popular underlying resources have a relatively evenly distributed selection rate. For an informational query, the search results are shown as above. For a navigational query, only the top result referencing the corresponding resource is shown.
  • query suggestions that cause search results that include images or other content that may distract users can be penalized in the ranking of query suggestions.
  • Search results that provide rich content tend to attract users' attention, even if the content is not related to the users' current interests, resulting in artificial topic drift.
  • the queries that cause these search results to be generated can be ranked lower than query suggestions that cause less distracting search results to appear (e.g., search results with primarily text content.
  • the queries are not re-ranked, but the rich content is suppressed, e.g., images and video clips are not shown in the search results.
  • the re-ranking or suppression is dependent on the corpus being searched. For example, if the user is search web documents, re-ranking or suppression can be enabled; conversely, if the user is searching images, re-ranking or suppression is not enabled.
  • query suggestions that result in the identification of content that is classified as sensitive content are not used for generating search results.
  • resources that are identified by a query suggestion can be checked by porn classifiers, and if a threshold number of the resources are classified as including porn (e.g., 10%, or any of the resources that would be referenced in the first page of search results, etc.), then the search results are not provided for the query suggestion.
  • the search results are generated but are not shown automatically. Instead, a button is shown that indicates the content that will be shown is potentially sensitive, and the user must confirm that the search results can be shown.
  • the client when the client detects that characters are no longer being deleted from the query input field 504 , the client can check a local cache for query suggestions and/or search results for the current query input in the query input field 504 .
  • the client can provide a query suggestion request R It to the search service and receive query suggestions QS It and search results SR Italy .
  • the client can render the query suggestions QS It in the query suggestion box 506 a and store the query suggestions QS It and the search results SR Italy in a local cache.
  • the client detects input of the character “h” in the query input field 504 and provides a query suggestion request R Ith to the search service, receives the query suggestions QS Ith , and renders the query suggestions in the query suggestion box 506 b .
  • the edit timer fires, and then expires after a period of inactivity.
  • the client determines that the current query input is “It” and checks the local cache for query suggestions and search results associated with the query input “It.” Locating the query suggestions QS It and the search results SR Italy , the client device then presents the query suggestions QS It and the search results SR Italy (e.g., the search results 514 a - c ) in the search interface 502 . If the search results for the query suggestion Italy are not in the local cache, then the client can request the search results from the search service, or the search service can provide the search results, in response to a prediction criterion being met.
  • requests for query suggestions are not sent with each keystroke, but instead are sent according to a period time, e.g., every 150 milliseconds, or some other time period. This reduces the load on the search service while only causing minimal changes in the user experience.
  • the search service monitors the client device connection, and does not provide search results for a query suggestion if the client device connection is experiencing a high latency or is severely bandwidth constrained. Instead, the search service provides search results in a conventional manner, i.e., when the user issues a request for a specific query.
  • the client device is configured to progressively render the search results as they are received.
  • a search result that were previously provided and displayed can remain displayed when new search results are received for a new query suggestion, provide the search result that was previously provided and displayed is relevant to the new query suggestion.
  • FIG. 5I shows a web browser page 530 in which the user has entered the character “e” in the address bar.
  • Suggestions 532 are shown in response.
  • Two of the suggestions, 532 b and 532 c are URLs for web pages, as indicated by the web icon adjacent to the URLs.
  • Two other suggestions, 532 a and 532 d are search query suggestions, as indicated by the search icon adjacent the search query suggestions.
  • an input device such as an arrow key
  • the user can highlight one of the suggestions 532 , and corresponding content 536 is shown in a preview pane 538 .
  • the corresponding content is, for example, search result content or a web page preview.
  • a preview of the webpage at example1.com is shown in the preview pane 538 .
  • Another input such as an enter key input, causes the client device to navigate to the previewed web page.
  • search results are shown in the preview pane 538 .
  • the user may interact with the search results as described above. If the user selects a particular search result, then the client device navigates to the page referenced by the underlying search result. Should the user navigate back, e.g., by selecting the back browser command 540 , then the client device navigates to a search interface page of the search service, such as the page 500 shown in FIG. 5A .
  • a user can specify preferences for providing search results.
  • the preference can be stored locally on the client (e.g., in a cookie) and provided to the search service, or can be stored in a user account with the search service.
  • the preferences include a setting for when search results are to be provided (e.g., a slider or a Likert scale to adjust the prediction criterion, with a low value/setting resulting in search results not being presented or only being presented after long times (e.g., 10 seconds), and a high value/setting resulting in search results being provided with each keystroke).
  • the user can choose to render results with every set of query suggestions, or can choose that the system be very conservative and only render results after several keystrokes, after an n-second pause, etc.
  • the search service can use information about a user to determine when to provide search results to a client device. For example, the search service can determine that the time period between consecutive query suggestion requests, provided by a client associated with the user, is typically less than a threshold period of time. The search service can determine that the current time period between consecutive query suggestion requests from the client are being received at a time period that is greater than normal and provide search results to the client based on this difference.
  • the search service can store as history data the search results that were shown for query suggestions, and the query suggestions for which they were shown. These suggestions and results can be differentiated from search queries for which a user explicitly requested a search. Query suggestions for which only some positive user interaction was observed—e.g., a long pause, a selection of a search result—are persisted to the user's history, while other query suggestions that did not result in some positive user interaction are not persisted to the user's history. Thus, the user, by examining his or her history, can review queries that he or she actually entered, and can also review query suggestions for which he or she exhibited a positive user interaction. As with other user history data, the user's search history and selections are protected by one or more privacy features to protect the user's privacy.
  • FIG. 6 is a flow diagram of a process 600 for providing search results responsive to a query suggestion to a client device.
  • the process 600 can be used by the search engine to provide search results to the client device.
  • the search engine receives from a client device a request for a search resource ( 602 ). For example, the search engine receives a request from the client device for a web search page.
  • the search engine provides to the client device in response to the request the search resource ( 604 ).
  • the search resource includes interface instructions that cause the client device to generate a search interface that includes a query input field.
  • the search engine provides the web search page, including HTML and scripts, to the client device.
  • the client device can use the HTML and scripts to render a search interface that includes a query input field.
  • the search engine receives query suggestion requests from the client device ( 606 ).
  • Each query suggestion request is generated in response to a query input in the query input field.
  • the client device can generate a query suggestion request from query input entered into the query input field by a user.
  • the client device can provide the query suggestion request to the search engine.
  • the client device can determine that a word or part of a word is entered into the query input field and generate a query suggestion request for each character in the word or in the partial word.
  • the client device can generate a query suggestion request for every n characters, or each word input into the query input field.
  • the search engine In response to each query suggestion request, the search engine provides query suggestions responsive to the request ( 608 ). For example, the search engine identifies query suggestions responsive to the query suggestion request and provides the query suggestions to the client device. The search engine can identify the highest ranked query suggestion or the query suggestion with the highest probability of being selected by a user. The search engine can identify search results associated with the identified query suggestion and start a timer associated with the search results.
  • the search engine determines if a prediction criterion is met ( 610 ).
  • the prediction criterion is independent of a user selection of a query suggestion provided in response to one or more query suggestion requests.
  • the prediction criterion is a probability threshold, a timer expiration, or a combination of the probability threshold and timer expiration. If the search engine determines that the probability associated with the identified query suggestion is above a threshold probability, the prediction criterion is met. If the search engine determines that a timer associated with one of the search results has expired (e.g., one of the search results associated with one of the query suggestion requests), the prediction criterion is met.
  • the timer can be associated with the search results for the most recent query suggestion request or search results from a query suggestion request received before the most recent query suggestion request.
  • the search engine provides search results to the client device ( 612 ).
  • the search results are responsive to one of the query suggestions provided in response to the query suggestion request or one or more previous query suggestion requests.
  • the search engine provides the search results associated with the identified query suggestion to the client if the probability criterion is met.
  • the search engine can provide the search results associated with a timer if the timer expired. If the search engine receives another query suggestion request from the client device, the process 600 continues for the next query suggestion request ( 608 ).
  • the search engine does not provide search results to the client device ( 612 ). For example, if the search engine determines that the prediction criterion is not met, the search engine does not provide search results to the client device. If the search engine receives another query suggestion request, the process 600 continues for the next query suggestion request ( 608 ). While the search engine does not receive another query suggestion request, the process 600 continues to check whether a prediction criterion is met ( 610 ). In some implementations, the process 600 stops if the search engine determines that a prediction criterion cannot be met until another query suggestion request is received from the client device.
  • process 600 The order of steps in the process 600 described above is illustrative only, and processing of query suggestion requests can be done in different orders.
  • the process 600 above describes providing query suggestions responsive to the query suggestion request, and then determining if a prediction criterion is met, the process 600 can be modified so that the process 600 determines if a prediction criterion is met and then provides query suggestions to the client, either with or without the search results.
  • the process 600 can include additional steps or some of the steps of the process 600 can be divided into multiple steps, as described above with reference to FIGS. 2-5E .
  • the process 600 can adjust a timer associated with search results when the search engine identifies the same search results as being responsive to two consecutive query suggestion requests.
  • the process 600 can include shortening or extending the duration of the timer.
  • the process 600 can include increasing a probability percent associated with a query suggestion that is responsive to two consecutive query suggestion requests.
  • the process 600 can include adjusting the prediction criteria based on the resources available to the search engine.
  • portions of the process 600 can be performed in the client device.
  • the client device can determine if prediction criteria are met, and request corresponding search results from the search engine when the prediction criteria are met.
  • the search engine provides the search results and client device renders the search results as described above.
  • the search results can be provided with each set of query suggestions, and when a prediction criterion is met, the client device renders the search results that are stored in the browser cache.
  • FIG. 7 is a flow diagram of a process 700 for determining if a prediction criterion is met.
  • the process 700 can be used by the search engine 110 to provide search results to the client device 106 or by the search service 404 to provide search results to the client 402 .
  • the process 700 can be performed as a part of the process 600 or as a separate process.
  • a search service performing the process 600 can perform the process 700 instead of steps 612 - 616 .
  • the search engine identifies a query suggestion with the highest respective probability relative to the probabilities of the other query suggestions provided ( 702 ). For example, each query suggestion has a respective probability of being selected by a user.
  • the search engine receives a query suggestion request from a client device and identifies query suggestions responsive to the request. The search engine can identify the query suggestion with the highest respective probability and identify search results responsive to the identified query suggestion.
  • the search engine determines if the highest respective probability exceeds a threshold probability ( 704 ).
  • the threshold probability is greater than zero.
  • the search engine has a predetermined threshold probability.
  • the search engine compares the highest respective probability with the threshold probability to determine if the highest respective probability exceeds the threshold probability.
  • the search engine has more than one threshold probability and the search engine determines the current serving delay criteria associated with the client that will be provided with the query suggestions, as described above.
  • the search engine provides search results to a client device responsive to the identified query suggestion ( 706 ).
  • the search results provided are responsive to the query suggestion with the highest respective probability. For example, if the highest respective probability exceeds the threshold probability, the search engine provides the search results associated with the identified query suggestion to the client.
  • the search engine provides the query suggestions, including the identified query suggestion and the other query suggestions, to the client device with the search results.
  • the search engine does not provide search results to a client device ( 708 ).
  • the search engine receives another query suggestion request and the process 700 continues to identify a query suggestion with the highest probability for the query suggestions responsive to the received query suggestion request ( 702 ).
  • FIG. 8 is a flow diagram of a process 800 for indicating a query suggestion for which search results are responsive.
  • the process 800 can be used by the client to receive query suggestions from the search engine.
  • the client generates a search interface that includes a query input field ( 802 ).
  • the client receives HTML and scripts from the search service and generates a web search page.
  • the web search page includes a query input field for receiving character input from a user.
  • the client provides query characters input in the query input field to a search service as a first query suggestion request ( 804 ). For example, the client receives query characters in the query input field from a user. The client sends the first query suggestion request to the search service in response to receiving the query characters. The client can send a query suggestion request for every character, every n characters, or every word entered into the query input field.
  • the client receives first query suggestions in response to the first query suggestion request ( 806 ). For example, the client receives the first query suggestions from the search service in response to the first query suggestion request.
  • the client renders the first query suggestions according to an order ( 808 ).
  • the first query suggestions can include a ranking order.
  • the client can present the first query suggestions in the search interface according to the ranking order.
  • the client can render the first query suggestions below the query input field.
  • the client can render the first query suggestions to the left or the right of the query input field.
  • the client generates an indication in the search interface that indicates the first query suggestion for which first search results are responsive ( 810 ).
  • the generation of the indication is performed in response to the client receiving the first search results responsive to one of the first query suggestions.
  • the first search results are received independent of user selection of one of the query suggestions and independent of sending a completed query to the search service. For example, the first search results are received independent from the user selecting one of the first query suggestions, selecting a search or a send button, or pressing enter.
  • the indication can be presentation of differentiated text in the query input field after the query character input.
  • the differentiated text and the query character input can form the word or words for which the first search results are responsive.
  • the indication can be highlighting the first query suggestion in a query suggestion box.
  • the process 800 can include additional steps or some of the steps of the process 800 can be divided into multiple steps, as described above with reference to FIGS. 2-5E .
  • FIG. 9 is a flow diagram of a process 900 for displaying second search results responsive to a second query suggestion.
  • the process 900 can be used by the client to receive query suggestions from the search service.
  • the client can perform the process 900 after performing the process 800 .
  • the client displays first search results in substantial synchronization with an indication in an interface that indicates a first query suggestion for which the first search results are responsive ( 902 ). For example, the client presents the first search results in a search interface at the same time that the differentiated text is presented in the query input field. In one example, the client presents the indication of the first query suggestion and after a delay of one second or less presents the first search results.
  • the client receives second search results responsive to a second query suggestion ( 904 ).
  • the second query search results are received subsequent in time to the first query suggestion for which the first search results are displayed.
  • the client may have provided a second query suggestion request, received the query suggestions, and prediction criterion for the second query suggestion is met.
  • the client generates an indication in the interface that indicates the second query suggestion ( 906 ). For example, the client can present differentiated text in a query input field or highlight the second query suggestion in a query suggestion box.
  • the client removes the first search results ( 908 ). For example, the client fades out the first search results, removing the first search results from the search interface.
  • the client displays the second search results in substantial synchronization with generating the indication in the interface that indicates the second query suggestion ( 910 ). For example, the client displays the second search results one second after presentation of the differentiated text or highlighting of the second query suggestion. In some implementations, the second search results fade in on the search interface while the first search results fade out from the search interface.
  • process 900 The order of steps in the process 900 described above is illustrative only, and processing of query suggestion requests can be done in different orders. For example, while the process 900 above describes removing the first search results, then displaying the second search results, the process 900 can be modified so that the first search results are removed at the same time that the second search results are displayed.
  • the process 900 can include additional steps or some of the steps of the process 900 can be divided into multiple steps, as described above with reference to FIGS. 2-5E .
  • a query suggestion when a query suggestion is selected, keywords related to the query suggestions are also selected, and advertisement targeted to the keywords are provided with the search results in response to a predication criterion being met.
  • the query input is a query stem
  • the query stem presents, at most, an ambiguous signal of the user's interest.
  • the query stem “p” is the first letter of many different words and queries that cover many different subjects and interests.
  • the advertisement management system 130 stores campaign data 132 that indicates whether an advertiser desires to target advertisements with search results that are provided in response to a prediction criterion being met.
  • the advertisement management system 130 allows advertisers to “opt out” of targeting in these situations.
  • the advertiser can select several targeting options.
  • the first targeting option targets advertisements based on keywords associated with query suggestions that are selected for providing search results in response to a prediction criterion being met.
  • the first targeting option selects advertisements in a manner similar to the way advertisements are selected for queries that are explicitly entered by the user (e.g., when the user enters a query and selects a search command to submit the query to the search engine 110 ).
  • an impression can be counted for the advertisement.
  • impressions may only be counted in the presence of the signal that a user has stopped typing and is viewing the search results and/or advertisements.
  • the second targeting option targets advertisements based on query stems. This targeting option is especially useful for advertisers that are attempting to meet one or more advertising objectives through the use of display advertisements.
  • a display advertisement is an advertisement that is not necessarily directed to driving traffic to a particular landing page, but instead is directed to some other advertising goal, such as building brand recognition.
  • Query Stem targeting can also be used for other advertising objectives as well, including driving traffic to particular landing page.
  • FIG. 10 depicts a web search page 1000 displaying query suggestions 1006 for a query stem (e.g., the letter “g”), search results 1010 responsive to one of the query suggestions (e.g., query suggestion 1004 ), and advertisements 1022 and 1024 responsive to the query stem.
  • Search results 1010 are responsive to the query suggestion 1004 (i.e., the query “gondola”), and are selected and shown as described above.
  • the advertisements are responsive to the query stem “g.”
  • the advertisers have submitted send bids for word stems.
  • Each word stem is one or more characters ordered in a sequence and to which one or more characters can be attached to the end of the sequence to form a word.
  • the letter “g” is a word stem for gondola, Gmail, games, and Gondwana, and other words.
  • query stems can also include two or more letters, e.g., a, aa, ab, etc.
  • Each stem bid is a bid for a corresponding word stem and corresponding to a price an advertiser pays for display of an advertisement targeted to the corresponding word stem.
  • Example pricing schemes are described with respect to FIG. 11 below.
  • the targeting of a corresponding word stem foreign advertisement is independent of keyword targeting.
  • a bid for an advertisement is submitted when a query stem is received, and the query stem need not correspond to a key word on which the advertiser has bid.
  • the advertisement management system 130 in response to receiving the query stem, identifies word stems that match the query stem, and provide corresponding stem bids of the matching word stems as bids to an advertisement auction for advertisement slots for displaying advertisements. For example, in FIG. 10 , there are m advertisement slots available in the area 1014 , and there is one advertisement slot available in the area 1012 . Separate auctions can be conducted for both areas, or, alternatively, a single auction can be conducted for the slots in both areas.
  • the advertisement management system 130 receives elected advertisements that are determined to have won an advertisement slot in the auction, and provides the selected advertisements for display in the advertisement slots on the client device.
  • the advertisement management system 130 can implement processes to reduce the number of auction operations. For example, it may not be practical to run an auction for each character that is input for query stem and which results in a request for query suggestions. For example, the user may type very quickly when inputting a complete query, and thus it would be inefficient to conduct advertisement auctions for each query input.
  • the advertisement management system 130 receives the query stem only in response to the search engine 110 (or the advertisement management system 130 ) determining that a prediction criterion is met. Accordingly, in these implementations, an advertisement auction is only run when search results are provided in response to a prediction criterion being met. In other implementations, the advertisement management system 130 provides the selected advertisements only in response to the search engine (or the advertisement management system 130 ) determining that a prediction criterion is met.
  • advertisements shown in FIG. 10 are described as having been provided in response to an auction based on stem bids, advertisements can also be provided in response to an auction based on keywords associated with a selected query suggestion. For example, some of the advertisements 1024 may have been selected based on stem bids for the query stem “g”, and some of the advertisements may been selected based on keyword bids associated with keywords for the query suggestion “gondola.”
  • advertisers can manually select query stems and input bids for each query stem.
  • the advertisement management system 130 can provide a user interface that shows the most frequent query stems that occur, and frequency measures indicating the relative frequency of occurrence of each query stem.
  • An advertiser can select various query stems and provide bids for each one.
  • the query stems and, optionally, stem bids for the query stems can be determined automatically from the keywords that an advertiser is currently bidding upon.
  • the advertisement management system can receive keywords on which an advertiser has bid.
  • the advertisement management system 130 selects one or more characters that form a stem of the keywords and that do not include all of the characters of the keywords. For example, assume an advertiser has bid on the keyword “aardvark.”
  • the advertisement management system 130 can select the query stems “a”, “aa”, and “aar.”
  • the advertisement management system 130 can determine which query stems to select from the particular keyword. In some implementations, the advertisement management system selects up to a maximum n-gram length of query stems. For example, if the advertiser management system 130 is configured to select up to quad-grams for query stems, then for the keyword aardvark, the query stems a, aa, aar, and aard are selected.
  • the advertisement management system can also take into account the frequency the query stems. For example, query stems that occur below a threshold frequency can be precluded from automatic selection. Returning to the aardvark example, assume that the query stems “aar” and “aard” only occurred a frequency that is below the threshold frequency. Accordingly, the advertisement management system 130 does not select these to query stems for automatic bidding. The advertiser may, however, manually select these query stems.
  • the advertisement management system identifies a keyword bid for the keyword from which the stem was derived, and determines the stem bid based on the identified keyword bid.
  • the stem bid is proportional to the keyword bid.
  • a stem bid can be a percentage of a corresponding keyword bid, e.g., 10%.
  • the stem bids can all be set to an initial fixed price, and the advertiser can manually adjust the prices.
  • the advertiser is charged an amount that is based on the stem bid when the advertisement is shown with the search results.
  • a user may frequently receive search results but may not otherwise view the search results review the advertisements are provided with the search results.
  • the user may be a slow typist.
  • an advertiser may not want to pay an auction cost for every occurrence of an advertisement being shown with the search results, as there may be many times that a user does not view the advertisements.
  • the advertisement management system 130 defines stem bids according to a cost per unit time of display. Using stem bids that are cost per unit time of display, the advertisement management system 130 can charge advertisers based on the amount of time that their advertisements were displayed.
  • the advertisement management system 130 receives data defining a duration that the selected advertisements were displayed in the advertisement slots on the client device after advertisements are provided to the client device.
  • the duration can be determined by instructions on the client device that are provided with the search resource, e.g., a timer that starts when the advertisements are rendered, and stops when new advertisements are received or when the search results are removed from display.
  • the duration can be determined by the search engine 110 or the advertisement management system 130 based on the time differential between previous advertisements being sent and a set of current advertisements being sent. Other ways determining the duration can also be implemented.
  • the advertisement management system 130 determines an amount proportional to the stem bid of the selected advertisement and the duration, and charges each advertiser of a selected advertisement the amount for the selected advertisement.
  • FIG. 11 is a graph 1100 of illustrating an amount based on a relationship between a stem bid and a display duration.
  • the horizontal axis represents time, and the vertical axis represents a cost incurred per unit time, e.g., the cost per second.
  • time T 0 is the time at which an advertisement is rendered on the client device.
  • no cost is incurred.
  • the function can be a first-degree polynomial function defining a relationship between the stem bid and unit time, as indicated by the line 1104 .
  • the function can be an n-degree polynomial function defining a relationship between the stem bid and unit time, with n being greater than 1, as indicated by the curves 1106 and 1108 .
  • Other functions can also be used, e.g., a sigmoid function defining a relationship between the stem bid and unit time.
  • the advertisement management system 130 can define a minimum display duration. This minimum duration is greater than 0 seconds, and if an advertisement is displayed only for less than the minimum duration, then no charges are incurred. For example, assume the time T 0 is 100 ms, indicating that the advertisement has been displayed for at least 100 ms. After 100 ms, charges are incurred, as indicated by the amounts corresponding to the function lines 1104 , 1106 and 1108 .
  • the advertisement management system 130 can find a maximum display duration.
  • the maximum display duration is the time after which no additional charges are incurred for display the advertisement. For example, assume T 1 represents the maximum display duration, e.g., 5 seconds. As long as and advertisements displayed for a time less than the maximum display duration, the amount charged to the advertiser continues to increase and tell the maximum display duration is reached. Thereafter, the amount is a maximum amount that does not increase further. For example, the maximum amount CPI is reached at the time T 1 . In some implementations, this maximum amount may correspond to an amount that an advertiser is typically charged for an impression. Other maximum amounts can also be used.
  • the advertisements that are provided with search results persist in the display even after the corresponding search results are removed.
  • advertisements are provided for a query stem can persist until the occurrence of another prediction criterion being met. For example, assume that the user continues to provide query inputs into the input field of the user interface shown in FIG. 10 , typing the query stem “gar”. The search results 1010 may disappear, is there no longer relevant to the current query input; however, the advertisements may persist until the occurrence of another prediction event. In the event that advertisements have been provided for both query stems and keywords, e.g., some of the advertisement shown in FIG.
  • FIG. 12 is a flow diagram of a process 1200 for selecting advertisements responsive to a query stem.
  • the process 1200 can be implemented in the advertisement management system 130 .
  • the process 1200 defines word stems ( 1202 ).
  • the advertisement management system defines word stems, each of which are were more characters ordered in the sequence into which one or more characters can be attached to the end of the sequence to former word.
  • the process 1200 receive stem bids for the word stems ( 1204 ).
  • the advertisement management system receive stem bids from advertisers.
  • the process 1200 receives a query stem from a client device ( 1206 ).
  • the advertisement management system receives a query stem is a query input of one or more characters ordered in an input sequence that defines the order in which the one or more characters were input as the query input.
  • the process 1200 identifies word stems that match the query stem ( 1208 ). For example, the advertisement management system 130 identifies only word stems that exactly match the query stem received. In some implementations, the advertisement management system can also identify word stems for which the query stem constitutes a stem for the word stem. For example, if the query stem “a” is received, the word stems “a”, “aa,” and “ab” can be determined to match the word stem.
  • the process 1200 provides a corresponding stem bids of the matching word stems is bids to an advertisement auction for advertisement slots for displaying advertisements ( 1210 ).
  • the advertisement management system 130 can access the campaign data 132 to retrieve the stem bids from advertisers that have been on the matching word stems.
  • the process 1200 provides selected advertisements that are determined to be have one an advertisement slots in the auction ( 1212 ).
  • the advertisement management system 130 selects advertisements for which the stem bids resulted in the highest auction scores.
  • the process 1200 provides the selected advertisements for display in the advertisement slots on the client device ( 1214 ).
  • the advertisement management system transmits the advertisements to the client device, and the client device, in response to receiving the advertisements, renders them with the search results.
  • the search engine 110 stores historical data in the form of click logs and query logs.
  • the search engine 110 can also store historical data indicating when search results resulted in a user interaction. For example, if search results are provided and displayed in response to a prediction criterion, and then no additional query suggestion requests or search commands are received, the search engine 110 interprets this as a signal that the results satisfy the user's information needs to some degree. Accordingly, a “valid prediction count” for each of the search results shown can updated.
  • the prediction count for each search result can be compared to its overall “prediction count”, i.e., the number of times the search result was provided in response to a prediction criterion being met, to determine a valid prediction rate.
  • the valid prediction rates can be used as a quality signal that is indicative of the quality of the underlying resources referenced by the search results.
  • a valid prediction count for a search result is only counted if there is a specific interaction with the search result. For example, a valid prediction count is only counted when a “mouse over” or selection of a search result is recorded.
  • the search engine 110 determines which resources referenced by search results are often selected from query suggestions that stem from the current query input.
  • the resources with the highest resource metric e.g., the top N resources that are most often selected, or the top N resources as ranked by a function of their selections, their importance in a resource graph, and, optionally, other metrics
  • the resource metrics of the resources are aggregated in such a manner that selections of the resources can result in search results referencing resources that would otherwise be responsive to different query suggestions that stem from the current query input.
  • FIG. 13 is a block diagram 1300 illustrating the selection of resources for a query input 1302 in response to a prediction criteria being met and based on resource metrics R.
  • query suggestions for which the query input is a stem are selected.
  • the set of query suggestions 1312 - 1316 are selected.
  • all query suggestions that stem from the query input 1302 are selected.
  • a subset of all query suggestions that stem from the query input 1302 are selected. The subset can be based on, for example, the query suggestions that are most often selected by users, or query suggestions that most often result in user interactions (e.g., selections of search results), or query suggestions with the highest confidence measures and or quality measures.
  • the search system 110 identifies sets of resources that are referenced by search results responsive to the query suggestions. For example, the first set of resources 1322 are referenced by search results responsive to the query suggestion 1312 ; the second set of resources 1324 are referenced by search results responsive to the query suggestion 1314 ; and the final set of resources 1326 are referenced by search results responsive to the query suggestion 1316 .
  • the sets of resources are not exclusive sets, i.e., a particular resource can be included in two sets, as indicated by the first resource in the set 1322 the second resource in the set 1324 be shaded to indicate they are the same resource.
  • the search system 110 then aggregates a quality metric R for each of the referenced resources in each set of resources.
  • quality metrics can be used.
  • the metric can be a count of the number of times a particular resource was selected from the search results belonging to the sets 1322 - 1326 ; or can be a selection rate that is a ratio of the number of times particular resource was selected from the search results belonging to the sets 1322 - 1326 to the number of times the search results were presented; or can be a value based on the a function of selections, importance in a resource graph, and, optionally, other metrics.
  • the search engine 110 identifies the resources with the highest resource metric, and, in response to a prediction criteria being met, provides search results referencing these resources in response to the query input 1302 .
  • search results referencing the top N resources, as ranked by their respective resource metrics can be provided for the query input 1302 .
  • a subset of the search results provided can reference the top one or two resources as determined by the resource metric R, and the remaining search results provided can be for a top ranked query suggestion.
  • Similar processes can be used for advertisements that are provided with search results. For example, if no additional query suggestion requests follow the display of the search results, and no additional search commands are received, then an impression for each advertisement that is shown with the search results is counted.
  • the advertisements remain if they are still determined to be relevant to the query suggestion for which the search results are provided.
  • the search engine 110 may be in data communication with the advertisement management system 130 .
  • the search system/advertisement management system determines whether the advertisements that were previously provided are relevant to the search results current being provided. If so, the advertisements that are relevant remain displayed. The advertisements that are not relevant can be removed, and the available advertisement slots that result can be separately auctioned for new, relevant advertisements.
  • Embodiments of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them.
  • Embodiments of the subject matter described in this specification can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a tangible program carrier for execution by, or to control the operation of, data processing apparatus.
  • the tangible program carrier can be computer-readable medium, such as a machine-readable storage device, a machine-readable storage substrate, a memory device, or a combination of one or more of them.
  • data processing apparatus encompasses all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers.
  • the apparatus can include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.
  • a computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.
  • a computer program does not necessarily correspond to a file in a file system.
  • a program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub-programs, or portions of code).
  • a computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.
  • the processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output.
  • the processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit).
  • processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer.
  • a processor will receive instructions and data from a read-only memory or a random access memory or both.
  • the essential elements of a computer are a processor for performing instructions and one or more memory devices for storing instructions and data.
  • a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks.
  • mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks.
  • a computer need not have such devices.
  • a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, to name just a few.
  • PDA personal digital assistant
  • GPS Global Positioning System
  • Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.
  • semiconductor memory devices e.g., EPROM, EEPROM, and flash memory devices
  • magnetic disks e.g., internal hard disks or removable disks
  • magneto-optical disks e.g., CD-ROM and DVD-ROM disks.
  • the processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.
  • embodiments of the subject matter described in this specification can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer.
  • a display device e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor
  • keyboard and a pointing device e.g., a mouse or a trackball
  • Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input.
  • Embodiments of the subject matter described in this specification can be implemented in a computing system that includes a back-end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front-end component, e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back-end, middleware, or front-end components.
  • the components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (“LAN”) and a wide area network (“WAN”), e.g., the Internet.
  • LAN local area network
  • WAN wide area network
  • the computing system can include clients and servers.
  • a client and server are generally remote from each other and typically interact through a communication network.
  • the relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

Abstract

Methods, systems, and apparatus, including computer program products, for serving advertisements responsive to partial queries. In an aspect, a method includes receiving stem bids for words stems, each stem bid being a bid for a corresponding word stem and corresponding to a price an advertiser pays for display of an advertisement targeted to the corresponding word stem, and wherein the targeting to the corresponding word stem is independent of keyword targeting; receiving a query stem from a client device; in response to receiving the query stem: identifying word stems that match the query stem, providing the corresponding stem bids of the matching word stems as bids to an advertisement auction for advertisement slots for displaying advertisements, and receiving selected advertisements that are determined to have won an advertisement slot in the auction; and providing the selected advertisements for display in the advertisement slots on the client device.

Description

    CROSS-REFERENCE TO RELATED APPLICATIONS
  • This application claims priority under 35 USC §119(e) to U.S. Patent Application Ser. No. 61/375,222, filed on Aug. 19, 2010, the entire contents of which are hereby incorporated by reference.
  • BACKGROUND
  • This specification describes technologies relating to providing content responsive to query stems.
  • The Internet provides access to a wide variety of resources, for example, video files, image files, audio files, or Web pages, including content for particular subjects, book articles, or news articles. A search system can select one or more resources in response to receiving a search query. A search query is data that a user submits to a search engine to satisfy the user's informational needs. The search queries are usually in the form of text, e.g., one or more query terms. The search system selects and scores resources based on their relevance to the search query and on their importance relative to other resources to provide search results that link to the selected resources. The search results are typically ordered according to the scores and presented according to this order.
  • In addition to search results, an advertising system may provide advertisements that are responsive to the query. For example, the search query may be used in keyword targeting to identify advertisements that are responsive to the query. These advertisements are provided with the search results.
  • Search systems can provide search suggestions to users to help users satisfy their informational needs. As used herein, the term “query suggestion” is a suggested data for a query that can be used to refine a search or refine a search strategy. Some search systems provide query suggestions in the form of a list of query suggestions as the user is typing a query. The user can select one of the query suggestions from the list without having to type the entire query suggestion. A client device typically sends suggestion requests to a search engine with each keystroke, and the search engine provides the query suggestions with prefixes that match the entered characters. Once received, the client device displays these suggestions for user selection.
  • Some search systems also provide search results as the user is entering a query. However, while a user is entering a query, the query is not complete. Accordingly, some keyword advertising targeting processes cannot rely on the incomplete query to identify advertisements.
  • SUMMARY
  • This specification describes technologies relating to providing search query suggestions and search results related to the search query suggestions.
  • In general, one aspect of the subject matter described in this specification can be embodied in methods that include the actions defining word stems, each word stem being one or more characters ordered in a sequence and to which one or more characters can be attached to the end of the sequence to form a word; receiving stem bids for the words stems, each stem bid being a bid for a corresponding word stem and corresponding to a price an advertiser pays for display of an advertisement targeted to the corresponding word stem, and wherein the targeting to the corresponding word stem is independent of keyword targeting; receiving a query stem from a client device, the query stem being a query input from a client device and being one or more characters ordered in an input sequence that defines an order in which the one or more characters were input as the query input; in response to receiving the query stem: identifying word stems that match the query stem, providing the corresponding stem bids of the matching word stems as bids to an advertisement auction for advertisement slots for displaying advertisements, and receiving selected advertisements that are determined to have won an advertisement slot in the auction; and providing the selected advertisements for display in the advertisement slots on the client device.
  • Particular embodiments of the subject matter described in this specification can be implemented so as to realize one or more of the following advantages. Advertisements can be targeted on query stems, allowing advertisers with objectives such as building brand recognition, to serve advertisements on otherwise ambiguous query inputs. The advertisements can be charged based on the duration which the advertisements were displayed, thereby allowed advertisers to better manage their advertising ROI.
  • Each of the advantages above is optional and can be achieved separately or in combination.
  • The details of one or more embodiments of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • FIG. 1 is a block diagram of an example environment in which a search engine provides search services.
  • FIG. 2 is a timing diagram of an example process for determining if a timer expires and search results should be provided to a client.
  • FIG. 3 is a timing diagram of an example process in which a confirmation of a query suggestion results in a prediction criterion being met.
  • FIG. 4 is a timing diagram of a process for monitoring multiple timers.
  • FIGS. 5A-5I depict a web search page displaying query suggestions and search results responsive one of the query suggestions.
  • FIG. 6 is a flow diagram of a process for providing search results responsive to a query suggestion to a client device.
  • FIG. 7 is a flow diagram of a process for determining if a prediction criterion is met.
  • FIG. 8 is a flow diagram of a process for indicating a query suggestion for which search results are responsive.
  • FIG. 9 is a flow diagram of a process for displaying second search results responsive to a second query suggestion.
  • FIG. 10 depicts a web search page displaying query suggestions for a query stem, search results responsive to one of the query suggestions, and advertisements responsive to the query stem.
  • FIG. 11 is a graph of illustrating an amount based on a relationship between a stem bid and a display duration.
  • FIG. 12 is a flow diagram of a process for selecting advertisements responsive to a query stem.
  • FIG. 13 is a block diagram illustrating the selection of resources for a query input in response to a prediction criteria being met and based on resource metrics.
  • Like reference numbers and designations in the various drawings indicate like elements.
  • DETAILED DESCRIPTION §1.0 Overview
  • This specification describes technologies relating to providing query suggestions in response to a query suggestion request and providing search results for at least one of the suggestions if a prediction criterion is met. When the prediction criterion is met, search results are provided to a client device associated with the query suggestion request and presented in a search interface. The search results are provided to the client device without receiving a search result request (e.g., without a user selecting one of the query suggestions or causing the client device to send a search request to the search engine).
  • The prediction criterion can be a probability, a timeout or some other signal of likely user interest. For example, a determination can be made that the probability a user will select one of the query suggestions is above a threshold value. Search results associated with the query suggestion with the probability above the threshold value can be provided to the client device. Alternatively, a determination can be made that a timeout occurred and search results associated with a highest ranked query suggestion (e.g., a most frequent query suggestion or a query suggestion with the highest probability of being selected) can be provided to the client device. Alternatively, detection of a language boundary (e.g., a syllable, a word, or a bi-gram) can cause a prediction criterion to be met.
  • Various user interface techniques are used to create a user experience in which the search results appear and disappear in a smooth and unobtrusive manner.
  • §1.1 Example Environment
  • FIG. 1 is a block diagram of an example environment 100 in which a search engine 110 provides search services. A computer network 102, such as a local area network (LAN), wide area network (WAN), the Internet, or a combination thereof, connects publisher web sites 104, client devices 106, and the search engine 110. The online environment 100 may include many thousands of publisher web sites 104 and client devices 106.
  • A web site 104 is a one or more web page resources 105 associated with a domain name, and each web site is hosted by one or more servers. An example web site is a collection of web pages formatted in hypertext markup language (HTML) that can contain text, graphic images, multimedia content, and programming elements, such as scripts. Each web site 104 is maintained by a publisher, e.g., an entity that manages and/or owns the web site.
  • A resource is any data that can be provided by the publisher 104 over the network 102 and that is associated with a resource address. Resources include HTML pages, word processing documents, and portable document format (PDF) documents, images, video, and feed sources, to name just a few. The resources can include content, such as words, phrases, pictures, and so on, and may include embedded information (such as meta information and hyperlinks) and/or embedded instructions (such as JavaScript scripts).
  • A client device 106 is an electronic device that is under the control of a user and is capable of requesting and receiving resources over the network 102. Example client devices 106 include personal computers, mobile communication devices, and other devices that can send and receive data over the network 102. A client device 106 typically includes a user application, such as a web browser, to facilitate the sending and receiving of data over the network 102. The web browser can enable a user to display and interact with text, images, videos, music and other information typically located on a web page at a website on the world wide web or a local area network.
  • To facilitate searching of these resources 105, the search engine 110 identifies the resources by crawling the publisher web sites 104 and indexing the resources provided by the publisher web sites 104. The indexed and, optionally, cached copies of the resources are stored in an indexed cache 112.
  • The client devices 106 submit search queries 109 to the search engine 110. The search queries 109 are submitted in the form of a search request that includes the search request and, optionally, a unique identifier that identifies the client device 106 that submits the request. The unique identifier can be data from a cookie stored at the client device, or a user account identifier if the user maintains an account with the search engine 110, or some other identifier that identifies the client device 106 or the user using the client device.
  • In response to the search request, the search engine 110 uses the indexed cache 112 to identify resources that are relevant to the queries. The search engine 110 identifies the resources in the form of search results 111 and returns the search results to the client devices 106 in search results page resource. A search result is data generated by the search engine 110 that identifies a resource that satisfies a particular search query, and includes a resource locator for the resource. An example search result can include a web page title, a snippet of text extracted from the web page, and the URL of the web page.
  • The search results are ranked based on scores related to the resources identified by the search results, such as information retrieval (“IR”) scores, and optionally a separate ranking of each resource relative to other resources (e.g., an authority score). In some implementations, the IR scores are computed from dot products of feature vectors corresponding to a query and a resource, and the ranking of the search results is based on relevance scores that are a combination of the IR scores and authority scores. The search results are ordered according to these scores and provided to the client device according to the order.
  • The client devices 106 receive the search results pages and render the pages for presentation to users. In response to the user selecting a search result at a client device 106, the client device 106 requests the resource identified by the resource locator included in the selected search result. The publisher of the web site 104 hosting the resource receives the request for the resource from the client device 106 and provides the resource to the requesting client device 106.
  • In some implementations, the queries 109 submitted from client devices 106 are stored in query logs 114. Click data for the queries and the web pages referenced by the search results are stored in click logs 116. The query logs 114 and the click logs 116 define search history data 117 that include data from and related to previous search requests associated with unique identifiers. The click logs define actions taken responsive to search results provided by the search engine 110. The query logs 114 and click logs 116 can be used to map queries submitted by the client devices to web pages that were identified in search results and the actions taken by users (i.e., that data are associated with the identifiers from the search requests so that a search history for each identifier can be accessed). The click logs 116 and query logs 114 can thus be used by the search engine to determine the sequence of queries submitted by the client devices, the actions taken in response to the queries, and how often the queries are submitted.
  • An advertisement management system 130 facilitates the provisioning of advertisements with the resources 105. In particular, the advertisement management system 130 allows advertisers to define targeting rules that take into account attributes of the particular user to provide targeted advertisements for the users. Example targeting rules include keyword targeting, in which advertiser provide bids for keywords that are present in either search queries or webpage content. Advertisements that are associated with keywords having bids that result in an advertisement slot being awarded in response to an auction are selected for displaying in the advertisement slots.
  • These targeted advertisements can be provided for many different resources, such as the resources 105 of the publishers 104, on a search results page resource, and, as will be described in more detail below, with a resource that includes the search engine interface 120. For example, a resource 105 from the syndication publisher 104 includes instructions that cause the client device to request advertisements from the advertisement management system 130. The request includes a publisher identifier and, optionally, keyword identifiers related to the content of the resource. The advertisement management system 130, in turn, provides targeted advertisements to the particular user. When a user selects an advertisement, the client device 106 generates a request for a landing page of the advertisement, which is typically a webpage of the advertiser.
  • With respect to a search results page, the client device renders the search results page and sends a request to the advertisement management system 130, along with one or more keywords related to the query that the user provide to the search engine 110. However, as will be described in more detail below, in some implementations, advertisements can be based on keywords related to query suggestions, and the advertisements can be provided with the search results that are provided in response to a prediction criterion being met. In other variations, which are also described in more detail below, advertisements can also be provided based on a current query input that does not constitute a completed query input, e.g., advertisements can be provided based on a single character input, or on a current input that forms a stem for many different words. In a manner similar to keyword targeting, advertiser submit stem bids for word stems. When a query input is a query stem that matches a word stem, advertisements that are associated with query stems having stem bids that result in an advertisement slot being awarded in response to an auction are selected for displaying in advertisement slots.
  • The advertisement management system 130 includes a data storage system that stores campaign data 132 and performance data 134. The campaign data 132 stores advertisements, targeting information, and budgeting information for advertisers. The performance data 134 stores data indicating the performance of the advertisements that are served. Such performance data can include, for example, click through rates for advertisements, the number of impressions for advertisements, and the number of conversions for advertisements. Other performance data can also be stored. In some implementations, the performance data 134 can be partitioned based on queries that are actually entered by users (e.g., queries that a user types and then selects a “search” command) and on queries that are suggested. As the performance of advertisements may vary based on the query type (e.g., actually entered queries and suggested queries), the advertisements that are selected for a query consisting of particular words can vary on whether the query is actually entered or suggested.
  • The campaign data 132 and the performance data 134 are used as input parameters to an advertisement auction. In particular, the advertisement management system 130, in response to each request for advertisements, conducts an auction to select advertisements that are provided in response to the request. The advertisements are ranked according to a score that, in some implementations, is proportional to a value based on an advertisement bid and one or more parameters specified in the performance data 134.
  • §1.2 Query Suggestions
  • In addition to providing search results in response to queries, the search engine 110, by use of a query suggestion system 118, provides query suggestions 113 to a client device. The query suggestions 113 can be provided by the search engine 110 in response to a query suggestion request from the client device. The query suggestion system 118 accesses the search history data 117 and determines a list of query suggestions based on search queries that have the query characters as a stem (or, alternatively or in addition, queries that are related by topic or co-occurrence). In some implementations, the query suggestions are based on search history data 117 associated with the unique identifier of the client device that submitted the query suggestion request. For example, the query suggestions can be suggestions based on frequency co-occurrence and query stemming of queries stored in query logs 114, and then filtered according to the particular search history data 117 associated with the unique identifier.
  • To illustrate, assume a client device requests a search resource from the search engine 110. The search engine 110 provides the requested search resource and interface instructions to the client device. The search resource and interface instructions cause the client device to generate a search interface 120 that includes a query input field 122. For example, the client device can launch a web browser and request a web, an image, or a video search page from the search engine 110. In some implementations, the client device can request a search page that searches a combination of text, images, and/or videos. The search engine 110 can provide HTML and scripts that cause the client device to generate the search interface 120 with the query input field 122.
  • The query input field 122 can receive query characters from a user, e.g., keystroke inputs, and provides each input to the search engine 110 in the form of a query suggestion request. In response to the query suggestion request, the query suggestion system 118 identifies and ranks query suggestions according to an order from highest rank to a lowest rank, and provides the client device with the query suggestions 113. For example, the query suggestion system 118 can access the search history data 117 and determine query suggestions based on search queries that have the query characters “ba” as a query stem, e.g., “bank,” “banksy,” “Bankrate,” “ball,” etc.
  • Various ranking algorithms can be used. In some implementations, the query suggestions are ranked based on the probability of the query suggestion being selected by a user. In some implementations, the query suggestion with the highest rank is the shortest word with the highest probability of being selected by the user. In some implementations, the query suggestions are ranked or re-ranked so natural extensions are shown and grouped together.
  • The client device receives the query suggestions 113 and renders the first query suggestions according to an order. The first query suggestions can be presented in a query suggestion box 124 a according to an order from a highest rank to a lowest rank. For example, the client device can present the first query suggestions in the query suggestion box 124 a in the search interface 120. The query suggestions can be presented in the query suggestion box 124 a such that the first query suggestion (e.g., “bank”) has the highest probability of being selected by a user, the second query suggestion (e.g., “banksy”) has the second highest probability, and so on.
  • §2.0 Search Results for Predicted Query Completions
  • In response to the query suggestion request and providing the query suggestions, the search engine 110 determines if a prediction criterion is met. The prediction criterion is independent of a user selection on the client device of a query suggestion provided in the first query suggestions, and is independent of user selection on the client device to generate a search request, e.g., selection of a search button or the user pressing enter. When a prediction criterion is met, search results responsive to one of the query suggestions are sent to the client device and displayed.
  • For a time based prediction criterion, the search engine 110 can determine that the prediction criterion is met when a timer initialized in response to the query suggestion request expires after a predefined time period and no additional query suggestion requests are received during the predefined time period. For example, the search engine 110 can receive the first query suggestion request and initialize and start a timer that expires after one second. The search engine 110 can monitor the timer and determine that the prediction criterion is met if the timer expires after two seconds and before another query suggestion is received. If the timer expires before another query suggestion is sent, then search results responsive to one of the query suggestions are sent to the client device and displayed. Alternatively, if the relevant search results are already present on the client device (e.g., cached), then the search results stored in the cache are displayed.
  • In some implementations, probability data indicates, for each query suggestion, a respective probability of being selected by the user. Thus, in some implementations, the prediction criterion can be based on this probability, or can be combined with a time-based prediction criterion. The search engine 110 can identify the query suggestion with the highest respective probability relative to the probabilities of the other query suggestions provided to the client device, and then determine that the prediction criterion is met when the highest respective probability of the query suggestion exceeds a threshold probability that is greater than zero. For example, the search engine 110 can determine that the query suggestion “bank” has a higher probability of being selected than the query suggestions “banksy,” “Bankrate,” and “ball.” The search engine 110 compares the probability of the query suggestion “bank,” e.g., five percent, with a threshold probability, e.g., three percent, and determines that search results associated with the query suggestion “bank” should be sent to the client device and displayed, or, alternatively, displayed from the cache if they are already present on the client device.
  • In some implementations, the query suggestions can also be ordered so that natural extensions are shown and grouped together. In these implementations, the shortest query suggestion in a group can be presented first, even though the shortest query suggestion may not have the highest probability of being selected by a user. For example, “Bankrate” can have a probability of five percent and “bank” can have a probability of four percent of being selected by the user. The search engine 110 can compare the probability of the query suggestion “bank” to the threshold probability and determine that search results associated with the query suggestion “bank” are to be sent to the client device.
  • The prediction criterion can also be based on a language model. For example, a prediction criterion can be met if the search engine 110 detects a language boundary in the query input associated with the first query suggestion request. The language boundary can be a syllable, a word, or a bi-gram, to name a few examples.
  • The providing and displaying of search results in response to prediction criteria being met is illustrated in FIG. 1 in the four time-sequenced representations 120-1-120-4 of the search resource 120. In the representation 120-1, the user has input the partial query “ba”, and a search criterion has been met. Accordingly, the search results SR11, SR12, SR13 . . . are provided and displayed, as indicated in the representation 120-2.
  • As used herein, a “partial query” is a query input that can be completed to form completed query of one or more words. A partial query can thus be one or more keyboard inputs that define a prefix or stem of a word, or can also be an entire word or words. For example, the letter “a” is a partial query, is the letter “a” is the stem for the words “aardvark,” “Appalachia,” etc. The partial query itself can also be a word, such as the letter “a” is a word. Likewise, the word “New” is also a partial query, as it can be combined with “York”, “London”, etc., to form other queries, and the partial query “New York” can be combined with “Mets” or “Yankees” to form other queries.
  • When a partial query is a word stem of one or more characters ordered in a sequence and to which one or more characters can be attached to the end of the sequence to form a word, the partial query is also referred to as a query stem. Typically, a query stem does not constitute a complete word, or constitutes only a word of few characters, e.g., “a” or “an”. Query stems can be targeted separately from keywords, as will be discussed in more detail below.
  • When search results are to be displayed in response to a prediction criterion being met, the client device, executing the instructions, generates an indication in the search interface that indicates the query suggestion for which the results are responsive. In some implementations, the indication can be an automatic completion of a query input in the query input field 122 indicating the query suggestion for which the search results are responsive. The automatic completion includes differentiated text 126 a appended to the query input in the query input field 122. For example, the differentiated text in the query input field 122 can be presented with a gray scale background. In some implementations, the differentiated text 126 a has a color background. In some implementations, the font of the differentiated text 126 a can have a different font than the query input, e.g., bold, italics, font type, etc. The cursor in the text input field, however, remains at the last actual input, i.e., after the letter “a” in the stem “ba”, so that user may resume or continue typing to add to the stem “ba”.
  • Other indications can also be used. For example, the query suggestion “bank” can be highlighted in the query suggestion box 124 a, e.g., with a box or a background color.
  • The client device can display the first search results in substantial synchronization with the indication in the search interface 120 that indicates the first query suggestion for which the first search results are responsive. For example, the client device can display the search results “SR11,” “SR12,” “SR13,” etc., in the search interface 120 at the same time that the differentiated text 126 a is displayed in the search interface 120. In another example, the search results are presented after the differentiated text is shown, e.g., after a short delay (e.g., one second or less).
  • As described above, the interface instructions can cause the client device to generate a query suggestion request in response to each character input in the query input field 122. Suppose, for example, the user continues typing, and enters the letter “r”, as shown in the representation 120-3. The query input field 122 can receive the character “r” to form a query input of “bar.” The scripts can receive the query input of “bar” and cause the client device to provide “bar” as a subsequent query suggestion request to the search engine 110 (or, alternatively, only the letter “r” if the search engine 110 stores a current state of the input of the client device). Because the partial query input “bar” does not match the predicted query completion “bank”, the client device removes the search results SR11, SR12, SR13, etc., from view.
  • The search engine 110 receives the query suggestion request and provides second query suggestions responsive to the request from the client device. For example, the query suggestion system 118 can determine the second query suggestions based on the search history data 117 and the query input of “bar”, and the client device renders the second query suggestions according to the order of the query suggestions. For example, the client device presents the query suggestions in a query suggestion box 124 b in the order “hart,” “barbie,” “barney.”
  • The search engine 110 then determine if a prediction criterion for the second query suggestions is met. Assuming the prediction criterion is met, the search engine 110 provides another set of search results to the client device responsive to a second query suggestion from the second query suggestions. For example, the search engine 110 can determine that a timer associated with the second query suggestions expired after a predefined time period and before another query suggestion request was received from the client device.
  • The client device receives the search results and generates an indication in the search interface 120 that indicates the second query suggestion. For example, the client device can present differentiated text 126 b appended to the query input in the query input field 122, and displays the second search results “SR21,” “SR22,” “SR23,” etc., in substantial synchronization with generating the indication in the interface that indicates the second query suggestion.
  • §2.1 Example Timing Diagrams
  • FIG. 2 is a timing diagram 200 of an example process for determining if a timer expires and search results should be provided to a client. The timing diagram 200 depicts a process associated with a client 202 and a search service 204 as time elapses downward along a vertical axis t. The timer can be initiated in response to receiving a request for query suggestions, or can be initiated in response to the search service 204 providing the suggestions. In the examples that follow, the timer is initiated when the query suggestions are provided. Time periods relative to other events can also be used to define a prediction criterion, e.g., a time period after search suggestions have been presented but have not receive user interaction, or a time period between consecutive query suggestion requests, to name a few examples.
  • At a time t0, the client 202 provides a first query suggestion request Rb to the search service 204 in response to receiving a query input character “b”. For example, the client 202 receives the query input character “b” from a query input field in a search interface.
  • The search service 204 identifies first query suggestions QSb responsive to the first query suggestion request Rb, and at time t1 provides the suggestions to the client 202. Additionally, the search service 204 determines that the query suggestion “bank” is the highest ranked query suggestion in the first query suggestions QSb and identifies search results SRbank for the query “bank.”
  • The search service 204 initializes and starts a timer T1 associated with search results SRbank. The timer T1 is set to expire after a predefined time period, e.g., at time t3, and the search service 204 monitors the timer T1 for an expiration timeout. The search service 204 provides first query suggestions QSb to the client 202 and the client 202 renders the first query suggestions QSb in the search interface. If the timer T1 expires and the search results SRbank are still responsive to the most recent query suggestion request, the search service 204 will provide the search results SRbank to the client 202.
  • At time t2, the client 202 provides a second query suggestion request Rbr to the search service 204. For example, the query input field receives a second character “r” as input from a user and the combined query input of “br” is used for the second query suggestion request Rbr.
  • At time t3, the timer T1 expires and the search service 204 determines that the search results SRbank are not responsive to the second query suggestion request Rbr. For example, the second character in “bank” is not “r” so the user was not typing “bank” into the query input field. Alternatively, the timer T1 is disabled at time t2 when the search service 204 determines that the search results SRbank are not responsive to the second query suggestion request Rbr. In either case, this results in the search service 204 not providing the search results SRbank.
  • At time t4, the search service 204 provides second query suggestions QSbr responsive to the second query suggestion request Rbr. The search service 204 determines that the query suggestion “brink” is the highest ranked query suggestion in the second query suggestions QSbr and identifies search results SRbrink. The search service 204 initializes and starts a timer T2 associated with search results SRbrink, where the timer will expire at time t5. The client 202 renders the second query suggestions QSbr in the search interface after they are received.
  • At time t5, the search service 204 determines that the timer T2 has expired and no query suggestion requests have been received since time t4, when T2 was initialized, and no other inputs have been received from the client device (e.g., an explicit command to perform a search for a particular query). The search service 204 provides the second search results SRbrink to the client 202 and the client 202 renders the second search results SRbrink in the search interface along with an indication of the query suggestion “brink” associated with the search results SRbrink.
  • FIG. 3 is a timing diagram of an example process in which a confirmation of a query suggestion results in a prediction criterion being met. The timing diagram 300 depicts a process associated with a client 302 and a search service 304 as time elapses downward along a vertical axis t.
  • The processes and steps with respect to times t0 and t1 are similar to those described with respect to FIG. 2 above. However, at time t3, prior to time t4, which is the expiration timeout for the timer T1, the search service 304 receives second query suggestions QSba. The second query suggestions QSba includes the query suggestion “bank” with the highest probability of being selected by a user. Here, the second search suggestion request, i.e., “ba”, can be interpreted as a confirmation that the search results SRbank are of interest to the user. Accordingly, the search service 304 interprets this as meeting a prediction criterion (i.e., receiving a confirmation of a query for pending search results for that query). Accordingly, the search results SRbank are sent with the query suggestions QSba before the timer T1 expires.
  • More than one prediction criterion can be used. For example, FIG. 4 is a timing diagram 400 of a process for monitoring multiple timers. The timing diagram 400 depicts a process associated with a client 402 and a search service 404 as time elapses downward along a vertical axis t. In FIG. 4, the search service 404 is able to process two or more sets of pending search results simultaneously and decide which set to send to the client device 402.
  • At time t0, the client 402 receives the character “b” entered in a query input field and input by a user. The client 402 provides a first query suggestion request Rb to the search service 404.
  • At time t1, the search service 404 sends first query suggestions QSb, which includes the highest ranked query suggestion “bank” in response to the first query suggestion request Rb. The search service 404 identifies first search results SRbank responsive to the query suggestion “bank.” The search service 404 initializes and starts a timer T1 associated with the first search results SRbank. The timer T1 is initialized with a first duration that expires at time t5. The search service 404 provides the first query suggestions QSb to the client 402 and the client 402 renders the first query suggestions QSb in a search interface that includes the query input field.
  • At time t2, the client 402 provides a second query suggestion request Rba to the search service 404 in response to the user entering “a” in the query input field. The second query suggestion request Rba is received less than a threshold time period after the first query suggestion request Rb. For example, the query input field is receiving query input characters from a user who is a fast typist.
  • At time t3, the search service 404 sends second query suggestions QSba, which again has “bank” as the highest ranked query suggestion in response to the second query suggestion request Rba. However, the query suggestions for the input “ba” also have a very highly rated query suggestion “bart.” Accordingly, the search service 404 starts a second timer T2 for search results SRbart. If no additional suggestion requests are received, then at the expiration of the timer T1 the search results SRbank are provided. However, if the user enters the letter “r”, then the search results for SRbank are purged, the timer T2 continues, however, if no additional requests are received, and when the timer T2 expires, the search results SRbart are provided.
  • The process of queuing multiple sets of search results can be extended for subsequent inputs. For example, if the client 402 sends the third query suggestion request Rbar (not shown), the search service 404 may identify the next query suggestion with the highest rank in the third query suggestions QSbar, for example “barn” and third search results SRbarn associated with the query suggestion “barn.” The search service 404 can initialize and start a timer T3 (not shown) associated with the third search results SRbarn. The search results SRbart and SRbarn, and, optionally, their respective timers, are then handled in a manner similar to the way the search results SRbank and SRbart and the timers T1 and T2 were handled.
  • In other implementations, if the time period between query suggestion requests remains short (e.g., less than a threshold time period Tz apart), the client 402 does not present search results in the search interface, regardless of whether the query suggestion request confirms a set of search results SR. This is because fast typing is interpreted as a signal that the user has a specific intent to enter a particular query, and thus does not need search results in the form of suggestions.
  • For example, suppose the user knows a particular phrase that he or she intends to enter, e.g., “banking bill held up in house of representatives.” The user may type this phrase quickly and then select a “search” command in the interface 120. During this time, the user is likely to ignore any search results that are provided, and thus the search service 404 need not provide search results. For example, if at time tx after t2, the search service 404 receives a third query suggestion request Rban, and the third query suggestion request Rban is less than a threshold time period apart from the second query suggestion request Rba, the search service 404 can extend the timer T1 to time ty so that search results are not sent to the client 402 at time t4. As long as the user continues to type quickly, the timer T1, or any other timers for any other search results SR that may be pending, can continue to be extended.
  • In some implementations, the time periods associated with consecutive query suggestion requests can be customized to a user. For example, the search service 404 can determine that a first client used by a first user provides query suggestion requests faster than the standard predetermined time period, e.g., the user may be a fast typist compared to many other users. The standard time period can be, for example, a half a second, and the search service can determine that the first client is providing query suggestion requests every third of a second. The search service can use the customized time period (e.g., a third of a second) to determine when to provide search results to the first client. In one example, the search service 404 can identify a second client based on a client identifier. The search service 404 can use client search history information associated with the second client to determine a customized time period. This threshold time can be reduced to a minimum amount, e.g., an amount that is greater than the threshold time period Tz.
  • The example time periods above are illustrative and not exhaustive, and other time periods can also be used. For example, when the search service 404 determines that consecutive query suggestion requests are received less than a threshold time period apart, the search service 404 does not check to see if prediction criterion is met until after a time period greater than the threshold time period has passed after receiving the last query suggestion request. Conversely, a pause between query suggestion requests (e.g., greater than the threshold time period) may indicate that one of the query suggestions interests the user and the search service 404 should send search results for the highest ranked query suggestion or the query suggestion with the highest probability of being selected by the user.
  • In alternate implementations, the search service 404 provides the search results SR to the client device automatically with the query suggestions, and the timers are processed locally on the client device. Accordingly, the search service 404 need not monitor the state of the client device, as the client device 402 is monitoring its own state with local timers. The search results that are provided are not immediately rendered, and include metadata indicating the query suggestion to which they are responsive. At a later time (e.g., at the expiration of a timer on the client device), the client 402 searches the local cache for the relevant search results.
  • In some implementations, the search service 404 does not provide search results to the client device 402 when the search results have been sent to the client device 402 previously. For example, the search service 404 can store data (e.g., that expires after the end of the session) indicating the search results that have been provided to the client device 402 and the length of the cache timer associated with the results. If the search service 404 determines that a prediction criterion is met for search results that were already sent to the client device 402 (e.g., during the same session), the search service 404 does not provide the search results to the client. Examples of such data indicating the search results that have been sent include a query suggestion for which search results were sent or the search results for the query suggestion, or other data that identifies the query suggestions and search results that were previously sent to the client device.
  • In some implementations, the query suggestions and/or the search results can also be set to expire from the local cache after a predetermined amount of time. The expiration time can be based on the time sensitivity of query suggestions. Generally, topical queries (e.g., news, pop culture) are more time sensitive than non-topical queries (e.g., history, medicine, etc.). For example, query suggestions related to news can be set to expire after one hour, and query suggestions related to roman history can be set to expire after one day (or longer).
  • As described above, other prediction criterion can also be used, e.g., the probability of a query suggestion being selected being in excess of a threshold, or the current query input meeting a language boundary criterion, such as a word, a syllable, or an n-gram. The search service 404 (or the client 402) may implement any one of the above prediction criterion either alone or in combination with each other.
  • §2.2 Example Search Result Serving Delay Criteria
  • In some implementations, the search service 404 can determine that the search service 404 is unable to provide search results for all query suggestion requests received or all prediction criteria met, e.g., based on limited resources. The search service 404 can invoke serving delay criteria in order to determine when to provide search results. For example, the search service 404 can prioritize query suggestion requests based on timers, the time period between consecutive query suggestion requests, and/or probabilities that a user will select a query suggestion request. Similarly, the search service 404 can determine that there is extra bandwidth or resources that are available, and the search service 404 can remove or reduce the serving delay criteria. In some implementations, the search service 404 can invoke different delay criteria for different clients based on the number of query suggestion requests received from respective clients, the number of prediction criteria met for respective clients, based on the time interval between consecutive query suggestion requests for a respective client, and/or based on the time interval between the determination of consecutive prediction criteria being met for a respective client.
  • In some implementations, the search service 404 can adjust the duration of timers associated with search results. For example, if the search service 404 determines that it is unable to provide search results to some clients based on the current prediction criteria, the search service 404 can increase a global time period for all timers that determine when search results are served to a client.
  • In some implementations, the search service 404 can provide search results to a client every n times a prediction criterion is met or a query suggestion request is received. For example, the search service 404 can determine that there are not available resources to provide search results to the client for every query suggestion request but there are enough resources to send search results to the client for every other query suggestion request from all clients.
  • In some implementations, the search service 404 can increase the probability threshold associated with the probability of a user selecting a query suggestion so that the search service 404 provides search results for fewer query suggestions based on the probability threshold being met. For example, the search service 404 may determine that it cannot serve search results for all pending search suggestions in which the prediction criterion is met, and thus may increase the probability threshold to a value that reduces the likelihood of the prediction criterion being met to a level for which all requests can again be served for which the prediction criterion is met.
  • In some implementations, the duration of a timer associated with search results is based on the probability associated with a query suggestion the search results are responsive to. For example, search results responsive to a query suggestion with a high probability of being selected by a user can be associated with a timer having a short duration, e.g., zero seconds, 100 milliseconds, or some other short duration, and search results responsive to a query suggestion with a low probability of being selected by a user can be associated with timer having a longer duration, e.g., a half a second. In some implementations, the duration is inversely proportional to the probability according to a function, such as a logarithmic function.
  • In some implementations, the search service 404 throttles the provision of search results to the client 402 based on the bandwidth or data connection capabilities of the connection with the client 402. In general, the higher the bandwidth or better the data connection capability (e.g., lag time), the more aggressive the search service is in providing search results. For example, the search service 404 can determine that a client has a high bandwidth connection and thus provides the client with search results in response to each query suggestion request received. Alternatively, the search service 404 determines that a client has a low bandwidth connection and provides the client with search results after a timer with a long duration expires.
  • §3.0 User Interface Examples
  • FIGS. 5A-5I depict a web search page 500 displaying query suggestions and search results responsive one of the query suggestions. The web search page 500 can be provided by the search engine or the search service to the client device or client, described above, in response to a request for a search resource (e.g., a search page).
  • The web search page 500 includes a search interface 502. The search interface 502 includes a query input field 504 and can display search results responsive to a query suggestion associated with a search query in the query input field 504. The web search page 500 can display web, image, video, product, or document search results, to name a few examples.
  • FIG. 5A shows a query suggestion box 506 a with query suggestions QSIt. The query suggestions “Italy,” “Italy map,” “Italian,” and “Italian History” are query suggestions responsive to the query input “It” in the query input field. For example, a user has entered the text “It” into the query input field but is unsure about the specific spelling of the word “Italy” or the topic that they are searching for (e.g., “Italy” or “Italian”). The search engine can provide query suggestions QSIt in the query suggestion box 506 a in order to help the user choose a search query.
  • The query suggestions QSIt can be ranked according to an order. For example, the query suggestions can be ranked based on the probability that the user will select the respective query suggestion. The ranking of the query suggestions according to probability is indicated by the numerical values 508 a adjacent to the query suggestions. These values are illustrative only and are not shown in the actual user interface. In the example ranking, “Italy” has the highest probability and a rank of one, “Italian” has the second probability and a highest rank of two, “Italy map” has the third probability and a highest rank of three, and “Italian History” has the lowest probability and a rank of four.
  • The search engine can re-rank the query suggestions QSIt so natural query extensions are presented together in a group. For example, according to the re-ranking, “Italy” has the highest rank, “Italy map” (with the same first term) has the second rank, “Italian” has the third rank, and “Italian History” has the fourth rank.
  • FIG. 5B shows the progression from FIG. 5A in response to a prediction criterion being met. In FIG. 5B, the search results are received at the client and the differentiated text “aly” 510 a is presented in the query input field 504. The input query combined with the differentiated text “aly” 510 a form one of the query suggestions (“Italy”) presented in the query suggestion box 506 a, e.g., the query suggestion with the highest rank.
  • Search results 514 a-c are presented in the search interface 502. The search results 514 a-c are responsive to the query suggestion “Italy” 512 a. For example, the search engine receives a query suggestion request RIt from the client device rendering the web search page 500 (e.g., a laptop displaying the web search page 500). The search engine identifies and provides query suggestions QSIt to the client device and the client device presents the query suggestions QSIt in the query suggestion box 506 a according to a rank. The search engine determines that a prediction criterion is met and sends the search results SRItaly associated with the query suggestion “Italy” 512 a to the web browser presenting the web search page 500. The web search page 500 presents the differentiated text “aly” 510 a and the search results 514 a-c in the search interface 502.
  • In some implementations, the client can render a proper subset of the search results. For example, the search service provides the search results SRItaly to the client, and the client only renders the search result 514 a without rendering the search results 514 b-c. After a predetermined time period of inactivity after rendering 514 a, e.g., one second, which may indicate the user is interested in the search result 514 a, the remaining search results may be rendered.
  • FIG. 5C shows the progression from FIG. 5B where the user enters the character “h” in the query input field 504 so that the query input presented is “Ith.” Here, the partial query “Ith” is not a stem (or otherwise part of) the query suggestions QIt. Accordingly, the search results 514 a-c are not responsive to the query suggestions QSIth. Thus, when the client device presents the query suggestions QSIth in the query suggestion box 506 b, the client device also fades out the search results 514 a-c, as indicated by the dashed outlines of the search results 514 a-c. The fading effect can be implemented by a script or other programming instructions.
  • Other ways of removing search results can also be used. For example, the search results 514 a-c are immediately removed from the search interface 502; they can be grayed out (e.g., the search results are changed to gray or another color and become partially transparent); or they can fade from view.
  • In other implementations, when the search results 514 a-c are no longer responsive to the query input, the search results remain in the search interface 502 in grayed-out form, but the search interface no longer allows user interaction (e.g., selection) of the search results 514 a-c in grayed out form. In an alternative implementation, the user can interact (e.g., mouse over, select, etc.) with the search results, and various actions can occur as a result. For example, mousing over a grayed out search result will cause it to appear in original form, and the user may select the search result to request the underlying resource it references. Alternatively, all of the search results may return to their original, selectable form, and the user query in the query input may revert to the query suggestion for which the search results are relevant. For example, in FIG. 5C, if the user were to mouse over search result 514 a, the search results 514 a-c would return to their normal form, and the query input would revert to “Italy”.
  • Returning now to the example of FIG. 5C, the query suggestions QSIth include the suggestions “Ithaca College,” “Ithaca,” and “Ithaca NY”, and are re-ranked as shown. For example, the search engine can rank the query suggestions QSIth according to the length of the query suggestion or so that natural extensions are shown and grouped together.
  • FIG. 5D shows the progression from FIG. 5C when a prediction criterion is met for the query suggestion “Ithaca” 512 b and search results are presented on the search interface 502. For example, the client device receives search results SRIthaca that are responsive to the query suggestion “Ithaca” 512 b. The client device can present the differentiated text “aca” 510 b in the query input field 504. The differentiated text “aca” 510 b are the characters from the query suggestion “Ithaca” 512 b that are not already present in the query input field 504.
  • In the examples above, search results are removed or added in response to query suggestion requests. However, there are additional ways in which search results can be show in the absence of any additional user input. One example way is by use of a presentation timeout, which occurs when a user takes no action for a specified period of time, e.g., 30 seconds, after search results are shown.
  • FIG. 5E shows the progression from FIG. 5D after a presentation timeout. A presentation timeout can occur when search results are presented in the search interface 502 for a predetermined time period and no user interaction within the search interface 502 is detected. For example, the search results 516 a-b were presented in the search interface 502 and user interaction with the search results 516 a-b during the predetermined time period was not detected. In response to the presentation timeout, the search interface 502 can automatically select a different query suggestion or re-rank the query suggestions, and notify the search engine of the selection. For example, the query suggestion “Ithaca NY” 512 c can be switched with the query suggestion “Ithaca” so that “Ithaca NY” 512 c is the highest ranked query suggestion in the query suggestion box 506 c. In response to the notification, the search engine provides search results (e.g., search results 518 a-c) for the newly selected query suggestion (e.g., “Ithaca NY”). In a variation of this implementation, instead of selecting a new query suggestion, the search interface 502 requests a new set of search results (e.g., a next page of search results, such as results 11-20 if results 1-10 are displayed) in response to the presentation timeout.
  • FIG. 5E also shows another example of how search results can be transitioned out of the search interface. Here, the search results 516 a-c fade out and move to the edge of the search interface 502 while the search results 518 a-c fade in and appear in the search interface 502. In other implementations, the search results 518 a-c scroll onto the search interface 502 while the search results 516 a-c scroll off the search interface 502.
  • In some implementations, a presentation timeout does not occur if the web search page 500 is minimized or no longer in focus (i.e., no longer the active view port). For example, a user can receive an email and switch to an email client. As the web search page 500 is no longer in focus, the timer for the presentation timeout is suspended. When the search page 500 returns to being the active view port, the timer resumes its countdown.
  • §3.1 Additional User Interface Features and Search Service Features
  • The user interface features and search service features described above do not constitute and exhaustive set of examples. Other user interface features and search service features can also be implements. Additional examples of user interface features and search service features are described in the following sections.
  • §3.1.1 Additional Query Input Field (“Searchbox”) Features
  • In some implementations, selection of the tab key allows a user to add or complete a single word in the query input field 504 at a time. For example, if the user pressed the tab key a first time in FIG. 5D, the portion of the differentiated text “aca” would be added to the query input “Ith” changing the query input to “Ithaca” and the updated differentiated text to “_NY.” Pressing the tab key a second time would change the query input to “Ithaca NY” and there would no longer be any differentiated text. Alternatively, the user may select from the query suggestions 512 b.
  • In some implementations, if the client detects an unexpected “space” character (e.g. two consecutive spaces after a complete word or space that creates an unexpected word break in a word), the current input is provided as a query. For example, the input “goog” may results in the suggestions of “google maps” or “google translate.” However, if the next query input is a space character, creating an input of “goog”, the input “goog” is used as a query for search results.
  • In some implementations, the client device will revert to the query input field when a user resumes typing. For example, the search results that are provided may scroll off the web page. If the user of the client devices uses a mouse to scroll down and review search results, and then resumes typing the query input, the client device will scroll up to query input field to resume modifying the current query input.
  • In some implementations, the query input field includes a command (e.g., an “x” button) on a side. Selection of the command clears the query input, and can additional send a notification to the search service to kill all pending searches and timers related to the query suggestions.
  • In some implementations, the status field can be displayed adjacent (e.g., on top of) the query input field. The status field can indicate the predicted query suggestion for which the current search results being shown. As the user continues to type, and the search results change, the status indicator can also change when a different query suggestion is used for the search results.
  • In some implementations, selecting an enter command enters the predicted completion query as the search query. In these implementations, to enter the current query input, the user selects an alternate key, such as a backspace, escape, or enters an extra (or unexpected) space character.
  • In some implementations, search results are not automatically displayed. Instead, one or more indicators are used to indicate when search results are ready to be displayed. For example, for each search query suggestion, a small round circle may be displayed and the circle is red when there are no search results to display for the suggestions, and the circle changes to green when search results are available to display. For example, circles 508 b in FIG. 5E, which represent rank and are not actually displayed, can instead be color coded red and green and actually be displayed. The user may use an arrow key or tab key, or mouse over, to change focus to certain suggestions, and when the focus is on a particular selection with a green indicator, the search results are displayed. Similarly, a progress bar that moves along as the user types, behind the text of the query input field, is displayed, and catches up with the cursor position in the query input text when the search results are received. Other indicators, such as bars in sequence (similar to a signal strength indicator), or the query input field being highlighted (e.g., the border changing to a blue color), can be used.
  • In some implementations, selecting a right arrow key causes a top-ranked query suggestion to be entered into the query input field. The user may then resume typing by adding to the suggestion. For example, a user may desire to enter the query “Mississippi blues festival.” Instead of typing in Mississippi, the user may type “Miss→blues festival.”
  • In some implementations, a partial query input, or a query stem, can be provided as a query to the search engine 110. In a first variation of these implementations, the partial query input can be provided to the search engine 110 as an additional query input in addition to the selected query suggestion. For example, for the partial query input “ama”, the search engine 110 may select the query suggestion “amazon rain forest” for identifying search results. However, the search engine may also conduct a search on the partial query input “ama”, and thus the search results that are shown in response to a prediction criteria being met include search results responsive to the query suggestion “amazon rain forest” and search results responsive to the query input “ama.” The respective search results for the query suggestion and the query input can be distinguished as described above with respect to FIG. 5F, e.g., color coded, etc.
  • In the second variation of these implementations, the search engine 110 will only process the partial query input if a confidence measure or quality measure associated with the query suggestions are below respective thresholds. For example, for the partial query input “klf”, the search engine 110 may determine that none of the query suggestions meet a confidence measure or equality measure. Accordingly, the search engine conducts a search only on the query “klf.” An example confidence measure is a frequency of occurrence of the query suggestion, a probability that the query suggestion will be selected by the user, or some other measure indicative that the query suggestion is a likely completion of the query input. An example quality measure for query suggestion is a value indicating the quality of the underlying resources referenced by search results responsive to the query suggestion. For example, if a query suggestion only identifies resources that are of little importance, e.g., resources that ranked low in an authority graph, the query suggestion can have a very low quality measure. Conversely, if the query suggestion identifies resources that are of high importance, e.g., resources are ranked very high in authority graph, the query suggestion can have a very high quality measure.
  • §3.1.2 Additions Suggestion Features
  • In some implementations, suggestions that are personalized based on the user's history, along with the search results that are provided, can be protected (e.g., encrypted). Such protection can be used to prevent replay attacks and other exploits that may compromise the user's privacy. For example, data for cookie exchanges, personalized suggestions and search results can be provided over a secure socket layer (SSL).
  • In some implementations, an on-screen virtual keyboard can be displayed (e.g., such as in the case of a mobile device user interface), and the virtual keyboard color code certain keys that correspond to likely completions by letter.
  • In some indications, indicators are used to indicate the probability of completion for each query suggestion, e.g., a bar shown to the right of the suggestion. Thus the user can visualize a confidence level shown for each suggestion.
  • In some implementations, the user may use an arrow key or tab key, or mouse over, to change focus to certain suggestions. When the focus changes to a particular suggestion, search results for that suggestion are shown. If the user does not select a suggestion, the user may resume typing to change focus back to the current query input.
  • In some implementations, query suggestions are continually formed from a last word or term of a query. The last word or term of a query is used as a basis for a new query suggestion, and thus the number of terms in the query suggestions can remain unbounded.
  • In some implementations, a query suggestion can include a command that, when selected, causes the display of a first (e.g., most highly ranked) search result for the query. For example, in FIG. 5F, mousing over (or otherwise selecting) one of the circles next to one of the search results 522 b will cause one of the corresponding search results 524 a, 524 b or 524 c to appear. Alternatively, selecting the circle will cause the client device to request the underlying resource referenced by the corresponding search result. For example, selecting the circle next to the search result “Ithaca NY” causes the client device to request the resource referenced by the search result 524 a.
  • §3.1.3 Additional Presentation Features
  • In some implementations, if a character is deleted from the query input (e.g., with the backspace or delete key), the search engine does not provide query suggestions to the client device, and additional query suggestions are not displayed, until a character is again added to the query input field 504.
  • In some implementations, if a character is deleted from the query input (e.g., by using a cut option), the search engine does not provide query suggestions to the client device and query suggestions are not displayed until the search engine determines that a character has not been deleted from the query input field 504 for a predetermined amount of time. For example, if the search interface 502 determines that changes to the query input are the removal of characters, the client device does not send a query suggestion request to the search engine until three seconds after the last character has been deleted. As with the presentation timer, the client device uses an edit timer in order to determine when to send another query suggestion request after character deletion or backspace.
  • In some implementations, when the search interface 502 does not detect interaction with the search results after a predetermined amount of time, search results for two or more of the query suggestions can be presented in the search interface. For example, the search result 518 a is responsive to “Ithaca,” the search result 518 b is responsive to “Ithaca NY,” and the search results 518 c is responsive to “Ithaca College.” Indication of the query suggestion associated with the respective search results can be presented to the side of the search results. For example, the search interface 502 can include a box to the left of each search result, where the box contains the text of the query suggestion associated with the search result.
  • In some implementations, multiple query suggestions can have a high priority or the same priority. When this occurs, at least one search result responsive to each of the multiple query suggestions that have the high priority or probability is rendered. FIG. 5G shows two query suggestions 512 d-e and search results 520 a-c responsive to the two query suggestions 512 d-e. For example, both of the two query suggestions 512 d-e have a high probability of being selected by a user. In these implementations, the background of the query suggestions (or the text of the query suggestions) are color coded in a query suggestion box 506 d and the background of the search results 520 a-c (or text of the search results) are color coded to match the query suggestion the results are responsive to. For example, the search results 520 a-b are responsive to the query suggestion “Ithaca College” 512 e and the search result 520 c is responsive to the query suggestion “Ithaca” 512 d.
  • Accordingly, the search results that are provided can be based on the top predicted query, or on several of the top predicted queries, e.g., grouped as in FIG. 5G, or in other implementations, based on the top results (e.g., the top 1 or 2 results) for each predicted query given the user's typed prefix.
  • In some implementations, pressing a particular key, such as a page down key, or placing the mouse in the scroll bar to scroll down, causes the client device to page through search results that are shown for a query suggestion while still maintaining the current cursor position in the query input. Thus, the user may resume typing at any time to continue to develop the query.
  • In some implementations, search results that are shown for suggestions do not disappear when the user continues to type. Instead, new search results may “scroll in” to the search page, e.g., from the top, and old search results may “scroll out” of the search page, e.g., to the bottom. Search results that are in the processing of scrolling downward and which are not search results that would be found for a current suggestion or query input are marked with an indication that indicates that the search results are “stale”, i.e., not relevant to the current query suggestion. An example of this interface is shown in FIG. 5H. Search results 526 are moving in a downward direction, as indicated by the arrow 527. The search results 526 a and 526 b are relevant to the query suggestion “Ithaca NY”, while the search results 526 c and 526 d are relevant to the query suggestion “Italy.” Accordingly, an indication, such as the search results 526 c and 526 d becoming transparent, or some other indicator, is generated to notify the user that the search results are stale.
  • In some implementations, the number of search results that are shown for a query suggestion vary, depending on whether the query suggestion is a navigational query or an informational query. A navigational query is a query that usually has only one satisfactory result, as indicated by a very high relevance score, by traffic patters, or a combination of relevance and traffic patterns. An informational query is a query that has many search results for which several or more the most popular underlying resources have a relatively evenly distributed selection rate. For an informational query, the search results are shown as above. For a navigational query, only the top result referencing the corresponding resource is shown.
  • In some implementations, query suggestions that cause search results that include images or other content that may distract users can be penalized in the ranking of query suggestions. Search results that provide rich content tend to attract users' attention, even if the content is not related to the users' current interests, resulting in artificial topic drift. Thus, in some implementations, the queries that cause these search results to be generated can be ranked lower than query suggestions that cause less distracting search results to appear (e.g., search results with primarily text content. In alternate implementations, the queries are not re-ranked, but the rich content is suppressed, e.g., images and video clips are not shown in the search results. In variations of these implementations, the re-ranking or suppression is dependent on the corpus being searched. For example, if the user is search web documents, re-ranking or suppression can be enabled; conversely, if the user is searching images, re-ranking or suppression is not enabled.
  • In some implementations, query suggestions that result in the identification of content that is classified as sensitive content, e.g., pornographic content as identified by one or more pre-existing porn classifiers, are not used for generating search results. For example, resources that are identified by a query suggestion can be checked by porn classifiers, and if a threshold number of the resources are classified as including porn (e.g., 10%, or any of the resources that would be referenced in the first page of search results, etc.), then the search results are not provided for the query suggestion. In alternate implementations, the search results are generated but are not shown automatically. Instead, a button is shown that indicates the content that will be shown is potentially sensitive, and the user must confirm that the search results can be shown.
  • §3.1.4 Additional Resource Management Features
  • In some implementations, when the client detects that characters are no longer being deleted from the query input field 504, the client can check a local cache for query suggestions and/or search results for the current query input in the query input field 504. For example, the client can provide a query suggestion request RIt to the search service and receive query suggestions QSIt and search results SRItaly. The client can render the query suggestions QSIt in the query suggestion box 506 a and store the query suggestions QSIt and the search results SRItaly in a local cache. The client then detects input of the character “h” in the query input field 504 and provides a query suggestion request RIth to the search service, receives the query suggestions QSIth, and renders the query suggestions in the query suggestion box 506 b. Thereafter, the user presses a backspace key to remove the character “h.” The edit timer fires, and then expires after a period of inactivity. The client then determines that the current query input is “It” and checks the local cache for query suggestions and search results associated with the query input “It.” Locating the query suggestions QSIt and the search results SRItaly, the client device then presents the query suggestions QSIt and the search results SRItaly (e.g., the search results 514 a-c) in the search interface 502. If the search results for the query suggestion Italy are not in the local cache, then the client can request the search results from the search service, or the search service can provide the search results, in response to a prediction criterion being met.
  • In some implementations, requests for query suggestions are not sent with each keystroke, but instead are sent according to a period time, e.g., every 150 milliseconds, or some other time period. This reduces the load on the search service while only causing minimal changes in the user experience.
  • In some implementations, the search service monitors the client device connection, and does not provide search results for a query suggestion if the client device connection is experiencing a high latency or is severely bandwidth constrained. Instead, the search service provides search results in a conventional manner, i.e., when the user issues a request for a specific query.
  • In some implementations, the client device is configured to progressively render the search results as they are received. In variations of this implementation, a search result that were previously provided and displayed can remain displayed when new search results are received for a new query suggestion, provide the search result that was previously provided and displayed is relevant to the new query suggestion.
  • §3.1.5 Mode Integration
  • In some implementations, the suggestion service and rendering of search results can be integrated in other web browser modes. For example, FIG. 5I shows a web browser page 530 in which the user has entered the character “e” in the address bar. Suggestions 532 are shown in response. Two of the suggestions, 532 b and 532 c, are URLs for web pages, as indicated by the web icon adjacent to the URLs. Two other suggestions, 532 a and 532 d, are search query suggestions, as indicated by the search icon adjacent the search query suggestions. By use of an input device, such as an arrow key, the user can highlight one of the suggestions 532, and corresponding content 536 is shown in a preview pane 538. The corresponding content is, for example, search result content or a web page preview.
  • For example, if the user highlights suggestion 532 b, as indicated by the dashed box 534 a, then a preview of the webpage at example1.com is shown in the preview pane 538. Another input, such as an enter key input, causes the client device to navigate to the previewed web page.
  • Conversely, if the user highlights suggestion 532 d, as indicated by the dashed box 534 b, then search results are shown in the preview pane 538. The user may interact with the search results as described above. If the user selects a particular search result, then the client device navigates to the page referenced by the underlying search result. Should the user navigate back, e.g., by selecting the back browser command 540, then the client device navigates to a search interface page of the search service, such as the page 500 shown in FIG. 5A.
  • §3.1.6 Logging, Preference and History Features
  • In some implementations, a user can specify preferences for providing search results. The preference can be stored locally on the client (e.g., in a cookie) and provided to the search service, or can be stored in a user account with the search service. The preferences include a setting for when search results are to be provided (e.g., a slider or a Likert scale to adjust the prediction criterion, with a low value/setting resulting in search results not being presented or only being presented after long times (e.g., 10 seconds), and a high value/setting resulting in search results being provided with each keystroke). For example, the user can choose to render results with every set of query suggestions, or can choose that the system be very conservative and only render results after several keystrokes, after an n-second pause, etc.
  • In some implementations, the search service can use information about a user to determine when to provide search results to a client device. For example, the search service can determine that the time period between consecutive query suggestion requests, provided by a client associated with the user, is typically less than a threshold period of time. The search service can determine that the current time period between consecutive query suggestion requests from the client are being received at a time period that is greater than normal and provide search results to the client based on this difference.
  • In some implementations, the search service can store as history data the search results that were shown for query suggestions, and the query suggestions for which they were shown. These suggestions and results can be differentiated from search queries for which a user explicitly requested a search. Query suggestions for which only some positive user interaction was observed—e.g., a long pause, a selection of a search result—are persisted to the user's history, while other query suggestions that did not result in some positive user interaction are not persisted to the user's history. Thus, the user, by examining his or her history, can review queries that he or she actually entered, and can also review query suggestions for which he or she exhibited a positive user interaction. As with other user history data, the user's search history and selections are protected by one or more privacy features to protect the user's privacy.
  • §4.0 Example Query Server Process Flows
  • FIG. 6 is a flow diagram of a process 600 for providing search results responsive to a query suggestion to a client device. The process 600 can be used by the search engine to provide search results to the client device.
  • The search engine receives from a client device a request for a search resource (602). For example, the search engine receives a request from the client device for a web search page.
  • The search engine provides to the client device in response to the request the search resource (604). The search resource includes interface instructions that cause the client device to generate a search interface that includes a query input field. For example, the search engine provides the web search page, including HTML and scripts, to the client device. The client device can use the HTML and scripts to render a search interface that includes a query input field.
  • The search engine receives query suggestion requests from the client device (606). Each query suggestion request is generated in response to a query input in the query input field. For example, the client device can generate a query suggestion request from query input entered into the query input field by a user. The client device can provide the query suggestion request to the search engine. The client device can determine that a word or part of a word is entered into the query input field and generate a query suggestion request for each character in the word or in the partial word. Alternatively, the client device can generate a query suggestion request for every n characters, or each word input into the query input field.
  • In response to each query suggestion request, the search engine provides query suggestions responsive to the request (608). For example, the search engine identifies query suggestions responsive to the query suggestion request and provides the query suggestions to the client device. The search engine can identify the highest ranked query suggestion or the query suggestion with the highest probability of being selected by a user. The search engine can identify search results associated with the identified query suggestion and start a timer associated with the search results.
  • The search engine determines if a prediction criterion is met (610). The prediction criterion is independent of a user selection of a query suggestion provided in response to one or more query suggestion requests. For example, the prediction criterion is a probability threshold, a timer expiration, or a combination of the probability threshold and timer expiration. If the search engine determines that the probability associated with the identified query suggestion is above a threshold probability, the prediction criterion is met. If the search engine determines that a timer associated with one of the search results has expired (e.g., one of the search results associated with one of the query suggestion requests), the prediction criterion is met. The timer can be associated with the search results for the most recent query suggestion request or search results from a query suggestion request received before the most recent query suggestion request.
  • If the prediction criterion is met, the search engine provides search results to the client device (612). The search results are responsive to one of the query suggestions provided in response to the query suggestion request or one or more previous query suggestion requests. For example, the search engine provides the search results associated with the identified query suggestion to the client if the probability criterion is met. The search engine can provide the search results associated with a timer if the timer expired. If the search engine receives another query suggestion request from the client device, the process 600 continues for the next query suggestion request (608).
  • If the prediction criterion is not met, the search engine does not provide search results to the client device (612). For example, if the search engine determines that the prediction criterion is not met, the search engine does not provide search results to the client device. If the search engine receives another query suggestion request, the process 600 continues for the next query suggestion request (608). While the search engine does not receive another query suggestion request, the process 600 continues to check whether a prediction criterion is met (610). In some implementations, the process 600 stops if the search engine determines that a prediction criterion cannot be met until another query suggestion request is received from the client device.
  • The order of steps in the process 600 described above is illustrative only, and processing of query suggestion requests can be done in different orders. For example, while the process 600 above describes providing query suggestions responsive to the query suggestion request, and then determining if a prediction criterion is met, the process 600 can be modified so that the process 600 determines if a prediction criterion is met and then provides query suggestions to the client, either with or without the search results.
  • In some implementations, the process 600 can include additional steps or some of the steps of the process 600 can be divided into multiple steps, as described above with reference to FIGS. 2-5E. For example, the process 600 can adjust a timer associated with search results when the search engine identifies the same search results as being responsive to two consecutive query suggestion requests. The process 600 can include shortening or extending the duration of the timer. In one example, the process 600 can include increasing a probability percent associated with a query suggestion that is responsive to two consecutive query suggestion requests. In one example, the process 600 can include adjusting the prediction criteria based on the resources available to the search engine.
  • Although described as being implemented in a server, portions of the process 600 can be performed in the client device. For example, the client device can determine if prediction criteria are met, and request corresponding search results from the search engine when the prediction criteria are met. The search engine, in turn, provides the search results and client device renders the search results as described above. Alternatively, the search results can be provided with each set of query suggestions, and when a prediction criterion is met, the client device renders the search results that are stored in the browser cache.
  • FIG. 7 is a flow diagram of a process 700 for determining if a prediction criterion is met. The process 700 can be used by the search engine 110 to provide search results to the client device 106 or by the search service 404 to provide search results to the client 402. The process 700 can be performed as a part of the process 600 or as a separate process. For example, a search service performing the process 600 can perform the process 700 instead of steps 612-616.
  • The search engine identifies a query suggestion with the highest respective probability relative to the probabilities of the other query suggestions provided (702). For example, each query suggestion has a respective probability of being selected by a user. In some implementations, the search engine receives a query suggestion request from a client device and identifies query suggestions responsive to the request. The search engine can identify the query suggestion with the highest respective probability and identify search results responsive to the identified query suggestion.
  • The search engine determines if the highest respective probability exceeds a threshold probability (704). The threshold probability is greater than zero. For example, the search engine has a predetermined threshold probability. The search engine compares the highest respective probability with the threshold probability to determine if the highest respective probability exceeds the threshold probability. In some implementations, the search engine has more than one threshold probability and the search engine determines the current serving delay criteria associated with the client that will be provided with the query suggestions, as described above.
  • If the probably exceeds the threshold, the search engine provides search results to a client device responsive to the identified query suggestion (706). The search results provided are responsive to the query suggestion with the highest respective probability. For example, if the highest respective probability exceeds the threshold probability, the search engine provides the search results associated with the identified query suggestion to the client. In some implementations, the search engine provides the query suggestions, including the identified query suggestion and the other query suggestions, to the client device with the search results.
  • Conversely, if the probably doe not exceed the threshold, the search engine does not provide search results to a client device (708).
  • In some implementations, the search engine receives another query suggestion request and the process 700 continues to identify a query suggestion with the highest probability for the query suggestions responsive to the received query suggestion request (702).
  • §5.0 Example Client Query Process Flows
  • FIG. 8 is a flow diagram of a process 800 for indicating a query suggestion for which search results are responsive. The process 800 can be used by the client to receive query suggestions from the search engine.
  • The client generates a search interface that includes a query input field (802). For example, the client receives HTML and scripts from the search service and generates a web search page. The web search page includes a query input field for receiving character input from a user.
  • The client provides query characters input in the query input field to a search service as a first query suggestion request (804). For example, the client receives query characters in the query input field from a user. The client sends the first query suggestion request to the search service in response to receiving the query characters. The client can send a query suggestion request for every character, every n characters, or every word entered into the query input field.
  • The client receives first query suggestions in response to the first query suggestion request (806). For example, the client receives the first query suggestions from the search service in response to the first query suggestion request.
  • The client renders the first query suggestions according to an order (808). For example, the first query suggestions can include a ranking order. The client can present the first query suggestions in the search interface according to the ranking order. The client can render the first query suggestions below the query input field. In some implementations, the client can render the first query suggestions to the left or the right of the query input field.
  • The client generates an indication in the search interface that indicates the first query suggestion for which first search results are responsive (810). The generation of the indication is performed in response to the client receiving the first search results responsive to one of the first query suggestions. The first search results are received independent of user selection of one of the query suggestions and independent of sending a completed query to the search service. For example, the first search results are received independent from the user selecting one of the first query suggestions, selecting a search or a send button, or pressing enter. The indication can be presentation of differentiated text in the query input field after the query character input. The differentiated text and the query character input can form the word or words for which the first search results are responsive. In other implementations, the indication can be highlighting the first query suggestion in a query suggestion box.
  • In some implementations, the process 800 can include additional steps or some of the steps of the process 800 can be divided into multiple steps, as described above with reference to FIGS. 2-5E.
  • FIG. 9 is a flow diagram of a process 900 for displaying second search results responsive to a second query suggestion. The process 900 can be used by the client to receive query suggestions from the search service. In some implementations, the client can perform the process 900 after performing the process 800.
  • The client displays first search results in substantial synchronization with an indication in an interface that indicates a first query suggestion for which the first search results are responsive (902). For example, the client presents the first search results in a search interface at the same time that the differentiated text is presented in the query input field. In one example, the client presents the indication of the first query suggestion and after a delay of one second or less presents the first search results.
  • The client receives second search results responsive to a second query suggestion (904). The second query search results are received subsequent in time to the first query suggestion for which the first search results are displayed. For example, the client may have provided a second query suggestion request, received the query suggestions, and prediction criterion for the second query suggestion is met.
  • The client generates an indication in the interface that indicates the second query suggestion (906). For example, the client can present differentiated text in a query input field or highlight the second query suggestion in a query suggestion box.
  • The client removes the first search results (908). For example, the client fades out the first search results, removing the first search results from the search interface.
  • The client displays the second search results in substantial synchronization with generating the indication in the interface that indicates the second query suggestion (910). For example, the client displays the second search results one second after presentation of the differentiated text or highlighting of the second query suggestion. In some implementations, the second search results fade in on the search interface while the first search results fade out from the search interface.
  • The order of steps in the process 900 described above is illustrative only, and processing of query suggestion requests can be done in different orders. For example, while the process 900 above describes removing the first search results, then displaying the second search results, the process 900 can be modified so that the first search results are removed at the same time that the second search results are displayed.
  • In some implementations, the process 900 can include additional steps or some of the steps of the process 900 can be divided into multiple steps, as described above with reference to FIGS. 2-5E.
  • §6.0 Advertisement Bidding and Management
  • In some implementations, when a query suggestion is selected, keywords related to the query suggestions are also selected, and advertisement targeted to the keywords are provided with the search results in response to a predication criterion being met. However, in these implementations, when the query input is a query stem, the query stem presents, at most, an ambiguous signal of the user's interest. For example, the query stem “p” is the first letter of many different words and queries that cover many different subjects and interests.
  • Thus, in some implementations, the advertisement management system 130 stores campaign data 132 that indicates whether an advertiser desires to target advertisements with search results that are provided in response to a prediction criterion being met. The advertisement management system 130 allows advertisers to “opt out” of targeting in these situations.
  • Should the advertiser decide to opt-in to targeting advertisements with search results that are provided in response to a prediction criterion being met, the advertiser can select several targeting options. The first targeting option targets advertisements based on keywords associated with query suggestions that are selected for providing search results in response to a prediction criterion being met. The first targeting option selects advertisements in a manner similar to the way advertisements are selected for queries that are explicitly entered by the user (e.g., when the user enters a query and selects a search command to submit the query to the search engine 110). Each time an advertisement is provided, an impression can be counted for the advertisement. Alternatively, as described in section 7 below, impressions may only be counted in the presence of the signal that a user has stopped typing and is viewing the search results and/or advertisements.
  • The second targeting option targets advertisements based on query stems. This targeting option is especially useful for advertisers that are attempting to meet one or more advertising objectives through the use of display advertisements. A display advertisement is an advertisement that is not necessarily directed to driving traffic to a particular landing page, but instead is directed to some other advertising goal, such as building brand recognition. Query Stem targeting, however, can also be used for other advertising objectives as well, including driving traffic to particular landing page.
  • FIG. 10 depicts a web search page 1000 displaying query suggestions 1006 for a query stem (e.g., the letter “g”), search results 1010 responsive to one of the query suggestions (e.g., query suggestion 1004), and advertisements 1022 and 1024 responsive to the query stem. Search results 1010 are responsive to the query suggestion 1004 (i.e., the query “gondola”), and are selected and shown as described above.
  • The advertisements are responsive to the query stem “g.” In particular, the advertisers have submitted send bids for word stems. Each word stem is one or more characters ordered in a sequence and to which one or more characters can be attached to the end of the sequence to form a word. For example, the letter “g” is a word stem for gondola, Gmail, games, and Gondwana, and other words. Although a query stem of only one letter shown in FIG. 10, query stems can also include two or more letters, e.g., a, aa, ab, etc.
  • The advertisers can review the query stems by use of the user interface, and provide stem bids for query stems that they desire to bid upon. Each stem bid is a bid for a corresponding word stem and corresponding to a price an advertiser pays for display of an advertisement targeted to the corresponding word stem. Example pricing schemes are described with respect to FIG. 11 below.
  • Typically, the targeting of a corresponding word stem foreign advertisement is independent of keyword targeting. In other words, a bid for an advertisement is submitted when a query stem is received, and the query stem need not correspond to a key word on which the advertiser has bid.
  • The advertisement management system 130, in response to receiving the query stem, identifies word stems that match the query stem, and provide corresponding stem bids of the matching word stems as bids to an advertisement auction for advertisement slots for displaying advertisements. For example, in FIG. 10, there are m advertisement slots available in the area 1014, and there is one advertisement slot available in the area 1012. Separate auctions can be conducted for both areas, or, alternatively, a single auction can be conducted for the slots in both areas.
  • Once the auctioning is complete, the advertisement management system 130 receives elected advertisements that are determined to have won an advertisement slot in the auction, and provides the selected advertisements for display in the advertisement slots on the client device.
  • The advertisement management system 130 can implement processes to reduce the number of auction operations. For example, it may not be practical to run an auction for each character that is input for query stem and which results in a request for query suggestions. For example, the user may type very quickly when inputting a complete query, and thus it would be inefficient to conduct advertisement auctions for each query input.
  • Thus, in some implementations, the advertisement management system 130 receives the query stem only in response to the search engine 110 (or the advertisement management system 130) determining that a prediction criterion is met. Accordingly, in these implementations, an advertisement auction is only run when search results are provided in response to a prediction criterion being met. In other implementations, the advertisement management system 130 provides the selected advertisements only in response to the search engine (or the advertisement management system 130) determining that a prediction criterion is met.
  • Although the advertisements shown in FIG. 10 are described as having been provided in response to an auction based on stem bids, advertisements can also be provided in response to an auction based on keywords associated with a selected query suggestion. For example, some of the advertisements 1024 may have been selected based on stem bids for the query stem “g”, and some of the advertisements may been selected based on keyword bids associated with keywords for the query suggestion “gondola.”
  • In some implementations, advertisers can manually select query stems and input bids for each query stem. For example, the advertisement management system 130 can provide a user interface that shows the most frequent query stems that occur, and frequency measures indicating the relative frequency of occurrence of each query stem. An advertiser can select various query stems and provide bids for each one.
  • In other implementations, the query stems and, optionally, stem bids for the query stems can be determined automatically from the keywords that an advertiser is currently bidding upon. For example, the advertisement management system can receive keywords on which an advertiser has bid. For each keyword, the advertisement management system 130 selects one or more characters that form a stem of the keywords and that do not include all of the characters of the keywords. For example, assume an advertiser has bid on the keyword “aardvark.” The advertisement management system 130 can select the query stems “a”, “aa”, and “aar.”
  • There are a variety of ways of the advertisement management system 130 can determine which query stems to select from the particular keyword. In some implementations, the advertisement management system selects up to a maximum n-gram length of query stems. For example, if the advertiser management system 130 is configured to select up to quad-grams for query stems, then for the keyword aardvark, the query stems a, aa, aar, and aard are selected.
  • In some implementations, the advertisement management system can also take into account the frequency the query stems. For example, query stems that occur below a threshold frequency can be precluded from automatic selection. Returning to the aardvark example, assume that the query stems “aar” and “aard” only occurred a frequency that is below the threshold frequency. Accordingly, the advertisement management system 130 does not select these to query stems for automatic bidding. The advertiser may, however, manually select these query stems.
  • To determine the stem bids for the automatically selected query stems, the advertisement management system identifies a keyword bid for the keyword from which the stem was derived, and determines the stem bid based on the identified keyword bid. In some implementations, the stem bid is proportional to the keyword bid. For example, a stem bid can be a percentage of a corresponding keyword bid, e.g., 10%. In other implementations, the stem bids can all be set to an initial fixed price, and the advertiser can manually adjust the prices.
  • In some implementations, the advertiser is charged an amount that is based on the stem bid when the advertisement is shown with the search results. However, a user may frequently receive search results but may not otherwise view the search results review the advertisements are provided with the search results. For example, the user may be a slow typist. Thus, an advertiser may not want to pay an auction cost for every occurrence of an advertisement being shown with the search results, as there may be many times that a user does not view the advertisements.
  • Accordingly, in some implementations, the advertisement management system 130 defines stem bids according to a cost per unit time of display. Using stem bids that are cost per unit time of display, the advertisement management system 130 can charge advertisers based on the amount of time that their advertisements were displayed.
  • In some implementations, the advertisement management system 130 receives data defining a duration that the selected advertisements were displayed in the advertisement slots on the client device after advertisements are provided to the client device. The duration can be determined by instructions on the client device that are provided with the search resource, e.g., a timer that starts when the advertisements are rendered, and stops when new advertisements are received or when the search results are removed from display. Alternatively, the duration can be determined by the search engine 110 or the advertisement management system 130 based on the time differential between previous advertisements being sent and a set of current advertisements being sent. Other ways determining the duration can also be implemented. For each selected advertisement that was displayed, the advertisement management system 130 determines an amount proportional to the stem bid of the selected advertisement and the duration, and charges each advertiser of a selected advertisement the amount for the selected advertisement.
  • FIG. 11 is a graph 1100 of illustrating an amount based on a relationship between a stem bid and a display duration. The horizontal axis represents time, and the vertical axis represents a cost incurred per unit time, e.g., the cost per second. For example, assume time T0 is the time at which an advertisement is rendered on the client device. At time T0, no cost is incurred. As time continues, because can increase according to a function defined relationship between the stem bid in unit time. A variety of functions can be used. For example, the function can be a first-degree polynomial function defining a relationship between the stem bid and unit time, as indicated by the line 1104. Alternatively, the function can be an n-degree polynomial function defining a relationship between the stem bid and unit time, with n being greater than 1, as indicated by the curves 1106 and 1108. Other functions can also be used, e.g., a sigmoid function defining a relationship between the stem bid and unit time.
  • In some implementations, the advertisement management system 130 can define a minimum display duration. This minimum duration is greater than 0 seconds, and if an advertisement is displayed only for less than the minimum duration, then no charges are incurred. For example, assume the time T0 is 100 ms, indicating that the advertisement has been displayed for at least 100 ms. After 100 ms, charges are incurred, as indicated by the amounts corresponding to the function lines 1104, 1106 and 1108.
  • In some implementations, the advertisement management system 130 can find a maximum display duration. The maximum display duration is the time after which no additional charges are incurred for display the advertisement. For example, assume T1 represents the maximum display duration, e.g., 5 seconds. As long as and advertisements displayed for a time less than the maximum display duration, the amount charged to the advertiser continues to increase and tell the maximum display duration is reached. Thereafter, the amount is a maximum amount that does not increase further. For example, the maximum amount CPI is reached at the time T1. In some implementations, this maximum amount may correspond to an amount that an advertiser is typically charged for an impression. Other maximum amounts can also be used.
  • In some implementations, the advertisements that are provided with search results persist in the display even after the corresponding search results are removed. In these implementations, advertisements are provided for a query stem can persist until the occurrence of another prediction criterion being met. For example, assume that the user continues to provide query inputs into the input field of the user interface shown in FIG. 10, typing the query stem “gar”. The search results 1010 may disappear, is there no longer relevant to the current query input; however, the advertisements may persist until the occurrence of another prediction event. In the event that advertisements have been provided for both query stems and keywords, e.g., some of the advertisement shown in FIG. 10 were provided for the query stem g, while other advertisements were provided for keywords associated with the word “gondola,” the advertisements based on the keywords may disappear with the search results, but the advertisements associated with the query stem “g” may persist. This behavior can be implemented in instructions that are provided with the search engine resource.
  • FIG. 12 is a flow diagram of a process 1200 for selecting advertisements responsive to a query stem. The process 1200 can be implemented in the advertisement management system 130.
  • The process 1200 defines word stems (1202). For example, the advertisement management system defines word stems, each of which are were more characters ordered in the sequence into which one or more characters can be attached to the end of the sequence to former word.
  • The process 1200 receive stem bids for the word stems (1204). For example, the advertisement management system receive stem bids from advertisers.
  • The process 1200 receives a query stem from a client device (1206). For example, the advertisement management system receives a query stem is a query input of one or more characters ordered in an input sequence that defines the order in which the one or more characters were input as the query input.
  • The process 1200 identifies word stems that match the query stem (1208). For example, the advertisement management system 130 identifies only word stems that exactly match the query stem received. In some implementations, the advertisement management system can also identify word stems for which the query stem constitutes a stem for the word stem. For example, if the query stem “a” is received, the word stems “a”, “aa,” and “ab” can be determined to match the word stem.
  • The process 1200 provides a corresponding stem bids of the matching word stems is bids to an advertisement auction for advertisement slots for displaying advertisements (1210). For example, the advertisement management system 130 can access the campaign data 132 to retrieve the stem bids from advertisers that have been on the matching word stems.
  • The process 1200 provides selected advertisements that are determined to be have one an advertisement slots in the auction (1212). For example, the advertisement management system 130 selects advertisements for which the stem bids resulted in the highest auction scores.
  • The process 1200 provides the selected advertisements for display in the advertisement slots on the client device (1214). For example, the advertisement management system transmits the advertisements to the client device, and the client device, in response to receiving the advertisements, renders them with the search results.
  • §7.0 Search Result Selections and Impression Attribution
  • As described above, the search engine 110 stores historical data in the form of click logs and query logs. The search engine 110 can also store historical data indicating when search results resulted in a user interaction. For example, if search results are provided and displayed in response to a prediction criterion, and then no additional query suggestion requests or search commands are received, the search engine 110 interprets this as a signal that the results satisfy the user's information needs to some degree. Accordingly, a “valid prediction count” for each of the search results shown can updated. The prediction count for each search result can be compared to its overall “prediction count”, i.e., the number of times the search result was provided in response to a prediction criterion being met, to determine a valid prediction rate. As different search results are provided for different suggestions, over time, highly relevant search results will emerge as having high valid prediction rates. Accordingly, the valid prediction rates can be used as a quality signal that is indicative of the quality of the underlying resources referenced by the search results.
  • In a variation of this implementation, a valid prediction count for a search result is only counted if there is a specific interaction with the search result. For example, a valid prediction count is only counted when a “mouse over” or selection of a search result is recorded.
  • In another implementation, the search engine 110 determines which resources referenced by search results are often selected from query suggestions that stem from the current query input. In response to the prediction criteria being met, the resources with the highest resource metric (e.g., the top N resources that are most often selected, or the top N resources as ranked by a function of their selections, their importance in a resource graph, and, optionally, other metrics) are provided as search results for the query input. The resource metrics of the resources are aggregated in such a manner that selections of the resources can result in search results referencing resources that would otherwise be responsive to different query suggestions that stem from the current query input.
  • The selections are illustrated in FIG. 13, which is a block diagram 1300 illustrating the selection of resources for a query input 1302 in response to a prediction criteria being met and based on resource metrics R. For the query input “a”, query suggestions for which the query input is a stem are selected. For example, as shown in FIG. 13, the set of query suggestions 1312-1316 are selected. In some implementations, all query suggestions that stem from the query input 1302 are selected. In other implementations, a subset of all query suggestions that stem from the query input 1302 are selected. The subset can be based on, for example, the query suggestions that are most often selected by users, or query suggestions that most often result in user interactions (e.g., selections of search results), or query suggestions with the highest confidence measures and or quality measures.
  • From the selected query suggestions, the search system 110 identifies sets of resources that are referenced by search results responsive to the query suggestions. For example, the first set of resources 1322 are referenced by search results responsive to the query suggestion 1312; the second set of resources 1324 are referenced by search results responsive to the query suggestion 1314; and the final set of resources 1326 are referenced by search results responsive to the query suggestion 1316. The sets of resources are not exclusive sets, i.e., a particular resource can be included in two sets, as indicated by the first resource in the set 1322 the second resource in the set 1324 be shaded to indicate they are the same resource.
  • The search system 110 then aggregates a quality metric R for each of the referenced resources in each set of resources. A variety of quality metrics can be used. For example, the metric can be a count of the number of times a particular resource was selected from the search results belonging to the sets 1322-1326; or can be a selection rate that is a ratio of the number of times particular resource was selected from the search results belonging to the sets 1322-1326 to the number of times the search results were presented; or can be a value based on the a function of selections, importance in a resource graph, and, optionally, other metrics.
  • The search engine 110 identifies the resources with the highest resource metric, and, in response to a prediction criteria being met, provides search results referencing these resources in response to the query input 1302. For example, search results referencing the top N resources, as ranked by their respective resource metrics, can be provided for the query input 1302. Alternatively, a subset of the search results provided can reference the top one or two resources as determined by the resource metric R, and the remaining search results provided can be for a top ranked query suggestion.
  • Similar processes can be used for advertisements that are provided with search results. For example, if no additional query suggestion requests follow the display of the search results, and no additional search commands are received, then an impression for each advertisement that is shown with the search results is counted.
  • With both attribution models above, if there is a user interaction responsive to the search results or an advertisement (e.g., a selection of a search result or an advertisement), corresponding impressions and the click-through are recorded.
  • In some implementations, when the client updates the search results presented in the search interface 502 after receiving new search results, the advertisements remain if they are still determined to be relevant to the query suggestion for which the search results are provided. For example, the search engine 110 may be in data communication with the advertisement management system 130. For each query suggestion and search result set, the search system/advertisement management system determines whether the advertisements that were previously provided are relevant to the search results current being provided. If so, the advertisements that are relevant remain displayed. The advertisements that are not relevant can be removed, and the available advertisement slots that result can be separately auctioned for new, relevant advertisements.
  • 8.0 Additional Implementation Details
  • Embodiments of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a tangible program carrier for execution by, or to control the operation of, data processing apparatus. The tangible program carrier can be computer-readable medium, such as a machine-readable storage device, a machine-readable storage substrate, a memory device, or a combination of one or more of them.
  • The term “data processing apparatus” encompasses all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.
  • A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub-programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.
  • The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit).
  • Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for performing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, to name just a few.
  • Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.
  • To provide for interaction with a user, embodiments of the subject matter described in this specification can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input.
  • Embodiments of the subject matter described in this specification can be implemented in a computing system that includes a back-end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front-end component, e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (“LAN”) and a wide area network (“WAN”), e.g., the Internet.
  • The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
  • While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

Claims (24)

What is claimed is:
1. A computer-implemented method, comprising:
defining word stems, each word stem being one or more characters ordered in a sequence and to which one or more characters can be attached to the end of the sequence to form a word;
receiving stem bids for the words stems, each stem bid being a bid for a corresponding word stem and corresponding to a price an advertiser pays for display of an advertisement targeted to the corresponding word stem, and wherein the targeting to the corresponding word stem is independent of keyword targeting;
receiving a query stem from a client device, the query stem being a query input from a client device and being one or more characters ordered in an input sequence that defines an order in which the one or more characters were input as the query input;
in response to receiving the query stem:
identifying word stems that match the query stem;
providing the corresponding stem bids of the matching word stems as bids to an advertisement auction for advertisement slots for displaying advertisements;
receiving selected advertisements that are determined to have won an advertisement slot in the auction; and
providing the selected advertisements for display in the advertisement slots on the client device.
2. The method of claim 1, wherein receiving a query stem comprises receiving the query stem only in response to determining that a prediction criterion is met, the prediction criterion being independent of a user selection of a query suggestion provided in response to the query stem and independent of receiving a completed query from the client device.
3. The method of claim 1, wherein providing the selected advertisements for display in the advertisement slots on the client device comprises providing the selected advertisements only in response to determining that a prediction criterion is met, the prediction criterion being independent of a user selection of a query suggestion provided in response to the query stem and independent of receiving a completed query from the client device.
4. The method of claim 1, wherein:
each stem bid defines cost per unit time of display; and
further comprising:
receiving data defining a duration that the selected advertisements were displayed in the advertisement slots on the client device;
for each selected advertisement, determining an amount proportional to the stem bid of the selected advertisement and the duration; and
charging each advertiser of a selected advertisement the amount for the selected advertisement.
5. The method of claim 4, wherein determining an amount proportional to the stem bid of the selected advertisement and the duration comprises:
determining if the duration exceeds a minimum duration that is greater than zero;
determining the amount to be zero if the duration does not exceed the minimum duration; and
determining the amount to be proportional to the stem bid of the selected advertisement and the duration if the duration does exceed the minimum duration.
6. The method of claim 4, wherein determining an amount proportional to the stem bid of the selected advertisement and the duration comprises:
determining if the duration exceeds a maximum duration;
determining the amount to be proportional to the stem bid of the selected advertisement and the duration if the duration does exceed the maximum duration; and
determining the amount to be a maximum amount if the duration does exceed the maximum duration.
7. The method of claim 4, wherein determining an amount proportional to the stem bid of the selected advertisement and the duration comprises determining an amount based on a first-degree polynomial function defining a relationship between the stem bid and unit time.
8. The method of claim 4, wherein determining an amount proportional to the stem bid of the selected advertisement and the duration comprises determining an amount based on an n-degree polynomial function defining a relationship between the stem bid and unit time, wherein n is greater than one.
9. The method of claim 4, wherein determining an amount proportional to the stem bid of the selected advertisement and the duration comprises determining an amount based on a sigmoid function defining a relationship between the stem bid and unit time.
10. The method of claim 1, wherein defining word stems comprises, for each of a plurality of advertisers:
receiving keywords on which the advertiser has bid; and
for each keyword, selecting one or more characters that form a stem of the keywords and that do not include all of the characters of the keywords.
11. The method of claim 10, wherein receiving stem bids for the words stems comprises:
for each word stem, identifying a keyword bid for the keyword from which the word stem was derived; and
determining the stem bid based on the identified keyword bid, the stem bid being proportional to the keyword bid.
12. A computer-implemented method, comprising:
defining word stems, each word stem being one or more characters ordered in a sequence and to which one or more characters can be attached to the end of the sequence to form a word;
receiving stem bids for the words stems, each stem bid being a bid for a corresponding word stem and corresponding to a price an advertiser pays for display of an advertisement targeted to the corresponding word stem;
receiving a query stem from a client device, the query stem being a query input from a client device and being one or more characters ordered in an input sequence that defines an order in which the one or more characters were input as the query input, and each query stem is received independent of a user selection of a query suggestion provided in response to the query stem and independent of receiving a completed query from the client device:
in response to receiving the query stem:
identifying word stems that match the query stem;
providing the corresponding stem bids of the matching word stems as bids to an advertisement auction for advertisement slots for displaying advertisements;
receiving selected advertisements that are determined to have won an advertisement slot in the auction; and
providing the selected advertisements for display in the advertisement slots on the client device.
13. A system, comprising:
a data processing apparatus; and
a computer storage medium encoded with a computer program, the program comprising data processing apparatus instructions that when executed by the data processing apparatus cause the data processing apparatus to perform operations comprising:
receiving stem bids for the words stems, each stem bid being a bid for a corresponding word stem and corresponding to a price an advertiser pays for display of an advertisement targeted to the corresponding word stem, and wherein the targeting to the corresponding word stem is independent of keyword targeting;
receiving a query stem from a client device, the query stem being a query input from a client device and being one or more characters ordered in an input sequence that defines an order in which the one or more characters were input as the query input:
in response to receiving the query stem:
identifying word stems that match the query stem;
providing the corresponding stem bids of the matching word stems as bids to an advertisement auction for advertisement slots for displaying advertisements;
receiving selected advertisements that are determined to have won an advertisement slot in the auction; and
providing the selected advertisements for display in the advertisement slots on the client device.
14. The system of claim 13, wherein receiving a query stem comprises receiving the query stem only in response to determining that a prediction criterion is met, the prediction criterion being independent of a user selection of a query suggestion provided in response to the query stem and independent of receiving a completed query from the client device.
15. The system of claim 13, wherein providing the selected advertisements for display in the advertisement slots on the client device comprises providing the selected advertisements only in response to determining that a prediction criterion is met, the prediction criterion being independent of a user selection of a query suggestion provided in response to the query stem and independent of receiving a completed query from the client device.
16. The system of claim 13, wherein:
each stem bid defines cost per unit time of display; and
the data processing apparatus instructions cause the data processing apparatus to perform operations further comprising:
receiving data defining a duration that the selected advertisements were displayed in the advertisement slots on the client device;
for each selected advertisement, determining an amount proportional to the stem bid of the selected advertisement and the duration; and
charging each advertiser of a selected advertisement the amount for the selected advertisement.
17. The system of claim 16, wherein determining an amount proportional to the stem bid of the selected advertisement and the duration comprises:
determining if the duration exceeds a minimum duration that is greater than zero;
determining the amount to be zero if the duration does not exceed the minimum duration; and
determining the amount to be proportional to the stem bid of the selected advertisement and the duration if the duration does exceed the minimum duration.
18. The system of claim 16, wherein determining an amount proportional to the stem bid of the selected advertisement and the duration comprises:
determining if the duration exceeds a maximum duration;
determining the amount to be proportional to the stem bid of the selected advertisement and the duration if the duration does exceed the maximum duration; and
determining the amount to be a maximum amount if the duration does exceed the maximum duration.
19. The system of claim 16, wherein determining an amount proportional to the stem bid of the selected advertisement and the duration comprises determining an amount based on a first-degree polynomial function defining a relationship between the stem bid and unit time.
20. The system of claim 16, wherein determining an amount proportional to the stem bid of the selected advertisement and the duration comprises determining an amount based on an n-degree polynomial function defining a relationship between the stem bid and unit time, wherein n is greater than one.
21. The system of claim 16, wherein determining an amount proportional to the stem bid of the selected advertisement and the duration comprises determining an amount based on a sigmoid function defining a relationship between the stem bid and unit time.
22. The system of claim 13, wherein defining word stems comprises, for each of a plurality of advertisers:
receiving keywords on which the advertiser has bid; and
for each keyword, selecting one or more characters that form a stem of the keywords and that do not include all of the characters of the keywords.
23. The system of claim 22, wherein receiving stem bids for the words stems comprises:
for each word stem, identifying a keyword bid for the keyword from which the word stem was derived; and
determining the stem bid based on the identified keyword bid, the stem bid being proportional to the keyword bid.
24. A non-transitory computer memory storage apparatus storing software data processing apparatus instructions that when executed by a data processing apparatus cause the data processing apparatus to perform operations comprising:
receiving stem bids for the words stems, each stem bid being a bid for a corresponding word stem and corresponding to a price an advertiser pays for display of an advertisement targeted to the corresponding word stem, and wherein the targeting to the corresponding word stem is independent of keyword targeting;
receiving a query stem from a client device, the query stem being a query input from a client device and being one or more characters ordered in an input sequence that defines an order in which the one or more characters were input as the query input:
in response to receiving the query stem:
identifying word stems that match the query stem;
providing the corresponding stem bids of the matching word stems as bids to an advertisement auction for advertisement slots for displaying advertisements;
receiving selected advertisements that are determined to have won an advertisement slot in the auction; and
providing the selected advertisements for display in the advertisement slots on the client device.
US13/213,909 2010-08-19 2011-08-19 Query stem advertising Abandoned US20120047025A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US13/213,909 US20120047025A1 (en) 2010-08-19 2011-08-19 Query stem advertising

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US37522210P 2010-08-19 2010-08-19
US13/213,909 US20120047025A1 (en) 2010-08-19 2011-08-19 Query stem advertising

Publications (1)

Publication Number Publication Date
US20120047025A1 true US20120047025A1 (en) 2012-02-23

Family

ID=44543862

Family Applications (7)

Application Number Title Priority Date Filing Date
US13/213,880 Active US8700653B2 (en) 2010-08-19 2011-08-19 Predictive query completion and predictive search results
US13/213,904 Active US8706750B2 (en) 2010-08-19 2011-08-19 Predictive query completion and predictive search results
US13/213,909 Abandoned US20120047025A1 (en) 2010-08-19 2011-08-19 Query stem advertising
US14/195,116 Abandoned US20140181135A1 (en) 2010-08-19 2014-03-03 Predictive query completion and predictive search results
US15/210,535 Active US9953076B2 (en) 2010-08-19 2016-07-14 Predictive query completion and predictive search results
US15/915,945 Active 2032-09-03 US11620318B2 (en) 2010-08-19 2018-03-08 Predictive query completion and predictive search results
US18/129,807 Pending US20230394072A1 (en) 2010-08-19 2023-03-31 Predictive query completion and predictive search results

Family Applications Before (2)

Application Number Title Priority Date Filing Date
US13/213,880 Active US8700653B2 (en) 2010-08-19 2011-08-19 Predictive query completion and predictive search results
US13/213,904 Active US8706750B2 (en) 2010-08-19 2011-08-19 Predictive query completion and predictive search results

Family Applications After (4)

Application Number Title Priority Date Filing Date
US14/195,116 Abandoned US20140181135A1 (en) 2010-08-19 2014-03-03 Predictive query completion and predictive search results
US15/210,535 Active US9953076B2 (en) 2010-08-19 2016-07-14 Predictive query completion and predictive search results
US15/915,945 Active 2032-09-03 US11620318B2 (en) 2010-08-19 2018-03-08 Predictive query completion and predictive search results
US18/129,807 Pending US20230394072A1 (en) 2010-08-19 2023-03-31 Predictive query completion and predictive search results

Country Status (6)

Country Link
US (7) US8700653B2 (en)
EP (2) EP2606441A1 (en)
AU (2) AU2011291549B2 (en)
CA (2) CA2808803C (en)
DE (2) DE212011100019U1 (en)
WO (2) WO2012024585A1 (en)

Cited By (57)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013000062A1 (en) * 2011-06-29 2013-01-03 The C-Factor Incorporated Method and apparatus for determining and delivering targeted content to a mobile platform
US20130018914A1 (en) * 2005-06-21 2013-01-17 W.W. Grainger, Inc. System and method for facilitating use of a selection guide
US20130174023A1 (en) * 2011-12-30 2013-07-04 Microsoft Corporation Presentation of rich search results in delineated areas
US20130246456A1 (en) * 2012-03-15 2013-09-19 Alibaba Group Holding Limited Publishing Product Information
US20130346496A1 (en) * 2012-06-26 2013-12-26 Yoelle Maarek System and method of predicting community member responsiveness
US20140129973A1 (en) * 2012-11-08 2014-05-08 Microsoft Corporation Interaction model for serving popular queries in search box
US20140280289A1 (en) * 2013-03-12 2014-09-18 Microsoft Corporation Autosuggestions based on user history
US20140324902A1 (en) * 2013-04-25 2014-10-30 Microsoft Corporation Automatic generation of a collection of content
US20140344250A1 (en) * 2013-05-20 2014-11-20 Microsoft Corporation Enhanced search refinement for personal information services
US20150012560A1 (en) * 2013-07-03 2015-01-08 Google Inc. Methods and systems for providing potential search queries that may be targeted by one or more keywords
US20150154316A1 (en) * 2013-12-02 2015-06-04 Qbase, LLC Search suggestions of related entities based on co-occurrence and/or fuzzy-score matching
US20150161143A1 (en) * 2012-06-01 2015-06-11 Zte Corporation Input processing method and device
US20150169643A1 (en) * 2012-05-14 2015-06-18 Google Inc. Providing supplemental search results in repsonse to user interest signal
US20150213041A1 (en) * 2013-03-15 2015-07-30 Google Inc. Search suggestion rankings
US20150234872A1 (en) * 2013-04-12 2015-08-20 Hitachi, Ltd. Computer, data processing method, and non-transitory storage medium
US9128982B2 (en) * 2010-12-23 2015-09-08 Nhn Corporation Search system and search method for recommending reduced query
US9135307B1 (en) * 2012-12-27 2015-09-15 Google Inc. Selectively generating alternative queries
US20150379570A1 (en) * 2014-06-30 2015-12-31 Wal-Mart Stores, Inc. Presenting Advertisement Content During Searches Of Digital Receipts
US9230013B1 (en) * 2013-03-07 2016-01-05 International Business Machines Corporation Suffix searching on documents
US20160019291A1 (en) * 2014-07-18 2016-01-21 John R. Ruge Apparatus And Method For Information Retrieval At A Mobile Device
US20160147421A1 (en) * 2014-11-24 2016-05-26 Facebook, Inc. Dynamic Status Indicator
US9361406B1 (en) * 2013-12-27 2016-06-07 Google Inc. Query completions
US20160182520A1 (en) * 2013-07-26 2016-06-23 Beijing Qihoo Technology Company Limited Method, apparatus and system for client accessing authenticated web address
US20160224531A1 (en) 2015-01-30 2016-08-04 Splunk Inc. Suggested Field Extraction
US20160283546A1 (en) * 2015-03-26 2016-09-29 International Business Machines Corporation Query strength indicator
US20170308289A1 (en) * 2016-04-20 2017-10-26 Google Inc. Iconographic symbol search within a graphical keyboard
US9827714B1 (en) 2014-05-16 2017-11-28 Google Llc Method and system for 3-D printing of 3-D object models in interactive content items
WO2018005903A1 (en) * 2016-06-30 2018-01-04 Zowdow, Inc. Systems and methods for enhanced search, content, and advertisement delivery
US9875287B1 (en) * 2014-05-21 2018-01-23 Google Inc. Providing a notification in place of content item
US20180024994A1 (en) * 2015-06-15 2018-01-25 Microsoft Technology Licensing, Llc Contextual language generation by leveraging language understanding
US9916368B2 (en) 2013-12-02 2018-03-13 QBase, Inc. Non-exclusionary search within in-memory databases
US9922344B1 (en) * 2012-02-24 2018-03-20 Google Llc Serving advertisements based on partial queries
US20180107953A1 (en) * 2015-09-17 2018-04-19 Tencent Technology (Shenzhen) Company Limited Content delivery method, apparatus, and storage medium
CN108073633A (en) * 2016-11-17 2018-05-25 百度在线网络技术(北京)有限公司 The bid of keyword and matching way setting method and device
US20180150507A1 (en) * 2015-01-30 2018-05-31 Splunk Inc. Command entry list for modifying a search query
US20190102479A1 (en) * 2017-09-29 2019-04-04 Korea Testing Certification (KTC) Plant material recommendation apparatus in use environment
USD852829S1 (en) * 2013-04-05 2019-07-02 Thales Avionics, Inc. Display screen or portion thereof with graphical user interface
US10397346B2 (en) 2014-11-24 2019-08-27 Facebook, Inc. Prefetching places
US10423638B2 (en) 2017-04-27 2019-09-24 Google Llc Cloud inference system
US10671182B2 (en) * 2014-10-16 2020-06-02 Touchtype Limited Text prediction integration
US10726037B2 (en) 2015-01-30 2020-07-28 Splunk Inc. Automatic field extraction from filed values
US10846316B2 (en) 2015-01-30 2020-11-24 Splunk Inc. Distinct field name assignment in automatic field extraction
US10863354B2 (en) 2014-11-24 2020-12-08 Facebook, Inc. Automated check-ins
US10896175B2 (en) 2015-01-30 2021-01-19 Splunk Inc. Extending data processing pipelines using dependent queries
US10949419B2 (en) 2015-01-30 2021-03-16 Splunk Inc. Generation of search commands via text-based selections
US10956502B2 (en) * 2018-05-18 2021-03-23 Yandex Europe Ag Method of and system for recommending fresh search query suggestions on search engine
US11030192B2 (en) 2015-01-30 2021-06-08 Splunk Inc. Updates to access permissions of sub-queries at run time
US11068452B2 (en) 2015-01-30 2021-07-20 Splunk Inc. Column-based table manipulation of event data to add commands to a search query
US20210397609A1 (en) * 2020-06-22 2021-12-23 S&P Global Automated information retrieval system and semantic parsing
US20220051288A1 (en) * 2012-03-01 2022-02-17 Google Llc Presenting options for content delivery
US11328324B1 (en) * 2013-08-30 2022-05-10 Groupon, Inc. Systems and methods for providing promotions based on search queries
US11327651B2 (en) * 2020-02-12 2022-05-10 Facebook Technologies, Llc Virtual keyboard based on adaptive language model
US11354308B2 (en) 2015-01-30 2022-06-07 Splunk Inc. Visually distinct display format for data portions from events
US11354514B2 (en) * 2017-11-14 2022-06-07 International Business Machines Corporation Real-time on-demand auction based content clarification
US11442924B2 (en) 2015-01-30 2022-09-13 Splunk Inc. Selective filtered summary graph
US11544248B2 (en) 2015-01-30 2023-01-03 Splunk Inc. Selective query loading across query interfaces
US11615073B2 (en) 2015-01-30 2023-03-28 Splunk Inc. Supplementing events displayed in a table format

Families Citing this family (146)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8232973B2 (en) 2008-01-09 2012-07-31 Apple Inc. Method, device, and graphical user interface providing word recommendations for text input
US11048765B1 (en) 2008-06-25 2021-06-29 Richard Paiz Search engine optimizer
US9600919B1 (en) 2009-10-20 2017-03-21 Yahoo! Inc. Systems and methods for assembling and/or displaying multimedia objects, modules or presentations
US8856115B1 (en) 2009-12-07 2014-10-07 Google Inc. Framework for suggesting search terms
US11423018B1 (en) 2010-04-21 2022-08-23 Richard Paiz Multivariate analysis replica intelligent ambience evolving system
US11379473B1 (en) 2010-04-21 2022-07-05 Richard Paiz Site rank codex search patterns
US10936687B1 (en) 2010-04-21 2021-03-02 Richard Paiz Codex search patterns virtual maestro
WO2012024585A1 (en) 2010-08-19 2012-02-23 Othar Hansson Predictive query completion and predictive search results
US8812733B1 (en) 2010-08-19 2014-08-19 Google Inc. Transport protocol independent communications library
KR20120021056A (en) * 2010-08-31 2012-03-08 삼성전자주식회사 Method for providing search service to store search result temporarily and display apparatus applying the same
KR101719268B1 (en) 2010-09-02 2017-03-23 삼성전자주식회사 Method for providing search service interconvertable search window and image display window and display apparatus applying the same
US8364709B1 (en) * 2010-11-22 2013-01-29 Google Inc. Determining word boundary likelihoods in potentially incomplete text
US8473507B2 (en) * 2011-01-14 2013-06-25 Apple Inc. Tokenized search suggestions
US9336329B2 (en) * 2011-05-18 2016-05-10 Koninklijke Philips N.V. Performing a search for a document
US8762356B1 (en) 2011-07-15 2014-06-24 Google Inc. Detecting change in rate of input reception
US8645825B1 (en) 2011-08-31 2014-02-04 Google Inc. Providing autocomplete suggestions
US9129606B2 (en) * 2011-09-23 2015-09-08 Microsoft Technology Licensing, Llc User query history expansion for improving language model adaptation
US10387503B2 (en) 2011-12-15 2019-08-20 Excalibur Ip, Llc Systems and methods involving features of search and/or search integration
US10504555B2 (en) 2011-12-20 2019-12-10 Oath Inc. Systems and methods involving features of creation/viewing/utilization of information modules such as mixed-media modules
US10296158B2 (en) 2011-12-20 2019-05-21 Oath Inc. Systems and methods involving features of creation/viewing/utilization of information modules such as mixed-media modules
US8880992B2 (en) * 2012-01-10 2014-11-04 Google Inc. Method and apparatus for animating transitions between search results
US9026631B2 (en) 2012-01-24 2015-05-05 International Business Machines Corporation Business-to-business social network
US11099714B2 (en) 2012-02-28 2021-08-24 Verizon Media Inc. Systems and methods involving creation/display/utilization of information modules, such as mixed-media and multimedia modules
US9495454B2 (en) * 2012-03-08 2016-11-15 Chih-Pin TANG User apparatus, system and method for dynamically reclassifying and retrieving target information object
US9378283B2 (en) * 2012-04-23 2016-06-28 Excalibur Ip, Llc Instant search results with page previews
US9843823B2 (en) 2012-05-23 2017-12-12 Yahoo Holdings, Inc. Systems and methods involving creation of information modules, including server, media searching, user interface and/or other features
US8671106B1 (en) * 2012-05-23 2014-03-11 Google Inc. Indicators for entities corresponding to search suggestions
CN103455507B (en) * 2012-05-31 2017-03-29 国际商业机器公司 Search engine recommends method and device
US10176633B2 (en) * 2012-06-05 2019-01-08 Apple Inc. Integrated mapping and navigation application
US9886794B2 (en) 2012-06-05 2018-02-06 Apple Inc. Problem reporting in maps
US9230556B2 (en) 2012-06-05 2016-01-05 Apple Inc. Voice instructions during navigation
US9111380B2 (en) 2012-06-05 2015-08-18 Apple Inc. Rendering maps
US9997069B2 (en) 2012-06-05 2018-06-12 Apple Inc. Context-aware voice guidance
US9418672B2 (en) 2012-06-05 2016-08-16 Apple Inc. Navigation application with adaptive instruction text
US9482296B2 (en) 2012-06-05 2016-11-01 Apple Inc. Rendering road signs during navigation
US8965696B2 (en) 2012-06-05 2015-02-24 Apple Inc. Providing navigation instructions while operating navigation application in background
US10417289B2 (en) 2012-06-12 2019-09-17 Oath Inc. Systems and methods involving integration/creation of search results media modules
US10303723B2 (en) 2012-06-12 2019-05-28 Excalibur Ip, Llc Systems and methods involving search enhancement features associated with media modules
WO2013188603A2 (en) * 2012-06-12 2013-12-19 Yahoo! Inc Systems and methods involving search enhancement features associated with media modules
US9875284B1 (en) * 2012-06-18 2018-01-23 A9.Com, Inc. Displaying search completion results
WO2014014374A1 (en) 2012-07-19 2014-01-23 Yandex Europe Ag Search query suggestions based in part on a prior search
US9424233B2 (en) 2012-07-20 2016-08-23 Veveo, Inc. Method of and system for inferring user intent in search input in a conversational interaction system
US9465833B2 (en) 2012-07-31 2016-10-11 Veveo, Inc. Disambiguating user intent in conversational interaction system for large corpus information retrieval
US9135335B2 (en) * 2012-12-03 2015-09-15 International Business Machines Corporation Filmstrip-based query suggestions
US9384279B2 (en) 2012-12-07 2016-07-05 Charles Reed Method and system for previewing search results
US9430782B2 (en) * 2012-12-17 2016-08-30 Facebook, Inc. Bidding on search results for targeting users in an online system
US20140172564A1 (en) * 2012-12-17 2014-06-19 Facebook, Inc. Targeting objects to users based on queries in an online system
US9305114B2 (en) * 2012-12-17 2016-04-05 Microsoft Technology Licensing, Llc Building long search queries
US9037598B1 (en) * 2013-01-25 2015-05-19 Google Inc. Variable query generation
US20140214820A1 (en) * 2013-01-31 2014-07-31 Chacha Search, Inc. Method and system of creating a seach query
US11741090B1 (en) 2013-02-26 2023-08-29 Richard Paiz Site rank codex search patterns
US11809506B1 (en) 2013-02-26 2023-11-07 Richard Paiz Multivariant analyzing replicating intelligent ambience evolving system
US20140280016A1 (en) * 2013-03-15 2014-09-18 Hugh Evan Williams Autocomplete-based advertisements
US9558508B2 (en) * 2013-03-15 2017-01-31 Microsoft Technology Licensing, Llc Energy-efficient mobile advertising
EP3640938A1 (en) 2013-05-07 2020-04-22 Veveo, Inc. Incremental speech input interface with real time feedback
US9633080B2 (en) 2013-05-28 2017-04-25 Microsoft Technology Licensing, Llc Hierarchical entity information for search
US20140358962A1 (en) * 2013-06-04 2014-12-04 Microsoft Corporation Responsive input architecture
US9298779B1 (en) 2013-07-16 2016-03-29 Google Inc. Combining content with a search result
US9342839B2 (en) 2013-07-16 2016-05-17 Google Inc. Combining content with a search result
US9043425B2 (en) * 2013-07-29 2015-05-26 Google Inc. Resource locator remarketing
US9619499B2 (en) 2013-08-07 2017-04-11 International Business Machines Corporation Hardware implementation of a tournament tree sort algorithm
US9251218B2 (en) 2013-08-07 2016-02-02 International Business Machines Corporation Tunable hardware sort engine for performing composite sorting algorithms
US9830354B2 (en) 2013-08-07 2017-11-28 International Business Machines Corporation Accelerating multiple query processing operations
US9495418B2 (en) 2013-08-07 2016-11-15 International Business Machines Corporation Scalable acceleration of database query operations
US8631325B1 (en) 2013-08-09 2014-01-14 Zoomdata, Inc. Real-time data visualization of streaming data
US10185748B1 (en) * 2013-08-22 2019-01-22 Evernote Corporation Combining natural language and keyword search queries for personal content collections
RU2592393C2 (en) * 2013-08-30 2016-07-20 Общество С Ограниченной Ответственностью "Яндекс" System and method for completing user's query and providing a response to a query
US20150081653A1 (en) * 2013-09-13 2015-03-19 Yahoo! Inc. Type free search assist
US20150088921A1 (en) * 2013-09-20 2015-03-26 Ebay Inc. Search guidance
US20150106358A1 (en) * 2013-10-14 2015-04-16 Google Inc. Systems and methods for conveying search results within an email application
US10061835B2 (en) * 2013-10-28 2018-08-28 Motorola Solutions, Inc. Establishing user-confidence levels of data inputs
US20150142891A1 (en) * 2013-11-19 2015-05-21 Sap Se Anticipatory Environment for Collaboration and Data Sharing
US9940664B2 (en) * 2013-12-30 2018-04-10 Tencent Technology (Shenzhen) Company Limited Information input method, account information input method and terminal
US10311054B2 (en) * 2014-01-08 2019-06-04 Red Hat, Inc. Query data splitting
US9984165B2 (en) * 2014-02-13 2018-05-29 Amadeus S.A.S. Increasing search result validity
US8862646B1 (en) 2014-03-25 2014-10-14 PlusAmp, Inc. Data file discovery, visualization, and importing
US11194868B1 (en) * 2014-04-29 2021-12-07 Google Llc Providing supplemental information in news search
US9471570B2 (en) * 2014-04-30 2016-10-18 Excalibur Ip, Llc Method and system for user selection of query suggestions
TWI748266B (en) * 2014-05-23 2021-12-01 南韓商三星電子股份有限公司 Search method, electronic device and non-transitory computer-readable recording medium
US11314826B2 (en) 2014-05-23 2022-04-26 Samsung Electronics Co., Ltd. Method for searching and device thereof
US9990433B2 (en) 2014-05-23 2018-06-05 Samsung Electronics Co., Ltd. Method for searching and device thereof
US11061893B2 (en) * 2014-05-30 2021-07-13 Apple Inc. Multi-domain query completion
US10540365B2 (en) * 2014-05-30 2020-01-21 Apple Inc. Federated search
US10204096B2 (en) * 2014-05-30 2019-02-12 Apple Inc. Device, method, and graphical user interface for a predictive keyboard
WO2015195971A1 (en) * 2014-06-18 2015-12-23 Google Inc. Methods, systems, and media for searching for video content
RU2014131311A (en) * 2014-07-29 2016-02-20 Общество С Ограниченной Ответственностью "Яндекс" METHOD (OPTIONS) FOR GENERATING THE SEARCH RESULTS PAGE, SERVER USED IN IT, AND METHOD FOR DETERMINING THE POSITION OF A WEB PAGE IN THE LIST OF WEB PAGES
US10691698B2 (en) * 2014-11-06 2020-06-23 International Business Machines Corporation Automatic near-real-time prediction, classification, and notification of events in natural language systems
US9852136B2 (en) 2014-12-23 2017-12-26 Rovi Guides, Inc. Systems and methods for determining whether a negation statement applies to a current or past query
US10310813B2 (en) 2014-12-29 2019-06-04 International Business Machines Corporation Hardware implementation of a tournament tree sort algorithm using an external memory
US9727218B2 (en) * 2015-01-02 2017-08-08 Microsoft Technology Licensing, Llc Contextual browser frame and entry box placement
US20160196348A1 (en) * 2015-01-07 2016-07-07 International Business Machines Corporation Re-ordering search results based on viewed pages
US10469346B2 (en) 2015-01-30 2019-11-05 Splunk Inc. Correlating performance data of client and host to identify performance issue of a third device
US9854049B2 (en) 2015-01-30 2017-12-26 Rovi Guides, Inc. Systems and methods for resolving ambiguous terms in social chatter based on a user profile
US9251276B1 (en) 2015-02-27 2016-02-02 Zoomdata, Inc. Prioritization of retrieval and/or processing of data
US9817871B2 (en) 2015-02-27 2017-11-14 Zoomdata, Inc. Prioritized retrieval and/or processing of data via query selection
US10169489B2 (en) * 2015-03-02 2019-01-01 International Business Machines Corporation Query disambiguation in a question-answering environment
US10157192B2 (en) * 2015-03-20 2018-12-18 Oath Inc. System and method for photo scene searching
US10713594B2 (en) 2015-03-20 2020-07-14 Salesforce.Com, Inc. Systems, methods, and apparatuses for implementing machine learning model training and deployment with a rollback mechanism
US9135559B1 (en) 2015-03-20 2015-09-15 TappingStone Inc. Methods and systems for predictive engine evaluation, tuning, and replay of engine performance
FR3035246A1 (en) * 2015-04-20 2016-10-21 Qwant METHOD OF DISPLAYING INFORMATION COMPLEMENTARY TO A SEARCH ENTRY CARRIED OUT IN A FIRST DATABASE
US10073886B2 (en) 2015-05-27 2018-09-11 International Business Machines Corporation Search results based on a search history
US10503764B2 (en) * 2015-06-01 2019-12-10 Oath Inc. Location-awareness search assistance system and method
RU2632269C2 (en) 2015-09-24 2017-10-03 Общество С Ограниченной Ответственностью "Яндекс" Method, server and electronic device for generation of clauses for search query
US20180329625A1 (en) * 2015-11-05 2018-11-15 Jason Griffin Word typing touchscreen keyboard
RU2015156695A (en) 2015-12-29 2017-07-05 Общество С Ограниченной Ответственностью "Яндекс" Method and system for processing a prefix associated with a search query
US11222064B2 (en) 2015-12-31 2022-01-11 Ebay Inc. Generating structured queries from images
US10305828B2 (en) 2016-04-20 2019-05-28 Google Llc Search query predictions by a keyboard
US9965530B2 (en) 2016-04-20 2018-05-08 Google Llc Graphical keyboard with integrated search features
US10140017B2 (en) 2016-04-20 2018-11-27 Google Llc Graphical keyboard application with integrated search
US10222957B2 (en) 2016-04-20 2019-03-05 Google Llc Keyboard with a suggested search query region
US10078673B2 (en) 2016-04-20 2018-09-18 Google Llc Determining graphical elements associated with text
US10609183B2 (en) * 2016-04-21 2020-03-31 Oath Inc. Content sharing recommendations
JP2017204018A (en) * 2016-05-09 2017-11-16 富士通株式会社 Search processing method, search processing program and information processing device
US11334800B2 (en) * 2016-05-12 2022-05-17 International Business Machines Corporation Altering input search terms
EP3403201A4 (en) * 2016-06-02 2019-01-09 Samsung Electronics Co., Ltd. Method and electronic device for predicting response
US10664157B2 (en) 2016-08-03 2020-05-26 Google Llc Image search query predictions by a keyboard
KR102017853B1 (en) * 2016-09-06 2019-09-03 주식회사 카카오 Method and apparatus for searching
US20180081893A1 (en) * 2016-09-19 2018-03-22 Ebay Inc. Prediction-based instant search
US9942312B1 (en) 2016-12-16 2018-04-10 Zoomdata, Inc. System and method for facilitating load reduction at a landing zone
US11170014B2 (en) * 2016-12-29 2021-11-09 Google Llc Method and system for preview of search engine processing
US10956519B2 (en) * 2017-06-29 2021-03-23 Cisco Technology, Inc. Fine-grained encrypted access to encrypted information
KR101945234B1 (en) * 2017-07-14 2019-02-08 (주)인터버드 Method for Searching Semiconductor Parts Using Algorithm of Eliminating Last Alphabet
US11016964B1 (en) * 2017-09-12 2021-05-25 Amazon Technologies, Inc. Intent determinations for content search
US10339192B1 (en) 2017-11-30 2019-07-02 Growpath, Inc. Systems and methods for matching buzzwords in a client management system
US11475082B1 (en) * 2017-12-15 2022-10-18 Palantir Technologies Inc. Systems and methods for context-based keyword searching
FR3080472B1 (en) 2018-04-19 2021-06-04 Amadeus Sas CHECKING THE GENERATION OF MULTIPLE ENTRY SEARCH RESULTS
US11126630B2 (en) * 2018-05-07 2021-09-21 Salesforce.Com, Inc. Ranking partial search query results based on implicit user interactions
US11086857B1 (en) * 2018-05-15 2021-08-10 Intuit Inc. Method and system for semantic search with a data management system
US11210728B2 (en) 2018-05-25 2021-12-28 Walmart Apollo, Llc Systems and methods for searching retail products and locations using a universal search bar
US11514036B2 (en) * 2018-08-14 2022-11-29 Jpmorgan Chase Bank, N.A. Method and system for self-learning natural language predictive searching
US10936823B2 (en) 2018-10-30 2021-03-02 International Business Machines Corporation Method and system for displaying automated agent comprehension
US11250039B1 (en) * 2018-12-06 2022-02-15 A9.Com, Inc. Extreme multi-label classification
CN109901896A (en) * 2018-12-06 2019-06-18 华为技术有限公司 Multi-task processing method in a kind of man-machine interactive system and man-machine interactive system
US11170026B1 (en) 2019-05-28 2021-11-09 Intuit Inc. System and method for identifying questions of users of a data management system
US11409805B2 (en) 2019-05-30 2022-08-09 AdMarketplace Computer implemented system and methods for implementing a search engine access point enhanced for suggested listing navigation
US11194467B2 (en) 2019-06-01 2021-12-07 Apple Inc. Keyboard management user interfaces
US11347756B2 (en) * 2019-08-26 2022-05-31 Microsoft Technology Licensing, Llc Deep command search within and across applications
US11042558B1 (en) 2019-09-06 2021-06-22 Tableau Software, Inc. Determining ranges for vague modifiers in natural language commands
US11423066B2 (en) 2019-09-12 2022-08-23 Intuit Inc. System and method for reducing user query ambiguity through chatbot clarifying questions
US11481111B2 (en) 2020-04-24 2022-10-25 Microsoft Technology Licensing, Llc Utilization of predictive gesture analysis for preloading and executing application components
US11900046B2 (en) 2020-08-07 2024-02-13 Microsoft Technology Licensing, Llc Intelligent feature identification and presentation
US20220075941A1 (en) * 2020-09-08 2022-03-10 Beijing Voyager Technology Co., Ltd. Artificial intelligence system and method for providing auto-complete suggestions
US11416136B2 (en) 2020-09-14 2022-08-16 Apple Inc. User interfaces for assigning and responding to user inputs
US11698933B1 (en) 2020-09-18 2023-07-11 Tableau Software, LLC Using dynamic entity search during entry of natural language commands for visual data analysis
US11301631B1 (en) 2020-10-05 2022-04-12 Tableau Software, LLC Visually correlating individual terms in natural language input to respective structured phrases representing the natural language input
US11914644B2 (en) * 2021-10-11 2024-02-27 Microsoft Technology Licensing, Llc Suggested queries for transcript search

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070255693A1 (en) * 2006-03-30 2007-11-01 Veveo, Inc. User interface method and system for incrementally searching and selecting content items and for presenting advertising in response to search activities
US20090119289A1 (en) * 2004-06-22 2009-05-07 Gibbs Kevin A Method and System for Autocompletion Using Ranked Results

Family Cites Families (101)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6009459A (en) 1997-01-10 1999-12-28 Microsoft Corporation Intelligent automatic searching for resources in a distributed environment
US6377965B1 (en) 1997-11-07 2002-04-23 Microsoft Corporation Automatic word completion system for partially entered data
US6006225A (en) * 1998-06-15 1999-12-21 Amazon.Com Refining search queries by the suggestion of correlated terms from prior searches
US6779060B1 (en) 1998-08-05 2004-08-17 British Telecommunications Public Limited Company Multimodal user interface
US6370518B1 (en) 1998-10-05 2002-04-09 Openwave Systems Inc. Method and apparatus for displaying a record from a structured database with minimum keystrokes
US6411950B1 (en) 1998-11-30 2002-06-25 Compaq Information Technologies Group, Lp Dynamic query expansion
US6751606B1 (en) 1998-12-23 2004-06-15 Microsoft Corporation System for enhancing a query interface
US6895558B1 (en) * 2000-02-11 2005-05-17 Microsoft Corporation Multi-access mode electronic personal assistant
US6578022B1 (en) * 2000-04-18 2003-06-10 Icplanet Corporation Interactive intelligent searching with executable suggestions
US6948129B1 (en) * 2001-02-08 2005-09-20 Masoud S Loghmani Multi-modal, multi-path user interface for simultaneous access to internet data over multiple media
EP2287819A3 (en) * 2001-02-09 2012-03-28 Yosef Mintz Improved method and system for mapping traffic predictions with respect to telematics and route guidance applications
US7778816B2 (en) * 2001-04-24 2010-08-17 Microsoft Corporation Method and system for applying input mode bias
US7752326B2 (en) 2001-08-20 2010-07-06 Masterobjects, Inc. System and method for utilizing asynchronous client server communication objects
US8112529B2 (en) 2001-08-20 2012-02-07 Masterobjects, Inc. System and method for asynchronous client server session communication
US20090006543A1 (en) * 2001-08-20 2009-01-01 Masterobjects System and method for asynchronous retrieval of information based on incremental user input
US7437349B2 (en) * 2002-05-10 2008-10-14 International Business Machines Corporation Adaptive probabilistic query expansion
US7676452B2 (en) * 2002-07-23 2010-03-09 International Business Machines Corporation Method and apparatus for search optimization based on generation of context focused queries
US7146362B2 (en) * 2002-08-28 2006-12-05 Bpallen Technologies Llc Method and apparatus for using faceted metadata to navigate through information resources
US8005919B2 (en) 2002-11-18 2011-08-23 Aol Inc. Host-based intelligent results related to a character stream
US7487185B2 (en) 2003-02-24 2009-02-03 Microsoft Corporation Technique for changing views of web content
US6947930B2 (en) * 2003-03-21 2005-09-20 Overture Services, Inc. Systems and methods for interactive search query refinement
US7051023B2 (en) * 2003-04-04 2006-05-23 Yahoo! Inc. Systems and methods for generating concept units from search queries
US7181445B2 (en) * 2003-09-05 2007-02-20 Bellsouth Intellectual Property Corporation Aggregating, retrieving, and providing access to document visuals
US7523096B2 (en) 2003-12-03 2009-04-21 Google Inc. Methods and systems for personalized network searching
US7451131B2 (en) * 2003-12-08 2008-11-11 Iac Search & Media, Inc. Methods and systems for providing a response to a query
CN101268504A (en) * 2004-02-25 2008-09-17 爱普乐技术公司 Apparatus for providing multi-mode digital input
US7836044B2 (en) 2004-06-22 2010-11-16 Google Inc. Anticipated query generation and processing in a search engine
US7483880B2 (en) * 2004-09-30 2009-01-27 Microsoft Corporation User interface for database display
US7499940B1 (en) 2004-11-11 2009-03-03 Google Inc. Method and system for URL autocompletion using ranked results
US20060161520A1 (en) * 2005-01-14 2006-07-20 Microsoft Corporation System and method for generating alternative search terms
US7461059B2 (en) 2005-02-23 2008-12-02 Microsoft Corporation Dynamically updated search results based upon continuously-evolving search query that is based at least in part upon phrase suggestion, search engine uses previous result sets performing additional search tasks
US7788248B2 (en) 2005-03-08 2010-08-31 Apple Inc. Immediate search feedback
US20060218115A1 (en) * 2005-03-24 2006-09-28 Microsoft Corporation Implicit queries for electronic documents
US7647312B2 (en) * 2005-05-12 2010-01-12 Microsoft Corporation System and method for automatic generation of suggested inline search terms
US20060271426A1 (en) 2005-05-31 2006-11-30 Microsoft Corporation Posted price market for online search and content advertisements
US7725496B2 (en) * 2005-06-17 2010-05-25 Computer Associates Think, Inc. System and method for identifying application resources
US7725485B1 (en) * 2005-08-01 2010-05-25 Google Inc. Generating query suggestions using contextual information
US7516124B2 (en) 2005-12-20 2009-04-07 Yahoo! Inc. Interactive search engine
US7672932B2 (en) * 2005-08-24 2010-03-02 Yahoo! Inc. Speculative search result based on a not-yet-submitted search query
US7844599B2 (en) * 2005-08-24 2010-11-30 Yahoo! Inc. Biasing queries to determine suggested queries
US7747639B2 (en) 2005-08-24 2010-06-29 Yahoo! Inc. Alternative search query prediction
US7921109B2 (en) 2005-10-05 2011-04-05 Yahoo! Inc. Customizable ordering of search results and predictive query generation
US7962479B2 (en) * 2005-11-09 2011-06-14 Yahoo! Inc. System and method for generating substitutable queries
US8694530B2 (en) * 2006-01-03 2014-04-08 Textdigger, Inc. Search system with query refinement and search method
US7676460B2 (en) * 2006-03-03 2010-03-09 International Business Machines Corporation Techniques for providing suggestions for creating a search query
KR100792699B1 (en) * 2006-03-17 2008-01-11 엔에이치엔(주) Method and system for automatically completed general recommended word and advertisement recommended word
KR100754768B1 (en) * 2006-04-06 2007-09-03 엔에이치엔(주) System and method for providing recommended word of adjustment each user and computer readable recording medium recording program for implementing the method
BRPI0713114A2 (en) * 2006-05-19 2012-04-17 My Virtual Model Inc simulation assisted search
US8301616B2 (en) * 2006-07-14 2012-10-30 Yahoo! Inc. Search equalizer
US7890499B1 (en) 2006-07-28 2011-02-15 Google Inc. Presentation of search results with common subject matters
US7716201B2 (en) * 2006-08-10 2010-05-11 Yahoo! Inc. Method and apparatus for reconstructing a search query
US20080109401A1 (en) 2006-09-12 2008-05-08 Microsoft Corporation Presenting predetermined search results with query suggestions
US20100312787A1 (en) * 2006-11-04 2010-12-09 Nokia Corporation Control of Search Scope
US7630970B2 (en) * 2006-11-28 2009-12-08 Yahoo! Inc. Wait timer for partially formed query
US7630978B2 (en) * 2006-12-14 2009-12-08 Yahoo! Inc. Query rewriting with spell correction suggestions using a generated set of query features
US20080147653A1 (en) * 2006-12-15 2008-06-19 Iac Search & Media, Inc. Search suggestions
US20080270932A1 (en) * 2006-12-15 2008-10-30 Iac Search & Media, Inc. Toolbox editing
US8255382B2 (en) 2007-06-20 2012-08-28 Boopsie, Inc. Dynamic menus for multi-prefix interactive mobile searches
US7680778B2 (en) 2007-01-19 2010-03-16 Microsoft Corporation Support for reverse and stemmed hit-highlighting
US20080189290A1 (en) * 2007-02-06 2008-08-07 Yahoo! Inc., A Delaware Corporation Method and System for Representing an Exploration of Resources Over a Communication Network
US20080195577A1 (en) * 2007-02-09 2008-08-14 Wei Fan Automatically and adaptively determining execution plans for queries with parameter markers
US7996786B2 (en) * 2007-03-05 2011-08-09 Microsoft Corporation Dynamically rendering visualizations of data sets
US20080221983A1 (en) * 2007-03-06 2008-09-11 Siarhei Ausiannik Network information distribution system and a method of advertising and search for supply and demand of products/goods/services in any geographical location
US7668823B2 (en) * 2007-04-03 2010-02-23 Google Inc. Identifying inadequate search content
US20080256056A1 (en) * 2007-04-10 2008-10-16 Yahoo! Inc. System for building a data structure representing a network of users and advertisers
US7689535B2 (en) * 2007-05-30 2010-03-30 Red Hat, Inc. Method for providing a unified view of a domain model to a user
US20090043741A1 (en) 2007-08-09 2009-02-12 Dohyung Kim Autocompletion and Automatic Input Method Correction for Partially Entered Search Query
US20090083232A1 (en) * 2007-09-24 2009-03-26 Taptu Ltd. Search results with search query suggestions
US20090094551A1 (en) 2007-10-05 2009-04-09 International Business Machines Corporation Enhancements to prevent split entries in the event of a window focus shift
US20090094211A1 (en) * 2007-10-05 2009-04-09 Fujitsu Limited Implementing an expanded search and providing expanded search results
US8694483B2 (en) * 2007-10-19 2014-04-08 Xerox Corporation Real-time query suggestion in a troubleshooting context
US7818334B2 (en) * 2007-10-22 2010-10-19 Microsoft Corporation Query dependant link-based ranking using authority scores
US20090171929A1 (en) * 2007-12-26 2009-07-02 Microsoft Corporation Toward optimized query suggeston: user interfaces and algorithms
US8131731B2 (en) * 2007-12-27 2012-03-06 Microsoft Corporation Relevancy sorting of user's browser history
US9000957B2 (en) 2008-01-13 2015-04-07 Aberra Molla Ethiopic character entry
US7984004B2 (en) * 2008-01-17 2011-07-19 Microsoft Corporation Query suggestion generation
US8959104B2 (en) * 2008-03-21 2015-02-17 Microsoft Corporation Presenting query suggestions based upon content items
US9798806B2 (en) * 2008-03-31 2017-10-24 Excalibur Ip, Llc Information retrieval using dynamic guided navigation
US8090738B2 (en) * 2008-05-14 2012-01-03 Microsoft Corporation Multi-modal search wildcards
US8082278B2 (en) * 2008-06-13 2011-12-20 Microsoft Corporation Generating query suggestions from semantic relationships in content
US8768909B2 (en) 2008-06-19 2014-07-01 Tropare, Inc. Dynamic menus for multi-prefix interactive mobile searches using predictive text to yield targeted advertisements
US20090327236A1 (en) * 2008-06-27 2009-12-31 Microsoft Corporation Visual query suggestions
US8745051B2 (en) * 2008-07-03 2014-06-03 Google Inc. Resource locator suggestions from input character sequence
US8010537B2 (en) * 2008-08-27 2011-08-30 Yahoo! Inc. System and method for assisting search requests with vertical suggestions
US20100076948A1 (en) * 2008-09-09 2010-03-25 International Business Machines Corporation System and method for interfacing search request and corresponding search result
US8370329B2 (en) * 2008-09-22 2013-02-05 Microsoft Corporation Automatic search query suggestions with search result suggestions from user history
US20100125809A1 (en) * 2008-11-17 2010-05-20 Fujitsu Limited Facilitating Display Of An Interactive And Dynamic Cloud With Advertising And Domain Features
US20100131481A1 (en) * 2008-11-26 2010-05-27 Honeywell International Inc. Methods for locating an item when a search mode is not selected
US20100146012A1 (en) * 2008-12-04 2010-06-10 Microsoft Corporation Previewing search results for suggested refinement terms and vertical searches
US20100185644A1 (en) * 2009-01-21 2010-07-22 Microsoft Corporatoin Automatic search suggestions from client-side, browser, history cache
US8275759B2 (en) 2009-02-24 2012-09-25 Microsoft Corporation Contextual query suggestion in result pages
CA2772638C (en) * 2009-08-31 2018-02-13 Google Inc. Framework for selecting and presenting answer boxes relevant to user input as query suggestions
US9405841B2 (en) * 2009-10-15 2016-08-02 A9.Com, Inc. Dynamic search suggestion and category specific completion
US8631004B2 (en) * 2009-12-28 2014-01-14 Yahoo! Inc. Search suggestion clustering and presentation
US8560561B2 (en) * 2010-01-07 2013-10-15 Microsoft Corporation Location aware search page elements
US8732171B2 (en) * 2010-01-28 2014-05-20 Microsoft Corporation Providing query suggestions
US8782556B2 (en) 2010-02-12 2014-07-15 Microsoft Corporation User-centric soft keyboard predictive technologies
US8341222B2 (en) * 2010-04-02 2012-12-25 Microsoft Corporation Text suggestion framework with client and server model
US8560562B2 (en) * 2010-07-22 2013-10-15 Google Inc. Predictive query suggestion caching
WO2012024585A1 (en) 2010-08-19 2012-02-23 Othar Hansson Predictive query completion and predictive search results
US20120167009A1 (en) 2010-12-22 2012-06-28 Apple Inc. Combining timing and geometry information for typing correction

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090119289A1 (en) * 2004-06-22 2009-05-07 Gibbs Kevin A Method and System for Autocompletion Using Ranked Results
US20070255693A1 (en) * 2006-03-30 2007-11-01 Veveo, Inc. User interface method and system for incrementally searching and selecting content items and for presenting advertising in response to search activities

Cited By (99)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130018914A1 (en) * 2005-06-21 2013-01-17 W.W. Grainger, Inc. System and method for facilitating use of a selection guide
US9128982B2 (en) * 2010-12-23 2015-09-08 Nhn Corporation Search system and search method for recommending reduced query
WO2013000062A1 (en) * 2011-06-29 2013-01-03 The C-Factor Incorporated Method and apparatus for determining and delivering targeted content to a mobile platform
US20130174023A1 (en) * 2011-12-30 2013-07-04 Microsoft Corporation Presentation of rich search results in delineated areas
US9286390B2 (en) * 2011-12-30 2016-03-15 Microsoft Technology Licensing, Llc Presentation of rich search results in delineated areas
US10956935B1 (en) * 2012-02-24 2021-03-23 Google Llc Serving advertisements based on partial queries
US11687968B1 (en) 2012-02-24 2023-06-27 Google Llc Serving advertisements based on partial queries
US9922344B1 (en) * 2012-02-24 2018-03-20 Google Llc Serving advertisements based on partial queries
US20220051288A1 (en) * 2012-03-01 2022-02-17 Google Llc Presenting options for content delivery
US20130246456A1 (en) * 2012-03-15 2013-09-19 Alibaba Group Holding Limited Publishing Product Information
US9665622B2 (en) * 2012-03-15 2017-05-30 Alibaba Group Holding Limited Publishing product information
US20150169643A1 (en) * 2012-05-14 2015-06-18 Google Inc. Providing supplemental search results in repsonse to user interest signal
US20150161143A1 (en) * 2012-06-01 2015-06-11 Zte Corporation Input processing method and device
US10535041B2 (en) * 2012-06-26 2020-01-14 Oath Inc. System and method of predicting community member responsiveness
US20130346496A1 (en) * 2012-06-26 2013-12-26 Yoelle Maarek System and method of predicting community member responsiveness
US20140129973A1 (en) * 2012-11-08 2014-05-08 Microsoft Corporation Interaction model for serving popular queries in search box
US9135307B1 (en) * 2012-12-27 2015-09-15 Google Inc. Selectively generating alternative queries
US9230013B1 (en) * 2013-03-07 2016-01-05 International Business Machines Corporation Suffix searching on documents
US20140280289A1 (en) * 2013-03-12 2014-09-18 Microsoft Corporation Autosuggestions based on user history
WO2014164420A1 (en) * 2013-03-12 2014-10-09 Microsoft Corporation Autosuggestions based on user history
US20150213041A1 (en) * 2013-03-15 2015-07-30 Google Inc. Search suggestion rankings
USD852829S1 (en) * 2013-04-05 2019-07-02 Thales Avionics, Inc. Display screen or portion thereof with graphical user interface
US20150234872A1 (en) * 2013-04-12 2015-08-20 Hitachi, Ltd. Computer, data processing method, and non-transitory storage medium
US20140324902A1 (en) * 2013-04-25 2014-10-30 Microsoft Corporation Automatic generation of a collection of content
US9805124B2 (en) * 2013-04-25 2017-10-31 Microsoft Technology Licensing, Llc Automatic generation of a collection of content
US20140344250A1 (en) * 2013-05-20 2014-11-20 Microsoft Corporation Enhanced search refinement for personal information services
US20150012560A1 (en) * 2013-07-03 2015-01-08 Google Inc. Methods and systems for providing potential search queries that may be targeted by one or more keywords
US9378517B2 (en) * 2013-07-03 2016-06-28 Google Inc. Methods and systems for providing potential search queries that may be targeted by one or more keywords
US20160182520A1 (en) * 2013-07-26 2016-06-23 Beijing Qihoo Technology Company Limited Method, apparatus and system for client accessing authenticated web address
US11328324B1 (en) * 2013-08-30 2022-05-10 Groupon, Inc. Systems and methods for providing promotions based on search queries
US9916368B2 (en) 2013-12-02 2018-03-13 QBase, Inc. Non-exclusionary search within in-memory databases
US9230041B2 (en) * 2013-12-02 2016-01-05 Qbase, LLC Search suggestions of related entities based on co-occurrence and/or fuzzy-score matching
US9613166B2 (en) 2013-12-02 2017-04-04 Qbase, LLC Search suggestions of related entities based on co-occurrence and/or fuzzy-score matching
US20150154316A1 (en) * 2013-12-02 2015-06-04 Qbase, LLC Search suggestions of related entities based on co-occurrence and/or fuzzy-score matching
US9639581B1 (en) * 2013-12-27 2017-05-02 Google Inc. Query completions
US9361406B1 (en) * 2013-12-27 2016-06-07 Google Inc. Query completions
US10698901B1 (en) * 2013-12-27 2020-06-30 Google Llc Query completions
US11693863B1 (en) * 2013-12-27 2023-07-04 Google Llc Query completions
US9827714B1 (en) 2014-05-16 2017-11-28 Google Llc Method and system for 3-D printing of 3-D object models in interactive content items
US10596761B2 (en) 2014-05-16 2020-03-24 Google Llc Method and system for 3-D printing of 3-D object models in interactive content items
US11347763B2 (en) * 2014-05-21 2022-05-31 Google Llc Providing a notification in place of content item
US9875287B1 (en) * 2014-05-21 2018-01-23 Google Inc. Providing a notification in place of content item
US20180107721A1 (en) * 2014-05-21 2018-04-19 Google Llc Providing a notification in place of content item
US10339565B2 (en) * 2014-06-30 2019-07-02 Walmart Apollo, Llc Presenting advertisement content during searches of digital receipts
US20150379570A1 (en) * 2014-06-30 2015-12-31 Wal-Mart Stores, Inc. Presenting Advertisement Content During Searches Of Digital Receipts
US20160019291A1 (en) * 2014-07-18 2016-01-21 John R. Ruge Apparatus And Method For Information Retrieval At A Mobile Device
US10671182B2 (en) * 2014-10-16 2020-06-02 Touchtype Limited Text prediction integration
US10503377B2 (en) * 2014-11-24 2019-12-10 Facebook, Inc. Dynamic status indicator
US20160147421A1 (en) * 2014-11-24 2016-05-26 Facebook, Inc. Dynamic Status Indicator
US10863354B2 (en) 2014-11-24 2020-12-08 Facebook, Inc. Automated check-ins
US10397346B2 (en) 2014-11-24 2019-08-27 Facebook, Inc. Prefetching places
US11442924B2 (en) 2015-01-30 2022-09-13 Splunk Inc. Selective filtered summary graph
US20160224531A1 (en) 2015-01-30 2016-08-04 Splunk Inc. Suggested Field Extraction
US11907271B2 (en) 2015-01-30 2024-02-20 Splunk Inc. Distinguishing between fields in field value extraction
US11868364B1 (en) 2015-01-30 2024-01-09 Splunk Inc. Graphical user interface for extracting from extracted fields
US11841908B1 (en) 2015-01-30 2023-12-12 Splunk Inc. Extraction rule determination based on user-selected text
US11741086B2 (en) 2015-01-30 2023-08-29 Splunk Inc. Queries based on selected subsets of textual representations of events
US11615073B2 (en) 2015-01-30 2023-03-28 Splunk Inc. Supplementing events displayed in a table format
US11573959B2 (en) 2015-01-30 2023-02-07 Splunk Inc. Generating search commands based on cell selection within data tables
US11544248B2 (en) 2015-01-30 2023-01-03 Splunk Inc. Selective query loading across query interfaces
US11544257B2 (en) 2015-01-30 2023-01-03 Splunk Inc. Interactive table-based query construction using contextual forms
US10726037B2 (en) 2015-01-30 2020-07-28 Splunk Inc. Automatic field extraction from filed values
US10846316B2 (en) 2015-01-30 2020-11-24 Splunk Inc. Distinct field name assignment in automatic field extraction
US11531713B2 (en) 2015-01-30 2022-12-20 Splunk Inc. Suggested field extraction
US10877963B2 (en) * 2015-01-30 2020-12-29 Splunk Inc. Command entry list for modifying a search query
US10896175B2 (en) 2015-01-30 2021-01-19 Splunk Inc. Extending data processing pipelines using dependent queries
US10915583B2 (en) 2015-01-30 2021-02-09 Splunk Inc. Suggested field extraction
US10949419B2 (en) 2015-01-30 2021-03-16 Splunk Inc. Generation of search commands via text-based selections
US20180150507A1 (en) * 2015-01-30 2018-05-31 Splunk Inc. Command entry list for modifying a search query
US11409758B2 (en) 2015-01-30 2022-08-09 Splunk Inc. Field value and label extraction from a field value
US11030192B2 (en) 2015-01-30 2021-06-08 Splunk Inc. Updates to access permissions of sub-queries at run time
US11068452B2 (en) 2015-01-30 2021-07-20 Splunk Inc. Column-based table manipulation of event data to add commands to a search query
US11354308B2 (en) 2015-01-30 2022-06-07 Splunk Inc. Visually distinct display format for data portions from events
US11222014B2 (en) 2015-01-30 2022-01-11 Splunk Inc. Interactive table-based query construction using interface templates
US11341129B2 (en) 2015-01-30 2022-05-24 Splunk Inc. Summary report overlay
US20160283546A1 (en) * 2015-03-26 2016-09-29 International Business Machines Corporation Query strength indicator
US9864775B2 (en) * 2015-03-26 2018-01-09 International Business Machines Corporation Query strength indicator
US10706237B2 (en) * 2015-06-15 2020-07-07 Microsoft Technology Licensing, Llc Contextual language generation by leveraging language understanding
US20180024994A1 (en) * 2015-06-15 2018-01-25 Microsoft Technology Licensing, Llc Contextual language generation by leveraging language understanding
US10621516B2 (en) * 2015-09-17 2020-04-14 Tencent Technology (Shenzhen) Company Limited Content delivery method, apparatus, and storage medium
US20180107953A1 (en) * 2015-09-17 2018-04-19 Tencent Technology (Shenzhen) Company Limited Content delivery method, apparatus, and storage medium
US20170308289A1 (en) * 2016-04-20 2017-10-26 Google Inc. Iconographic symbol search within a graphical keyboard
US20190278817A1 (en) * 2016-06-30 2019-09-12 Zowdow, Inc. Systems and methods for enhanced search, content, and advertisement delivery
CN109923541A (en) * 2016-06-30 2019-06-21 周道有限公司 The system and method sent for enhancing search, content and advertisement
WO2018005903A1 (en) * 2016-06-30 2018-01-04 Zowdow, Inc. Systems and methods for enhanced search, content, and advertisement delivery
CN108073633A (en) * 2016-11-17 2018-05-25 百度在线网络技术(北京)有限公司 The bid of keyword and matching way setting method and device
US11403314B2 (en) 2017-04-27 2022-08-02 Google Llc Cloud inference system
US11734292B2 (en) 2017-04-27 2023-08-22 Google Llc Cloud inference system
US10423638B2 (en) 2017-04-27 2019-09-24 Google Llc Cloud inference system
US10678859B2 (en) * 2017-09-29 2020-06-09 Korea Testing Certification (KTC) Plant material recommendation apparatus in use environment
CN109583984A (en) * 2017-09-29 2019-04-05 财团法人韩国机械电气电子试验研究院 Equipment and materials recommendation apparatus in use environment
US20190102479A1 (en) * 2017-09-29 2019-04-04 Korea Testing Certification (KTC) Plant material recommendation apparatus in use environment
US11354514B2 (en) * 2017-11-14 2022-06-07 International Business Machines Corporation Real-time on-demand auction based content clarification
US10956502B2 (en) * 2018-05-18 2021-03-23 Yandex Europe Ag Method of and system for recommending fresh search query suggestions on search engine
US20220261150A1 (en) * 2020-02-12 2022-08-18 Facebook Technologies, Llc Virtual keyboard based on adaptive language model
US11327651B2 (en) * 2020-02-12 2022-05-10 Facebook Technologies, Llc Virtual keyboard based on adaptive language model
US11899928B2 (en) * 2020-02-12 2024-02-13 Meta Platforms Technologies, Llc Virtual keyboard based on adaptive language model
US11620282B2 (en) * 2020-06-22 2023-04-04 S&P Global Inc. Automated information retrieval system and semantic parsing
US20210397609A1 (en) * 2020-06-22 2021-12-23 S&P Global Automated information retrieval system and semantic parsing

Also Published As

Publication number Publication date
US8700653B2 (en) 2014-04-15
US20230394072A1 (en) 2023-12-07
AU2011291549B2 (en) 2015-03-19
CA2808856A1 (en) 2012-02-23
EP2606440A1 (en) 2013-06-26
US20180203863A1 (en) 2018-07-19
CA2808803A1 (en) 2012-02-23
US20160321270A1 (en) 2016-11-03
DE212011100019U1 (en) 2012-04-05
AU2011291549A1 (en) 2013-03-14
CA2808803C (en) 2018-11-06
AU2011291544B2 (en) 2015-03-26
US20120047135A1 (en) 2012-02-23
US20140181135A1 (en) 2014-06-26
US11620318B2 (en) 2023-04-04
US9953076B2 (en) 2018-04-24
AU2011291544A1 (en) 2013-03-14
WO2012024580A1 (en) 2012-02-23
EP2606441A1 (en) 2013-06-26
WO2012024585A1 (en) 2012-02-23
DE212011100017U1 (en) 2012-04-03
US20120047134A1 (en) 2012-02-23
US8706750B2 (en) 2014-04-22

Similar Documents

Publication Publication Date Title
US20230394072A1 (en) Predictive query completion and predictive search results
US8060524B2 (en) History answer for re-finding search results
US9613132B2 (en) Method of and system for displaying a plurality of user-selectable refinements to a search query
US20140280015A1 (en) Serving advertisements for search preview based on user intents
EP2777005B1 (en) Dynamic determination of number of served advertisements
US11593906B2 (en) Image recognition based content item selection
CN106462613B (en) Ranking suggestions based on user attributes
US11250716B2 (en) Network system for contextual course recommendation based on third-party content
US20150278687A1 (en) User device side predicted performance measure adjustments
US20140156623A1 (en) Generating and displaying tasks
AU2017216516B2 (en) Predictive query completion and predictive search results
US20170251070A1 (en) Multiple User Interest Profiles
US8666810B1 (en) System and method for online unique users and frequency estimation for advertisements on a video sharing website based on auction history results
US20200401638A1 (en) Method of and system for generating search query completion suggestion on search engine

Legal Events

Date Code Title Description
AS Assignment

Owner name: GOOGLE INC., CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:STROHMAN, TREVOR;REEL/FRAME:027084/0033

Effective date: 20111010

AS Assignment

Owner name: GOOGLE INC., CALIFORNIA

Free format text: CORRECTIVE ASSIGNMENT TO CORRECT THE ATTORNEY DOCKET NUMBER OF 16113-540001 PREVIOUSLY RECORDED ON REEL 027084 FRAME 0033. ASSIGNOR(S) HEREBY CONFIRMS THE ATTORNEY DOCKET NUMBER AS 16113-2540001;ASSIGNOR:STROHMAN, TREVOR;REEL/FRAME:027159/0509

Effective date: 20111010

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION

AS Assignment

Owner name: GOOGLE LLC, CALIFORNIA

Free format text: CHANGE OF NAME;ASSIGNOR:GOOGLE INC.;REEL/FRAME:044142/0357

Effective date: 20170929