WO2003094489A1 - Method and system for rapid navigation in aural user interface - Google Patents

Method and system for rapid navigation in aural user interface Download PDF

Info

Publication number
WO2003094489A1
WO2003094489A1 PCT/FI2002/000363 FI0200363W WO03094489A1 WO 2003094489 A1 WO2003094489 A1 WO 2003094489A1 FI 0200363 W FI0200363 W FI 0200363W WO 03094489 A1 WO03094489 A1 WO 03094489A1
Authority
WO
WIPO (PCT)
Prior art keywords
mode
menu
aural
user interface
items
Prior art date
Application number
PCT/FI2002/000363
Other languages
French (fr)
Inventor
Jukka-Pekka Salmenkaita
Original Assignee
Nokia Corporation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nokia Corporation filed Critical Nokia Corporation
Priority to PCT/FI2002/000363 priority Critical patent/WO2003094489A1/en
Priority to JP2004502598A priority patent/JP2005524122A/en
Priority to AU2002251113A priority patent/AU2002251113A1/en
Publication of WO2003094489A1 publication Critical patent/WO2003094489A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/038Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16Sound input; Sound output
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/22Procedures used during a speech recognition process, e.g. man-machine dialogue
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04MTELEPHONIC COMMUNICATION
    • H04M3/00Automatic or semi-automatic exchanges
    • H04M3/42Systems providing special services or facilities to subscribers
    • H04M3/487Arrangements for providing information services, e.g. recorded voice services or time announcements
    • H04M3/493Interactive information services, e.g. directory enquiries ; Arrangements therefor, e.g. interactive voice response [IVR] systems or voice portals

Definitions

  • the invention relates to user interfaces for accessing digital devices and services. More specifically, the invention is a method for enhancing the usability of voice and multimodal user interfaces.
  • the user can choose from two modes for navigation through menu structures, one of which is a standard mode for normal use and the other a rapid mode for experienced use.
  • menu hierarchies both (a) to inform the user of alternative selection options as well as (b) to provide the means for the user to navigate through the menu layers to the desired selection option.
  • the menu hierarchies are presented in visual form. In situations, e.g. when hands-free and eyes-free use of digital devices and services are desired, a vis- ual presentation of the menu hierarchies is not feasible. Situations like these occur e.g. in cars and other vehicles or in case of people with visual impairments, few to mention.
  • a menu hierarchy can be presented in aural form using for example text-to-speech (TTS) synthesis or predefined voice prompts to inform a user of alternative selection options.
  • TTS text-to-speech
  • an aural or voice user interface is suitable for informing the user of the alternative selection options, it has a drawback to be relative slow in presenting information to the user.
  • this slowness is justified due the need to inform the user of the alternative selection options and to initiate him into the menu structure.
  • the user learns the menu structure, and the slowness brings a major disconvenience on the user. Nevertheless, a need for navigating through the menu layers remains, but the user would prefer faster means of going through the menu layers.
  • the first way is to accelerate a speed of the entire TTS synthesis or voice prompts presentation. This means that the aural presentation of the menu remains in the original form but pronunciation is uttered faster which may exacerbate understand- ing the menu items.
  • Another known way to make navigation process faster is to provide option for the end user to customize the user interface by creating manual short-cuts. This is well-known and widely used in the field of web browsing.
  • Still another way to speed up the navigation process is to provide automatic short-cuts to digital services for the end user. The latter two ways are mainly used in visual user interfaces and they expect rather many steps to be taken by the user to reconcile the menu structure with the short-cuts.
  • TTS Text-to-Speech
  • the object of the invention is to provide a method and system which allows rapid voice scrolling of menu items to the user in aural user interfaces.
  • the objective of the invention is achieved by activating the rapid mode in which mode voice prompts corresponding to menu items are shortened significantly and hence the user interface is faster than in prior art. In the standard mode when the rapid mode is deactivated full-length voice prompts are used.
  • the advantage of the invention is that those users who are well familiar with the menu structure easily activate and use the rapid mode to avoid themselves a slowness of the standard mode user interface. Meanwhile in unfamiliar usage situations the users may choose navigating through menu items in the standard mode.
  • the invention enhances the usability of aural and multimodal user interfaces providing for experienced users a faster way to present menu options, if desired. Otherwise the user can continue with standard mode in normal order. Also an advantage of this invention is that it can be used in association with other methods and systems to speed up navigation process.
  • the method of present invention relates to a method for navigating in user interfaces for accessing digital devices and services, the device comprising at least an aural or multimodal user interface and connection to the network device, is characterized in that, it comprises steps, in which a second mode for an aural user interface is activated and said second mode comprising different and shorter content than the first mode.
  • the system of present invention relates to a system for navigating in aural user in- terfaces for accessing digital devices and services, the device comprising means for at least an aural or multimodal user interface and means for connection to the network device, is characterized in that, it comprises means for activating and deactivating a second mode for an aural user interface.
  • a network device is a server in a network or a computer device in a network using peer-to-peer connection.
  • One preferred embodiment of the invention is to use TTS engine embedded in the terminal device.
  • the TTS engine can be in the network and accordingly the TTS engine should provide both standard and rapid prompts to the terminal device or, alternatively, either standard or rapid prompts depending on the current mode of the terminal device.
  • One embodiment of the invention is to use predefined voice prompts both in standard and rapid mode.
  • the voice prompts in the rapid mode are just short audio signals e.g. beeps.
  • the rapid mode is activated by a special key or by a long press of menu up and menu down key.
  • the rapid mode is activated by the long press on menu up/down function key for a fast forward/reverse operation and deactivated by releasing said menu up/down key for a standard forward/reverse operation.
  • Fig 1. is a flow diagram of a method for selecting a mode according to the invention.
  • Fig 2. is a flow diagram of a method for rapid navigation in aural user interfaces according to one embodiment of the invention.
  • Fig 3a is a flow diagram of a method for rapid navigation in aural user interfaces according to another embodiment of the invention.
  • Fig 3b is a flow diagram of a method for rapid navigation in aural user interfaces according to an optional embodiment of the invention.
  • Fig 4. is a block diagram of a system for rapid navigation in aural user interfaces according to one embodiment of the invention.
  • Fig 5. is a block diagram of a system for rapid navigation in aural user interfaces according to another embodiment of the invention.
  • an aural or voice user interface has two modes for navigating through menu structures and the two modes are a standard or normal mode and a rapid mode.
  • a standard mode is called a "first mode”
  • a rapid mode is called a "second mode”.
  • menu options can be presented in aural form using e.g. voice prompts generated by text-to-speech (TTS) synthesis or predefined voice prompts.
  • TTS text-to-speech
  • voice prompts corresponding to menu items are shortened significantly by presenting only e.g. a first syllable of each voice prompt.
  • the second mode is deacti- vated the first mode is used where voice prompts corresponding to menu items are presented as full-length voice prompts.
  • Fig 1. shows a flow diagram of the two modes for navigating in aural user interfaces according to the invention.
  • the first mode is depicted by block 11 and the second mode depicted by block 12 and the change between these two blocks is activated by giving a sign by the user e.g. by pressing a separate key, by a long press of the menu up/down key, by a special voice key or by giving a voice command.
  • the selection of the mode is independent of the actual location of the navigation process i.e. the user may change the mode whenever he wants to do so.
  • timeout situation which occurs when e.g. a large block of operations may disconnect a network con- nection automatically after a fixed period of time, the mode automatically changes to the first (normal) mode. So, after timeout the first mode is a default mode.
  • Fig 2. shows a flow diagram of a method for navigating in aural user interfaces according to one embodiment of the invention.
  • the user starts a navigation process in the terminal device according to step 100 and a main menu is available for selecting a desired menu option as shown in step 102.
  • the menu is fully voice-enabled and presented to the user using a TTS engine in eyes-free situation e.g. while driving a car.
  • the top-level menu structure of the main menu can be in the written form (in aural form in reality) the following:
  • step 104 in figure 2 he can make a choice for navigating in the standard (first) or rapid (second) mode. If he wants to change the mode from the default mode which normally is the first mode, in step 106 he can select the second mode by giving a sign e.g. by pressing or releasing (stop pressing) a separate key, by a long press of the menu up/down key or releasing the menu up/down key, by a special voice key or by giving a voice command. He can conventionally select a direction to browse the menu up or down. Lets presume the user's selection in step 106 is a long press of menu down key. In step 108 the next item is fetched from the menu.
  • step 110 it is checked which of the two modes have been selected.
  • the menu item is uttered in the shortened form e.g. the first syllable of the text "messages" can be "me” according to step 112. If this is the selection desired, the user gives a sign by pressing a key or by giving a voice command corresponding to "yes” in step 114 to move on to step 116 where the right item is selected. In this example "me” is not the right selection and the user gives a sign by pressing a key or by giving a voice command corresponding to "no" in step 114 to proceed to step 124 for the next new menu item.
  • step 124 the user returns from step 124 through step 126 to step 106 again. If the user wants to stop the navigation process or if he wants to select a new menu or submenu he continues from step 124 to step 128. If all the items of the menu are navigated, he also continues from step 126 to step 128 for a new selection of a menu.
  • the user had an idea that the item "task list” is somewhere around the middle in the menu structure. This means that there is no need to change the mode in step 104 and according to the invention the user continues in the second mode through menu items 1 to 5 and these items are uttered e.g. "me - ca - pro - se - ga” instead of "messages - call register - profiles - settings - games", before the right selection option would be presented to him.
  • the user can fast-forward the first five or so items, and listen in detail only that area of the menu in which he is reasonably confident the right menu item choice is in.
  • the user can decide to change mode from the second to the first mode in step 104.
  • the selection is made in step 106 deactivating the second mode by giving a sign e.g. by pressing a separate key, by a long press of the menu up/down key or releasing said key, by a special voice key or by giving a voice command.
  • This sign can be any of the aforementioned and it can be independent of the previous sign used to change the mode if more than one sign is in use for changing mode. In case if only one sign is in use for this purpose in the terminal device the user gives the same sign once again to change the mode.
  • step 110 After deactivating the second mode in step 106 the condition expressed in step 110 is not met and the selection will be the standard mode i.e. the first mode according to step 118.
  • the next menu item is number 6 "calculator”.
  • the next menu item is expressed in the first mode in step 118 in the full-length form "calculator". In this case it is not the right choice and according to step 120 the user goes on the menu structure through steps 124 and 126.
  • step 104 is the possibility to change the mode, but in this case the user doesn't do so and continues with the present mode.
  • step 118 the next item is uttered "task list" which is the right selection to the user and he selects this item in step 122 to add a note to the task list "task list”.
  • task list which is the right selection to the user and he selects this item in step 122 to add a note to the task list "task list”.
  • the first seven menu items are presented in top-level menu up direction in the form "me - ca - pro - se - ga - calculator - task list” according to the invention.
  • step 124 and 126) whether he wants to make a new selection of menu in step 128 or end navigation in step 129.
  • the loop comprising steps 104 and 106 can situate anywhere in the flow diagram i.e. the mode change can be performed whenever the user wants to do so. This same loop is also depicted in figure 1.
  • One preferred embodiment of the invention is to activate the second mode by a long press of the menu up/down key is steps 104 and 106 according to figure 2. While keeping the menu up/down key pressed down for a period of time t, rapid voice scroll in the second mode is executed in a loop according to steps 108, 110, 112, 114, (116), 124, and 126. After time period t the menu up/down key is released which means that a sign according to steps 104 and 106 is given by stopping a long press of the menu up/down key (i.e. releasing the key) and the mode is changed to the first mode. Then, standard voice scroll in the first mode is executed in a loop according to steps 108, 110, 118, 120, (122), 124 and 126 until the mode is changed again in steps 104 and 106 whenever the user wants to do so.
  • a first syllable of each item in the second mode can be replaced by a simple audio signal such as a beep "di".
  • the first seven menu items are presented in top-level menu up direction in the form "di - di - di - di - di - calculator - task list" according to the invention.
  • a beep other audio signals or aural characteristics generated by the TTS engine or predefined voice prompts e.g. a tone, pitch or any combination of aforementioned can be linked to each menu item or each group of menu items.
  • one embodiment of the invention is a fast voice scroll of menu items in a mobile phone user interface.
  • Selected menu items can be fast-forwarded or fast- reversed using the rapid (second) mode navigation according the invention and then the right selection option can be presented in the standard (first) mode. This facilitates the menu navigation especially in hands-free and eyes-free usage situations.
  • Another embodiment of the invention is to use the information of past device and service usage behavior of the user as a basis for recommendations for the right selection option.
  • the most likely option can be presented in different tone, pitch or other voice characteristics to facilitate option selection.
  • This kind of enhancement is possible in terminal devices that implement a "recommendation engine" for recommending services e.g. mobile services (m-services).
  • a menu of music items of a voice-enabled MP3 player which is either embedded as one application in a mobile phone or as a separate accessory device.
  • the user can select between different pieces of music selecting from a menu of music items e.g. names of pieces of music.
  • a menu of music items e.g. names of pieces of music.
  • all the names are provided to user interface via TTS synthesis the process is very slow.
  • he will remember a rough order of the pieces.
  • the menu items are news headlines in the user interface of the Internet browser or alike application in terminal device, the headlines can be arranged in a way that similar topics are next to each other.
  • the user can either listen to the headlines in the first mode or radiply voice scroll over topic areas not interesting him in the second mode according to the invention.
  • the menu items can be email headers prefixed with the name of the sender.
  • the user can rapidly voice scroll over mes- sages from senders not interesting him at the moment.
  • the menu items can be a calendar or task list entries arranged in order of due time and date.
  • the user can rapidly voice scroll to entries corresponding to his rough time of interest e.g. from notes for today to notes for the next days to notes for next months.
  • Figure 3a shows a flow diagram of another embodiment of invention.
  • an attribute corresponding to the menu items or groups of menu items in the aural user interface in the second mode loop in step 111 This attribute is selected by the user by a predefined sign e.g. by pressing or releasing a key or a combination of keys or by giving a voice command corresponding to the attribute.
  • the attribute can be certain menu items or group of items in the menu structure corresponding to its position in the menu defined by serial numbers or alpha-betic order of items, certain types of news headlines or music items, names, email addresses or headers prefixed with the name of senders, time or date or any other kind of similar attribute selection criteria.
  • the attribute can also correspond to information of the previous usage behavior of the user.
  • the attributes or group of items selected by the attribute are run automatically in the second mode, without needing any measures taken by the user, until said attribute is not valid anymore, i.e. the menu has been scrolled down to an item that does not fulfill the attribute criteria.
  • the attribute is valid for the item or a group of items in the second mode navigation in step 113
  • the "attribute loop" is run through in the second mode according step 112.
  • the attribute is invalid the user is asked according the embodiment of the invention described in figure 2 to select or not to select a new mode in this case the first mode navigation according steps 104 and 106.
  • Figure 3b shows a flow diagram of an optional embodiment of invention.
  • an attribute corresponding to the menu items or groups of menu items in the aural user interface in the second mode loop in step 111 This attribute is selected by the user by a predefined sign e.g. by pressing or releasing a key or a combination of keys or by giving a voice command corresponding to the attribute.
  • the attribute is certain menu items or group of items in the menu structure corresponding to successive prompts having similar contents.
  • the menu items can be email headers prefixed with the name of the sender.
  • the selection for an attribute for rapid (second) mode is made in step 111.
  • the selection for the attribute is the name of the sender of the email message.
  • the user can rapidly voice scroll over messages from senders not interesting him at the moment as shown in figure 3a.
  • the messages are expressed by the name of sender in the second e.g. in the chronological order where the latest message is presented first.
  • the voice prompt in the second mode could be "John". If he has sent five email messages it is presented in form "John- John- John- John- John” according to the rapid mode navigation illustrated in figure 3a.
  • the attribute for the rapid mode is a voice prompt "John".
  • the latest message from John is presented in the second mode as a voice prompt "John” according to steps 113, 115 and 112 in figure 3b.
  • this second mode prompt "John” be- haves like a "virtual" first mode prompt, because similar successive prompts are to follow.
  • the prompts with similar contents in succession are replaced by a "virtual" second mode prompt e.g. by audio signal "bib” according to step 117 where the "rapid rapid mode” is automatically activated after a voice prompt "John” is presented as an menu item for the latest message.
  • the prompts in rapid rapid mode can be preferable similar or different audio signals or aural characteristics generated by the TTS engine or predefined voice prompts e.g. a tone, pitch or any combination of aforementioned can be linked to each menu item or each group of menu items. All other embodiments of the invention described in association with figure 2 are also feasible with this attribute feature. Accordingly, the aforementioned attrib- ute option according to figure 3a is also applicable in the standard mode loop in figure 2 according to the invention.
  • FIG 4 is illustrated a block diagram of a system for rapid navigation in aural user interfaces according to one embodiment of the invention.
  • the TTS engine 302 is embedded into the terminal device 30 which also comprises the user interface 300 for at least aural or multimodal input and output, the processor unit 304 associated with the memory 307, DSP unit 305 for signal processing and receiver or transceiver unit 338 for receiving (and transmitting) radio frequency signal transmitted by the application server 35 through the network 33.
  • the processor unit 304 operates to present menu items to the user via the user interface and and the memory unit 307 stores the menu items and optional attributes.
  • Other means that are required for presenting the menu items to the user are loudspeakers, microphones and display associated with suitable drivers illustrated by box 301.
  • the recommendation engine (not depicted) can be embedded in association with TTS engine to the terminal device.
  • Box 306 depicts input means for acti- vating and deactivating the second mode e.g. by pressing or releasing a separate key, by a long press of menu up/down key or releasing the key, or by using a voice key by giving a voice command.
  • the TTS engine 302 is arranged so that there is a separate "pipeline" for the first mode navigation 310 and for the second mode navigation 312 and means for selecting the right mode 314 and 316 according to the se- lection made in box 306. These means illustrated by boxes 314 and 316 can be e.g. switches.
  • the arrangement comprises at least an application program 350 and a transmitter 351 in association with the server 35.
  • Figure 5 illustrates a block diagram of a system for rapid navigation in aural user interfaces according to another embodiment of the invention.
  • the TTS engine 454 is embedded into the network side 43 into the application server 45 which also comprises an application program 450, a processor unit 455 and associ- ated memory 451 for signal processing and transmitter or transceiver unit 452 for transmitting (and receiving) radio frequency signal via network 43 to the terminal device 40.
  • the recommendation engine (not depicted) can be embedded in association with TTS engine.
  • the TTS engine 454 is arranged so that there is a separate "pipeline" for the first mode navigation 456 and for the second mode navigation 458 and means for selecting the right mode, boxes 457 and 459. These means can be e.g. switches.
  • the terminal device 40 comprises the user interface 400 for at least aural or multimodal input and output, processor unit 404 for presenting menu items to the user, memory 407 for storing menu items, DSP unit 405 for signal processing and receiver or transceiver unit 402 for receiving (and transmitting) radio frequency signal from the application server 45 through the network 43.
  • Box 406 depicts input means for activating and deactivating the second mode e.g. by pressing or releasing a separate key, by a long press of menu up/down key or releasing said menu up/down key, or by using a voice key or by giving a voice command.
  • the signal processing is arranged in box 454 so that it provides both the first mode and second mode prompts or, alternatively, depending on the current mode of the terminal device either the first mode or second mode prompts to the box 405 of the terminal device 40.
  • the current mode of navigation is in accordance with the selection made in box 406.
  • Other means that are required for presenting the menu items to the user are loudspeakers, microphones and display associated with suitable drivers illustrated by box 401.
  • the present invention is an optional user interface enhancement for the end user that can be used with several other known methods to speed up the navigation process in user interfaces. As examples of these methods here is mentioned three of them.
  • the first way is to accelerate a speed of the entire TTS synthesis or voice prompts pres- entation.
  • the second known way to make navigation process faster is to provide option for the end user to customize the user interface by creating manual short-cuts as well-known in the field of web browsing.
  • the third way to speed up the navigation process is to provide automatic short-cuts to digital services for the end user. At least all these above-mentioned alternatives can be used complementary to the embodiments of the present invention.
  • WO 01/45086 System and Method of Voice Browsing for Mobile Terminals Using Dual-Mode Wireless Connection

Abstract

The invention allows experienced users to rapidly navigate through audio or multimodal menu structures in aural user interfaces for accessing digital devices and services. The aural user interface has two modes for navigating through menu structures, one of which is a standard mode for normal use (118) and the other a rapid mode (112) to speed up the navigation process. The mode can be changed (104, 106) whenever during the navigation process by the user.

Description

Method and System for Rapid Navigation in Aural User Interface
TECHNICAL FIELD OF THE INVENTION
The invention relates to user interfaces for accessing digital devices and services. More specifically, the invention is a method for enhancing the usability of voice and multimodal user interfaces. The user can choose from two modes for navigation through menu structures, one of which is a standard mode for normal use and the other a rapid mode for experienced use.
BACKGROUND OF THE INVENTION
User interfaces for digital devices and services typically use menu hierarchies both (a) to inform the user of alternative selection options as well as (b) to provide the means for the user to navigate through the menu layers to the desired selection option. Typically, the menu hierarchies are presented in visual form. In situations, e.g. when hands-free and eyes-free use of digital devices and services are desired, a vis- ual presentation of the menu hierarchies is not feasible. Situations like these occur e.g. in cars and other vehicles or in case of people with visual impairments, few to mention.
In such situations as mentioned above a menu hierarchy can be presented in aural form using for example text-to-speech (TTS) synthesis or predefined voice prompts to inform a user of alternative selection options. However, although an aural or voice user interface is suitable for informing the user of the alternative selection options, it has a drawback to be relative slow in presenting information to the user. When a user is navigating through the menu structure for the first time, this slowness is justified due the need to inform the user of the alternative selection options and to initiate him into the menu structure. After several repeated times of navigating the menu the user learns the menu structure, and the slowness brings a major disconvenience on the user. Nevertheless, a need for navigating through the menu layers remains, but the user would prefer faster means of going through the menu layers.
There are known several ways to speed up the navigation process in user interfaces. The first way is to accelerate a speed of the entire TTS synthesis or voice prompts presentation. This means that the aural presentation of the menu remains in the original form but pronunciation is uttered faster which may exacerbate understand- ing the menu items. Another known way to make navigation process faster is to provide option for the end user to customize the user interface by creating manual short-cuts. This is well-known and widely used in the field of web browsing. Still another way to speed up the navigation process is to provide automatic short-cuts to digital services for the end user. The latter two ways are mainly used in visual user interfaces and they expect rather many steps to be taken by the user to reconcile the menu structure with the short-cuts. It is also possible to arrange recommendations of most wanted menu items based on the earlier user behavior but this upsets the original menu structure and complicates navigating through the menu items infrequently used. However, all these above-mentioned alternatives can be used complementary to the present invention described in section Summary of the invention.
One known method to accelerate a TTS client application is described in the document US 6188983 "Method for Dynamically Altering Text-to-Speech (TTS) Attributes of a TTS Engine not Inherently Capable of Dynamic Attribute Alteration" [1]. This document makes known a method that enables a TTS client application to change e.g. pitch and speed while playback is in progress. This capability can be used when TTS engines don't allow these modifications to be made dynamically. This method is restricted to adjust TTS parameters like pitch and speed of TTS playback by the user without stopping playback. This method is targeted for general purpose use in connection with TTS playback where the text can be read faster or slower depending on user request but the contents of a text remain in the original form. Simply, this means that the user has to listen to the same litany of voice prompts (and text) every time again and again whenever navigating the menu items. This is inconvenient for the users that are well familiar with the contents of a voice prompt (and text).
Another known method for using TTS or predefined audio prompts in user interfaces is described in the document WO 01/45086 entitled "System and Method of Voice Browsing for Mobile Terminals Using Dual-Mode Wireless Connection" [2]. Here, interactive voice response services are used by means of a voice mode and a data mode for alternately transmitting voice and data between the mobile terminal and the server application. During one call there is a capability of swapping between speech and data. The speech content, user input and command vary according to the particular voice application. The main idea in this method is to use a limited sized grammar to improve speech recognition in general on terminal-side but it doesn't influence the navigation properties e.g. speed in the aural user interface itself. Again, this means that the user has to listen through the same speech every time which is inconvenient for the users that are well familiar with the contents of a speech.
Therefore there is a need for especially experienced users to improve the navigation characteristics in aural user interfaces for accessing digital devices and services. To assure user-friendly hands-free and eyes-free operation of the user interface a flexible and fast navigation process is an essential need.
SUMMARY OF THE INVENTION
The object of the invention is to provide a method and system which allows rapid voice scrolling of menu items to the user in aural user interfaces. The objective of the invention is achieved by activating the rapid mode in which mode voice prompts corresponding to menu items are shortened significantly and hence the user interface is faster than in prior art. In the standard mode when the rapid mode is deactivated full-length voice prompts are used.
The advantage of the invention is that those users who are well familiar with the menu structure easily activate and use the rapid mode to avoid themselves a slowness of the standard mode user interface. Meanwhile in unfamiliar usage situations the users may choose navigating through menu items in the standard mode. The invention enhances the usability of aural and multimodal user interfaces providing for experienced users a faster way to present menu options, if desired. Otherwise the user can continue with standard mode in normal order. Also an advantage of this invention is that it can be used in association with other methods and systems to speed up navigation process.
The method of present invention relates to a method for navigating in user interfaces for accessing digital devices and services, the device comprising at least an aural or multimodal user interface and connection to the network device, is characterized in that, it comprises steps, in which a second mode for an aural user interface is activated and said second mode comprising different and shorter content than the first mode.
The system of present invention relates to a system for navigating in aural user in- terfaces for accessing digital devices and services, the device comprising means for at least an aural or multimodal user interface and means for connection to the network device, is characterized in that, it comprises means for activating and deactivating a second mode for an aural user interface. According to the present invention a network device is a server in a network or a computer device in a network using peer-to-peer connection.
One preferred embodiment of the invention is to use TTS engine embedded in the terminal device. In some embodiments the TTS engine can be in the network and accordingly the TTS engine should provide both standard and rapid prompts to the terminal device or, alternatively, either standard or rapid prompts depending on the current mode of the terminal device. One embodiment of the invention is to use predefined voice prompts both in standard and rapid mode. In one simplified embodiment of the invention the voice prompts in the rapid mode are just short audio signals e.g. beeps. In some embodiments of the invention the rapid mode is activated by a special key or by a long press of menu up and menu down key. In some embodiments of the invention the rapid mode is activated by the long press on menu up/down function key for a fast forward/reverse operation and deactivated by releasing said menu up/down key for a standard forward/reverse operation.
Some embodiments of the invention are described in the dependent claims.
BRIEF DESCRIPTION OF THE DRAWINGS
The foregoing and other objects, features and advantages of this invention will be apparent from the following more particular description of the preferred embodiments of the invention as illustrated in the accompanying drawings.
Fig 1. is a flow diagram of a method for selecting a mode according to the invention.
Fig 2. is a flow diagram of a method for rapid navigation in aural user interfaces according to one embodiment of the invention.
Fig 3a. is a flow diagram of a method for rapid navigation in aural user interfaces according to another embodiment of the invention.
Fig 3b. is a flow diagram of a method for rapid navigation in aural user interfaces according to an optional embodiment of the invention.
Fig 4. is a block diagram of a system for rapid navigation in aural user interfaces according to one embodiment of the invention.
Fig 5. is a block diagram of a system for rapid navigation in aural user interfaces according to another embodiment of the invention. DETAILED DESCRIPTION
According to the invention an aural or voice user interface has two modes for navigating through menu structures and the two modes are a standard or normal mode and a rapid mode. In this document a standard mode is called a "first mode" and a rapid mode is called a "second mode". In user interfaces menu options (menu items) can be presented in aural form using e.g. voice prompts generated by text-to-speech (TTS) synthesis or predefined voice prompts. When the second mode is activated, voice prompts corresponding to menu items are shortened significantly by presenting only e.g. a first syllable of each voice prompt. When the second mode is deacti- vated the first mode is used where voice prompts corresponding to menu items are presented as full-length voice prompts.
Fig 1. shows a flow diagram of the two modes for navigating in aural user interfaces according to the invention. The first mode is depicted by block 11 and the second mode depicted by block 12 and the change between these two blocks is activated by giving a sign by the user e.g. by pressing a separate key, by a long press of the menu up/down key, by a special voice key or by giving a voice command. The selection of the mode is independent of the actual location of the navigation process i.e. the user may change the mode whenever he wants to do so. In case of timeout situation which occurs when e.g. a large block of operations may disconnect a network con- nection automatically after a fixed period of time, the mode automatically changes to the first (normal) mode. So, after timeout the first mode is a default mode.
Fig 2. shows a flow diagram of a method for navigating in aural user interfaces according to one embodiment of the invention. The user starts a navigation process in the terminal device according to step 100 and a main menu is available for selecting a desired menu option as shown in step 102. Lets presume that the menu is fully voice-enabled and presented to the user using a TTS engine in eyes-free situation e.g. while driving a car. In case of a mobile phone as a terminal device the top-level menu structure of the main menu can be in the written form (in aural form in reality) the following:
(1) messages
(2) call register
(3) profiles
(4) settings (5) games
(6) calculator
(7) task list
(8) calendar
(9) infrared
(10) radio
(11) extra functions
(12) services
The user would like to add a note on the item (7) "task list". However, he doesn't' remember whether said item is called "task list", "notebook", "post-it stickers", "to- do-list", or any number of similar expressions. But as an experienced user he does remember that the desired item is somewhere around the middle in the menu structure.
According to the invention the user has alternative ways to proceed. In step 104 in figure 2 he can make a choice for navigating in the standard (first) or rapid (second) mode. If he wants to change the mode from the default mode which normally is the first mode, in step 106 he can select the second mode by giving a sign e.g. by pressing or releasing (stop pressing) a separate key, by a long press of the menu up/down key or releasing the menu up/down key, by a special voice key or by giving a voice command. He can conventionally select a direction to browse the menu up or down. Lets presume the user's selection in step 106 is a long press of menu down key. In step 108 the next item is fetched from the menu. In step 110 it is checked which of the two modes have been selected. In case of the second mode the menu item is uttered in the shortened form e.g. the first syllable of the text "messages" can be "me" according to step 112. If this is the selection desired, the user gives a sign by pressing a key or by giving a voice command corresponding to "yes" in step 114 to move on to step 116 where the right item is selected. In this example "me" is not the right selection and the user gives a sign by pressing a key or by giving a voice command corresponding to "no" in step 114 to proceed to step 124 for the next new menu item. If there are any items left in the same menu, the user returns from step 124 through step 126 to step 106 again. If the user wants to stop the navigation process or if he wants to select a new menu or submenu he continues from step 124 to step 128. If all the items of the menu are navigated, he also continues from step 126 to step 128 for a new selection of a menu.
In this example the user had an idea that the item "task list" is somewhere around the middle in the menu structure. This means that there is no need to change the mode in step 104 and according to the invention the user continues in the second mode through menu items 1 to 5 and these items are uttered e.g. "me - ca - pro - se - ga" instead of "messages - call register - profiles - settings - games", before the right selection option would be presented to him. In other words, according to the invention the user can fast-forward the first five or so items, and listen in detail only that area of the menu in which he is reasonably confident the right menu item choice is in.
When in the area of the menu where the desired item most probably locates, the user can decide to change mode from the second to the first mode in step 104. The selection is made in step 106 deactivating the second mode by giving a sign e.g. by pressing a separate key, by a long press of the menu up/down key or releasing said key, by a special voice key or by giving a voice command. This sign can be any of the aforementioned and it can be independent of the previous sign used to change the mode if more than one sign is in use for changing mode. In case if only one sign is in use for this purpose in the terminal device the user gives the same sign once again to change the mode. After deactivating the second mode in step 106 the condition expressed in step 110 is not met and the selection will be the standard mode i.e. the first mode according to step 118. Lets presume that after changing mode according to steps 104 and 106 the next menu item is number 6 "calculator". Then the next menu item is expressed in the first mode in step 118 in the full-length form "calculator". In this case it is not the right choice and according to step 120 the user goes on the menu structure through steps 124 and 126. Then again in step 104 is the possibility to change the mode, but in this case the user doesn't do so and continues with the present mode. Then in step 118 the next item is uttered "task list" which is the right selection to the user and he selects this item in step 122 to add a note to the task list "task list". So in this example, to this far the first seven menu items are presented in top-level menu up direction in the form "me - ca - pro - se - ga - calculator - task list" according to the invention. After this the user can choose in step 124 (and 126) whether he wants to make a new selection of menu in step 128 or end navigation in step 129. It should be noted that in figure 2 the loop comprising steps 104 and 106 can situate anywhere in the flow diagram i.e. the mode change can be performed whenever the user wants to do so. This same loop is also depicted in figure 1.
One preferred embodiment of the invention is to activate the second mode by a long press of the menu up/down key is steps 104 and 106 according to figure 2. While keeping the menu up/down key pressed down for a period of time t, rapid voice scroll in the second mode is executed in a loop according to steps 108, 110, 112, 114, (116), 124, and 126. After time period t the menu up/down key is released which means that a sign according to steps 104 and 106 is given by stopping a long press of the menu up/down key (i.e. releasing the key) and the mode is changed to the first mode. Then, standard voice scroll in the first mode is executed in a loop according to steps 108, 110, 118, 120, (122), 124 and 126 until the mode is changed again in steps 104 and 106 whenever the user wants to do so.
According to one embodiment of the invention in the second mode a first syllable of each item can be replaced by a simple audio signal such as a beep "di". In this case the first seven menu items are presented in top-level menu up direction in the form "di - di - di - di - di - calculator - task list" according to the invention. According to some embodiments of the invention instead of a beep other audio signals or aural characteristics generated by the TTS engine or predefined voice prompts e.g. a tone, pitch or any combination of aforementioned can be linked to each menu item or each group of menu items.
To summarize, one embodiment of the invention is a fast voice scroll of menu items in a mobile phone user interface. Selected menu items can be fast-forwarded or fast- reversed using the rapid (second) mode navigation according the invention and then the right selection option can be presented in the standard (first) mode. This facilitates the menu navigation especially in hands-free and eyes-free usage situations.
Another embodiment of the invention is to use the information of past device and service usage behavior of the user as a basis for recommendations for the right selection option. The most likely option can be presented in different tone, pitch or other voice characteristics to facilitate option selection. This kind of enhancement is possible in terminal devices that implement a "recommendation engine" for recommending services e.g. mobile services (m-services).
According to still one embodiment of the invention is a menu of music items of a voice-enabled MP3 player which is either embedded as one application in a mobile phone or as a separate accessory device. The user can select between different pieces of music selecting from a menu of music items e.g. names of pieces of music. However, if all the names are provided to user interface via TTS synthesis the process is very slow. Moreover, in particular in case of experienced user, he will remember a rough order of the pieces. In this case according to the invention it is possible to fast-forward or fast-reverse those areas of the menu where it is not likely the desired music item exists.
Still some other embodiments of the invention are to follow. In case the menu items are news headlines in the user interface of the Internet browser or alike application in terminal device, the headlines can be arranged in a way that similar topics are next to each other. Thus the user can either listen to the headlines in the first mode or radiply voice scroll over topic areas not interesting him in the second mode according to the invention. In case of email the menu items can be email headers prefixed with the name of the sender. Thus the user can rapidly voice scroll over mes- sages from senders not interesting him at the moment. In case of personal organizers, mobile phones, PDAs or other similar devices the menu items can be a calendar or task list entries arranged in order of due time and date. Thus according to the invention the user can rapidly voice scroll to entries corresponding to his rough time of interest e.g. from notes for today to notes for the next days to notes for next months.
Figure 3a shows a flow diagram of another embodiment of invention. In addition to the embodiment of the invention described in figure 2, there is a possibility to set an attribute corresponding to the menu items or groups of menu items in the aural user interface in the second mode loop in step 111. This attribute is selected by the user by a predefined sign e.g. by pressing or releasing a key or a combination of keys or by giving a voice command corresponding to the attribute. The attribute can be certain menu items or group of items in the menu structure corresponding to its position in the menu defined by serial numbers or alpha-betic order of items, certain types of news headlines or music items, names, email addresses or headers prefixed with the name of senders, time or date or any other kind of similar attribute selection criteria. The attribute can also correspond to information of the previous usage behavior of the user. When the attribute option is set in step 111, the items or group of items selected by the attribute are run automatically in the second mode, without needing any measures taken by the user, until said attribute is not valid anymore, i.e. the menu has been scrolled down to an item that does not fulfill the attribute criteria. When the attribute is valid for the item or a group of items in the second mode navigation in step 113, the "attribute loop" is run through in the second mode according step 112. When the attribute is invalid the user is asked according the embodiment of the invention described in figure 2 to select or not to select a new mode in this case the first mode navigation according steps 104 and 106. All the other steps illustrated by dash line relating to the embodiment of figure 3a are the same as depicted in figure 2. The attribute can be defined among menu items in different ways e.g. it can be a group of items beginning with the same letter or presented by same audio signal (beep) or some other way alike. All other embodiments of the invention described in association with figure 2 are also feasible with this attribute feature. Accordingly, the aforementioned attribute option according to figure 3a is also applicable in the standard mode loop in figure 2 according to the invention.
Figure 3b shows a flow diagram of an optional embodiment of invention. In addition to the embodiment of the invention described in figure 2, there is a possibility to set an attribute corresponding to the menu items or groups of menu items in the aural user interface in the second mode loop in step 111. This attribute is selected by the user by a predefined sign e.g. by pressing or releasing a key or a combination of keys or by giving a voice command corresponding to the attribute. In this embodiment the attribute is certain menu items or group of items in the menu structure corresponding to successive prompts having similar contents. E.g. in case of email messages the menu items can be email headers prefixed with the name of the sender. According to the embodiment of the invention illustrated in figure 3b, the selection for an attribute for rapid (second) mode is made in step 111. Lets presume in this example that the selection for the attribute is the name of the sender of the email message. Now, the user can rapidly voice scroll over messages from senders not interesting him at the moment as shown in figure 3a. There might be many messages from the same sender and the messages are expressed by the name of sender in the second e.g. in the chronological order where the latest message is presented first. For the name "John Smith" the voice prompt in the second mode could be "John". If he has sent five email messages it is presented in form "John- John- John- John- John" according to the rapid mode navigation illustrated in figure 3a. According to the optional embodiment illustrated in figure 3b the attribute for the rapid mode is a voice prompt "John". The latest message from John is presented in the second mode as a voice prompt "John" according to steps 113, 115 and 112 in figure 3b. According to the optional embodiment this second mode prompt "John" be- haves like a "virtual" first mode prompt, because similar successive prompts are to follow. In this case the prompts with similar contents in succession are replaced by a "virtual" second mode prompt e.g. by audio signal "bib" according to step 117 where the "rapid rapid mode" is automatically activated after a voice prompt "John" is presented as an menu item for the latest message. As a resault of this example, there is presented in the rapid mode navigation a litany of prompts "John-bib-bib- bib-bib" instead of repeating the name five times. The prompts in rapid rapid mode can be preferable similar or different audio signals or aural characteristics generated by the TTS engine or predefined voice prompts e.g. a tone, pitch or any combination of aforementioned can be linked to each menu item or each group of menu items. All other embodiments of the invention described in association with figure 2 are also feasible with this attribute feature. Accordingly, the aforementioned attrib- ute option according to figure 3a is also applicable in the standard mode loop in figure 2 according to the invention.
In figure 4 is illustrated a block diagram of a system for rapid navigation in aural user interfaces according to one embodiment of the invention. In this embodiment the TTS engine 302 is embedded into the terminal device 30 which also comprises the user interface 300 for at least aural or multimodal input and output, the processor unit 304 associated with the memory 307, DSP unit 305 for signal processing and receiver or transceiver unit 338 for receiving (and transmitting) radio frequency signal transmitted by the application server 35 through the network 33. The processor unit 304 operates to present menu items to the user via the user interface and and the memory unit 307 stores the menu items and optional attributes. Other means that are required for presenting the menu items to the user are loudspeakers, microphones and display associated with suitable drivers illustrated by box 301. As an option also the recommendation engine (not depicted) can be embedded in association with TTS engine to the terminal device. Box 306 depicts input means for acti- vating and deactivating the second mode e.g. by pressing or releasing a separate key, by a long press of menu up/down key or releasing the key, or by using a voice key by giving a voice command. The TTS engine 302 is arranged so that there is a separate "pipeline" for the first mode navigation 310 and for the second mode navigation 312 and means for selecting the right mode 314 and 316 according to the se- lection made in box 306. These means illustrated by boxes 314 and 316 can be e.g. switches. In the network side the arrangement comprises at least an application program 350 and a transmitter 351 in association with the server 35. Communication between the terminal device 30 and application server 35 in the network, or a computer device in a network using peer-to-peer connection, takes place by any known telecommunication system which is compliant with but not limited to, at least one of the following: TCP/IP, CDMA, HSCSD, GPRS, WCDMA, EDGE, UMTS, Bluetooth, Telsedic, Iridium, Inmarsat, WLAN, DIGI-TV and imode. Figure 5 illustrates a block diagram of a system for rapid navigation in aural user interfaces according to another embodiment of the invention. In this embodiment the TTS engine 454 is embedded into the network side 43 into the application server 45 which also comprises an application program 450, a processor unit 455 and associ- ated memory 451 for signal processing and transmitter or transceiver unit 452 for transmitting (and receiving) radio frequency signal via network 43 to the terminal device 40. As an option also the recommendation engine (not depicted) can be embedded in association with TTS engine. The TTS engine 454 is arranged so that there is a separate "pipeline" for the first mode navigation 456 and for the second mode navigation 458 and means for selecting the right mode, boxes 457 and 459. These means can be e.g. switches. The terminal device 40 comprises the user interface 400 for at least aural or multimodal input and output, processor unit 404 for presenting menu items to the user, memory 407 for storing menu items, DSP unit 405 for signal processing and receiver or transceiver unit 402 for receiving (and transmitting) radio frequency signal from the application server 45 through the network 43. Box 406 depicts input means for activating and deactivating the second mode e.g. by pressing or releasing a separate key, by a long press of menu up/down key or releasing said menu up/down key, or by using a voice key or by giving a voice command. The signal processing is arranged in box 454 so that it provides both the first mode and second mode prompts or, alternatively, depending on the current mode of the terminal device either the first mode or second mode prompts to the box 405 of the terminal device 40. The current mode of navigation is in accordance with the selection made in box 406. Other means that are required for presenting the menu items to the user are loudspeakers, microphones and display associated with suitable drivers illustrated by box 401. Communication between the terminal device 40 and application server 45, or a computer device in a network using peer-to-peer connection, takes place by any known telecommunication system which is compliant with but not limited to, at least one of the following: TCP/IP, CDMA, HSCSD, GPRS, WCDMA, EDGE, UMTS, Bluetooth, Telsedic, Mdium, Inmarsat, WLAN, DIGI-TV and imode.
The present invention is an optional user interface enhancement for the end user that can be used with several other known methods to speed up the navigation process in user interfaces. As examples of these methods here is mentioned three of them. The first way is to accelerate a speed of the entire TTS synthesis or voice prompts pres- entation. The second known way to make navigation process faster is to provide option for the end user to customize the user interface by creating manual short-cuts as well-known in the field of web browsing. The third way to speed up the navigation process is to provide automatic short-cuts to digital services for the end user. At least all these above-mentioned alternatives can be used complementary to the embodiments of the present invention.
While presently preferred embodiments of the invention have been shown and de- scribed in particularity, those skilled in the art will recognize that the invention is not limited to the embodiments described herein. The invention may be otherwise embodied within the spirit and scope of the idea as set forth in the appended claims.
CITED DOCUMENTS
[1] US 6188983: "Method for Dynamically Altering Text-to-Speech (TTS) Attrib- utes of a TTS Engine not Inherently Capable of Dynamic Attribute Alteration"
[2] WO 01/45086: "System and Method of Voice Browsing for Mobile Terminals Using Dual-Mode Wireless Connection"

Claims

Claims
1. A method for navigating in aural user interfaces for accessing digital devices and services, comprising steps, in which
- menu items of said aural user interface is initiated (100, 102),
- a first mode (11, 118) with a first audio content for said aural user interface is presented (102), and
- a second mode (12, 112) for said aural user interface is presented for activation (104, 106); said second mode being an alternative choice to said first mode; said second mode comprising of modified audio content, the modification being based on the audio content in said first mode.
2. A method according to claim 1, wherein the first mode is automatically selected when said second mode is not activated.
3. A method according to claim 1, wherein the first mode is automatically selected after a timeout situation.
4. A method according to claim 1, wherein the first mode and second mode is interchanged by the user, regardless of the step of navigation process.
5. A method according to claim 1, wherein menu items in the first mode are presented in full-length form generated by text-to-speech (TTS) synthesis, predefined voice prompts or any other aural means.
6. A method according to claim 1, wherein menu items in the second mode are presented in shortened form generated by text-to-speech (TTS) synthesis, predefined voice prompts or any other aural means compared to the first mode.
7. A method according to claim 1, wherein an audio content of said second mode is different and shorter than an audio content of said first mode.
8. A method according to claim 1 or 6, wherein said second mode comprises shortened form of menu items, wherein said shortened form is one of the following: a syllable of a prompt, an uttered number of an item, a tone, a pitch, a beep, any aural characteristic, any audio signal referring to the items.
9. A method according to claim 1, comprising further steps, in which - an attribute linked to the second mode of the aural user interface is selected (HI), and
- the second mode for the aural user interface is selected when said attribute is valid (112, 113).
10. A method according to any of claims 1, 8 or 9, wherein said attribute linked to the second mode corresponding to the menu items or a group of items, is defined for successive items with similar contents and similar voice prompts in the second mode, so that a plurality of said voice prompts are replaced by a shorter audio or aural signal (117) during the second mode, except the first voice prompt of similar prompts which is presented in the second mode (112) according to the original selection (104, 106).
11. A method according to claim 10, wherein said shorter audio or aural signals (117) during the second mode are identical or different.
12. A method according to claim 1, wherein said attribute corresponding to the menu item or a group of items is based on its position in the menu, information of the previous usage behavior of the user, type of audio signal referring to shortened form of items or any similar attribute selection criteria, and said attribute is linked to said second mode.
13. A method according to claim 12, wherein the most likely options are presented in different tone, pitch or any similar type audio or voice characteristic when the attribute is based on the previous usage behavior of the user.
14. A method according to claim 1, wherein the menu comprises a menu item of an aural user interface, wherein the item is one of the following: an item in aural user interface of a mobile phone menu, a music item of a MP3 player menu, an email header of an email menu, a news headline of Internet service menu, a calendar or task list of a communicator or PDA menu, any other item of a menu of digital devices or services.
15. A method according to claim 1, used for hands-free and eyes-free usage of digital devices and services.
16. A system for navigating in aural user interfaces for accessing digital devices and services, the terminal device (30, 40) comprising means for at least said aural user interface (300, 400) and means for connection (338, 351, 402, 452) to the network device (35, 45), comprising
- means for initiating menu items of said aural user interface (304, 307, 404, 407),
- means for selecting a first mode with a first audio content for said aural user interface (310, 314, 316, 456, 457, 459), and
- means for activating and deactivating a second mode for said aural user interface (306, 310, 314, 316, 406, 458, 457, 459), said second mode being an alternative choice to the first mode, and said second mode comprising of modified audio content, the modification being based on the audio content in said first mode.
17. A system according to claim 16, comprising means for automatically selecting the first mode when said second mode is not activated.
18. A system according to claim 16, wherein the first mode is automatically se- lected after a timeout situation.
19. A system according to claim 16, comprising means (300, 301, 304, 305, 400, 401, 404, 405) for presenting the menu items in full-length form in the first mode generated by text-to-speech (TTS) synthesis, predefined voice prompts or any other aural means.
20. A system according to claim 16, comprising means (300, 301, 304, 305, 400, 401, 404, 405) for presenting menu items in shortened form in the second mode generated by text-to-speech (TTS) synthesis, predefined voice prompts or any other aural means.
21. A system according to claim 16, wherein an audio content of said second mode is different and shorter than an audio content of said first mode.
22. A system according to claim 16, comprising a TTS engine (302) embedded in the terminal device (30).
23. A system according to claim 16, comprising a TTS engine (454) in the network side (43, 45) and the TTS engine comprises means (456, 457, 458, 459) for providing voice prompts both in the first mode and in the second mode, or, alterna- tively, either in the first mode or in the second mode depending on the current mode of the terminal device (40).
24. A system according to claim 16, characterized in that, it comprises means (306, 406) for automatically activating the second mode.
25. A system according to claim 16, comprising means (306, 406) for activating and deactivating the second mode, wherein said means is one of the following: a press of a separate key, a release of a separate key, a long press of the menu up/down key, a release of the menu up/down key, a special voice key, a voice command.
26. A system according to claim 16, wherein the long press on menu up/down function key (306, 406) activates a fast forward/reverse operation and the release of said menu up/down key activates a standard forward/reverse operation.
27. A system according to claim 16, further comprising means for selecting an attribute linked to the second mode and evaluating said attribute for the right selection (304, 307, 404, 407).
28. A system according to claim 16, wherein the attribute based on the information of the previous usage behavior of the user is generated by the recommendation engine embedded in the terminal device.
29. A system according to claim 16, wherein the aural user interface comprises means for multimodal user interface (300, 400).
30. A system according to claim 16, used for hands-free and eyes-free usage of digital devices and services.
PCT/FI2002/000363 2002-04-29 2002-04-29 Method and system for rapid navigation in aural user interface WO2003094489A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
PCT/FI2002/000363 WO2003094489A1 (en) 2002-04-29 2002-04-29 Method and system for rapid navigation in aural user interface
JP2004502598A JP2005524122A (en) 2002-04-29 2002-04-29 Fast navigation method and system in auditory user interface
AU2002251113A AU2002251113A1 (en) 2002-04-29 2002-04-29 Method and system for rapid navigation in aural user interface

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/FI2002/000363 WO2003094489A1 (en) 2002-04-29 2002-04-29 Method and system for rapid navigation in aural user interface

Publications (1)

Publication Number Publication Date
WO2003094489A1 true WO2003094489A1 (en) 2003-11-13

Family

ID=29286315

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/FI2002/000363 WO2003094489A1 (en) 2002-04-29 2002-04-29 Method and system for rapid navigation in aural user interface

Country Status (3)

Country Link
JP (1) JP2005524122A (en)
AU (1) AU2002251113A1 (en)
WO (1) WO2003094489A1 (en)

Cited By (100)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1665222A2 (en) * 2004-06-03 2006-06-07 Leapfrog Enterprises, Inc., Pepi Ross, Director of Legal Affairs User created interactive interface
CN100351763C (en) * 2004-09-24 2007-11-28 三星电子株式会社 Method and apparatus for providing a user control interface in audio multistreaming
GB2467432A (en) * 2009-01-30 2010-08-04 Apple Inc Audio user interface with single sensing element for displayless electronic device
US8952887B1 (en) 2001-06-20 2015-02-10 Leapfrog Enterprises, Inc. Interactive references to related application
US9262612B2 (en) 2011-03-21 2016-02-16 Apple Inc. Device access using voice authentication
US9318108B2 (en) 2010-01-18 2016-04-19 Apple Inc. Intelligent automated assistant
US9330720B2 (en) 2008-01-03 2016-05-03 Apple Inc. Methods and apparatus for altering audio output signals
US9338493B2 (en) 2014-06-30 2016-05-10 Apple Inc. Intelligent automated assistant for TV user interactions
US9483461B2 (en) 2012-03-06 2016-11-01 Apple Inc. Handling speech synthesis of content for multiple languages
US9495129B2 (en) 2012-06-29 2016-11-15 Apple Inc. Device, method, and user interface for voice-activated navigation and browsing of a document
US9535906B2 (en) 2008-07-31 2017-01-03 Apple Inc. Mobile device having human language translation capability with positional feedback
US9582608B2 (en) 2013-06-07 2017-02-28 Apple Inc. Unified ranking with entropy-weighted information for phrase-based semantic auto-completion
US9620104B2 (en) 2013-06-07 2017-04-11 Apple Inc. System and method for user-specified pronunciation of words for speech synthesis and recognition
US9626955B2 (en) 2008-04-05 2017-04-18 Apple Inc. Intelligent text-to-speech conversion
US9633660B2 (en) 2010-02-25 2017-04-25 Apple Inc. User profiling for voice input processing
US9633674B2 (en) 2013-06-07 2017-04-25 Apple Inc. System and method for detecting errors in interactions with a voice-based digital assistant
US9646609B2 (en) 2014-09-30 2017-05-09 Apple Inc. Caching apparatus for serving phonetic pronunciations
US9646614B2 (en) 2000-03-16 2017-05-09 Apple Inc. Fast, language-independent method for user authentication by voice
US9668121B2 (en) 2014-09-30 2017-05-30 Apple Inc. Social reminders
US9697820B2 (en) 2015-09-24 2017-07-04 Apple Inc. Unit-selection text-to-speech synthesis using concatenation-sensitive neural networks
US9715875B2 (en) 2014-05-30 2017-07-25 Apple Inc. Reducing the need for manual start/end-pointing and trigger phrases
US9721566B2 (en) 2015-03-08 2017-08-01 Apple Inc. Competing devices responding to voice triggers
US9760559B2 (en) 2014-05-30 2017-09-12 Apple Inc. Predictive text input
US9785630B2 (en) 2014-05-30 2017-10-10 Apple Inc. Text prediction using combined word N-gram and unigram language models
US9798393B2 (en) 2011-08-29 2017-10-24 Apple Inc. Text correction processing
US9818400B2 (en) 2014-09-11 2017-11-14 Apple Inc. Method and apparatus for discovering trending terms in speech requests
US9842105B2 (en) 2015-04-16 2017-12-12 Apple Inc. Parsimonious continuous-space phrase representations for natural language processing
US9842101B2 (en) 2014-05-30 2017-12-12 Apple Inc. Predictive conversion of language input
US9858925B2 (en) 2009-06-05 2018-01-02 Apple Inc. Using context information to facilitate processing of commands in a virtual assistant
US9865280B2 (en) 2015-03-06 2018-01-09 Apple Inc. Structured dictation using intelligent automated assistants
US9886953B2 (en) 2015-03-08 2018-02-06 Apple Inc. Virtual assistant activation
US9886432B2 (en) 2014-09-30 2018-02-06 Apple Inc. Parsimonious handling of word inflection via categorical stem + suffix N-gram language models
US9899019B2 (en) 2015-03-18 2018-02-20 Apple Inc. Systems and methods for structured stem and suffix language models
US9934775B2 (en) 2016-05-26 2018-04-03 Apple Inc. Unit-selection text-to-speech synthesis based on predicted concatenation parameters
US9953088B2 (en) 2012-05-14 2018-04-24 Apple Inc. Crowd sourcing information to fulfill user requests
US9966068B2 (en) 2013-06-08 2018-05-08 Apple Inc. Interpreting and acting upon commands that involve sharing information with remote devices
US9966065B2 (en) 2014-05-30 2018-05-08 Apple Inc. Multi-command single utterance input method
US9972304B2 (en) 2016-06-03 2018-05-15 Apple Inc. Privacy preserving distributed evaluation framework for embedded personalized systems
US9971774B2 (en) 2012-09-19 2018-05-15 Apple Inc. Voice-based media searching
US10043516B2 (en) 2016-09-23 2018-08-07 Apple Inc. Intelligent automated assistant
US10049663B2 (en) 2016-06-08 2018-08-14 Apple, Inc. Intelligent automated assistant for media exploration
US10049668B2 (en) 2015-12-02 2018-08-14 Apple Inc. Applying neural network language models to weighted finite state transducers for automatic speech recognition
US10057736B2 (en) 2011-06-03 2018-08-21 Apple Inc. Active transport based notifications
US10067938B2 (en) 2016-06-10 2018-09-04 Apple Inc. Multilingual word prediction
US10074360B2 (en) 2014-09-30 2018-09-11 Apple Inc. Providing an indication of the suitability of speech recognition
US10079014B2 (en) 2012-06-08 2018-09-18 Apple Inc. Name recognition system
US10078631B2 (en) 2014-05-30 2018-09-18 Apple Inc. Entropy-guided text prediction using combined word and character n-gram language models
US10083688B2 (en) 2015-05-27 2018-09-25 Apple Inc. Device voice control for selecting a displayed affordance
US10089072B2 (en) 2016-06-11 2018-10-02 Apple Inc. Intelligent device arbitration and control
US10101822B2 (en) 2015-06-05 2018-10-16 Apple Inc. Language input correction
US10127911B2 (en) 2014-09-30 2018-11-13 Apple Inc. Speaker identification and unsupervised speaker adaptation techniques
US10127220B2 (en) 2015-06-04 2018-11-13 Apple Inc. Language identification from short strings
US10169329B2 (en) 2014-05-30 2019-01-01 Apple Inc. Exemplar-based natural language processing
US10176167B2 (en) 2013-06-09 2019-01-08 Apple Inc. System and method for inferring user intent from speech inputs
US10186254B2 (en) 2015-06-07 2019-01-22 Apple Inc. Context-based endpoint detection
US10185542B2 (en) 2013-06-09 2019-01-22 Apple Inc. Device, method, and graphical user interface for enabling conversation persistence across two or more instances of a digital assistant
US10192552B2 (en) 2016-06-10 2019-01-29 Apple Inc. Digital assistant providing whispered speech
US10223066B2 (en) 2015-12-23 2019-03-05 Apple Inc. Proactive assistance based on dialog communication between devices
US10241644B2 (en) 2011-06-03 2019-03-26 Apple Inc. Actionable reminder entries
US10241752B2 (en) 2011-09-30 2019-03-26 Apple Inc. Interface for a virtual digital assistant
US10249300B2 (en) 2016-06-06 2019-04-02 Apple Inc. Intelligent list reading
US10255907B2 (en) 2015-06-07 2019-04-09 Apple Inc. Automatic accent detection using acoustic models
US10269345B2 (en) 2016-06-11 2019-04-23 Apple Inc. Intelligent task discovery
US10276170B2 (en) 2010-01-18 2019-04-30 Apple Inc. Intelligent automated assistant
US10283110B2 (en) 2009-07-02 2019-05-07 Apple Inc. Methods and apparatuses for automatic speech recognition
US10297253B2 (en) 2016-06-11 2019-05-21 Apple Inc. Application integration with a digital assistant
US10318871B2 (en) 2005-09-08 2019-06-11 Apple Inc. Method and apparatus for building an intelligent automated assistant
CN109885226A (en) * 2019-02-21 2019-06-14 山东浪潮云信息技术有限公司 A method of secondary navigation menu is realized based on Vue.js
US10354011B2 (en) 2016-06-09 2019-07-16 Apple Inc. Intelligent automated assistant in a home environment
US10356243B2 (en) 2015-06-05 2019-07-16 Apple Inc. Virtual assistant aided communication with 3rd party service in a communication session
US10366158B2 (en) 2015-09-29 2019-07-30 Apple Inc. Efficient word encoding for recurrent neural network language models
US10410637B2 (en) 2017-05-12 2019-09-10 Apple Inc. User-specific acoustic models
US10446141B2 (en) 2014-08-28 2019-10-15 Apple Inc. Automatic speech recognition based on user feedback
US10446143B2 (en) 2016-03-14 2019-10-15 Apple Inc. Identification of voice inputs providing credentials
US10482874B2 (en) 2017-05-15 2019-11-19 Apple Inc. Hierarchical belief states for digital assistants
US10490187B2 (en) 2016-06-10 2019-11-26 Apple Inc. Digital assistant providing automated status report
US10496753B2 (en) 2010-01-18 2019-12-03 Apple Inc. Automatically adapting user interfaces for hands-free interaction
US10509862B2 (en) 2016-06-10 2019-12-17 Apple Inc. Dynamic phrase expansion of language input
US10521466B2 (en) 2016-06-11 2019-12-31 Apple Inc. Data driven natural language event detection and classification
US10552013B2 (en) 2014-12-02 2020-02-04 Apple Inc. Data detection
US10553209B2 (en) 2010-01-18 2020-02-04 Apple Inc. Systems and methods for hands-free notification summaries
US10568032B2 (en) 2007-04-03 2020-02-18 Apple Inc. Method and system for operating a multi-function portable electronic device using voice-activation
US10567477B2 (en) 2015-03-08 2020-02-18 Apple Inc. Virtual assistant continuity
US10593346B2 (en) 2016-12-22 2020-03-17 Apple Inc. Rank-reduced token representation for automatic speech recognition
US10607141B2 (en) 2010-01-25 2020-03-31 Newvaluexchange Ltd. Apparatuses, methods and systems for a digital conversation management platform
US10659851B2 (en) 2014-06-30 2020-05-19 Apple Inc. Real-time digital assistant knowledge updates
US10671428B2 (en) 2015-09-08 2020-06-02 Apple Inc. Distributed personal assistant
US10679605B2 (en) 2010-01-18 2020-06-09 Apple Inc. Hands-free list-reading by intelligent automated assistant
US10691473B2 (en) 2015-11-06 2020-06-23 Apple Inc. Intelligent automated assistant in a messaging environment
US10706373B2 (en) 2011-06-03 2020-07-07 Apple Inc. Performing actions associated with task items that represent tasks to perform
US10705794B2 (en) 2010-01-18 2020-07-07 Apple Inc. Automatically adapting user interfaces for hands-free interaction
US10733993B2 (en) 2016-06-10 2020-08-04 Apple Inc. Intelligent digital assistant in a multi-tasking environment
US10747498B2 (en) 2015-09-08 2020-08-18 Apple Inc. Zero latency digital assistant
US10755703B2 (en) 2017-05-11 2020-08-25 Apple Inc. Offline personal assistant
US10791176B2 (en) 2017-05-12 2020-09-29 Apple Inc. Synchronization and task delegation of a digital assistant
US10810274B2 (en) 2017-05-15 2020-10-20 Apple Inc. Optimizing dialogue policy decisions for digital assistants using implicit feedback
US11010550B2 (en) 2015-09-29 2021-05-18 Apple Inc. Unified language modeling framework for word prediction, auto-completion and auto-correction
US11025565B2 (en) 2015-06-07 2021-06-01 Apple Inc. Personalized prediction of responses for instant messaging
US11217255B2 (en) 2017-05-16 2022-01-04 Apple Inc. Far-field extension for digital assistant services
US11587559B2 (en) 2015-09-30 2023-02-21 Apple Inc. Intelligent device identification

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0697780A2 (en) * 1994-08-19 1996-02-21 International Business Machines Corporation Voice response system
EP0973314A2 (en) * 1998-07-17 2000-01-19 Siemens Information and Communication Networks Inc. Apparatus and method for improving the user interface of integrated voice response systems
US6188983B1 (en) * 1998-09-02 2001-02-13 International Business Machines Corp. Method for dynamically altering text-to-speech (TTS) attributes of a TTS engine not inherently capable of dynamic attribute alteration
WO2001069917A2 (en) * 2000-03-10 2001-09-20 Ods Properties, Inc. Interactive wagering systems with streamlined telephone-based wagering

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0697780A2 (en) * 1994-08-19 1996-02-21 International Business Machines Corporation Voice response system
EP0973314A2 (en) * 1998-07-17 2000-01-19 Siemens Information and Communication Networks Inc. Apparatus and method for improving the user interface of integrated voice response systems
US6188983B1 (en) * 1998-09-02 2001-02-13 International Business Machines Corp. Method for dynamically altering text-to-speech (TTS) attributes of a TTS engine not inherently capable of dynamic attribute alteration
WO2001069917A2 (en) * 2000-03-10 2001-09-20 Ods Properties, Inc. Interactive wagering systems with streamlined telephone-based wagering

Cited By (136)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9646614B2 (en) 2000-03-16 2017-05-09 Apple Inc. Fast, language-independent method for user authentication by voice
US8952887B1 (en) 2001-06-20 2015-02-10 Leapfrog Enterprises, Inc. Interactive references to related application
EP1665222A4 (en) * 2004-06-03 2008-01-16 Leapfrog Entpr Inc User created interactive interface
EP1665222A2 (en) * 2004-06-03 2006-06-07 Leapfrog Enterprises, Inc., Pepi Ross, Director of Legal Affairs User created interactive interface
CN100351763C (en) * 2004-09-24 2007-11-28 三星电子株式会社 Method and apparatus for providing a user control interface in audio multistreaming
US10318871B2 (en) 2005-09-08 2019-06-11 Apple Inc. Method and apparatus for building an intelligent automated assistant
US10568032B2 (en) 2007-04-03 2020-02-18 Apple Inc. Method and system for operating a multi-function portable electronic device using voice-activation
US10381016B2 (en) 2008-01-03 2019-08-13 Apple Inc. Methods and apparatus for altering audio output signals
US9330720B2 (en) 2008-01-03 2016-05-03 Apple Inc. Methods and apparatus for altering audio output signals
US9626955B2 (en) 2008-04-05 2017-04-18 Apple Inc. Intelligent text-to-speech conversion
US9865248B2 (en) 2008-04-05 2018-01-09 Apple Inc. Intelligent text-to-speech conversion
US10108612B2 (en) 2008-07-31 2018-10-23 Apple Inc. Mobile device having human language translation capability with positional feedback
US9535906B2 (en) 2008-07-31 2017-01-03 Apple Inc. Mobile device having human language translation capability with positional feedback
US8862252B2 (en) 2009-01-30 2014-10-14 Apple Inc. Audio user interface for displayless electronic device
GB2467432A (en) * 2009-01-30 2010-08-04 Apple Inc Audio user interface with single sensing element for displayless electronic device
US9858925B2 (en) 2009-06-05 2018-01-02 Apple Inc. Using context information to facilitate processing of commands in a virtual assistant
US10475446B2 (en) 2009-06-05 2019-11-12 Apple Inc. Using context information to facilitate processing of commands in a virtual assistant
US10795541B2 (en) 2009-06-05 2020-10-06 Apple Inc. Intelligent organization of tasks items
US11080012B2 (en) 2009-06-05 2021-08-03 Apple Inc. Interface for a virtual digital assistant
US10283110B2 (en) 2009-07-02 2019-05-07 Apple Inc. Methods and apparatuses for automatic speech recognition
US10705794B2 (en) 2010-01-18 2020-07-07 Apple Inc. Automatically adapting user interfaces for hands-free interaction
US10553209B2 (en) 2010-01-18 2020-02-04 Apple Inc. Systems and methods for hands-free notification summaries
US9318108B2 (en) 2010-01-18 2016-04-19 Apple Inc. Intelligent automated assistant
US10276170B2 (en) 2010-01-18 2019-04-30 Apple Inc. Intelligent automated assistant
US11423886B2 (en) 2010-01-18 2022-08-23 Apple Inc. Task flow identification based on user intent
US10496753B2 (en) 2010-01-18 2019-12-03 Apple Inc. Automatically adapting user interfaces for hands-free interaction
US9548050B2 (en) 2010-01-18 2017-01-17 Apple Inc. Intelligent automated assistant
US10679605B2 (en) 2010-01-18 2020-06-09 Apple Inc. Hands-free list-reading by intelligent automated assistant
US10706841B2 (en) 2010-01-18 2020-07-07 Apple Inc. Task flow identification based on user intent
US10607140B2 (en) 2010-01-25 2020-03-31 Newvaluexchange Ltd. Apparatuses, methods and systems for a digital conversation management platform
US11410053B2 (en) 2010-01-25 2022-08-09 Newvaluexchange Ltd. Apparatuses, methods and systems for a digital conversation management platform
US10984326B2 (en) 2010-01-25 2021-04-20 Newvaluexchange Ltd. Apparatuses, methods and systems for a digital conversation management platform
US10984327B2 (en) 2010-01-25 2021-04-20 New Valuexchange Ltd. Apparatuses, methods and systems for a digital conversation management platform
US10607141B2 (en) 2010-01-25 2020-03-31 Newvaluexchange Ltd. Apparatuses, methods and systems for a digital conversation management platform
US10049675B2 (en) 2010-02-25 2018-08-14 Apple Inc. User profiling for voice input processing
US9633660B2 (en) 2010-02-25 2017-04-25 Apple Inc. User profiling for voice input processing
US9262612B2 (en) 2011-03-21 2016-02-16 Apple Inc. Device access using voice authentication
US10102359B2 (en) 2011-03-21 2018-10-16 Apple Inc. Device access using voice authentication
US11120372B2 (en) 2011-06-03 2021-09-14 Apple Inc. Performing actions associated with task items that represent tasks to perform
US10706373B2 (en) 2011-06-03 2020-07-07 Apple Inc. Performing actions associated with task items that represent tasks to perform
US10057736B2 (en) 2011-06-03 2018-08-21 Apple Inc. Active transport based notifications
US10241644B2 (en) 2011-06-03 2019-03-26 Apple Inc. Actionable reminder entries
US9798393B2 (en) 2011-08-29 2017-10-24 Apple Inc. Text correction processing
US10241752B2 (en) 2011-09-30 2019-03-26 Apple Inc. Interface for a virtual digital assistant
US9483461B2 (en) 2012-03-06 2016-11-01 Apple Inc. Handling speech synthesis of content for multiple languages
US9953088B2 (en) 2012-05-14 2018-04-24 Apple Inc. Crowd sourcing information to fulfill user requests
US10079014B2 (en) 2012-06-08 2018-09-18 Apple Inc. Name recognition system
US9495129B2 (en) 2012-06-29 2016-11-15 Apple Inc. Device, method, and user interface for voice-activated navigation and browsing of a document
US9971774B2 (en) 2012-09-19 2018-05-15 Apple Inc. Voice-based media searching
US9582608B2 (en) 2013-06-07 2017-02-28 Apple Inc. Unified ranking with entropy-weighted information for phrase-based semantic auto-completion
US9620104B2 (en) 2013-06-07 2017-04-11 Apple Inc. System and method for user-specified pronunciation of words for speech synthesis and recognition
US9966060B2 (en) 2013-06-07 2018-05-08 Apple Inc. System and method for user-specified pronunciation of words for speech synthesis and recognition
US9633674B2 (en) 2013-06-07 2017-04-25 Apple Inc. System and method for detecting errors in interactions with a voice-based digital assistant
US10657961B2 (en) 2013-06-08 2020-05-19 Apple Inc. Interpreting and acting upon commands that involve sharing information with remote devices
US9966068B2 (en) 2013-06-08 2018-05-08 Apple Inc. Interpreting and acting upon commands that involve sharing information with remote devices
US10176167B2 (en) 2013-06-09 2019-01-08 Apple Inc. System and method for inferring user intent from speech inputs
US10185542B2 (en) 2013-06-09 2019-01-22 Apple Inc. Device, method, and graphical user interface for enabling conversation persistence across two or more instances of a digital assistant
US10169329B2 (en) 2014-05-30 2019-01-01 Apple Inc. Exemplar-based natural language processing
US9785630B2 (en) 2014-05-30 2017-10-10 Apple Inc. Text prediction using combined word N-gram and unigram language models
US9842101B2 (en) 2014-05-30 2017-12-12 Apple Inc. Predictive conversion of language input
US9760559B2 (en) 2014-05-30 2017-09-12 Apple Inc. Predictive text input
US10078631B2 (en) 2014-05-30 2018-09-18 Apple Inc. Entropy-guided text prediction using combined word and character n-gram language models
US10497365B2 (en) 2014-05-30 2019-12-03 Apple Inc. Multi-command single utterance input method
US11133008B2 (en) 2014-05-30 2021-09-28 Apple Inc. Reducing the need for manual start/end-pointing and trigger phrases
US9715875B2 (en) 2014-05-30 2017-07-25 Apple Inc. Reducing the need for manual start/end-pointing and trigger phrases
US9966065B2 (en) 2014-05-30 2018-05-08 Apple Inc. Multi-command single utterance input method
US10659851B2 (en) 2014-06-30 2020-05-19 Apple Inc. Real-time digital assistant knowledge updates
US9668024B2 (en) 2014-06-30 2017-05-30 Apple Inc. Intelligent automated assistant for TV user interactions
US9338493B2 (en) 2014-06-30 2016-05-10 Apple Inc. Intelligent automated assistant for TV user interactions
US10904611B2 (en) 2014-06-30 2021-01-26 Apple Inc. Intelligent automated assistant for TV user interactions
US10446141B2 (en) 2014-08-28 2019-10-15 Apple Inc. Automatic speech recognition based on user feedback
US10431204B2 (en) 2014-09-11 2019-10-01 Apple Inc. Method and apparatus for discovering trending terms in speech requests
US9818400B2 (en) 2014-09-11 2017-11-14 Apple Inc. Method and apparatus for discovering trending terms in speech requests
US9646609B2 (en) 2014-09-30 2017-05-09 Apple Inc. Caching apparatus for serving phonetic pronunciations
US9986419B2 (en) 2014-09-30 2018-05-29 Apple Inc. Social reminders
US9668121B2 (en) 2014-09-30 2017-05-30 Apple Inc. Social reminders
US10074360B2 (en) 2014-09-30 2018-09-11 Apple Inc. Providing an indication of the suitability of speech recognition
US10127911B2 (en) 2014-09-30 2018-11-13 Apple Inc. Speaker identification and unsupervised speaker adaptation techniques
US9886432B2 (en) 2014-09-30 2018-02-06 Apple Inc. Parsimonious handling of word inflection via categorical stem + suffix N-gram language models
US10552013B2 (en) 2014-12-02 2020-02-04 Apple Inc. Data detection
US11556230B2 (en) 2014-12-02 2023-01-17 Apple Inc. Data detection
US9865280B2 (en) 2015-03-06 2018-01-09 Apple Inc. Structured dictation using intelligent automated assistants
US9721566B2 (en) 2015-03-08 2017-08-01 Apple Inc. Competing devices responding to voice triggers
US11087759B2 (en) 2015-03-08 2021-08-10 Apple Inc. Virtual assistant activation
US10311871B2 (en) 2015-03-08 2019-06-04 Apple Inc. Competing devices responding to voice triggers
US10567477B2 (en) 2015-03-08 2020-02-18 Apple Inc. Virtual assistant continuity
US9886953B2 (en) 2015-03-08 2018-02-06 Apple Inc. Virtual assistant activation
US9899019B2 (en) 2015-03-18 2018-02-20 Apple Inc. Systems and methods for structured stem and suffix language models
US9842105B2 (en) 2015-04-16 2017-12-12 Apple Inc. Parsimonious continuous-space phrase representations for natural language processing
US10083688B2 (en) 2015-05-27 2018-09-25 Apple Inc. Device voice control for selecting a displayed affordance
US10127220B2 (en) 2015-06-04 2018-11-13 Apple Inc. Language identification from short strings
US10356243B2 (en) 2015-06-05 2019-07-16 Apple Inc. Virtual assistant aided communication with 3rd party service in a communication session
US10101822B2 (en) 2015-06-05 2018-10-16 Apple Inc. Language input correction
US10255907B2 (en) 2015-06-07 2019-04-09 Apple Inc. Automatic accent detection using acoustic models
US10186254B2 (en) 2015-06-07 2019-01-22 Apple Inc. Context-based endpoint detection
US11025565B2 (en) 2015-06-07 2021-06-01 Apple Inc. Personalized prediction of responses for instant messaging
US11500672B2 (en) 2015-09-08 2022-11-15 Apple Inc. Distributed personal assistant
US10747498B2 (en) 2015-09-08 2020-08-18 Apple Inc. Zero latency digital assistant
US10671428B2 (en) 2015-09-08 2020-06-02 Apple Inc. Distributed personal assistant
US9697820B2 (en) 2015-09-24 2017-07-04 Apple Inc. Unit-selection text-to-speech synthesis using concatenation-sensitive neural networks
US10366158B2 (en) 2015-09-29 2019-07-30 Apple Inc. Efficient word encoding for recurrent neural network language models
US11010550B2 (en) 2015-09-29 2021-05-18 Apple Inc. Unified language modeling framework for word prediction, auto-completion and auto-correction
US11587559B2 (en) 2015-09-30 2023-02-21 Apple Inc. Intelligent device identification
US10691473B2 (en) 2015-11-06 2020-06-23 Apple Inc. Intelligent automated assistant in a messaging environment
US11526368B2 (en) 2015-11-06 2022-12-13 Apple Inc. Intelligent automated assistant in a messaging environment
US10049668B2 (en) 2015-12-02 2018-08-14 Apple Inc. Applying neural network language models to weighted finite state transducers for automatic speech recognition
US10223066B2 (en) 2015-12-23 2019-03-05 Apple Inc. Proactive assistance based on dialog communication between devices
US10446143B2 (en) 2016-03-14 2019-10-15 Apple Inc. Identification of voice inputs providing credentials
US9934775B2 (en) 2016-05-26 2018-04-03 Apple Inc. Unit-selection text-to-speech synthesis based on predicted concatenation parameters
US9972304B2 (en) 2016-06-03 2018-05-15 Apple Inc. Privacy preserving distributed evaluation framework for embedded personalized systems
US10249300B2 (en) 2016-06-06 2019-04-02 Apple Inc. Intelligent list reading
US10049663B2 (en) 2016-06-08 2018-08-14 Apple, Inc. Intelligent automated assistant for media exploration
US11069347B2 (en) 2016-06-08 2021-07-20 Apple Inc. Intelligent automated assistant for media exploration
US10354011B2 (en) 2016-06-09 2019-07-16 Apple Inc. Intelligent automated assistant in a home environment
US10509862B2 (en) 2016-06-10 2019-12-17 Apple Inc. Dynamic phrase expansion of language input
US11037565B2 (en) 2016-06-10 2021-06-15 Apple Inc. Intelligent digital assistant in a multi-tasking environment
US10733993B2 (en) 2016-06-10 2020-08-04 Apple Inc. Intelligent digital assistant in a multi-tasking environment
US10067938B2 (en) 2016-06-10 2018-09-04 Apple Inc. Multilingual word prediction
US10192552B2 (en) 2016-06-10 2019-01-29 Apple Inc. Digital assistant providing whispered speech
US10490187B2 (en) 2016-06-10 2019-11-26 Apple Inc. Digital assistant providing automated status report
US10521466B2 (en) 2016-06-11 2019-12-31 Apple Inc. Data driven natural language event detection and classification
US11152002B2 (en) 2016-06-11 2021-10-19 Apple Inc. Application integration with a digital assistant
US10089072B2 (en) 2016-06-11 2018-10-02 Apple Inc. Intelligent device arbitration and control
US10269345B2 (en) 2016-06-11 2019-04-23 Apple Inc. Intelligent task discovery
US10297253B2 (en) 2016-06-11 2019-05-21 Apple Inc. Application integration with a digital assistant
US10043516B2 (en) 2016-09-23 2018-08-07 Apple Inc. Intelligent automated assistant
US10553215B2 (en) 2016-09-23 2020-02-04 Apple Inc. Intelligent automated assistant
US10593346B2 (en) 2016-12-22 2020-03-17 Apple Inc. Rank-reduced token representation for automatic speech recognition
US10755703B2 (en) 2017-05-11 2020-08-25 Apple Inc. Offline personal assistant
US11405466B2 (en) 2017-05-12 2022-08-02 Apple Inc. Synchronization and task delegation of a digital assistant
US10410637B2 (en) 2017-05-12 2019-09-10 Apple Inc. User-specific acoustic models
US10791176B2 (en) 2017-05-12 2020-09-29 Apple Inc. Synchronization and task delegation of a digital assistant
US10482874B2 (en) 2017-05-15 2019-11-19 Apple Inc. Hierarchical belief states for digital assistants
US10810274B2 (en) 2017-05-15 2020-10-20 Apple Inc. Optimizing dialogue policy decisions for digital assistants using implicit feedback
US11217255B2 (en) 2017-05-16 2022-01-04 Apple Inc. Far-field extension for digital assistant services
CN109885226A (en) * 2019-02-21 2019-06-14 山东浪潮云信息技术有限公司 A method of secondary navigation menu is realized based on Vue.js

Also Published As

Publication number Publication date
JP2005524122A (en) 2005-08-11
AU2002251113A1 (en) 2003-11-17

Similar Documents

Publication Publication Date Title
WO2003094489A1 (en) Method and system for rapid navigation in aural user interface
EP2243088B1 (en) Methods and apparatus for implementing distributed multi-modal applications
US9070292B2 (en) Information providing system and vehicle-mounted apparatus
US6834265B2 (en) Method and apparatus for selective speech recognition
EP1113416B1 (en) User interface for text to speech conversion
US8996379B2 (en) Speech recognition text entry for software applications
US20110205149A1 (en) Multi-modal input system for a voice-based menu and content navigation service
KR100365860B1 (en) Method for transmitting message in mobile terminal
US20080221902A1 (en) Mobile browser environment speech processing facility
US20090030697A1 (en) Using contextual information for delivering results generated from a speech recognition facility using an unstructured language model
KR20040063170A (en) Ui with graphics-assisted voice control system
KR20070100837A (en) Methods and apparatus for automatically extending the voice vocabulary of mobile communications devices
JP2009509268A (en) Graphical user interface for electronics
WO2009088665A2 (en) Methods and apparatus for implementing distributed multi-modal applications
US6732078B1 (en) Audio control method and audio controlled device
WO2009146860A1 (en) User interface, device, computer readable medium and method for switching operating modes upon pressing a volume key
KR101335771B1 (en) Electronic Device With Touch Screen And Method Of Inputting Information Using Same
US20100022229A1 (en) Method for communicating, a related system for communicating and a related transforming part
KR100698194B1 (en) A mobile communication terminal, and a method for performing ttstext to speech function
KR101215369B1 (en) Method for selecting a menu and mobile terminal capable of implementing the same
JP4861102B2 (en) Web display control method and mobile communication terminal device
WO2005060595A2 (en) Mobile telephone with a speech interface
JP2000305749A (en) Information providing device by voice

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NO NZ OM PH PL PT RO RU SD SE SG SI SK SL TJ TM TN TR TT TZ UA UG US UZ VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
WWE Wipo information: entry into national phase

Ref document number: 2004502598

Country of ref document: JP

122 Ep: pct application non-entry in european phase