CN103890836A - Bluetooth or other wireless interface with power management for head mounted display - Google Patents

Bluetooth or other wireless interface with power management for head mounted display Download PDF

Info

Publication number
CN103890836A
CN103890836A CN201180050791.3A CN201180050791A CN103890836A CN 103890836 A CN103890836 A CN 103890836A CN 201180050791 A CN201180050791 A CN 201180050791A CN 103890836 A CN103890836 A CN 103890836A
Authority
CN
China
Prior art keywords
user
processor
input
devices according
state
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201180050791.3A
Other languages
Chinese (zh)
Other versions
CN103890836B (en
Inventor
J·J·雅各布森
C·帕金森
S·A·波姆博
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Kopin Corp
Original Assignee
Kopin Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Kopin Corp filed Critical Kopin Corp
Publication of CN103890836A publication Critical patent/CN103890836A/en
Application granted granted Critical
Publication of CN103890836B publication Critical patent/CN103890836B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/42203—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS] sound input device, e.g. microphone
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/26—Power supply means, e.g. regulation thereof
    • G06F1/32—Means for saving power
    • G06F1/3203—Power management, i.e. event-based initiation of a power-saving mode
    • G06F1/3206—Monitoring of events, devices or parameters that trigger a change in power modality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/26—Power supply means, e.g. regulation thereof
    • G06F1/32—Means for saving power
    • G06F1/3203—Power management, i.e. event-based initiation of a power-saving mode
    • G06F1/3234—Power saving characterised by the action undertaken
    • G06F1/325—Power saving in peripheral device
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/26—Power supply means, e.g. regulation thereof
    • G06F1/32—Means for saving power
    • G06F1/3203—Power management, i.e. event-based initiation of a power-saving mode
    • G06F1/3234—Power saving characterised by the action undertaken
    • G06F1/3287—Power saving characterised by the action undertaken by switching off individual functional units in the computer system
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012—Head tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/0304—Detection arrangements using opto-electronic means
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/4104—Peripherals receiving signals from specially adapted client devices
    • H04N21/4122—Peripherals receiving signals from specially adapted client devices additional display device, e.g. video projector
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/4223—Cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/439—Processing of audio elementary streams
    • H04N21/4394—Processing of audio elementary streams involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/443—OS processes, e.g. booting an STB, implementing a Java virtual machine in an STB or power management in an STB
    • H04N21/4436—Power management, e.g. shutting down unused components of the receiver
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0101—Head-up displays characterised by optical features
    • G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0179—Display position adjusting means not related to the information to be displayed
    • G02B2027/0187—Display position adjusting means not related to the information to be displayed slaved to motion of at least a part of the body of the user, e.g. head, eye
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

A Head Mounted Display (HMD) system that includes a wireless front end that interprets spoken commands and/or hand motions and/or body gestures to selectively activate subsystem components only as needed to carry out specific commands.

Description

For the bluetooth with power management or other wave points of head mounted display
the cross reference of related application
The application requires the U.S. Provisional Application number 61/384,419 submitted on September 20th, 2010 and the rights and interests of the U.S. Provisional Application submitted on September 20th, 2010 number 61/384,543.By reference whole instructions of above one or more application are combined in to this.
Technical field
Present disclosure relates to a kind of head mounted display (HMD) system or other video eyeglasses devices, and it has the wireless front end of the activation of controlling other system assembly.
Background technology
The miniscope of research and development can provide large format, high-resolution colour picture and video flowing in very little shape factor in the recent period.A purposes of this class display is head mounted display (HMD) device that is worn over user face or head, similar a pair of glasses or earphone.The electronic component being attached in such device has become very complicated, and can comprise now integrated data processor, wave point and other input equipments, as head tracking accelerometer, video camera, speech recognition circuit and software and other assemblies.
Summary of the invention
In preferred environment, head mounted display (HMD) system comprises high resolution micro display, (multiple) microphone and (multiple) loudspeaker, multiple satellite interface (as wave point), multiple radio network interface, multiple video camera (visible ray, infrared ray etc.), multiple audio frequency apparatus, multiple keyboard, multiple game console equipment and other peripheral hardwares.
Wave point (as bluetooth, WiFi, honeycomb or other wave points) becomes the front end of this system and controls other system assembly and when is activated.In the time that this HMD system switches on power for the first time, it can for example be similar to bluetooth earphone and work with minimum function, thereby the function of loudspeaker and microphone is only provided.Paired device (as the mobile phone of bluetooth compatibility (being Apple iPhone or Google Android smart phone)) can be for being fed to audio session and being fed audio session under hands-free mode.Under this pattern, other functions of this HMD system remain on park mode, until earphone identifies the verbal order that wakes other system assembly and function up.
For example, in the time that this blue tooth front end detects with specific audio-frequency assembly the data stream of importing into, this audio-frequency assembly is transferred to the speech identifying function that is presented as circuit or software.If it is that this front end will cause this embedded data processor to open any other peripherals, as video display to the verbal order of this system that needs video to export that this speech identifying function is determined this audio-frequency assembly.In the time no longer needing this video display, this processor and video driver circuit are put back to sleep pattern.
In another example, this user can say the order of more sophisticated functions that need to this plate borne processor.For example, this user can ask to read " pending " task list of today.This inventory can be by generating as the application program of the Microsoft Outlook moving on principal computer.Not only need to wake up this flush bonding processor and obtain this To Do List information to access local mass-memory unit thereby carry out this order, also need the word that generates the sound signal corresponding with this spoken word to sound circuit.
In another example, this user can ask to check as Microsoft Word file and be attached to the agenda file in web mail message.This may need to wake this flush bonding processor up, activates this local microenvironment escope, and connects with external network, for example, by bluetooth, WLAN (wireless local area network) (WLAN) circuit and/or cellular modem circuit.Then, these circuit will only be energized at that time.Then, this flush bonding processor and main frame (as personal computer) contact, thereby retrieval and display networks.
In order to support this function, this wireless front end has wherein comprised form or other logics, make it not only can carry out the function of voice to word, or convert posture to word, then can also explain these voice or posture order and seek advice from assembly that row need to switch on to carry out these oral or posture orders.
Accompanying drawing explanation
According to the following more specific description of example embodiment of the present invention, foregoing will be clearly, as shown in these accompanying drawings, wherein run through these different views, and identical Reference numeral refers to identical part.These figure not necessarily draw in proportion, but show embodiments of the invention emphatically.
Fig. 1 is head mounted display (HMD) system that comprises miniscope and other peripheral interfaces.
Fig. 2 is another skeleton view, show be worn over user's head installation the HMD system of video camera peripheral hardware.
Fig. 3 is high layer function figure, shows this HMD and conduct thereof as the purposes of the controller of the host-processor of personal computer.
Fig. 4 is the high-level block diagram of the assembly of this HMD component internal.
Fig. 5 is more detailed figure, shows the electronic package using in certain embodiments.
Fig. 6 is high-level flowchart, how works and controls other various HMD system components and when become an effective example as system controller thereby show this blue tooth front end.
Fig. 7 is the list of verbal order, posture order or headwork order and those parts that are activated of this subsystem.
Embodiment
The explanation to example embodiment below.
Fig. 1 shows head mounted display (HMD) system 100, and this system generally includes framework 1000, band 1002, back 1004, loudspeaker 1006, cantilever or arm 1008 and miniscope sub-component 1010.
HMD 100 can be the available Golden-i of Kopin Corporation of the Taunton based on from Massachusetts tMhands-free communication and control device.One side (a particularly side relative with miniscope sub-component 1010) of this device can comprise " hot shoe " peripheral port 1020, it can to accessory provide corresponding connection as what further explain below, so that user can delete the assembly of the various HMD of being attached to 100.
Fig. 2 is the view that is worn over the HMD 100 of user's head, and wherein video camera accessory 1060 has been placed in port one 020.Video camera 1060 can comprise Voice & Video sensing and two abilities of record, and can be for example in the little profile factor of similar " bullet cam ".As will be explained further below, this user provides user command (can be headwork or voice command) control the setting of display 1010, video camera 1020 and carry out other functions.
This HMD system can mainly as or independent personal computer or " smart phone " device and/or play a role as the Long-distance Control of host personal computer or other devices.For this purpose, being included in this housing is various electronic circuits, comprises the microcomputer as understood very soon, one or more wave point, the storer being associated or other memory storages, position, action and/or attitude sensor.These camera positions, action and/or attitude sensor are used to follow the tracks of this user's head or the position of hand, action and/or posture, and this microphone audio reception input command.
This wave point makes HMD system 100 can be used as smart phone itself or the Long-distance Control as Framework computing device 200 (referring to Fig. 3).For example, main frame 200 can be laptop computer, mobile phone, RIM Blackberry, Apple iPhone, Google Android or other smart phone equipment, has the computational complexity own similar or higher with Long-distance Control HMD system 100 and similar or larger network connectivty.Main frame 200 can further be connected to other networks, is connected to the Internet as connected 210 by wide area network (WAN).HMD system 100 and main frame 200 are by being connected to each other as the suitable wireless connections that provided by Bluetooth link or WLAN (wireless local area network) (WLAN) 150.
As shown in Figure 3, HMD system 100 receives the input from this user, as the phonetic entry via microphone, via the hand exercise of video camera or other optical sensors and/or via the headwork input of accelerometer or other headwork tracking circuits.By the software in system 100 (and/or with main frame 200 in software cooperation) order that these content translation apparatus for converting 100 and/or main frame 200 systems can be identified.If these contents are the orders for HMD system 100, it can carry them, and if they are the orders for main frame 200, they are passed bluetooth and/or WiFi interface 150 sends to main frame 200.Then, thus main frame 200 explains that according to its oneself operating system/application software these translated orders carry out various functions.
In these orders, may there is the order of obtaining with display file.
These orders may be to receive phone calls.
Other order may be that retrieval is from the information of internet and shown, as Email or webpage.
Other order may be in virtual monitor, to select the visual field, then returns to selected screen to this remote equipment.It is to be further understood that, the very large breadth of virtual viewing area may with in the operating system of operation on main frame 200 or the application software of interior operation be associated because offer the various voice commands of the display 1010 on HMD 100 and/or headwork and only selected a part for this large-scale virtual viewing area in this visual field.
About HMD 100 in response to phonetic entry being detected, action and/or posture and the more information of the function that can carry out, be that the title of submitting on May 5th, 2010 is " Long-distance Control (Remote Control of Host Application Using Motion and Voice Commands) of the host application program of usage operation and voice command " (sequence number 12/774,179, attorney 0717.2098-001) and be " by moving, the wireless hand-free formula with detachable accessory of body gesture or voice command control is calculated earphone (Wireless Hands-Free Computing Headset with Detachable Accessories Controlled by Motion, Body Gesture, or Vocal Commands) " (sequence number 13/018 in the title that on February 1st, 2011 submits to, 999, attorney 0717.2102-001) common unsettled U.S. Patent application in made explanation, wherein the full content of each file is combined in this by reference.
HMD system 100 can provide other functions to other accessories beyond video camera, as extra loudspeaker, extra video camera, light source and can put hot shoe 1020 into or self-powered battery in any other electronic package.
Fig. 4 is the high-level block diagram of the simplification of the non-limiting example embodiment of HMD system 100.These assemblies are integrated in the various places of HMD shell as shown in Figure 1.HMD system 100 can comprise a container (eye pod) assembly 4000, and this assembly comprises above-mentioned miniscope 4010 and one or more microphone 4020.One or more loudspeakers 4030 are placed near (referring to the item 1006 in Fig. 1) in the housing receiver of user's ear.System electronic element can be placed on the appropriate location (as back 1004) on this framework or in this framework, and comprises open media application platform (OMAP) processor 4110, power/audio companion chip 4102, display driver 4104, head-tracker 4105, circuit board 4106 and WLAN/blue tooth interface 4108.In this housing, be also provided with power supply, for example lithium ion battery 4200.
Head-tracker circuit 4106 can comprise the circuit that uses hall effect sensor, MIM diode, accelerometer, gyroscope and/or frequency converter to detect headwork and posture, as along or around transverse movement and the rotation posture of X-axis, Y-axis and Z axis.
HMD system 100 can also receive input from external input device (as wireless mouse, trace ball maybe can be passed through the keyboard of blue tooth interface 4108 wireless connections).Software in WLAN/BT front end 4108, OMAP 4100 and/or main frame can be for the gesture of explaining that this video camera detects.Video camera plate 4060 can optionally provide video input.
OMAP processor can comprise CPU (central processing unit), and may comprise the on-chip memory of nonvolatile memory and/or ROM (read-only memory) (ROM), such as random access memory (RAM).This OMAP can be by Texas Instruments, and Inc. sells and used processor or the renewal version of the Texas Instruments model OMAP 3530 of multimedia processor.This OMAP is normally more powerful than WLAN/BT interface 4108, the processor of power consumption more.
In this example, be also to provide audio frequency, USB, Keyboard Control and battery charging function from the available TPS 65950 power/audio companion chips of Texas Instruments to this system.
WLAN/BT interface 4108 can be the circuit of model LBEE 1W8NEC interface or the wireless module with the renewal of similar or stronger ability.
This display driver can be the display driver from the available model KCD-A 910 of Kopin Corporation of the Westborough of Massachusetts.
Also be can comprise model 113LV, 152LV, the acceptable miniscope of 230LV, WQVGA or other manufacturers from the available miniscope 4010 of Kopin.
NCS module 4400 is processed received microphone signal, speech identifying function to be provided and to produce sound signal to audio frequency companion chip 4102.
Fig. 5 is more detailed block scheme, shows the connectedness between the various assemblies of Fig. 4.
From then in figure, can more easily see, WLAN/BT front end 4108 can be used as UART interface.It is programmed to control OMAP processor 4100 and correspondingly energising or the off-position of audio frequency and power supply companion chip 4102, and is applied to the electric power of other peripheral functions (as video camera, display, WLAN modulator-demodular unit etc.).
Fig. 6 is the process flow diagram that illustrates several illustrative functions, and wherein WLAN/ blue tooth front end 4108 can be controlled to the energising of other system assembly.Fig. 7 is sample table, and this table can be kept determining when by front end 4108 power supply of the various devices of On/Off, and in the discussion of Fig. 6, carries out reference below.
In the first example, state 6000 times, system component may be at park mode at first, has only enabled the function (for example microphone 4020 and loudspeaker 4030) of bluetooth earphone.In the time that switching on power for the first time, HMD system 100 can enter this state 6000.Therefore, for example, at state 6100, call out if there is incoming call, these audio frequency input and output functions keep active (for example make microphone 4020 in opening and enable loudspeaker 4030) at state 6110.But at state 6120, OMAP 4100, display 4010, video camera 4060 and other assemblies remain on park mode.When end of calling, process the state 6000 that turns back to.
In another example, from dormant state 6000, this voicefrequency circuit can detect specific user voice command, as " reading my To Do List ".From then on state 6200, this OMAP is activated for 6210 times at state.Then, this OMAP can access local storage region (as SDRAM storer) at state 6220 and visit the file being associated with user's local task list, thereby obtains pending detailed task list on the same day.These To Do List data are returned to this audio interface at state 6240 (now this To Do List text conversion has become voice) from OMAP 4100, and then from the loudspeaker of earphone 100, are heard.Just get the hang of 6500 once complete this function, then now this system is returned to dormant state 6000, and now this system only plays a role as bluetooth earphone again, waits for another user command.
In another example, at state 6300, user says order and indicates her to wish to check the New York Times homepage on www.NYTimes.com.In the time receiving this order, this OMAP is activated at state 6310.WLAN interface is activated, and this OMAP can be given an order to this primary processor (described in conjunction with Fig. 3) thereby the HTML content of acquisition New York Times homepage.Also enable miniscope 1010 in 6330 a certain moment of getting the hang of, expected that this user wishes display web page.Can obtain by other modes this html web page of New York Times.com, if for example this earphone itself has the ability carried bluetooth, WLAN circuit (WiFi) or accessed internet by auxiliary cellular modem by plate.In any case, once obtain this content, then just at state 6350, it formatd and show on this miniscope.At a time, this user can send another and order to forbid this display (or this order may be overtime), again enters sleep mode 6000 in this moment.
Therefore, in the time that this bluetooth and/or WiFi front end detect with user voice command the audio stream importing into, this bluetooth and/or WiFi circuit wake this OMAP and other peripherals automatically up, but just carry out those actual necessary equipment of this order.Therefore, this bluetooth and/or WiFi front end, as the form reservation information with form or data base entries, indicate which peripherals and/or OMAP (if any) to need energising to carry out each in many different user orders.
In another example (referring to Fig. 7), this video camera can detect user's gesture, and provides input to control the energising of the specific components to system 100.Under this state, first enable these audio-frequency assemblies, but detected that verbal order is to read and conciliate the gesture of loosening one's grip by activating at least video camera 4060 and OMAP 4100.
In another example, this user can be head movement as input.Under this pattern, for example, in the time that this user is using head movement translation, convergent-divergent or otherwise navigating the virtual viewing area of large format, may only have headwork tracking 4106 and display is the associated component being activated.
Like this, the function of this HMD system is by suitable verbal order, posture order or from the order control of principal computer, and is explained by this bluetooth and/or WiFi interface, only this part of wakeup subsystem electronic component as required.When completing after this activity, another verbal order or overtimely this HMD system can be turned back to sleep pattern.Now, this bluetooth and/or WiFi circuit are got back to dormant state and are waited for external contact signal, as call, another voice command or by the computer interface order of WLAN.
The instruction of all patents of quoting at this, open application and list of references is all combined in this with its full content by reference.
Although shown particularly with reference to its example embodiment and described the present invention, but those of ordinary skills should be understood that, in the case of not departing from by the included scope of the present invention of claims, wherein can aspect form and details, carry out various changes.

Claims (19)

1. for a method for control head head mounted displays (HMD) system, described system comprises miniscope, user input device, first processor and two or more other peripheral circuits, and described method comprises:
Only enable described first processor and described user input device at the first state;
Detect user's input;
Use described processor that described user's input is interpreted as to verbal order or posture order; And
According to described verbal order or described posture order, enter the second state, wherein enable in two or more other peripheral hardwares selected peripheral hardware and forbid other peripheral hardwares.
2. method according to claim 1, wherein said two or more other peripheral hardwares comprise the second processor.
3. method according to claim 2, wherein said the second processor is than the more power consumption of described first processor.
4. method according to claim 1, wherein said two or more other peripheral hardwares are selected from lower group, and described group is made up of Wide Area Network interface, the second processor, display, video camera and headwork tracker.
5. method according to claim 2, wherein said user's input is the verbal order for access file, and applies electric power to described the second processor and described miniscope in described the second state, then shows described file.
6. method according to claim 5, wherein said user's input is the verbal order for accessed web page, and described the second state further makes wave point can be connected to network to obtain described webpage.
7. method according to claim 2, wherein said peripheral hardware is included in video camera and the second processor that described the first state is enabled, and described user input is gesture, and described the second processor is explained described gesture entering before described the second state.
8. method according to claim 2, wherein said peripheral hardware comprises wave point, and described the second state sends to main frame by described wave point by described user command.
9. method according to claim 7, wherein said user command is the request of an aspect for handling described display, and described in described the second status modifier, the information on display presents.
10. a Portable earphone device, comprising:
Miniscope, for showing visual information;
Two or more peripherals, provide one or more peripheral hardware input signals, at least one in described one or more peripheral hardware input signal indication sensor inputs or user action and/or Speech input;
First processor, for translating into one or more orders that detect by least one described peripheral hardware input signal of indication sensor input or user action and/or Speech input; And
Processor controller, for according to described one or more command selection that detect enable the one or more of described miniscope and described peripheral hardware.
11. devices according to claim 10, wherein said one or more peripherals comprises:
One or more microphones, for from described user's received audio signal, and
And described first processor further comprises:
Speech recognition device, for the treatment of sound signal with detect voice command.
12. devices according to claim 10, wherein said one or more peripherals is motion detector, and described motion detector provides two or more action inputs, described two or more action input indications are along the user action of two or more axles.
13. devices according to claim 12, wherein said motion detector is the video camera for detection of described user's hand and/or body gesture motion.
14. devices according to claim 12, wherein said motion detection is the head movement tracking equipment for detection of described user's head movement.
15. devices according to claim 1, wherein said peripheral hardware further comprises communication interface, described communication interface provides one or more wireless links between described Portable ear machine equipment and another data processing equipment.
16. devices according to claim 1, wherein said user command is processed to control the many aspects that present of visual information on described miniscope by described first processor.
17. devices according to claim 16, the wherein said user command control visual field.
18. devices according to claim 17, wherein said user command control zoom factor, shift factor or scale factor.
19. devices according to claim 16, wherein said user command is selected hyperlink item in web displaying.
CN201180050791.3A 2010-09-20 2011-09-15 The bluetooth with power management or other wave points for head mounted display Active CN103890836B (en)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US38441910P 2010-09-20 2010-09-20
US38454310P 2010-09-20 2010-09-20
US61/384,419 2010-09-20
US61/384,543 2010-09-20
PCT/US2011/051760 WO2012040030A2 (en) 2010-09-20 2011-09-15 Bluetooth or other wireless interface with power management for head mounted display

Publications (2)

Publication Number Publication Date
CN103890836A true CN103890836A (en) 2014-06-25
CN103890836B CN103890836B (en) 2017-03-15

Family

ID=45874290

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201180050791.3A Active CN103890836B (en) 2010-09-20 2011-09-15 The bluetooth with power management or other wave points for head mounted display

Country Status (4)

Country Link
EP (1) EP2617202A4 (en)
JP (1) JP5974008B2 (en)
CN (1) CN103890836B (en)
WO (1) WO2012040030A2 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104615243A (en) * 2015-01-15 2015-05-13 深圳市掌网立体时代视讯技术有限公司 Head-wearable type multi-channel interaction system and multi-channel interaction method
CN105797366A (en) * 2016-03-25 2016-07-27 中国传媒大学 Head-wearing type interactive audio game terminal based on sound source location
CN105828230A (en) * 2015-01-25 2016-08-03 哈曼国际工业有限公司 Headphones with integral image display
CN109074048A (en) * 2016-04-26 2018-12-21 克朗斯股份公司 The operating system of machine for food industry

Families Citing this family (60)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9952664B2 (en) 2014-01-21 2018-04-24 Osterhout Group, Inc. Eye imaging in head worn computing
US20150205111A1 (en) 2014-01-21 2015-07-23 Osterhout Group, Inc. Optical configurations for head worn computing
US9229233B2 (en) 2014-02-11 2016-01-05 Osterhout Group, Inc. Micro Doppler presentations in head worn computing
US9965681B2 (en) 2008-12-16 2018-05-08 Osterhout Group, Inc. Eye imaging in head worn computing
US9715112B2 (en) 2014-01-21 2017-07-25 Osterhout Group, Inc. Suppression of stray light in head worn computing
US9400390B2 (en) 2014-01-24 2016-07-26 Osterhout Group, Inc. Peripheral lighting for head worn computing
US9298007B2 (en) 2014-01-21 2016-03-29 Osterhout Group, Inc. Eye imaging in head worn computing
US9851787B2 (en) 2012-11-29 2017-12-26 Microsoft Technology Licensing, Llc Display resource management
US20140282273A1 (en) * 2013-03-15 2014-09-18 Glen J. Anderson System and method for assigning voice and gesture command areas
US9299194B2 (en) 2014-02-14 2016-03-29 Osterhout Group, Inc. Secure sharing in head worn computing
US10254856B2 (en) 2014-01-17 2019-04-09 Osterhout Group, Inc. External user interface for head worn computing
US20160019715A1 (en) 2014-07-15 2016-01-21 Osterhout Group, Inc. Content presentation in head worn computing
US11103122B2 (en) 2014-07-15 2021-08-31 Mentor Acquisition One, Llc Content presentation in head worn computing
US9671613B2 (en) 2014-09-26 2017-06-06 Osterhout Group, Inc. See-through computer display systems
US9829707B2 (en) 2014-08-12 2017-11-28 Osterhout Group, Inc. Measuring content brightness in head worn computing
US9594246B2 (en) 2014-01-21 2017-03-14 Osterhout Group, Inc. See-through computer display systems
US10191279B2 (en) 2014-03-17 2019-01-29 Osterhout Group, Inc. Eye imaging in head worn computing
US9575321B2 (en) 2014-06-09 2017-02-21 Osterhout Group, Inc. Content presentation in head worn computing
US9810906B2 (en) 2014-06-17 2017-11-07 Osterhout Group, Inc. External user interface for head worn computing
US11227294B2 (en) 2014-04-03 2022-01-18 Mentor Acquisition One, Llc Sight information collection in head worn computing
US9939934B2 (en) 2014-01-17 2018-04-10 Osterhout Group, Inc. External user interface for head worn computing
US9841599B2 (en) 2014-06-05 2017-12-12 Osterhout Group, Inc. Optical configurations for head-worn see-through displays
US9746686B2 (en) 2014-05-19 2017-08-29 Osterhout Group, Inc. Content position calibration in head worn computing
US9448409B2 (en) 2014-11-26 2016-09-20 Osterhout Group, Inc. See-through computer display systems
US10649220B2 (en) 2014-06-09 2020-05-12 Mentor Acquisition One, Llc Content presentation in head worn computing
US20150277118A1 (en) 2014-03-28 2015-10-01 Osterhout Group, Inc. Sensor dependent content position in head worn computing
US10684687B2 (en) 2014-12-03 2020-06-16 Mentor Acquisition One, Llc See-through computer display systems
US9529195B2 (en) 2014-01-21 2016-12-27 Osterhout Group, Inc. See-through computer display systems
US9494800B2 (en) 2014-01-21 2016-11-15 Osterhout Group, Inc. See-through computer display systems
US11487110B2 (en) 2014-01-21 2022-11-01 Mentor Acquisition One, Llc Eye imaging in head worn computing
US9532715B2 (en) 2014-01-21 2017-01-03 Osterhout Group, Inc. Eye imaging in head worn computing
US9651784B2 (en) 2014-01-21 2017-05-16 Osterhout Group, Inc. See-through computer display systems
US9740280B2 (en) 2014-01-21 2017-08-22 Osterhout Group, Inc. Eye imaging in head worn computing
US11892644B2 (en) 2014-01-21 2024-02-06 Mentor Acquisition One, Llc See-through computer display systems
US9836122B2 (en) 2014-01-21 2017-12-05 Osterhout Group, Inc. Eye glint imaging in see-through computer display systems
US9766463B2 (en) 2014-01-21 2017-09-19 Osterhout Group, Inc. See-through computer display systems
US20150205135A1 (en) 2014-01-21 2015-07-23 Osterhout Group, Inc. See-through computer display systems
US9651788B2 (en) 2014-01-21 2017-05-16 Osterhout Group, Inc. See-through computer display systems
US9753288B2 (en) 2014-01-21 2017-09-05 Osterhout Group, Inc. See-through computer display systems
US11669163B2 (en) 2014-01-21 2023-06-06 Mentor Acquisition One, Llc Eye glint imaging in see-through computer display systems
US11737666B2 (en) 2014-01-21 2023-08-29 Mentor Acquisition One, Llc Eye imaging in head worn computing
US9846308B2 (en) 2014-01-24 2017-12-19 Osterhout Group, Inc. Haptic systems for head-worn computers
US20150241964A1 (en) 2014-02-11 2015-08-27 Osterhout Group, Inc. Eye imaging in head worn computing
US9401540B2 (en) 2014-02-11 2016-07-26 Osterhout Group, Inc. Spatial location presentation in head worn computing
US20160187651A1 (en) 2014-03-28 2016-06-30 Osterhout Group, Inc. Safety for a vehicle operator with an hmd
US9672210B2 (en) 2014-04-25 2017-06-06 Osterhout Group, Inc. Language translation with head-worn computing
US10853589B2 (en) 2014-04-25 2020-12-01 Mentor Acquisition One, Llc Language translation with head-worn computing
US9423842B2 (en) 2014-09-18 2016-08-23 Osterhout Group, Inc. Thermal management for head-worn computer
US9651787B2 (en) 2014-04-25 2017-05-16 Osterhout Group, Inc. Speaker assembly for headworn computer
WO2015179877A2 (en) * 2014-05-19 2015-11-26 Osterhout Group, Inc. External user interface for head worn computing
US10663740B2 (en) 2014-06-09 2020-05-26 Mentor Acquisition One, Llc Content presentation in head worn computing
US9684172B2 (en) 2014-12-03 2017-06-20 Osterhout Group, Inc. Head worn computer display systems
JP6516464B2 (en) * 2014-12-15 2019-05-22 Kddi株式会社 Wearable search system
USD751552S1 (en) 2014-12-31 2016-03-15 Osterhout Group, Inc. Computer glasses
USD753114S1 (en) 2015-01-05 2016-04-05 Osterhout Group, Inc. Air mouse
US20160239985A1 (en) 2015-02-17 2016-08-18 Osterhout Group, Inc. See-through computer display systems
EP3104252B1 (en) * 2015-06-09 2020-11-11 Nokia Technologies Oy Transition of a head mounted display from operation in a first power mode to operation in a second power mode
US9298283B1 (en) 2015-09-10 2016-03-29 Connectivity Labs Inc. Sedentary virtual reality method and systems
TWI599966B (en) * 2016-05-10 2017-09-21 H P B Optoelectronic Co Ltd Gesture control modular system
USD864959S1 (en) 2017-01-04 2019-10-29 Mentor Acquisition One, Llc Computer glasses

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040104864A1 (en) * 2002-11-28 2004-06-03 Nec Corporation Glasses type display and controlling method thereof
WO2009054619A2 (en) * 2007-10-22 2009-04-30 Moon Key Lee Augmented reality computer device
CN101506868A (en) * 2006-09-08 2009-08-12 索尼株式会社 Display device and display method
US20100079356A1 (en) * 2008-09-30 2010-04-01 Apple Inc. Head-mounted display apparatus for retaining a portable electronic device with display

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH10312631A (en) * 1997-05-14 1998-11-24 Nec Niigata Ltd Sound reproducing device and information processing device
US6408257B1 (en) * 1999-08-31 2002-06-18 Xerox Corporation Augmented-reality display method and system
EP2276020A1 (en) * 1999-11-09 2011-01-19 Nuance Communications Austria GmbH Speech recognition method for activating a hyperlink of an internet page
US7899915B2 (en) * 2002-05-10 2011-03-01 Richard Reisman Method and apparatus for browsing using multiple coordinated device sets
JP2004349915A (en) * 2003-05-21 2004-12-09 Matsushita Electric Ind Co Ltd Remote control unit
KR100594117B1 (en) * 2004-09-20 2006-06-28 삼성전자주식회사 Apparatus and method for inputting key using biosignal in HMD information terminal
JP4364861B2 (en) * 2005-11-01 2009-11-18 株式会社ソニー・コンピュータエンタテインメント Information display device
JP2008054085A (en) * 2006-08-25 2008-03-06 Hitachi Ltd Broadcast receiving apparatus and starting method thereof
US8386259B2 (en) * 2006-12-28 2013-02-26 Intel Corporation Voice interface to NFC applications
US8378924B2 (en) * 2007-01-12 2013-02-19 Kopin Corporation Monocular display device
WO2008143870A1 (en) * 2007-05-14 2008-11-27 Kopin Corporation Mobile wireless display for accessing data from a host and method for controlling
WO2008144638A2 (en) * 2007-05-17 2008-11-27 Redstart Systems Inc. Systems and methods of a structured grammar for a speech recognition command system
IL183637A (en) * 2007-06-04 2013-06-27 Zvi Lapidot Distributed head-mounted display
JP5217386B2 (en) * 2007-11-21 2013-06-19 株式会社ニコン Control device, head mounted display device, program, and control method
US8416152B2 (en) 2008-06-11 2013-04-09 Honeywell International Inc. Method and system for operating a near-to-eye display

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040104864A1 (en) * 2002-11-28 2004-06-03 Nec Corporation Glasses type display and controlling method thereof
CN101506868A (en) * 2006-09-08 2009-08-12 索尼株式会社 Display device and display method
WO2009054619A2 (en) * 2007-10-22 2009-04-30 Moon Key Lee Augmented reality computer device
US20100079356A1 (en) * 2008-09-30 2010-04-01 Apple Inc. Head-mounted display apparatus for retaining a portable electronic device with display

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104615243A (en) * 2015-01-15 2015-05-13 深圳市掌网立体时代视讯技术有限公司 Head-wearable type multi-channel interaction system and multi-channel interaction method
CN105828230A (en) * 2015-01-25 2016-08-03 哈曼国际工业有限公司 Headphones with integral image display
CN105828230B (en) * 2015-01-25 2020-02-14 哈曼国际工业有限公司 Headphones with integrated image display
CN105797366A (en) * 2016-03-25 2016-07-27 中国传媒大学 Head-wearing type interactive audio game terminal based on sound source location
CN109074048A (en) * 2016-04-26 2018-12-21 克朗斯股份公司 The operating system of machine for food industry

Also Published As

Publication number Publication date
JP5974008B2 (en) 2016-08-23
EP2617202A2 (en) 2013-07-24
CN103890836B (en) 2017-03-15
JP2014503085A (en) 2014-02-06
WO2012040030A2 (en) 2012-03-29
EP2617202A4 (en) 2015-01-21
WO2012040030A3 (en) 2014-03-27

Similar Documents

Publication Publication Date Title
CN103890836A (en) Bluetooth or other wireless interface with power management for head mounted display
US8736516B2 (en) Bluetooth or other wireless interface with power management for head mounted display
KR102411124B1 (en) Electronic device and method for performing task using external electronic device in electronic device
US9122307B2 (en) Advanced remote control of host application using motion and voice commands
US8862186B2 (en) Lapel microphone micro-display system incorporating mobile information access system
CN102438063B (en) Mobile terminal and method of controlling operation of the mobile terminal
EP3411780B1 (en) Intelligent electronic device and method of operating the same
CN104049745A (en) Input control method and electronic device supporting the same
KR20110136587A (en) Mobile terminal and operation method thereof
CN104508591A (en) Headset computer with docking station
CN106293076A (en) Communication terminal and intelligent terminal's gesture identification method and device
US20150052375A1 (en) Information processing method and electronic device
KR102546510B1 (en) Method for providing information mapped between plurality inputs and electronic device supporting the same
CN110827820B (en) Voice awakening method, device, equipment, computer storage medium and vehicle
CN108694032A (en) Use the picture outputting method of external equipment and the electronic equipment for supporting it
JP2022501739A (en) Stylus pen detection method, system and related equipment
CN108073281A (en) The method and and its electronic equipment of content are provided
CN113782025B (en) Speech recognition method, device, terminal and storage medium
CN115985309A (en) Voice recognition method and device, electronic equipment and storage medium
KR101727900B1 (en) Mobile terminal and operation control method thereof
CN110531918A (en) A kind of processing method of music data, device, mobile terminal and storage medium
KR20120057256A (en) Mobile terminal and operation method thereof
CN109640217A (en) A kind of speaker control method and terminal device
CN111917959B (en) Camera control method and device, storage medium and smart watch
KR20110133860A (en) Mobile terminal and operation method thereof

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant