SYSTEM FOR WEARABLE GENERAL-PURPOSE 3-DIMENSIONAL INPUT
Description Technical Field
The present invention relates to a wearable general- purpose 3-dimensional input system; and more particularly, to a wearable general-purpose 3-dimensional input system for controlling various devices using single wearable device without using a required controlling device such as a remote controller, a mouse and a pen and a method thereof .
Background Art
As electronic appliances for home and office have been generalized, various input devices thereof have been introduced to control these electronic appliances .
As an example of the input device, a remote controller is used to control a television set and an audio device , and a keyboard and a mouse are used to control a personal computer (PC) . Furthermore, a touch screen or a pen such as stylus may be used for the personal digital assistance ( PDA) .
Recently, it is common to use two or three of TVs or PCs in single home . Therefore , the number of controlling devices such as the remote controller, the mouse and the pen also increases in proportional to the number of the TVs or PCs used .
Since there are usually many kinds of controlling devices around home , it is annoying to a user to find a proper controlling device such as a remote controller corresponding to a target device to control such as TV whenever the user wants to control the target device .
Meanwhile , the home network and the middleware have been generalized . The concepts of the home network and the
middleware have been applied to control the electronic household or office appliances , conveniently . That is , a user is allowed to control various information electronic appliances connected each others through the home network by transmitting a command to a target device . However, it still requires the input devices such as the remote controller, the mouse and the pen to input the command .
In order to overcome the limitation of the conventional input devices , there were many researches for developing the alternative input devices , for example , a mouse using an acceleration sensor, an input device having an acceleration sensor at a pen tip to sense the motion and having an pressure sensor to sense whether the pen touches the paper or not, and a pen for detecting motions using an acceleration sensor and transmitting the detected information about motion through Bluetooth .
For example , the pen using the acceleration sensor and Bluetooth includes a button and the pen is activated through pushing the button . Furthermore , there was another pen using 3-axes acceleration sensors and an optical sensor disclosed at U . S . Patent Publication No . 2002 /0148655A1. This pen enhances the accuracy of the pen movement estimation by compensating the acceleration with an angle formed between the pen and a plane using the optical sensor although the acceleration sensor is used .
The pens may additionally use the pressure sensor to determine whether the pen touches the plane or not to identify a beginning and an ending of stroke . An example of the alternative input device is a universal remote controller, which controls various electronic devices . However, a user should select a target appliance by pushing a corresponding button on the universal remote controller, whenever the user wants to control a TV, a VCR, and an audio system.
For example , if the audio system is connected to a TV, a VCR and a CD player and the user wants to change the channel of the TV and to adjust the volume of the audio system, the user must perform a sequence of operations such as selecting the TV, changing channel , selecting the audio system, and finally adj usting volume . It is very complicated and annoying to the user .
In order to overcome such a drawback, another conventional method of controlling various electronic appliances with the stored controlling information in a supplementary storage was introduced . That is , sequences of operations for the user' s specific activity are stored in the additional storage, and once the user selects a target activity by pushing a button, a corresponding sequence is searched and retrieved from the storage . Then, the remote controller executes the sequence of operation .
Further another conventional input device was introduced in Japan Patent Publication No . 2003-143683. The convention input device uses an acceleration sensor to sense the sequence of impacts made when a user taps obj ects or his body parts . The sensed sequence of impacts is transformed to a predetermined command to control a target device according to pre-defined mapping information that defines impact sequence patterns and related commands . As described above, these conventional methods and devices must be used to corresponding electronic appliances only although the universal remote controller is used .
In order to control a personal computer that requires complicated control mechanism, a mouse and a digitizing tablet are used . Also, a pen is generally used for portable devices such as a personal digital assistance (PDA) .
It is well known that three-dimensional mouse , pen or remote controller may be embodied by monitoring user' s motion using an inertial sensor such as an acceleration
sensor and a gyro sensor . However, related conventional technologies are limited to perform functions related to one of the mouse, the pen and the remote controller only . That is , the generality thereof is degraded . As an example of the conventional technology, a mouse using an inertial sensor uses buttons to input data as like as a typical mouse . A pen using an inertial sensor also uses a button or a pressure sensor to identify a beginning and an ending of stroke . Also, the optical sensor or the gyro sensor is additionally used to accurately identify the movement of the pen although the acceleration sensor is used to obtain information related to movement of the pen .
Such additional buttons and sensors are another limitation to reduce a size of a device or to make a device wearable to a user .
Also, the conventional technologies are operated as only one of input devices such as a mouse , a pen and a remote controller although the inertial sensor is used .
Disclosure Technical Problem
It is , therefore, an obj ect of the present invention to provide a wearable general-purpose three-dimensional input system that performs input functions of a remote controller , a mouse and a pen with only one device .
Technical Solution
In accordance with one aspect of the present invention, there is provided a wearable general-purpose three- dimensional wearable system including : a wearable input device having a structure wearable on a user for transmitting motion information of the user; and a host
device for analyzing the motion information received from the wearable input device and feeding back the result to the user .
The wearable general-purpose three-dimensional wearable system may further include a host connecting device for connecting the wearable input device to the host device through a wireless link .
Advantageous Effects
A wearable general-purpose three-dimensional wearable system according to the present invention can be embodied as an integrated input device that operated as a remote controller , a pen and a mouse according to user' s motion and the usage context .
A wearable general-purpose three-dimensional wearable system according to the present invention also can control various electronic appliances in a home network or in a ubiquitous environment using a single wearable device . Furthermore , a wearable general-purpose three- dimensional wearable system according to the present invention allows a user to naturally use without using additional buttons . Since it does not require any button, the device can be miniaturized . In a wearable general-purpose three-dimensional wearable system according to the present invention, a wearable device transits between the inactive state and the active state according to the user' s motion without using additional switches . Moreover, a wearable general-purpose three- dimensional wearable system according to the present invention senses motions for operating the device using three-axis acceleration sensor without requiring additional sensors . A wearable general-purpose three-dimensional wearable
system according to the present invention can be embodied as a ring to be wearable on a user' s finger to sense a handwriting motion of a user .
Description of Drawings
The above and other obj ects and features of the present invention will become apparent from the following description of the preferred embodiments given in conj unction with the accompanying drawings , in which :
Fig . 1 is a block diagram illustrating a wearable general-purpose three-dimensional wearable system according to a preferred embodiment of the present invention;
Fig . 2 is a block diagram of the host connecting device of the wearable general-purpose three-dimensional wearable system shown in Fig . 1 ;
Fig . 3 shows an example of using the wearable device 100 shown in Fig . 1 ;
Fig . 4 is a block diagram illustrating the wearable device 100 shown in Fig . 1 ;
Fig . 5 shows an example of graphic user interface displayed on a display according to a preferred embodiment of the present invention;
Fig . 6 is a block diagram illustrating the host 200 shown in Fig . 1 ;
Fig . 7 is a block diagram showing a motion analyzing unit of the host 200 according to the present invention;
Figs . 8 , 9 and 10 are graphs showing characteristic patterns of acceleration which are compared by the pattern extracting unit 901 ;
Fig . 11 is a view showing the relation between a user' s hand and the wearable device 100 making a motion for handwriting;
Fig . 12 is a graph showing a relation between the wearable device 100 and a plane for handwriting when a user
makes a motion for handwriting using the wearable device 100 ; and
Fig . 13 is a view showing an example of handwriting made using the wearable device 100.
Best Mode for the Invention
Other obj ects and aspects of the invention will become apparent from the following description of the embodiments with reference to the accompanying drawings , which is set forth hereinafter .
Fig . 1 is a block diagram illustrating a wearable general-purpose three-dimensional wearable system according to a preferred embodiment of the present invention .
As shown in Fig . 1 , the wearable general-purpose three-dimensional wearable system according to the present embodiment includes a wearable device 100 , a host 200 , a host connecting device 300 for connecting the wearable device 100 and the host 200 , and a display 120 connected to the host 200. The wearable general-purpose three- dimensional wearable system according to the present embodiment may further include a network 110.
The host 200 may be an information electronic appliance such as a personal computer ( PC ) , a personal digital assistance (PDA) or a set-top box . If the host 200 is capable of wireless communication such as the PC or the PDA, the host 200 may communicate with the wearable device directly without requiring the host connecting device 300. The wearable device 100 periodically senses user' s movement and transmits the information related the sensed user' s movement to the host 200.
The host 200 receives the transmitted information from the wearable device 100 , analyzes the received information and transforms the analyzed information to operations of
mouse , buttons of remote controller and movements of pen .
The host 200 also feeds back an operating mode and movements of the wearable device 100 through the display 120 to a user . The host 200 may control other device connected to the network 10 using the obtained control information through analyzing the movement of the wearable device .
Fig . 2 is a block diagram of the host connecting device of the wearable general-purpose three-dimensional wearable system shown in Fig . 1.
As shown in Fig . 2 , the host connecting device 300 includes an antenna 304 and a wireless processing unit 304 for receiving data from the wearable device 100 through a wireless link and processing the received data, a host I/F unit 303 for interfacing with the host 200 and a controlling unit 302 for controlling the wireless processing unit 301 and the host I/F unit 303 and transferring the information .
Fig . 3 shows an example of using the wearable device 100 shown in Fig . 1.
As shown in Fig . 3 , a user wears a ring type wearable device 100 according to the present invention to control devices 4 , which can be controlled by a remote controller 1 , a mouse 2 and a pen 3 , through a wireless link . Fig . 4 is a block diagram illustrating the wearable device 100 shown in Fig . 1.
As shown in Fig . 4 , the wearable device 100 includes an acceleration sensor 101 for sensing a user' s motion, a wireless processing unit 103 for transmitting the sensed information to the host device, a controlling unit 102 for receiving the information from the acceleration sensor and transmitting the received information to the wireless processing unit 103 , and an antenna 104 for transmitting/receiving signals . The acceleration sensor 101 , the controlling unit 102 and the wireless processing
unit 103 receives power from a power supply 105.
Fig . 5 shows an example of graphic user interface displayed on a display according to a preferred embodiment of the present invention . As shown in Fig . 5, the host 200 displays controls , i . e . , a dial 804 , a button 802 , a slid bar 803 and a text input window 809 , on the display 120.
Fig . 6 is a block diagram illustrating the host 200 shown in Fig . 1 , and Fig . 7 is a block diagram showing a motion analyzing unit of the host 200 according to the present invention .
As shown in Fig . 6 , the host includes a motion analyzing unit 205, a motion recognizing unit 204 , an application/GUI unit 207 and a handwriting recognizing unit 208.
As shown in Fig . 7 , the motion analyzing unit 205 of the host 200 includes a characteristic pattern extracting unit 901 , a tilt estimating unit 902 , a tilt compensating unit 903 , a state information managing unit 906 and integrating units 904 , 905 , 907 and 908.
Hereinafter, the wearable general-purpose three- dimensional wearable system according to a preferred embodiment of the present invention will be described in detail with reference to Figs . 4 through 7. The controlling unit 102 of the wearable device 100 constantly monitors motions of user using the acceleration sensor 101 in an inactive state of the wearable device 100. If the motion of user is sensed by the acceleration sensor 101 , the wearable device 100 transits from the inactive state to an active state . Then, the wearable device 100 transfers the motion information from the acceleration sensors of each axis 101 to the host 200 through the wireless processing unit 103 and the antenna 104.
The wearable device 100 may transit from the inactive state to the active state by comparing a value denoting a
size of the user' s movement outputted from the acceleration sensors 101 to a threshold value that is previously defined , The value denoting the size of the user' s movement may be obtained by squaring absolute values of accelerations of each axis , adding the squared results and calculating a square root of the added result .
In the present embodiment , repeated movements over the threshold in a predetermined time interval are defined as a motion to transit from the inactive state to the active state as shown in Table 1.
The wearable device 100 transfers three-dimensional motion information that is information about movements of each axis of the wearable device to the motion analyzing unit 205 and the motion recognizing unit 204. The motion analyzing unit 205 analyzes the three- dimensional motion information from the wearable device 100 with coordinates of the controls displayed on the display 120 and its operating mode for the wearable device 100.
If the operating mode for the wearable device 100 is a mouse , the motion analyzing unit 205 analyzes the three- dimensional motion information as two-dimensional movement or a mouse button state change . The motion analyzing unit 205 transfers the analyzed information to the application/GUI unit 207 to move a mouse pointer 806 according to the analyzed information .
On the contrary, if the operating mode of the wearable device transits from the mouse to a pen by selecting the text input window 809, the application/GUI unit 207 displays proper pointers 807 and 808 on the screen 801 , and the motion analyzing unit 205 transmits the two-dimensional motion information and information about a beginning of a stroke and an ending of stroke to the handwriting recognizing unit 208 to recognize the user' s handwriting .
The recognized character at the handwriting recognizing unit 208 is transferred to the application/GUI
unit 207 and displayed on the text input window 809.
The motion recognizing unit 204 compares the motion of the wearable device 100 to pre-defined motions using the operating mode information for the wearable device 100 provided by the motion analyzing unit 205 and pre-defined motion-command relation information .
If the motion information from the wearable device 100 is matched with one of the predefined motions , the command information of the matched predefined motion is transmitted to the application/GUI unit 207.
The handwriting recognition unit 208 and the motion recognizing unit 204 may be implemented based on various handwriting recognition and motion recognition algorithms which are widely known to those skilled in the art . Therefore , the detail thereof is omitted .
Since a preferable sensitivity differs according to a user, the sensitivity information set by the user to transit the operating mode is provided to the motion analyzing unit 205. The acceleration sensor 101 measures acceleration when the obj ect having the sensor moves and outputs the measured acceleration . The speed of movement can be calculated by integrating the measured acceleration . And the distance of movement can be calculated by integrating the speed . The acceleration sensor 101 includes an acceleration sensor senses three axes such as X, Y, and Z . Therefore , the speed and the distance made by an obj ect in the three- dimensional space can be obtained .
When the wearable device 100 is used as a mouse , the obtained speed and the obtained travel distance are transmitted to the application /GUI unit 207 with the button state information analyzed at the state information managing unit 906. As a result , the motion of the wearable device 100 is transformed to operations of a mouse on the display such as clicking of a mouse button or moving of a
mouse pointer .
If a user wears the wearable device 100 having the acceleration sensor on a finger and moves the finger with the wearable device 100 , different acceleration patterns are obtained according to the movements of the wearable device 100. That is, start and stop in free space, stop by bumping against an obj ect , changing movement direction into the opposite direction show the unique acceleration patterns . In the present invention, the characteristic pattern extracting unit 901 compares the inputted acceleration pattern with the predefined patterns and outputs the matched pattern .
A well known pattern matching algorithm in signal processing may be used for the determination of matching . The matched pattern is used to change the operation mode for the wearable device .
Figs . 8 , 9 and 10 are graphs showing characteristic patterns of acceleration, which are compared by the character pattern extracting unit 901.
The wearable device 100 periodically measures acceleration with the acceleration sensor 101 and transmits the measured acceleration information to the host 200.
As shown in Fig . 9 , if the acceleration is changed from + to - or from - to + more steeply than a predetermined steepness set by the user, the acceleration information is interpreted as that the wearable device 100 moves into the opposite direction .
As shown in Fig . 8 , if the acceleration is attenuated and oscillated after a sharp + or - acceleration, it is interpreted as that the wearable device is stopped by dumping against an obj ect .
As shown in Fig . 10 , if + or - acceleration is slowly shown, it is interpreted as that the wearable device 100 begins to move or stops after moving .
Fig . 9 is a view showing the relation between a user' s hand and the wearable device 100 making a motion for handwriting, and Fig . 10 is a graph showing a relation between the wearable device 100 and a plane for handwriting when a user makes a motion for handwriting using the wearable device 100.
In the conventional input system, the movement of the input system can be directly observed because the sensor is included in the mouse or the pen . However, the wearable device 100 according to the present invention cannot directly observe the movement of the pen when the user writes with the wearable device .
Fig . 11 is a view showing an example of handwriting made using the wearable device 100. As shown in Fig . 11 , when a user writes using a pen, the wearable device can only senses the smaller movement 504 compared to the movement 505 of a pen tip on the plane . As shown in Fig . 9, when the wearable device 100 is operated in the pen mode , the individual grip styles for a pen 603 are different from each other and the wearing styles for the wearable device on finger are different too . So, the tilt of the wearable device on the plane 601 is not constant .
That is , X, Y, Z axis 604 and 704 of the planes 605 and 701 are tilted to X, Y, Z axis 703 of the wearable device 100.
In order to accurately estimate a traj ectory of the pen movement on the plane 605 , an angle 705 between the pen movement plane 605 and the plane 604 of the wearable device 100 must be obtained . The accelerations of each axis are measured and the angle 105 is calculated from the measured acceleration .
At the time when the pen or the finger touches the plane, the acceleration pattern shown in Fig . 8 is observed . Such an acceleration pattern is detected from the
characteristic pattern extracting unit 901 and the input acceleration pattern is transmitted to the tilt estimating unit 902.
If maximum acceleration values of each axis that are observed from the pattern are x, y, and z , a vector D = (x, y, z ) becomes vertical to the plane on which the pen moves . The tilt estimating unit 902 calculates an angle formed by two vectors D= (x, y, z ) and S= ( O , 0 , z ' ) denoting a vector of a plane where the finger touches using a following Eq . 1 when the pattern of Fig . 8 is detected .
For example, if the X and the Y axes of the wearable device 100 are parallel to the plane where the pen moves , only movement of the z axis is detected when the pen touches the plane .
In this case , the movement of the pen on the plane can be observed by detecting movements on the X and the Y axes using the sensor of the wearable device 10'0 on the finger .
On the contrary, if the movement of the X axis is sensed only, the movements of the Y and the Z axes are detected to observe the movement of the pen to write a stroke .
As described above , if the movement of only one axis is sensed, the movement information of other two axes is fully reflected by the accuracy of the sensor because the movement of the pen is parallel to the plane . However, if movements of more than two axes are sensed, the acceleration of device detected by the sensor will be the exaggerated acceleration afterwards .
For example , if the X axis and the plane form the angle θ, the real acceleration of the X axis x is equal to x' cosθ where x' is a measured acceleration of the X axis .
The tilt compensating unit 903 compensates the acceleration by performing the shown equation for the acceleration measured by the sensor using the tilt calculated at the tilt estimating unit 902. The integrating unit 904 converts the compensated acceleration to a speed through integrating the compensated acceleration and the integrating unit 905 converts the speed to a distance through integrating the speed again .
The compensated speed for each axis and the distance are transmitted to the handwriting recognizing unit 208 with the stroke begin and end information obtained at the state information managing unit 906.
As described above , following input methods can be provided using the wearable general-purpose three- dimensional wearable system according to the present invention as follows . Since the following input methods are one of embodiments of the present invention, it may be modified according to a user .
<Start using device>
A user begins an interaction to the host 200 with a predefined motion such as shaking the wearable device 100 to the right and the left several times .
I f the controlling unit 102 of the wearable device 100 senses movement identical to the predefined motion under a predetermined condition, the controlling 102 realizes the detected movement as activation of the device . Therefore , the controlling 102 communicates to the host 200 and transmits the three-dimensional motion information outputted from the sensor .
The host 200 analyzes the three-dimensional motion information transmitted from the wearable device 100 as shown in Fig . 6.
In this case , the host 200 regards the wearable device 100 to perform functions of mouse . Therefore , the host 200
displays a mouse pointer on a display to notice that the user begins operations .
<0perate as mouse> When the user moves the wearable device 100 , the acceleration sensor detects the three-dimensional motion information thereof, and the wearable device 100 transmits the three-dimensional motion information to the host 200.
The host 200 analyzes the three-dimensional motion information of the wearable device 100 to find corresponding movement of the mouse . For example , the host 200 may analyze a motion of quickly moving the finger in the upward and the downward directions as a clicking operation of the mouse . If the user makes such a motion twice , the host regards that as the double clicking operation of the mouse .
If the user stops to move the finger after moving the finger up and down, the host regards that as activation of right button of the mouse .
<Select control>
When a mouse pointer is located on a predetermined control , the host 200 regards a motion of moving the finger up and down as the selection of the control . Then, if the user moves the finger to right and left , the host 200 regards that as controlling a dial or a sliding bar . If the user moves the finger in the upward direction, the host 200 regards that as release of selection . Meanwhile, if the user moves the finger up and down when the mouse point is on a text input window, the host 200 regards this motion to transit the operating state of the wearable device to a pen from the mouse .
<Operate as pen>
When the wearable device 100 is operated as the pen, the user is allowed to write texts by holding a pen or using the finger wearing the wearable device as the pen .
Herein, the acceleration sensor senses the motion of the wearable device 100 as a beginning of valid stork 501 by detecting a sudden stop motion generated from the wearable device 100 when the pen or the finger touches the plane .
Also, the acceleration sensor sets a reference plane 704 of the plane where the pen moves by calculating the three-dimensional accelerations when the acceleration sensor detects the sudden stop motion .
Then, detected movements of the wearable device 100 are analyzed as two-dimensional movements 505 on the reference plane and the movement information of stroke is indirectly calculated when the user makes a handwriting motion .
If the user moves the pen vertically or about vertically upwards from the reference plane , the host 200 regards this motion as ending of stroke 502.
The movements 504 from the beginning of stroke and the ending of stoke are transmitted to the handwriting recognizing unit and they are recognized as control operations such as texts , space or backspace . While the wearable device 100 is operated as the pen, movements 503 of the wearable device 100 before the beginning 501 of valid stroke and the ending 502 of the valid stroke are regarded as movements of mouse . Therefore , these movements are displayed as the movement of the mouse pointer on the display instead of the stroke 505.
<Return to mouse from pen>
If the user shakes the finger after the user moves the pen in up direction from the plane because of finishing inputting of text , the motions of the user is regarded as
returning to the mouse operating state .
<Remote controller mode>
When the wearable device 100 is operated as the mouse , the host 200 also determines whether the movements of the wearable device 100 are matched to the predefined motions for the remote controller .
When the user selects one of devices such as an audio system and a video player having functions previously defined according to the present invention, or when the user activates one of applications predefined according to the present invention, the host 200 determines whether the movement of the user is matched to operations defined according to the device or the application at first . I f the user' s movement is matched, this motion is regarded as the control command for the selected device and the control command is transferred to the device and the application .
If the user' s movement is not matched, this motion is regarded as moving the mouse pointer .
For example, if a TV is selected, a motion of stopping the finger after quickly moving the finger to right (left) is analyzed as reducing the volume, or a motion of stopping the finger after quickly moving the finger to upward (downward) may be analyzed as changing the channel .
The following Table 1 shows transition between the operation modes of a wearable device , conditions , a motion of finger for entering corresponding operation mode and meaning thereof according to a preferred embodiment of the present invention .
Table 1
<^> : move to left and right quickly <—> : move to left and right slowly or normally \ \ : move upward and downward slowly or normally Il : move downward and stop by dump obj ect T : move upward
Il T : sequence of movement that move downward, stop by dump obj ect and move upward .
A user may be allowed to select and define convenient motions as operations of wearable device under each operation mode and conditions . Therefore , the embodiment shown in table 1 may vary according to the user' s preference .
The above described method according to the present invention can be embodied as a program and stored on a computer readable recording medium. The computer readable recording medium is any data storage device that can store data which can be thereafter read by the computer system.
The computer readable recording medium includes a read-only memory (ROM) , a random-access memory (RAM) , a CD-ROM, a floppy disk, a hard disk and an optical magnetic disk .
While the present invention has been described with respect to certain preferred embodiments , it will be apparent to those skilled in the art that various changes and modifications may be made without departing from the scope of the invention as defined in the following claims .