WO2012104288A1

WO2012104288A1 - A device having a multipoint sensing surface

Info

Publication number: WO2012104288A1
Application number: PCT/EP2012/051531
Authority: WO
Inventors: Robert Skog; Justus Petersson
Original assignee: Telefonaktiebolaget L M Ericsson (Publ)
Priority date: 2011-02-03
Filing date: 2012-01-31
Publication date: 2012-08-09

Abstract

There is provided a device comprising a multipoint sensing surface and a gesture module. The multipoint sensing surface is for receiving inputs from one or more objects. The gesture module is configured to receive inputs from the multipoint sensing surface, and to match a gesture on the multipoint sensing surface to an action, the magnitude of the action determined by the number of contact points with which the gesture is made.

Description

A DEVICE HAVING

A MULTIPOINT SENSING SURFACE

Technical field

The present application relates to a device comprising a multipoint sensing surface; a method in a device having a multipoint sensing surface, and a computer-readable medium.

Background

Touch sensitive surfaces are popular device interfaces and can be found in the form of touch screens on smart-phones and as touch-pads on laptops. With the advent of capacitive, as opposed to resistive, touch sensors it became possible to detect multiple points of contact on the touch sensitive surface and from this the use of gestures in the user interface became more common.

An example of a simple gesture and action is pinch-to-zoom-out, whereby if two contact points such as those presented by a thumb and forefinger on the touch sensitive surface are brought together as a pinch the device responds by performing a zoom-out action on the currently displayed item, such as a picture or a webpage.

US Patent No. 7,340,077 to Gokturk et al. describes an arrangement where a camera is used to recognise a user's body position, which is used as an input into a related electronic device.

US Patent No. 6,191 ,773 to Maruno et al. describes an interface apparatus including means for recognizing the shape or movement of the hand of an operator, and using this to control the information displayed on the screen of a device.

US Patent Application Publication No. US 2010/0180298 by Kim et al.

describes a terminal device coupled to a broadcast receiving apparatus. The terminal device includes a communication unit which receives electronic program guide (EPG) information from the Internet, a display unit that displays EPG information, and a control unit that controls the broadcast receiving apparatus to perform an operation corresponding to a selection.

"A Touch Gesture Reference Guide" by C. Villamor et al., available at www.lukew.com, describes gestures used for touch commands, using gestures to support user actions, visual representations of gestures, and outlines of how popular software platforms support touch gestures.

An advantage of using gestures as part of the user interface of a device is that certain actions can be performed intuitively by the user. A problem with existing devices that use touch sensitive surfaces is that as the gestures increase in number and complexity, the user interface becomes less intuitive and less user friendly.

Summary

There is provided a device comprising a multipoint sensing surface and a gesture module. The multipoint sensing surface is for receiving inputs from one or more objects. The gesture module is configured to receive inputs from the multipoint sensing surface, and to match a gesture on the multipoint sensing surface to an action, the magnitude of the action determined by the number of contact points with which the gesture is made. By having the magnitude or strength of an action that is invoked by a gesture determined by the number of fingers with which the gesture is made, a simple and intuitive man-machine interface is provided.

The device may further comprise a display. The multipoint sensing surface may be mounted on the display. The device may further comprise a transmission means for sending instructions to a further device. The further device may be at least one of: a television, a set top box, a radio, a media player and a PC. The action may be to change channel on a further device, wherein the number of fingers with which the gesture is performed determines the number of channels changed. The direction of the gesture may determine the direction of channel change.

The action may be to skip a segment of media playback, wherein the number of fingers with which the gesture is performed determines the length of the skip. The direction of the gesture may determine the direction of the skip. The direction of the skip may be forwards in time or backwards in time.

The action may be to change the speed of media playback, wherein the number of fingers with which the gesture is performed determines the change in playback speed. The direction of the gesture may determine whether the playback speed is increased or decreased. The playback speed may be negative, sometimes referred to as "re-wind"

The action may be to move a cursor on a screen, wherein the number of fingers with which the gesture is performed determines the amount the cursor moves. In this way a multi-speed cursor is provided.

There is further provided a method, in a device having a multipoint sensing surface. The method comprises receiving inputs from one or more objects in contact with the multipoint sensing surface. The method further comprises matching a gesture on the multipoint sensing surface to an action. The method further comprises determining the magnitude of the action by the number of contact points with which the gesture is made.

The multipoint sensing surface may be mounted in front of a display. Such a multipoint sensing surface is substantially transparent.

The method may further comprise sending instructions to a further device, the instructions relating to the action corresponding to the gesture and the number of contact points with which the gesture is made. The action may be to change channel, wherein the number of fingers with which the gesture is performed determines the number of channels changed. The direction in which the gesture is made may determine the direction of channel change.

The gesture may be at least one of a horizontal swipe, a vertical swipe or a rotational swipe, or a combination thereof.

There is further still provided a computer-readable medium, carrying instructions, which, when executed by computer logic, causes said computer logic to carry out any of the methods defined herein.

There is further provided a remote control for selecting and controlling an electronic device. The remote control comprising: a touch-sensitive screen; a programmable digital signal processor in communication with the touch- sensitive screen; and a transmitter in communication with the programmable digital signal processor; wherein the processor is configured to determine a number of fingers touching the touch-sensitive screen, to set based on the number determined a pace corresponding to the electronic device, to identify a gesture made by the fingers on the screen, and to generate based on the gesture identified a corresponding command to be transmitted to the electronic device corresponding to the set pace.

Brief description of the drawings

A device having a multipoint sensing surface will now be described, by way of example only, with reference to the accompanying drawings, in which:

Figure 1 illustrates an embodiment of a device as described herein having a multipoint sensing surface;

Figure 2 illustrates a plurality of gestures which may be used to instruct a command to be sent to a further device;

Figure 3 shows some alternative gestures which may be used herein; Figure 4 illustrates a method incorporating the gestures described herein;

Figure 5 is a block diagram illustrating the components of the device described herein; and Figure 6 is a block diagram illustrating the components of an alternative device also described herein.

Detailed description

Figure 1 illustrates an embodiment of a device having a multipoint sensing surface. The device shown in figure 1 is a remote control comprising a user interface for receiving instructions from a user and a transmitting means for transmitting commands to at least one further device. Remote control 100 sends instructions to a television 1 10 and a radio 120. The instructions are transmitted from the remote control 100 using conventional wireless

communications techniques such as infrared, Bluetooth™, Wi-Fi or similar. Remote control 100 comprises a display 102 which has a transparent touch sensitive surface overlaid to create what is commonly termed a touch-screen. Remote control 100 additionally includes a plurality of hardware buttons 103.

A user may use remote control 100 to send commands to the further devices 1 10, 120 to control their operation. The remote control 100 thus allows a user to perform actions such as turn on a further device, and to change the channel the further device outputs. A typical remote control has only physical buttons for receiving input from a user, but remote controls with touch sensitive surfaces such as touch screens are known.

Figure 2 illustrates a plurality of gestures which may be used to instruct a channel change command to be sent to one of the further devices 1 10, 120. To which of the further devices 1 10, 120 that the gesture relates is determined by an aspect of the user interface of device 100. For example, in one embodiment the user interface of remote control 100 comprises a plurality of buttons at the top of display 102 each button relating to a further device 1 10, 120 that may be controlled. Before making a gesture on the touch screen 102 the user must ensure that the appropriate button is highlighted indicating the appropriate further device is selected.

The touch screen 102 of remote control 100 is capable of detecting multiple simultaneous contact points. In the embodiment shown remote control 100 comprises an LCD display with a capacitive touch surface overlaid. Figure 2A illustrates a two finger swipe to the right which is interpreted by remote control 100 as single channel up instruction. Figure 2B illustrates a two finger swipe to the left which is interpreted by remote control 100 as a single channel down instruction. While selecting a channel, switching between channels, or channel surfing, a user will typically wish to change from one channel to another channel multiple channels away in the displayed channel order. This may be achieved by the user inputting multiple consecutive channel up or channel down instructions. Remote control 100 presents an intuitive and more efficient method of providing these instructions by recognising a three finger swipe to the left or to the right as a multiple channel up or a multiple channel down instruction. This is illustrated in figure 2C which shows a three finger swipe to the right indicating a five channel up instruction; and figure 2D which illustrates a three finger swipe to the left indicating a five channels down instruction.

The number of channels that is skipped when a multiple channel change instruction is received is an implementation detail which will vary dependant upon the total number of channels available. For example, for a television with only eight channels a five channel change instruction is unlikely to be useful whereas a three channel change instruction is more likely to be used. Similarly, for a radio with one hundred channels a ten channel change instruction maybe more useful than a five channel change instruction. Figure 3 shows some alternative gestures to which the multiple channel change instruction may be assigned. Each of the gestures in figure 3 is shown made by two contact points which will typically be two fingers. Each of the gestures shown in figure 3 may also be made with one, three or even four fingers. Figures 3A and 3B illustrate a two finger swipe to the right and to the left respectively corresponding to the single channel change instructions in figures 2A and 2B. Figure 3C shows a two finger swipe up, and figure 3D shows a two finger swipe down. Figure 3E shows a two finger rotational swipe clockwise and Figure 3F shows a two finger rotational swipe anticlockwise. The channel up and channel down instruction described above in relation to remote control 100 may alternatively be implemented with vertical swipes (figures 3C and 3D respectively) or rotational swipes (figures 3E and 3F respectively). It should be further noted that where the size of the touch screen 102 of remote control 100 is sufficient three speed channel change instructions may be implemented. For example, a two finger swipe may correspond to a single channel change, a three finger swipe may corresponds to a four channel change and a four finger swipe may correspond to a ten channel change. It should be noted that the above described technique is not limited to channel change commands. The technique may also be applied to scrolling or skipping through a media presentation (such as a recorded or buffered TV programme, an audio file or an audio & video file), or scrolling through a list such as an electronic programme guide (EPG) or content catalogue. In the context of scrolling through a media presentation, a two finger swipe may correspond to scrolling at twice normal speed, whereas a three finger swipe may correspond to scrolling at four times normal speed. Alternatively, a two finger swipe may correspond to an increase of the scrolling speed by one step (from 1x to 2x, or from 2x to 4x), and a three finger swipe may correspond to an increase of the scrolling speed by two steps (from 1x to 4x, from 2x to 8x, or from 4x to 16x).

In the context of skipping through a presentation, a two finger swipe may correspond to skipping 5 seconds, whereas a three finger swipe may correspond to skipping 30 seconds. Similarly, when scrolling through a list a two finger swipe may correspond to a moving on to the next item on the list (or time slot in an EPG) and a three finger swipe may correspond to moving onto the fourth next item on the list (or time slot on the EPG). It should be noted that in the above examples the direction of the gesture may be used to determine the direction of the skipping or scrolling.

Furthermore, it should be noted that in the above, single finger gestures are not used for scrolling or skipping instructions as this may be reserved for another function in the user interface of remote control 100 such as, for example, cursor movement. It should be understood that where reserving of the single finger swipe is not require then the single finger swipe may be used for a single channel change, skip or scroll instructions with additional fingers used to indicate a higher speed of channel change, skip or scroll.

Further still, the technique described herein may be applied to cursor movement. This is particularly relevant to devices that have a separate touch sensitive surface and screen, or to devices which have a touch screen but where the cursor controlled by the touch sensitive surface of the touch screen is displayed on a screen other than the touch screen. The relation between amount of input movement and amount of cursor movement is determined by the cursor speed. For a given input movement a fast cursor moves further on the screen than a slow cursor. The technique described herein may be used to provide a multi-speed cursor whereby a cursor moves at one speed in response to a one finger gesture, and at a faster speed in response to a two finger gesture. Optionally, the cursor may move faster still in response to a three finger gesture. Alternatively, the cursor may move at one speed in response to a one finger gesture, and at a slower speed in response to a two finger gesture.

Optionally, the cursor may move slower still in response to a three finger gesture. This may be particularly useful where precision cursor control is occasionally required.

Figure 4 illustrates a method implemented by a device such as remote control 100 incorporating the gestures described above. At 400 the process starts and at 410 it is determined whether or not a touch input is detected. If no touch input is detected the process proceeds to a standby mode 405 in which the device may periodically determine whether touch inputs have been made to the multipoint sensing surface. If a touch input is detected at 410 the process proceeds to 420 where a determination is made as to the number of contact points that the touch input comprises. Then the process proceeds to 430 where the touch input is monitored and any movements of the contact points are detected. Once movement is detected a gesture is identified at 440 which best correlates with the detected movement. Thus, the process establishes what gesture was used and the number of contact points used to make it. At 450 the process performs the action corresponding to the input gesture and the number of contact points. The action may comprise transmitting a control instruction to a further device.

Figure 5 is a block diagram illustrating the components of remote control 100. Remote control 100 comprises a touch screen 510, a gesture module 520, a processor 530 and a graphics module 540. The gesture module 520 receives touch information from the touch sensitive surface of touch screen 510 and translates these signals into gesture information which is sent to a processor 530. Processor 530 is arranged to run instructions to allow the remote control 100 to perform as required. Processor 530 sends display information to a graphics module 540 which drives the display component of touch screen 510.

Figure 6 illustrates the structure of a device 600 which does not comprise of a touch screen but has separate touch and display surfaces. Device 600 comprises a touch surface 610, a gesture module 620, a processor 630, a graphics module 540, and a display 650. The gesture module 620 receives touch information from the touch surface 610 and translates these signals into gesture information which is sent to a processor 630. Processor 630 is arranged to run instructions to allow the device 600 to perform as required. Processor 630 sends display information to a graphics module 640 which drives the display 650.

It will be apparent to the skilled person that the exact order and content of the actions carried out in the method described herein may be altered according to the requirements of a particular set of execution parameters. Accordingly, the order in which actions are described and/or claimed is not to be construed as a strict limitation on order in which actions are to be performed.

Claims

1 . A device comprising:

a multipoint sensing surface for receiving inputs from one or more objects;

a gesture module configured to receive inputs from the multipoint sensing surface, and to match a gesture on the multipoint sensing surface to an action, the magnitude of the action determined by the number of contact points with which the gesture is made.

2. The device of claim 1 , further comprising a display.

3. The device of claim 2, wherein the multipoint sensing surface is mounted on the display.

4. The device of any preceding claim, further comprising a transmission means for sending instructions to a further device.

5. The device of claim 4, wherein the further device is at least one of: a television, a set top box, a radio, a media player and a PC.

6. The device of any preceding claim, wherein the action is to change channel, wherein the number of fingers with which the gesture is performed determines the number of channels changed.

7. The device of any of claims 1 to 5, wherein the action is to skip a segment of content playback, wherein the number of fingers with which the gesture is performed determines the length of the skip.

8. The device of any of claims 1 to 5, wherein the action is to change the speed of content playback, wherein the number of fingers with which the gesture is performed determines the change in playback speed.

9. The device of any of claims 1 to 5, wherein the action is to move a cursor on a screen, wherein the number of fingers with which the gesture is performed determines the amount the cursor moves.

10. The device of any preceding claim, wherein the gesture is at least one of a horizontal swipe, a vertical swipe or a rotational swipe, or a combination thereof.

1 1 . A method, in a device having a multipoint sensing surface, the method comprising:

receiving inputs from one or more objects in contact with the multipoint sensing surface;

matching a gesture on the multipoint sensing surface to an action; determining the magnitude of the action by the number of contact points with which the gesture is made.

12. The method of claim 1 1 , wherein the multipoint sensing surface is mounted on a display.

13. The method of claim 1 1 or 12, further comprising sending instructions to a further device, the instructions relating to the action corresponding to the gesture and the number of contact points with which the gesture is made.

14. The method of claim, 1 1 , 12 or 13, wherein the action is to change channel, wherein the number of fingers with which the gesture is performed determines the number of channels changed.

15. The method of claim 14, wherein the direction of the gesture

determines the direction of channel change.

16. The method of any of claims 1 1 to 15, wherein the gesture is at least one of a horizontal swipe, a vertical swipe or a rotational swipe, or a combination thereof.

17. A computer-readable medium, carrying instructions, which, when executed by computer logic, causes said computer logic to carry out any of the methods defined by claims 1 1 to 16.