US20130069860A1 - Organizational Tools on a Multi-touch Display Device - Google Patents
Organizational Tools on a Multi-touch Display Device Download PDFInfo
- Publication number
- US20130069860A1 US20130069860A1 US13/404,439 US201213404439A US2013069860A1 US 20130069860 A1 US20130069860 A1 US 20130069860A1 US 201213404439 A US201213404439 A US 201213404439A US 2013069860 A1 US2013069860 A1 US 2013069860A1
- Authority
- US
- United States
- Prior art keywords
- objects
- target element
- display device
- touch display
- organizational tool
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F3/04886—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
Abstract
A process for enabling objects displayed on a multi-input display device to be grouped together is disclosed that includes defining a target element that enables objects displayed on a multi-input display device to be grouped together through interaction with the target element. Engagement of an input mechanism with one of the target element and a particular one of the objects displayed on the multi-input display device is detected. Movement of the input mechanism is monitored while the input mechanism remains engaged with whichever one of the target element and the particular displayed object that the input mechanism engaged. A determination is made that at least a portion of a particular displayed object is overlapping at least a portion of a target element on the multi-input display device upon detecting disengagement of the input mechanism. As a consequence of disengagement and the overlap, processes are invoked that establish a relationship between the particular displayed object and a position on the target element and that causes transformations applied to the target element also to be applied to the particular displayed object while maintaining the relationship between the particular displayed object and the position on the target element.
Description
- This application claims the benefit of U.S. patent application Ser. No. 12/785,134, filed May 21, 2010, and titled “Organizational Tools on a Multi-touch Display Device,” which claims priority to U.S. Provisional Application No. 61/180,408, filed May 21, 2009, and titled “Organizational Tools on a Multi-touch Display Device”, the disclosures of which are considered part of (and are incorporated by reference in) the disclosure of this application.
- This disclosure relates to organizational tools on a multi-touch display device.
- Multi-touch display devices often adopt many of the characteristics of touch-screen display devices, and yet they are generally more sophisticated than traditional touch-screen display devices in that they are capable of detecting the presence and location of multiple touches on, within, or within the vicinity of the surface of the display area at the same time. Specifically, multi-point input computing systems receive, recognize, and act upon multiple inputs at the same time. Because multi-point input computing systems are capable of receiving, recognizing, and acting upon multiple inputs at the same time, multi-point input computing systems may enable multiple users to interact with individual systems at the same time, thereby providing for collaboration between the multiple users.
- Like traditional touch-screen display devices, some multi-touch display devices require that a user physically touch the surface of the display area with one or more fingers, styluses, and/or other mechanisms in order to engage the surface of the multi-touch display device, while other multi-touch display devices are capable of receiving input by detecting that one or more fingers, styluses, and/or other input mechanisms have engaged the surface of the multi-touch display device by hovering around, or otherwise in the vicinity of, the surface of the display area without requiring that the input mechanism actually make physical contact with the surface of the touch-screen display device.
- In a general aspect, a method for merging two sets of objects displayed on a multi-input display device is disclosed that includes defining a first target element that enables objects displayed on a multi-input display device to be grouped together through interaction with the first target element, visually displaying, on the multi-input display device, at least a portion of the first target element, defining a second target element that enables objects displayed on the multi-input display device to be grouped together through interaction with the second target element, visually displaying, on the multi-input display device, at least a portion of the second target element, and visually displaying, on the multi-point display device, a first set of objects, each constituent object of the first set of objects at least partially overlapping the first target element. The method also includes invoking processes that establish relationships between the constituent objects of the first set of objects and one or more corresponding positions on the first target element that are overlapped by the constituent objects of the first set of objects that cause the relationships between the constituent objects of the first set of objects and the corresponding positions on the first target element to be maintained when transformations are applied to the first target element such that transformations applied to the first target element also are applied to the constituent objects of the first set of objects. The method also includes visually displaying, on the multi-point display device, a second set of objects, each constituent object of the second set of objects at least partially overlapping the second target element. The method also includes invoking processes that establish relationships between the constituent objects of the first set of objects and one or more corresponding positions on the second target element that are overlapped by the constituent objects of the second set of objects that cause the relationships between the constituent objects of the second set of objects and the corresponding positions on the second target element to be maintained when transformations are applied to the second target element such that transformations applied to the second target element also are applied to the constituent objects of the second set of objects. The method also includes detecting that an input mechanism has engaged the second target element. The method also includes, in response to detecting that the input mechanism has engaged the second target element, monitoring movement of the input mechanism while the input mechanism remains engaged with the second target element. The method also includes translating the second target element and the constituent objects of the second set of objects as a function of the monitored movement of the input mechanism such that at least a portion of the second target element is overlapping the first target element. The method also includes, after translating the second target element and the constituent objects of the second set of objects, detecting that the input mechanism has disengaged the second target element. The method also includes determining, upon detecting that the input mechanism has disengaged the second target element, that the second target element is overlapping at least a portion of the first target element as a consequence of the translation of the second target element. The method also includes, in response to determining that the second target element is overlapping the portion of the first target element upon detecting that the input mechanism has disengaged the second target element, invoking processes that establish new relationships between the constituent objects of the second set of objects and one or more corresponding positions on the first target element that cause the relationships between the constituent objects of the second set of objects and the corresponding positions on the first target element to be maintained when transformations are applied to the first target element such that transformations applied to the first target element also are applied to the constituent objects of the second set of objects.
- In some implementations, the method may further include, after invoking processes that establish new relationships between the constituent objects of the second set of objects and one or more corresponding positions on the first target element that cause the relationships between the constituent objects of the second set of objects and the corresponding positions on the first target element to be maintained when transformations are applied to the first target element such that transformations applied to the first target element also are applied to the constituent objects of the second set of objects, detecting that an input mechanism has newly engaged the first target element. The method may further include, in response to detecting that the first target element has newly engaged the input mechanism, monitoring movement of the input mechanism while the input mechanism remains engaged with the first target element. The method may further include applying a transformation to the first target element as a function of the monitored movement of the input mechanism. The method may further include applying a corresponding transformation to the first set of objects as a consequence of applying the transformation to the first target element and the established relationships between the first set of objects and the corresponding positions on the first target element. The method may further include applying the corresponding transformation to the second set of objects as a consequence of applying the transformation to the first target element and the established relationships between the second set of objects and the corresponding positions on the first target element.
- In some implementations, translating the second target element and the constituent objects of the second set of objects as a function of the monitored movement of the input mechanism such that at least a portion of the second target element is overlapping the first target element may include visually displaying the second target element as overlapping at least a portion of the first target element and any constituent objects of the first set of objects for which relationships have been established with positions on the first target element that correspond to positions at which the second target element overlaps the first target element. Invoking processes that establish new relationships between the constituent objects of the second set of objects and one or more corresponding positions on the first target element that are overlapped by the constituent objects of the second set of objects that cause the relationships between the constituent objects of the second set of objects and the corresponding positions on the first target element to be maintained when transformations are applied to the first target element such that transformations applied to the first target element also are applied to the constituent objects of the second set of objects may include invoking a process that establishes a relationship between the second target element and one or more corresponding positions on the first target element that causes the relationship between the second target element and the corresponding positions on the first target element to be maintained when transformations are applied to the first target element such that transformations applied to the first target element also are applied to the second target element.
- In some implementations, translating the second target element and the constituent objects of the second set of objects as a function of the monitored movement of the input mechanism may include translating the second target element and the constituent objects of the second set of objects such that the entirety of the second target element is overlapping the first target element. Determining that the second target element is overlapping at least a portion of the first target element as a consequence of the translation of the second target element may include determining that the entirety of the second target element is overlapping the first target element. Invoking processes that establish new relationships between the constituent objects of the second set of objects and one or more corresponding positions on the first target element that are overlapped by the constituent objects of the second set of objects that cause the relationships between the constituent objects of the second set of objects and the corresponding positions on the first target element to be maintained when transformations are applied to the first target element in response to determining that the second target element is overlapping the first target element, may include, in response to determining that the entirety of the second target element is overlapping the first target element: terminating the relationships between the constituent objects of the second set of objects and the one or more corresponding positions on the second target element, and eliminating the second target element from display on the multi-input display device.
- In some implementations, the method may further include, after eliminating the second target element from display on the multi-input display device, detecting that an input mechanism has newly engaged the first target element. The method may further include, in response to detecting that the first target element has newly engaged the input mechanism, monitoring movement of the input mechanism while the input mechanism remains engaged with the first target element. The method may further include applying a transformation to the first target element as a function of the monitored movement of the input mechanism. The method may further include applying a corresponding transformation to the first set of objects as a consequence of applying the transformation to the first target element and the established relationships between the first set of objects and the corresponding positions on the first target element. The method may further include applying the corresponding transformation to the second set of objects as a consequence of applying the transformation to the first target element and the established relationships between the second set of objects and the corresponding positions on the first target element.
- In some implementations, invoking processes that establish new relationships between the constituent objects of the second set of objects and one or more corresponding positions on the first target element that are overlapped by the constituent objects of the second set of objects that cause the relationships between the constituent objects of the second set of objects and the corresponding positions on the first target element to be maintained when transformations are applied to the first target element such that transformations applied to the first target element also are applied to the constituent objects of the second set of objects may include determining that, as a consequence of having translated the second target element and the constituent objects of the second set of objects, a particular object of the second set of objects is overlapping at least a portion of another object of the first set of objects. invoking processes that establish new relationships between the constituent objects of the second set of objects and one or more corresponding positions on the first target element that are overlapped by the constituent objects of the second set of objects that cause the relationships between the constituent objects of the second set of objects and the corresponding positions on the first target element to be maintained when transformations are applied to the first target element such that transformations applied to the first target element also are applied to the constituent objects of the second set of objects may also include, as a consequence of determining that the particular object of the second set of objects is overlapping at least a portion of the other object of the first set of objects, transforming the particular object such that it does not overlap any object of the first set of objects but continues to overlap at least a portion of the first target element.
- In some implementations, the method may further include detecting that the input mechanism has engaged the second target element. The method may further include, in response to detecting that the input mechanism has engaged the second target element, invoking a process to determine if future movements by the input mechanism while the input mechanism remains engaged with the second target element are to terminate the relationship between the second set of objects and the corresponding positions on the first target element and the relationship between the second target element and the corresponding positions on the first target element. The method may further include, based on results of the process to determine if future movements by the input mechanism while the input mechanism remains engaged with the second target element are to terminate the relationship between the second set of objects and the corresponding positions on the first target element and the relationship between the second target element and the corresponding positions on the first target element, determining that future movements by the input mechanism are to terminate the relationship between the second set of objects and the corresponding positions on the first target element and the relationship between the second target element and the corresponding positions on the first target element. The method may further include, after determining that future movements by the input mechanism are to terminate the relationship between the second set of objects and the corresponding positions on the first target element and the relationship between the second target element and the corresponding positions on the first target element, detecting movement of the input mechanism while the input mechanism remains engaged with the second target element. The method may further include, as a consequence of detecting the movement of the input mechanism while the input mechanism remains engaged with the second target element and having determined that future movements by the input mechanism are to terminate the relationship between the second set of objects and the corresponding positions on the first target element and the relationship between the second target element and the corresponding positions on the first target element, terminating the relationship between the second set of objects and corresponding positions on the first target element and the relationship between the second target element and the corresponding positions on the first target element such that the relationships between the second set of objects and the corresponding positions on the first target element and the relationship between the second target element and the corresponding positions on the first target element are no longer maintained when transformations are applied to the first target element.
-
FIGS. 1A-1J are diagrams of a multi-touch display device configured to provide rectangular two-dimensional organizational tools to users of the multi-touch display device to enable the users to organize objects displayed by the multi-touch display device. -
FIG. 2 is a diagram of a multi-touch display device configured to provide one-dimensional linear organizational tools to users of the multi-touch display device to enable the users to organize objects displayed by the multi-touch display device. -
FIG. 3 is a diagram of a multi-touch display device configured to provide two-dimensional rotary organizational tools to users of the multi-touch display device to enable the users to organize objects displayed by the multi-touch display device. -
FIG. 4A is a flowchart of an example of a first process for attaching an object to a rectangular two-dimensional organizational tool. -
FIGS. 4B-4E illustrate four variations of the attachment process illustrated inFIG. 4A . -
FIG. 4F is a flowchart of an example of a second process for attaching an object to a rectangular two-dimensional organizational tool. -
FIGS. 4G and 4H illustrate two variations of the attachment process illustrated inFIG. 4F . -
FIG. 5A is a flowchart of an example of a process for detaching an object from a rectangular two-dimensional organizational tool. -
FIGS. 5B-5F illustrate five variations of the detachment process illustrated inFIG. 5A . -
FIG. 6A is a diagram illustrating a multi-touch display device performing transformations to an organizational tool and its attached object when the attached objects are attached to the organizational tool at two or more points. -
FIG. 6B is a diagram illustrating a multi-touch display device performing transformations to an organizational tool and its attached object when the attached objects are attached to the organizational tool at only one point. -
FIG. 7A is a flowchart of an example of a process for organizing objects attached to a rectangular two-dimensional organizational tool. -
FIGS. 7B and 7C illustrate two variations of the organizing process illustrated inFIG. 7A . -
FIG. 8A is a flowchart of an example of a process for merging/composing two rectangular two-dimensional organizational tools. -
FIG. 8B is a diagram illustrating a multi-touch display device composing two overlapping organizational tools and detaching two composed organizational tools. -
FIG. 8C is a diagram illustrating a multi-touch display device merging two overlapping organizational tools and splitting the single, merged organizational tool into multiple separate organizational tools. -
FIG. 9 is a diagram illustrating an alternative implementation of the rectangular two-dimensional organizational tool. -
FIG. 10 is a diagram of a multi-touch display device that provides a one-dimensional linear organizational tool that illustrates techniques for attaching objects to the one-dimensional linear organizational tool. -
FIG. 11 is a diagram of a multi-touch display device that provides a one-dimensional linear organizational tool that illustrates techniques for detaching objects from the one-dimensional linear organizational tool -
FIG. 12A is a flowchart of an example of a process for organizing objects attached to a linear one-dimensional organizational tool. -
FIGS. 12B and 12C are diagrams of a multi-touch display device that illustrate two different examples the multi-touch display device automatically adjusting the angular orientation of an object upon attaching the objects to a one-dimensional linear organizational tool. -
FIG. 13 is a diagram illustrating a multi-touch display device performing transformations to a one-dimensional linear organizational tool and its attached objects when the attached objects are attached to the organizational tool at only one point. -
FIGS. 14A-14E are diagrams illustrating a multi-touch display device performing transformations to a one-dimensional linear organizational tool and displaying a preview screen of the one-dimensional linear organizational tool in accordance with the transformations. -
FIG. 15 is a sequence of diagrams of amulti-touch display device 1500 that provides a one-dimensional linearorganizational tool 1524 and controls for increasing and/or decreasing the scale of one or more portions of the one-dimensional linear organizational tool. -
FIGS. 16A and 16B illustrate how different parts of a single attachment strip may be viewed through multiple organizational tools, possibly on more than one multi-touch display device. -
FIG. 17 is a diagram of a multi-touch display device that provides a two-dimensional rotary organizational tool that illustrates techniques for attaching objects to the two-dimensional rotary organizational tool. -
FIG. 18 is a diagram of a multi-touch display device that provides a two-dimensional rotary organizational tool that illustrates techniques for detaching objects from the two-rotary organizational tool. -
FIGS. 19( a)-19(d) are diagrams of a multi-touch display device that illustrate an example of the multi-touch display device automatically adjusting the angular orientation of an object upon attaching the object to a two-dimensional rotary organizational tool. -
FIGS. 19( e)-19(f) are diagrams of a multi-touch display device automatically adjusting the angular orientation of an object upon detaching the object from a two-dimensional rotary organizational tool. -
FIG. 20 is a diagram illustrating a multi-touch display device performing transformations to an organizational tool and its attached objects when the attached objects are attached to the organizational tool at only one point. -
FIG. 21 is a diagram illustrating a multi-touch display device providing controls for organizing objects attached to an organizational tool. - A multi-touch display device provides one or more organizational tools that enable a user to organize objects displayed by the multi-touch display device. Such objects displayed by the multi-touch display device may include, for example, images, videos, and/or documents. The multi-touch display device may be configured to concurrently provide multiple organizational tools such that each of the multiple organization tools may be used concurrently by one or multiple users of the multi-touch display device. The concurrent provision of multiple organizational tools on the same multi-touch display device (or on multiple logically related multi-touch display devices) may facilitate collaboration between (or at least concurrent use by) multiple users by, for example, providing each of the multiple users with one or more personal organizational tools for organizing displayed objects with which the user is working
-
FIGS. 1A-1J are illustrations of amulti-touch display device 100 configured to provide organizational tools tousers multi-touch display device 100 to enable the users to organize objects displayed by themulti-touch display device 100. Furthermore,FIGS. 1A-1J illustrate examples of different user interactions with the organizational tools provided by themulti-touch display device 100. - Referring to
FIG. 1A users multi-touch display device 100, which is displayingobjects organizational tools canvas 127. Each ofusers objects organizational tools canvas 127 by engaging the surface of themulti-touch display device 100 with, for example, his/her fingers. Among other manipulations,users objects organizational tools canvas 127. -
Organizational tools organizational tools organizational tool control strip multi-touch display device 100 than thecontrol strip 128. Thus,object 106, which is attached toorganizational tool 124 inFIG. 1B , is shown behindcontrol strip 128 in that figure. As a result, the control strip is displayed no matter how many objects are attached to the organizational tool or how cluttered the organizational tool becomes. In this way, a user will be able to interact with and manipulate the organizational tool despite the current number and position of attached objects. By engaging the control strip of the organizational tool, a user may manipulate the organizational tool by, for example, moving, rotating, or resizing the organizational tool through interaction with its control strip. - As discussed above, displayed objects may be attached by users to the
organizational tools multi-touch display device 100, for example, to facilitate organization of thecanvas 127 and/or to store such attached objects for later access. For example, referring again toFIG. 1B , themulti-touch display device 100 showsobjects organizational tool 124 in response to input received fromuser 102 by themulti-touch display device 100. Similarly, themulti-touch display device 100 shows object 118 attached toorganizational tool 126 in response to input received fromuser 104 by themulti-touch display device 100. - In some implementations, attaching displayed objects to organizational tools involves moving the displayed objects into the vicinity of the organizational tools. For example,
FIG. 1B showsuser 102 engaging themulti-touch display device 100 and movingobject 112 overorganizational tool 124 in order to attachobject 112 toorganizational tool 124 whileuser 104 is engaging themulti-touch display device 100, and it showsuser 104 engaging themulti-touch display device 100 and movingobject 122 overorganizational tool 126 to attachobject 122 toorganizational tool 126. Similarly, as shown inFIG. 1C , in response tousers objects organizational tools multi-touch display device 100 showsobjects organizational tools organizational tools - In some implementations,
canvas 127 may be infinite or at least have an extent that is larger than the visual display of themulti-touch display device 100. As such, themulti-touch display device 100 may provide controls to users of themulti-touch display device 100 that enable the users to instruct themulti-touch display device 100 to translate and/or scale thecanvas 127 to modify the region of thecanvas 127 that is visually displayed by themulti-touch display device 100. In such implementations, organizational tools provided by themulti-touch display device 100 and their attached objects may be immune to (e.g., unaffected by) certain transformations applied to thecanvas 127. Consequently, such organizational tools may serve as convenient mechanisms for storing objects at a desired position on themulti-touch display device 100 when transformations are applied to thecanvas 127. -
FIGS. 1D-1E illustrate the use oforganizational tools canvas 127. In them, objects 108, 110, 116, and 120 are not attached to either of theorganizational tools multi-touch display device 100treats objects canvas 127. As such, when themulti-touch display device 100 performs transformations oncanvas 127, themulti-touch display device 100 also may perform the same (or similar) transformations onobjects objects canvas 127 as thecanvas 127 is manipulated. - In
FIG. 1D ,user 102 is engaging the surface of themulti-touch display device 100 at a point on the canvas 127 (i.e., a point on themulti-touch display device 100 where themulti-touch display device 100 is not displaying an object or organizational tool.) As a result, themulti-touch display device 100 detects thatuser 102 is touching themulti-touch display device 100, interprets this touch as an input corresponding tocanvas 127 at the contact point, and associates the input withcanvas 127. While continuing to engage the surface of themulti-touch display device 100 with his finger,user 102 may drag his finger across the surface of the device to cause themulti-touch display device 100 to translate thecanvas 127. - As illustrated in
FIG. 1E ,user 102 is dragging his finger in a vertical direction along the surface toward the top ofmulti-touch display device 100. Having already associated the input withcanvas 127, themulti-touch display device 100 detects the vertical movement of the user's finger, from a first point as shown inFIG. 1D to a second point as shown inFIG. 1E , interprets this input as a request byuser 102 to translatecanvas 127 by a distance corresponding to the distance from the first point to the second point, and visually translatescanvas 127 andobjects multi-touch display device 100 translatesobjects canvas 127. Thus, objects 108, 110, 116, and 120 are moved in a vertical direction (indicated by the illustrated arrows) fromoriginal positions 108′, 110′, 116′, and 120′. - In this example,
organizational tools canvas 127, and, therefore, themulti-touch display device 100 holdsorganizational tools canvas 127 andobjects organizational tools canvas 127 allows theusers canvas 127 while providing theusers organizational tools - In some alternative implementations, the
multi-touch display device 100 may associateorganizational tools canvas 127 in a manner similar toobjects multi-touch display device 100 also applies the same (or similar) translations to theorganizational tools multi-touch display device 100 translatescanvas 127,multi-touch display device 100 may translateorganizational tools canvas 127, by a predetermined factor of the translation applied tocanvas 127. In other words,multi-touch display device 100 may translateorganizational tools canvas 127. Thus, a user may drag his finger from one side of the screen to the other, andmulti-touch display device 100 will translateobjects organizational tools - The
multi-touch display device 100 may provide controls for performing transformations (e.g., rotate, scale, translate) onorganizational tools FIGS. 1F-1K illustrate transformations being applied toorganizational tools - In
FIG. 1F ,user 102 has engaged the surface ofmulti-touch display device 100 at a point corresponding toorganizational tool 124 and is dragging his finger along themulti-touch display device 100 in a downward direction.Multi-touch display device 100 detects this input byuser 102, associates the input withorganizational tool 124, interprets the input as a request to translateorganizational tool 124, and translatesorganizational tool 124 as a function of the movement of the user's finger. Becauseobjects organizational tool 124,multi-touch display device 100 translatesobjects organizational tool 124. - As also illustrated in
FIG. 1F ,user 104 has engaged the surface ofmulti-touch display device 100 at a point corresponding toorganizational tool 126 and is dragging his finger across themulti-touch display device 100 in a leftward direction.Multi-touch display device 100 detects the input ofuser 104, associates the input withorganizational tool 126, interprets the input as a request to moveorganizational tool 126, and translatesorganizational tool 126 as a function of the movement of the user's finger. Again, sinceobjects organizational tool 126,multi-touch display device 100 translatesobjects organizational tool 126. As a consequence,user 104 may interact withmulti-touch display device 100 to repositionorganizational tool 126 near the particular section ofcanvas 127 that he is working with at any given time. - Notably, objects 108, 110, 116, and 118, which are not associated with either
organizational tool 124 ororganizational tool 126 remain unaffected by the manipulations illustrated inFIG. 1F . - By enabling user to translate
organizational tools - As illustrated in
FIGS. 1G-1H , themulti-touch display device 100 may provide controls for scalingorganizational tools User 102 is engaging the surface ofmulti-touch display device 100 with two fingers at distinct points along the surface of themulti-touch display device 100, each such point corresponding to a point on the surface oforganizational tool 124. Themulti-touch display device 100 detects these two concurrent inputs byuser 102 and associates them withorganizational tool 124. Likewise,user 104 is engaging the surface ofmulti-touch display device 100 with two fingers at distinct points along the surface of themulti-touch display device 100, each such point corresponding with a point on the surface oforganizational tool 126. Themulti-touch display device 100 detects these two concurrent inputs byuser 104 and associates them withorganizational tool 126. - As illustrated in
FIG. 1H , asuser 102 drags his fingers toward each other,multi-touch display device 100 detects the movements of the user's fingers and interprets it as a request to scaleorganizational tool 124. As a result,multi-touch display device 100 scales the size oforganizational tool 124 as a function of the detected movement. Thecloser user 102 drags his fingers together, the smallermulti-touch display device 100 displaysorganizational tool 124. Becauseobjects organizational tool 124,multi-touch display device 100 also scalesobjects organizational tool 124. Therefore, asmulti-touch display device 100 decreases the size oforganizational tool 124, it also decreases the size ofobjects objects organizational tool 124. - The
multi-touch display device 100 performs a similar process asuser 104 drags his fingers toward each other along the surface ofmulti-touch display device 100.Multi-touch display device 100 detects the input ofuser 104, interprets it as a request to scaleorganizational tool 126, and decreases the visual size oforganizational tool 126 as a function of the detected movement of the user's fingers. Becauseobjects organizational tool 126, when themulti-touch display device 100 scalesorganizational tool 126, themulti-touch display device 100 also decreases the size ofobjects objects organizational tool 126. - Scaling
organizational tools user 102 may prefer to focus on only a few objects at a time. After attaching the objects thatuser 102 is not currently working with toorganizational tool 124,multi-touch display device 100 may minimizeorganizational tool 124 and provideuser 102 with a cleaner and less distracting workspace. Accordingly,multi-touch display device 100 may provide a mechanism for minimizing the display of a vast number of objects while keeping them easily accessible to the user. As soon as the user wants to access one of the attached objects,multi-touch display device 100 may enable the organizational tool to be increased in size in a comparable manner to the method described above for reducing the size oforganizational tool - In alternative implementations, the
multi-touch display device 100 may scale the objects attached toorganizational tools organizational tools FIG. 1H , asuser 102 drags his fingers toward each other along the surface ofmulti-touch display device 100,multi-touch display device 100 may decrease the visual size ofobjects organizational tool 124. Such a scaling mechanism may serve as a convenient way to create more free space on an organizational tool without scaling the organizational tool itself. - While reducing the size of
organizational tools organizational tools organizational tools canvas 127 where they are not interfering with the work ofusers multi-touch display device 100 may enableorganizational tools canvas 127 that are partially or completely outside of the visual display of themulti-touch display device 100.FIGS. 1I and 1J are illustrative. - In
FIG. 1I ,user 102 is engaging the surface ofmulti-touch display device 100 at a point whereorganizational tool 124 is displayed.Multi-touch display device 100 detects this input byuser 102 and associates the input withorganizational tool 124. Similarly,user 104 is engaging the surface ofmulti-touch display device 100 at a point whereorganizational tool 126 is displayed. Consequently,multi-touch display device 100 detects the input byuser 104 and associates the input withorganizational tool 126. - As illustrated in
FIG. 1J ,multi-touch display device 100 detects thatuser 102 is dragging his finger along themulti-touch display device 100 in a downward direction, from a first point as shown inFIG. 1I to a second point as shown inFIG. 1J ,multi-touch display device 100 interprets the detected movement as a request to translateorganizational tool 124 by a distance corresponding to the distance from the first point to the second point, and, consequently, translatesorganizational tool 124 the detected distance or a function of the detected distance such that a portion of theorganizational tool 124 is translated to a region of thecanvas 127 outside of the visual display of themulti-touch display device 100. As a result, whileorganizational tool 124 continues to exist in its entirety, only a portion oforganizational tool 124 remains visually displayed by themulti-touch display device 100. Becauseobjects organizational tool 124, multi-touch display device translatesobjects organizational tool 124. - The
multi-touch display device 100 performs a similar process in response touser 104 dragging his finger in a rightward direction along the surface ofmulti-touch display device 100.Multi-touch display device 100 detects the input ofuser 104, from a first point as shown inFIG. 1I to a second point as shown inFIG. 1J , interprets it as a request to translateorganizational tool 126 by a distance corresponding to the distance from the first point to the second point, and translatesorganizational tool 124 the detected distance or a function of the detected distance toward the right edge of the surface. Ultimately, the multi-touch display device translates a portion oforganizational tool 126 to a region of thecanvas 127 that is outside of the visual display of themulti-touch display device 100. Becauseobjects organizational tool 126, as themulti-touch display device 100 translatesorganizational tool 126, the multi-touch display device also translatesobjects organizational tool 126. - By enabling
organizational tools canvas 127 that are partially or completely outside of the visual display of themulti-touch display device 100, themulti-touch display device 100 may provide a less cluttered workspace forusers multi-touch display device 100 may more prominently display those objects most pertinent tousers organizational tools - A variety of different organizational tools may be provided by a multi-touch display device. For example, as discussed above in connection with
FIGS. 1A-1J , a multi-touch display device may provide a rectangular two-dimensional organizational tool that enables manipulations similar to functionality provided by a cork bulletin board. Additionally or alternatively, a multi-touch display device may provide a one-dimensional, linear organizational tool that enables manipulations similar to functionality provided by a clothesline or a short-order cook ticket line, and/or a multi-touch display device may provide a two-dimensional rotary organizational tool that enables manipulations similar to functionality provided by a Lazy Susan. -
FIGS. 2( a) and 2(b) are illustrations of amulti-touch display device 200 providing a one-dimensional organizational tool that enables manipulations similar to functionality provided by a clothes line or a short-order cook ticket line. As illustrated inFIGS. 2( a) and 2(b),organizational tool 224 includes anattachment strip 228 and boundary handles 230 and 232. As will be described in greater detail below, theattachment strip 228 provides a mechanism for attaching objects toorganizational tool 224. For example, as illustrated inFIGS. 2( a) and 2(b), objects 206, 208, 210, 212, 214, and 216 are attached toorganizational tool 224 viaattachment strip 228. - In some implementations, the extent of
organizational tool 224 may be greater than the visual display oforganizational tool 224. For example, boundary handles 230 and 232 define the boundaries of the visual display oforganizational tool 224, but, as illustrated by the dashed lines inFIGS. 2( a) and 2(b), the extent oforganizational tool 224 extends beyond boundary handles 230 and 232 even though the visual display oforganizational tool 224 is confined to the region between boundary handles 230 and 232. - For example, referring to
FIG. 2( a),object 216 is attached toorganizational tool 224 viaattachment strip 228, but is not displayed because it is attached to theattachment strip 228 oforganizational tool 224 at a position that is outside of the visual display oforganizational tool 224 defined by boundary handles 230 and 232. Similarly, referring toFIG. 2( b), objects 206 and 208 are attached toorganizational tool 224 viaattachment strip 228, but not displayed because they are attached to theattachment strip 228 oforganizational tool 224 at a position that is outside of the visual display oforganizational tool 224 defined by boundary handles 230 and 232. - In some implementations,
attachment strip 228 may be of infinite extent. In alternative implementations,attachment strip 228 may have a finite extent. In order to enable a user to access portions oforganizational tool 224 that fall outside of the visual display oforganizational tool 224,multi-touch display device 200 provides controls for translatingattachment strip 228 in a side-to-side fashion. Additionally or alternatively, one or more of boundary handles 230 and 232 can be manipulated to increase (or decrease) the extent of the visual display oforganizational tool 224, thereby providing access to a larger (or smaller) number of attached objects. - Referring to
FIG. 2( a), a user is engaging the surface ofmulti-touch display device 200 with a finger 202 at a point corresponding toattachment strip 228.Multi-touch display device 200 detects the input by the user at point “a” and associates the input withattachment strip 228 oforganizational tool 224. Referring now toFIG. 2( b), as the user drags finger 202 to the left across the surface ofmulti-touch display device 200,multi-touch display device 200 detects the movement of the user's finger 202 from point “a” to point “b,” interprets the detected movement as a request to translateattachment strip 228 by a distance corresponding to the distance from point “a” to point “b,” and translatesattachment strip 228 by the detected distance or some function of the detected distance. Becauseobjects organizational tool 200 viaattachment strip 228,multi-touch display device 200 translatesobjects attachment strip 228. In so doing, themulti-touch display device 200 preserves the spatial relationships betweenobjects attachment strip 228. - As a result of the translation of
attachment strip 228, the portion of theattachment strip 228 that is located between boundary handles 230 and 232 has changed. In particular, the portion ofattachment strip 228 to which objects 206 and 208 are attached no longer lies within boundary handles 230 and 232, while the portion ofattachment strip 228 to which object 216 is attached now lies within boundary handles 230 and 232. Therefore,multi-touch display device 200 does not displayobjects object 216. - As discussed above, a multi-touch display device may also provide a two-dimensional rotary organizational tool.
FIGS. 3( a) and 3(b) are illustrations of amulti-touch display device 300 providing a two-dimensional rotary organizational tool that enables manipulations similar to functionality provided by a Lazy Susan. As illustrated inFIGS. 3( a) and 3(b),organizational tool 324 includes ancontrol strip 328. As will be described in greater detail below, thecontrol strip 328 provides a mechanism for manipulatingorganizational tool 324. Similar to the control strip described above with regard to the rectangular two-dimensional organizational tool, in some implementations,control strip 328 will appear on top of objects attached toorganizational tool 324 in order to provide a convenient way to manipulateorganizational tool 324. For example, as illustrated inFIGS. 3( a) and 3(b), objects 306 and 308 are attached toorganizational tool 324 and appear undercontrol strip 328. - In certain implementations,
multi-touch display device 300 may interpret input associated withcontrol strip 328 differently from input associated with other parts oforganizational tool 324. For instance,multi-touch display device 300 may interpret a single, moving input associated with a part oforganizational tool 324 other thancontrol strip 328 as a request to move the entirety oforganizational tool 324 in correspondence with the detected input, in a fashion similar to that discussed above with regard toFIG. 1F . However,multi-touch display device 300 may interpret a single, moving input associated with thecontrol strip 328 oforganizational tool 324 as a request to rotateorganizational tool 324 about its center. An input need not be directly at a point corresponding to controlstrip 328 in order formulti-touch display device 300 to associate the input with thecontrol strip 328. Rather, the input may correspond to a point within a predetermined distance ofcontrol strip 328. - Referring to
FIG. 3( a), a user is engaging the surface ofmulti-touch display device 300 with afinger 302 at a point corresponding to controlstrip 328.Multi-touch display device 300 detects the input by the user at point “a” and associates the input withcontrol strip 328 oforganizational tool 324. Referring now toFIG. 3( b), as the user dragsfinger 302 in a counterclockwise fashion across the surface ofmulti-touch display device 300,multi-touch display device 300 detects the movement of the user'sfinger 302 from point “a” to point “b,” interprets the detected movement as a request to rotateorganizational tool 324, and rotatesorganizational tool 324 by the detected distance or some function of the detected distance. Becauseobjects organizational tool 324,multi-touch display device 300 translatesobjects organizational tool 324 such thatobjects organizational tool 324. In so doing, themulti-touch display device 300 also preserves the spatial relationships betweenobjects - The following section describes the rectangular two-dimensional organizational tool, described above in
FIGS. 1A-1K , in greater detail. In particular, various methods for attaching objects to the organizational tool, detaching objects from the organizational tool, manipulating the organizational tool, and automatically arranging objects attached to the organizational tool will be discussed. - As discussed above, objects can be attached to a rectangular two-dimensional organizational tool, for example, for organizational and/or storage purposes. The multi-touch display device may provide various different techniques for attaching an object to the organizational tool.
-
FIG. 4A is aflowchart 470 of an example of a process for attaching an object to a rectangular two-dimensional organizational tool. The process illustrated inflowchart 470 may be performed by, for example, themulti-touch display device 100 ofFIGS. 1A-1K , themulti-touch display device 200 ofFIGS. 2( a)-2(b), or themulti-touch display device 300 ofFIGS. 3( a)-3(b). - Multiple objects are displayed by a multi-touch display device (472). In addition, the multi-touch display device displays an organizational tool (i.e., a target element) to facilitate the organization and/or storage of one or more displayed objects (474).
- Next, the organizational tool may be activated for attachment (476). When the organizational tool is not activated, the multi-touch display device may not permit objects to be attached to the organizational tool. In certain implementations, only when the multi-touch display device activates the organizational tool will the multi-touch display device attach objects to the organizational tool. In order to activate the organizational tool, the multi-touch display device may, for example, detect a predetermined sequence of inputs to a predetermined portion of the surface of the multi-touch display device (e.g., detecting an input to the surface of the multi-touch display device corresponding to a displayed button or detecting two quick tap inputs to the surface of the multi-touch display device corresponding to the control strip of the organizational tool being activated). Alternatively, a physical switch located either on the multi-touch display device itself or a remote control may be engaged in order for the multi-touch display device to activate the organizational tool. It is also contemplated that the organizational tool may always be activated for the purpose of attaching objects, in which case sub-process 476 need not be performed.
- After the organizational tool has been activated for attachment, to the extent necessary, the multi-touch display device detects that an input mechanism has engaged the surface of the multi-touch display device at a point corresponding to where the organizational tool or one of the displayed objects is displayed (478). In response, the multi-touch display device monitors movement by the input mechanism while the input mechanism remains engaged with the surface (480).
- As the input mechanism moves, the multi-touch display device interprets the detected movement as a request to translate the corresponding object or organizational tool, and updates the position of the object or organizational tool as a function of the detected movement (482). As long as the multi-touch display device continues to detect the input from the input mechanism, it will continue to monitor the movement of the input mechanism (484). When the multi-touch display device detects that the input mechanism has disengaged the surface of the multi-touch display device, the multi-touch display device determines whether the translated object overlaps the organizational tool or, in the case where the organizational tool is translated, whether any object not already attached to the organizational tool overlaps the organizational tool (486).
- If no currently displayed objects visually overlap the organizational tool, no objects are attached to the organizational tool (488). In contrast, if one or more displayed objects are determined to be overlapping the organizational tool, the multi-touch display device determines whether the disengagement of the surface of the multi-touch display device by the input mechanism should trigger an attachment of the object(s) overlapping the organizational tool to the organizational tool (490).
- In certain implementations, the determination of whether the disengagement should trigger an attachment is based on a stored rule or set of rules. For example, a rule may specify that a disengagement of the surface while the multi-touch display device concurrently detects an engagement by one or more separate input in a predetermined manner corresponding to the organizational tool will trigger an attachment of the object(s) overlapping the organizational tool to the organizational tool. Stated differently, the rule may require that the multi-touch display device detect that a user has continuously engaged the organizational tool through, for example, the control strip concurrent with an object being dragged over the organizational tool and released. An example of this rule is described in greater detail in connection with regard
FIG. 4C , below. - Alternatively or additionally, a rule may specify that a disengagement of the surface preceded by an engagement of the surface at a pressure greater than a predetermined threshold and at a point corresponding to where the object overlaps the organizational tool will trigger an attachment of the object(s) overlapping the organizational tool to the organizational tool. Stated differently, the rule may require that the multi-touch display device detect that the user has engaged an object, dragged the object over the organizational tool, and pressed down with increased pressure on the surface of the multi-touch display device before disengaging the surface. An example of this rule is described in greater detail with regard to
FIG. 4D , below. - Alternatively or additionally, a rule may specify that a disengagement of the surface preceded by an engagement associated with only the organizational tool and at a pressure greater that a predetermined threshold, such that the organizational tool is “pressed below” the level of the objects to be attached to the organizational tool, will trigger an attachment of the object(s) overlapping the organizational tool to the organizational tool. An example of this rule is described in greater detail with regard to
FIG. 4E , below. - If the multi-touch display device determines that the object(s) that overlap the organizational tool should not be attached to the organizational tool, no objects will be attached to the organizational tool as a result of the disengagement of the surface of the multi-touch display device by the input mechanism. In contrast, if the multi-touch display device determines that the object(s) that overlap the organizational tool should be attached to the organizational tool, the multi-touch display device invokes a process to attach the object(s) that overlap the organizational tool to the organizational tool such that future manipulations applied to the organizational tool also may be applied to the attached object(s) as a consequence of their attachment to the organization tool (492).
-
FIGS. 4B-4E illustrate four variations ofattachment process 470. In FIGS. 4B(a), 4C(a), 4D(a), and 4E(a),multi-touch display device 400 displays objects 402, 404, 406, and 408 andorganizational tool 410.Organizational tool 410 includescontrol strip 412. Furthermore, object 408 has already been attached toorganizational tool 410. - In some implementations, the multi-touch display device may require some sort of affirmative action on behalf of a user before attaching an object to a grouping tool so as to avoid inadvertently attaching an object when such an attachment is not desired. In such implementations, absent detecting the required affirmative action, the multi-touch display device may not attach an object to the grouping tool even when the object is translated to a position overlapping the grouping tool and then released. Referring to FIG. 4B(b), a user engages the surface of
multi-touch display device 400 at a point corresponding to object 402, drags finger 403 over the surface, and disengages the surface at a point whereobject 402 overlapsorganizational tool 410. -
Multi-touch display device 400 detects this input by finger 403, associates the input withobject 402, interprets the input as a request to translateobject 402, and translatesobject 402 as a function of the movement of the user's finger. In effect,multi-touch display device 400 updates the display ofobject 402 such that the finger 403 appears to remain engaged with the same point on the display even as the finger 403 moves about. After detecting the disengagement of finger 403,multi-touch display device 400 determines whether the translated object visually overlaps theorganizational tool 410. Here,multi-touch display device 400 detects thatobject 402 overlapsorganizational tool 410, and therefore determines whether the disengagement of finger 403 should trigger an attachment of theobject 402 toorganizational tool 410. In this example,multi-touch display device 400 employs a rule or set of rules to determine whether an attachment should be made. Additionally, the rule or set of rules have not been met, somulti-touch display device 400 does not attachobject 402 toorganizational tool 410. - In some implementations, a multi-touch display device may employ an attachment rule whereby the multi-touch display device attaches an object to an organizational tool in response to detecting that that a user has relinquished control of the object while the object is located at a position over the organization tool and while the control strip is engaged by an input mechanism. Referring to FIGS. 4C(a)-4C(b), user 414 has engaged the surface of
multi-touch display device 400 withfinger 418 at a point corresponding toorganizational tool 410. In addition, while continuing to engageorganizational tool 410, user 414 also engages the surface ofmulti-touch display device 400 withfinger 416 at a point corresponding to object 402 and dragsfinger 416 across the surface of the screen. Themulti-touch display device 400 detects the movement offinger 416, and translatesobject 402 to a point whereobject 402 overlapsorganizational tool 410 in response. - Thereafter,
finger 416 disengages the surface of themulti-touch display device 400. In response to detecting thatfinger 416 has disengaged the surface ofmulti-touch display device 400,multi-touch display device 400 determines whetherobject 402 is overlappingorganizational tool 410. As a consequence of detecting thatobject 402 overlapsorganizational tool 410,multi-touch display device 400 then determines whetherobject 402 should be attached toorganizational tool 410. Becausefinger 418 was engagingorganizational tool 400 at the time whenfinger 416 relinquished control of object 402 (or within a threshold period of time within the time at whichfinger 416 relinquished control of object 402),multi-touch display device 400 determines thatobject 402 should be attached toorganizational tool 410. Therefore,multi-touch display device 400 attachesobject 402 toorganizational tool 410. - In some implementations, a multi-touch display device may employ an attachment rule whereby the multi-touch display device attaches an object to an organizational tool in response to detecting that that a user has relinquished control of the object while the object is located at a position over the organization tool and after pressing down on the surface of the multi-touch display device at a point where the object overlaps the organizational tool. Referring to FIGS. 4D(a)-4D(b), user 414 has engaged the surface of
multi-touch display device 400 withfinger 416 at a point corresponding to object 402 and dragsfinger 416 across the surface of the screen. Themulti-touch display device 400 detects the movement of 416, and translatesobject 402 to a point whereobject 402 overlapsorganizational tool 410 in response. - Thereafter,
finger 416 presses down more firmly on the surface ofmulti-touch display device 400 and then disengages the surface. In response to detecting thatfinger 416 has disengaged the surface ofmulti-touch display device 400,multi-touch display device 400 determines whetherobject 402 is overlappingorganizational tool 410. As a consequence of detecting thatobject 402 overlapsorganizational tool 410,multi-touch display device 400 then determines whetherobject 402 should be attached toorganizational tool 410. Becausefinger 416 more firmly engaged the surface of multi-touch display device 400 (e.g., where the absolute contact pressure or the differential in contact pressure between the initial engagement and the pressure when the object overlapsorganizational tool 410 exceeds a predetermined threshold) before disengaging,multi-touch display device 400 determines thatobject 402 should be attached toorganizational tool 410. Therefore,multi-touch display device 400 attachesobject 402 toorganizational tool 410. - In some implementations, the
multi-touch display device 400 may employ an attachment rule that enables objects to be attached to the organizational tool as a consequence of the translation of the organizational tool as opposed to the translation of the object(s) to be attached. For example, the multi-touch display device may provide for the attachment of objects to the organizational tool by enabling a user to depress the organizational tool to a visual layer that is beneath the layer at which the object to be attached is displayed and then “dragging” the organizational tool beneath the object to be attached. Referring to FIGS. 4E(a)-4E(d), user 414 has engaged the surface ofmulti-touch display device 400 withfinger 416 at a point corresponding toorganizational tool 410 and dragsfinger 416 across the surface of the screen. Themulti-touch display device 400 detects the movement of 416, and translatesorganizational tool 410 to a point whereobjects organizational tool 410 in response. - In certain implementations,
multi-touch display device 400 may detect that the pressure with whichfinger 416 has engaged the surface exceeds a predetermined threshold in order to “drag”organizational tool 410 beneathobjects multi-touch display device 400 detects that the pressure does not exceed the threshold, it will translateorganizational tool 410 such thatorganizational tool 410overlaps objects organizational tool 410 is engaged,multi-touch display device 400 will “drag”organizational tool 410 beneathobjects - After translating
organizational tool 410,finger 416 disengages the surface ofmulti-touch display device 400. In response to detecting thatfinger 416 has disengaged the surface ofmulti-touch display device 400,multi-touch display device 400 determines whether any objects overlaporganizational tool 410. As a consequence of detecting thatobject organizational tool 410,multi-touch display device 400 then determines whetherobjects organizational tool 410. Becausefinger 416 engaged the surface ofmulti-touch display device 400 at a pressure necessary to “drag”organizational tool 410 beneath the objects before disengaging,multi-touch display device 400 determines thatobjects organizational tool 410. Therefore,multi-touch display device 400 attachesobjects organizational tool 410. - In some implementations, a multi-touch display device may enable a user to impart motion to a displayed object and, thereafter, the multi-touch display device may maintain the object in motion even after the user has relinquished control of the object by disengaging the multi-touch display device. Stated differently, the multi-touch display device may enable user to “flick” or “fling” an object across the multi-touch display device by initially engaging the surface of the multi-touch display device and then disengaging the surface of the multi-touch display device while the object is in motion. In such implementations, the multi-touch display device may employ an attachment rule whereby the multi-touch display device attaches objects to an organizational tool when the multi-touch display device detects that objects originally set into motion by a user have come to rest over an organizational tool. Furthermore, in such implementations, the multi-touch display device may apply friction to objects as they move across the multi-touch display device, and, in order to depict the organizational tools as “sticky” surfaces to which the objects can attach, the multi-touch display device may apply greater friction to objects as they move over an organizational tool than when the objects move over the canvas.
-
FIG. 4F is aflowchart 440 of a process for attaching objects that have been set into motion to an organizational tool. The process illustrated inflowchart 440 may be performed by, for example, themulti-touch display device 100 ofFIGS. 1A-1K , themulti-touch display device 200 ofFIGS. 2( a)-2(b), themulti-touch display device 300 ofFIGS. 3( a)-3(b), or themulti-touch display device 400 ofFIGS. 4B-4E . - Multiple objects are displayed by a multi-touch display device (442). In addition, the multi-touch display device displays an organizational tool (i.e., a target element) to facilitate the organization and/or storage of one or more displayed objects (444). Moreover, the multi-touch display device defines a coefficient of friction for each of the organizational tool and the canvas (446). These coefficients of friction are used to determine the movement of objects across the screen, described in greater detail below. In certain embodiments, the multi-touch display device defines a greater coefficient of friction for the organizational tool than the canvas in order to simulate that organizational tool is “stickier” than the canvas.
- Next, the organizational tool may be activated for attachment (448). The multi-touch display device may activate the organizational tool for attachment utilizing methods similar to those discussed above with regard to
step sub-process 476. It is also contemplated that the organizational tool may always be activated for the purpose of attaching objects, in whichcase step sub-process 448 need not be performed. - After the organizational tool has been activated for attachment, to the extent necessary, the multi-touch display device detects that an input mechanism has engaged the surface of the multi-touch display device at a point corresponding to where the organizational tool or one of the displayed objects is displayed (450). In response, the multi-touch display device monitors movement by the input mechanism while the input mechanism remains engaged with the surface (452).
- As the input mechanism moves, the multi-touch display device interprets the detected movement as a request to translate the corresponding object or organizational tool, and updates the position of the object or organizational tool as a function of the detected movement (454). As long as the multi-touch display device continues to detect the input from the input mechanism, it will continue to monitor the movement of the input mechanism (456). When the multi-touch display device detects that the input mechanism has disengaged the surface of the multi-touch display device, the multi-touch display device maintains the motion of the object as a function of the motion imparted by the input and the coefficient of friction of the surface(s) that the object overlaps (458). Stated differently, upon detecting the disengagement, the multi-touch display device continues updating the position of the object with the velocity the object had immediately prior to the disengagement, while slowing the velocity of the object based on the coefficient of friction of the surface(s) that the object overlaps. While the object is in motion, the multi-touch display device continuously determines the coefficient of friction to apply to the motion based on whether the object overlaps the canvas and/or the organizational tool. If the object solely overlaps either of the canvas or the organizational tool, the corresponding coefficient of friction is applied. If the object overlaps both the canvas and the organizational tool, the greater of the two coefficients of friction is applied.
- The multi-touch display device continuously decreases the stored velocity of the moving object with regard to the applied coefficient of friction, while continuously updating the display of the object with regard to the velocity, until the velocity reaches zero (460). Once the object comes to rest, the multi-touch display device determines whether the translated object visually overlaps the organizational tool (462). If the translated object does not visually overlap the organizational tool, no objects are attached to the organizational tool (464). In contrast, if the translated object is determined to be overlapping the organizational tool, the multi-touch display device determines whether the object coming to rest should trigger an attachment of the object (466). In certain implementations, the determination of whether the disengagement should trigger an attachment is based on a stored rule or set of rules. Rules similar to those discussed above with regard to 470 may be utilized.
- If the multi-touch display device determines that the object(s) that overlap the organizational tool should not be attached to the organizational tool, no objects will be attached to the organizational tool as a result of the disengagement of the surface of the multi-touch display device by the input mechanism. In contrast, if the multi-touch display device determines that the object(s) that overlap the organizational tool should be attached to the organizational tool, the multi-touch display device invokes a process to attach the object(s) that overlap the organizational tool to the organizational tool such that future manipulations applied to the organizational tool also may be applied to the attached object(s) as a consequence of their attachment to the organization tool (468).
-
FIGS. 4G-4H illustrate two variations ofattachment process 440. In FIGS. 4G(a) and 4H(a),multi-touch display device 400 displays objects 402, 404, 406, and 408 andorganizational tool 410 oncanvas 418.Object 408 has already been attached toorganizational tool 410. - In some implementations, the
multi-touch display device 400 may define a greater coefficient of friction for the organizational tool than for the canvas in order to simulate that the organizational tool is “stickier” than the canvas. Thus, as an object in motion that has been released begins to overlap the organizational tool,multi-touch display device 400 slows the motion of the object at a greater rate, as if the object were encountering a stickier surface. Referring to FIG. 4G(b), a user engages the surface ofmulti-touch display device 400 at a point corresponding to object 402, dragsfinger 416 over the surface, and disengages the surface at a point whereobject 402 only overlapscanvas 418. -
Multi-touch display device 400 detects this input byfinger 416, associates the input withobject 402, interprets the input as a request to translateobject 402, and translatesobject 402 as a function of the movement of the user's finger. In effect,multi-touch display device 400 updates the display ofobject 402 such that the point at which the user originally engages theobject 402 continues to be displayed at the point at which the input is currently detected. After detecting the disengagement offinger 416,multi-touch display device 400 maintains the motion ofobject 402 as a function of the velocity imparted by the input prior to disengagement and the coefficient of friction ofcanvas 418, which object 402 currently visually overlaps. Thus,multi-touch display device 400 slowsobject 402 with regard to the defined coefficient of friction ofcanvas 418. Referring to FIG. 4G(c), asobject 402 encountersorganizational tool 410, beginning to visually overlap it,multi-touch display device 400 begins to update the motion ofobject 402 with regard to the coefficient of friction oforganizational tool 410, because this coefficient of friction is greater than that of thecanvas 418. Thus,multi-touch display device 400 slowsobject 402 at an even faster rate. - Referring to FIG. 4G(d), once
object 402 has come to rest,multi-touch display device 400 determines whetherobject 402 visually overlaps theorganizational tool 410. Here,multi-touch display device 400 detects thatobject 402 overlapsorganizational tool 410, and therefore determines whetherobject 402 coming to rest should trigger an attachment of theobject 402 toorganizational tool 410. As described above, in certain implementations,multi-touch display device 400 employs a rule or set of rules to determine whether an attachment should be made. In this example, the rule to determine whether an attachment should be made is simply whether the object overlaps the organizational tool. Becauseobject 402 overlapsorganizational tool 410,multi-touch display device 400 determines thatobject 402 should be attached toorganizational tool 410. Therefore,multi-touch display device 400 attachesobject 402 toorganizational tool 410. - In some implementations, the
multi-touch display device 400 may define an equal coefficient of friction for both the organizational tool and the canvas. Thus, as an object in motion that has been released begins to overlap the organizational tool,multi-touch display device 400 will not change the rate at which the object is slowed. Referring to FIG. 4H(b), a user engages the surface ofmulti-touch display device 400 at a point corresponding to object 402, dragsfinger 416 over the surface, and disengages the surface at a point whereobject 402 only overlapscanvas 418. -
Multi-touch display device 400 detects this input byfinger 416, associates the input withobject 402, interprets the input as a request to translateobject 402, and translatesobject 402 as a function of the movement of the user's finger. In effect,multi-touch display device 400 updates the display ofobject 402 such that the point at which the user originally engages theobject 402 continues to be displayed at the point at which the input is currently detected. After detecting the disengagement offinger 416,multi-touch display device 400 maintains the motion ofobject 402 as a function of the velocity imparted by the input prior to disengagement and the coefficient of friction ofcanvas 418, which object 402 currently visually overlaps. Thus,multi-touch display device 400 slowsobject 402 with regard to the defined coefficient of friction ofcanvas 418. Referring to FIG. 4H(c), asobject 402 encountersorganizational tool 410, beginning to visually overlap it,multi-touch display device 400 updates the motion ofobject 402 with regard to the coefficient of friction oforganizational tool 410. However, because the coefficient of friction ofcanvas 418 andorganizational tool 410 are the same,multi-touch display device 400 slowsobject 402 at the same rate. In this way,multi-touch display device 400 continues the motion ofobject 402 for a greater period of time than would have been the case with regard toFIG. 4G . - Referring to FIG. 4H(d), once
object 402 has come to rest,multi-touch display device 400 determines whetherobject 402 visually overlaps theorganizational tool 410. Because the coefficient of friction does not change between the surfaces,multi-touch display device 400 may continue the motion ofobject 402 pastorganizational tool 410, such thatobject 402 does not overlaporganizational tool 410 when it comes to rest. Here, however,multi-touch display device 400 detects thatobject 402 overlapsorganizational tool 410, and therefore determines whetherobject 402 coming to rest should trigger an attachment of theobject 402 toorganizational tool 410. As described above, in certain implementations,multi-touch display device 400 employs a rule or set of rules to determine whether an attachment should be made. In this example, the rule to determine whether an attachment should be made is simply whether the object overlaps the organizational tool. Becauseobject 402 overlapsorganizational tool 410,multi-touch display device 400 determines thatobject 402 should be attached toorganizational tool 410. Therefore,multi-touch display device 400 attachesobject 402 toorganizational tool 410. -
FIG. 5A is aflowchart 500 of an example of a process for detaching an object from a rectangular two-dimensional organizational tool. The process illustrated inflowchart 500 may be performed by, for example, themulti-touch display device 100 ofFIGS. 1A-1K , themulti-touch display device 200 ofFIGS. 2( a)-2(b), themulti-touch display device 300 ofFIGS. 3( a)-3(b), or themulti-touch display device 300 ofFIGS. 4B-4E and 4G-4H. - The multi-touch display device detects that an input mechanism has engaged the surface of the multi-touch display device at a point corresponding to where the organizational tool or one of the displayed objects is displayed (502). In response, the multi-touch display device determines whether any future movements of the input mechanism should trigger a detachment of the object(s) corresponding to the input (504).
- In certain implementations, the determination of whether any future movements of the input mechanism should trigger a detachment is based on a stored rule or set of rules. For example, a rule may specify that an initial engagement of the surface corresponding to the organizational tool or one of the displayed objects while the multi-touch display device concurrently detects an engagement by one or more separate inputs in a predetermined manner corresponding to the organizational tool will trigger a detachment of the object(s) from the organization tool. Stated differently, the rule may require that the multi-touch display device detect that a user has continuously engaged the organizational tool through, for example, the control strip concurrent with an object being dragged over the organizational tool and released. An example of this rule is described in greater detail in connection with regard
FIG. 5D , below. - Alternatively or additionally, a rule may specify that corresponding to the organizational tool at a pressure greater that a predetermined threshold, such that the organizational tool is “pressed below” the level of the objects attached to the organizational tool, will trigger a detachment of the object(s) attached to the organizational tool. An example of this rule is described in greater detail with regard to
FIG. 5E , below. - If the multi-touch display device determines that the object(s) attached to the organizational tool should not be detached from the organizational tool, no objects will be detached from the organizational tool as a result of the engagement of the surface of the multi-touch display device by the input mechanism. In contrast, if the multi-touch display device determines that the object(s) attached to the organizational tool should be detached from the organizational tool, the multi-touch display device monitors the movement of input mechanism while the input mechanism remains engaged with the surface (506). As long as the multi-touch display device continues to detect the input from the input mechanism, it will continue to monitor the movement of the input mechanism. Once it detects a predetermined type and amount of movement of the input mechanism, the multi-touch display device invokes a process to detach the object(s) attached to the organizational tool such that future manipulations applied to the organizational tool will not be applied to the object(s) as a consequence of their detachment from the organization tool (508).
-
FIGS. 5B-5F illustrate five variations ofdetachment process 500. In FIGS. 5B(a), 5C(a), 5D(a), 5E(a), and 5F(a)multi-touch display device 510 displays objects 512, 514, 516, and 518 andorganizational tool 520.Object organizational tool 520. - In some implementations, the multi-touch display device may require some sort of affirmative action on behalf of a user before detaching an object from an organizational tool so as to avoid inadvertently detaching the object when such a detachment is not desired. In such implementations, absent detecting the required affirmative action, the multi-touch display device may not detach an object from the organizational tool even when an input corresponds to the object and is moved away from the organizational tool. Referring to FIG. 5B(b), a user engages the surface of
multi-touch display device 510 at a first point corresponding to object 512, and dragsfinger 522 over the surface, from the first point to a second point. -
Multi-touch display device 510 detects this input byfinger 522 and associates the input withobject 512. After detecting the engagement offinger 522,multi-touch display device 510 determines whether any future movements offinger 522 should trigger a detachment ofobject 512. In this example,multi-touch display device 510 employs a rule or set of rules to determine whether an detachment should be made, however, the rule or set of rules have not been met, somulti-touch display device 510 does not detachobject 512 fromorganizational tool 520. Becausemulti-touch display device 510 does not detachobject 512,multi-touch display device 510 interprets movement of the detected input as a request to translateobject 512 by a distance corresponding to the distance from the first point to the second point. However, asobject 512 continues to be attached toorganizational tool 520,multi-touch display device 510 will translateorganizational tool 520, and any other objects attached to it, in correspondence with any translation ofobject 512. - In some implementations, a multi-touch display device may employ a detachment rule whereby the multi-touch display device detaches an object from an organizational tool in response to detecting that a user has engaged the object in a predetermined manner. The predetermined manner of input may include, for example, a single point engagement of the surface of the multi-touch display device at a point corresponding to an attached object, or a predefined series of inputs at a predetermined point on an attached object. Referring to FIG. 5C(a), a user has engaged the surface of
multi-touch display device 510 withfinger 522 at a first point corresponding to object 512. Themulti-touch display device 510 detects the input offinger 522, and associates the input withobject 512. - After detecting the engagement of
finger 522,multi-touch display device 510 determines whether any future movements of thefinger 522 should trigger a detachment ofobject 512. Becausefinger 522 engages attachedobject 512 as a singular input,multi-touch display device 510 determines thatobject 512 should be detached fromorganizational tool 520. Thereafter, in reference to FIG. 5C(b), the user dragsfinger 522 across the surface of themulti-touch display device 510 in a rightward motion from the first point to a second point. In response to detecting the movement offinger 522,multi-touch display device 510 translatesobject 512 by a distance corresponding to the distance from the first point to the second point. In some implementations,multi-touch display device 510 detaches object 512 fromorganizational tool 520 whenmulti-touch display device 510 detects thatfinger 522 disengages the surface ofmulti-touch display device 510 and determines thatobject 512 no longer overlapsorganizational tool 520. In other implementations, themulti-touch display device 510 detaches object 512 fromorganizational tool 520 whenmulti-touch display device 510 detects any movement offinger 522 after determining that the movement should trigger a detachment ofobject 512. - In some implementations, a multi-touch display device may employ an detachment rule whereby the multi-touch display device detaches an object from an organizational tool in response to detecting that that a user has engaged the object while the object is attached to the organization tool and while the control strip is engaged by another input mechanism. Referring to FIG. 5D(a), a user has engaged the surface of
multi-touch display device 510 withfinger 522 at a first point corresponding toorganizational tool 520. In addition, while continuing to engageorganizational tool 520, the user also engages the surface ofmulti-touch display device 510 withfinger 524 at a point corresponding to object 512. Themulti-touch display device 510 detects the input offinger 522 and associates it withorganizational tool 520, and detects the input offinger 524 and associates it withobject 512. - After detecting the engagement of
finger 524,multi-touch display device 510 determines whether any future movements of thefinger 524 should trigger a detachment ofobject 512. Becausefinger 522 was engagingorganizational tool 510 at the time whenfinger 524 engaged object 512 (or within a threshold period of time within the time at whichfinger 524 engaged object 512),multi-touch display device 510 determines thatobject 512 should be detached fromorganizational tool 520. Thereafter, in reference to FIG. 5D(b), the user dragsfinger 524 across the surface of themulti-touch display device 510 in a rightward motion from the first point to a second point. In response to detecting the movement offinger 524,multi-touch display device 510 translatesobject 512 by a distance corresponding to the distance from the first point to the second point, and detachesobject 512. - In some implementations, the
multi-touch display device 510 may employ an detachment rule that enables objects to be detached from the organizational tool as a consequence of the translation of the organizational tool as opposed to the translation of the object(s) to be detached. For example, the multi-touch display device may provide for the detachment of objects from the organizational tool by enabling a user to depress the organizational tool to a visual layer that is beneath the layer at which the object(s) attached to the organizational tool are displayed and then “dragging” the organizational tool from under the object(s) to be detached. Referring to FIG. 5E(a), a has engaged the surface ofmulti-touch display device 510 withfinger 522 at a first point corresponding toorganizational tool 520. Themulti-touch display device 510 detects the input offinger 522, and associates the input withorganizational tool 520. - After detecting the engagement of
finger 522,multi-touch display device 510 determines whether any future movements of thefinger 522 should trigger a detachment of theobjects multi-touch display device 510 may detect whether the pressure with whichfinger 522 has engaged the surface exceeds a predetermined threshold in order to determine if any future movements of thefinger 522 should trigger a detachment of theobjects multi-touch display device 510 detects that the pressure does not exceed the threshold, it may translateorganizational tool 510 without detachingobjects objects organizational tool 520 is engaged,multi-touch display device 510 determines, regardless of the detected pressure, that any future movements of thefinger 522 should trigger a detachment of the attached objects. - In this example, because
multi-touch display device 510 detects that the pressure exceeds the predetermined threshold,multi-touch display device 510 determines that any future detected movements offinger 522 should trigger a detachment of theobjects finger 522 across the surface of themulti-touch display device 510 in a rightward motion from the first point to a second point. In response to detecting the movement offinger 524,multi-touch display device 510 translatesorganizational tool 520 by a distance corresponding to the distance from the first point to the second point. In certain implementations, oncemulti-touch display device 510 detects thatfinger 522 has disengaged the surface,multi-touch display device 510 may determine whether any objects not already attached toorganizational tool 520 overlaporganizational tool 520. If any non-attached objects do overlaporganizational tool 520,multi-touch display device 510 may attach the objects overlappingorganizational tool 520. In other implementations, oncemulti-touch display device 510 detects thatfinger 522 has disengaged the surface,multi-touch display device 510 will not attach any objects toorganizational tool 520, regardless of whether they overlaporganizational tool 520. - In some implementations, a multi-touch display device may employ an detachment rule whereby the multi-touch display device detaches object(s) from an organizational tool in response to detecting that that a user has engaged the organizational control at a specific point or region. Referring to FIG. 4F(a), a user has engaged the surface of
multi-touch display device 510 withfinger 522 at a point corresponding toorganizational tool 520, and more specifically at a point corresponding to a predetermined input target or button (e.g., a “Release All” button) onorganizational tool 520.Multi-touch display device 510 detects the input offinger 522 and associates it withorganizational tool 520. - After detecting the engagement of
finger 522,multi-touch display device 510 determines whether the engagement, disengagement, or future movement offinger 522 should trigger a detachment of theobjects finger 522 engagesmulti-touch display device 510 at a point corresponding to a “Release All” button,multi-touch display device 510 determines that the engagement offinger 522 should trigger a detachment of all attachedobjects organizational tool 520,multi-touch display device 510 may translate each ofobjects organizational tool 520. In such implementations,multi-touch display device 510 may also rotate each detached object such that the angular orientation of each detached object corresponds with the angular orientation ofmulti-display device 510. - In other implementations, as a result of triggering the detachment of all attached objects from
organizational tool 520,multi-touch display device 510 detachesobjects organizational tool 520 orobjects organizational tool 520, but are not attached toorganizational tool 520 for purposes of manipulation of eitherorganizational tool 520 or the objects themselves. - In additional implementations, as a result of triggering the detachment of all attached objects from
organizational tool 520,multi-touch display device 510 may, as shown in FIG. 5F(c), detachobjects organizational tool 520. In such an implementation,multi-touch display device 510 will continue to display 512, 514, 516, and 518 at the positions at which each object was displayed prior to being detached, howeverorganizational tool 520 will no longer be displayed. - As discussed above, one effect of an object being attached to an organizational tool is that when the multi-touch display device applies transformations to the organizational tool, the multi-touch display device also may apply transformations to the objects attached to the organizational tool as a consequence of their attachment. However, when the multi-touch display device applies a transformation to an organization tool, the transformations that the multi-touch display device applies to objects attached to the organizational tool may depend upon how the objects are attached to the organizational tool.
- In some cases, the multi-touch display device may attach an object to an organizational tool at only a single point. In such cases, transformations applied to the organizational tool only impact an attached object (i.e., cause a corresponding transformation to be applied to the attached object) if the transformations applied to the organizational tool impact the point upon the organizational tool at which the object is attached. Alternatively, in other cases, the multi-touch display device may attach an object to an organization tool at two or more points. In such cases, transformations applied to the organizational tool will be applied equally to the attached object.
- The manner in which all objects are attached to a organizational tool may be set for a given organizational tool by the user or they may be automatically preset upon creation by the multi-touch display device. Alternatively, the manner in which objects are attached to a organizational tool may be detected separately for each object upon being attached to the organizational tool.
-
FIG. 6A illustrates a multi-touch display device performing transformations to an organizational tool and its attached object when the attached objects are attached to the organizational tool at two or more points. As illustrated in FIG. 6A(a), objects 602, 604, 606, and 608 each are attached toorganizational tool 610 at two or more points. Therefore, when themulti-touch display device 600 rotatesorganizational tool 610 in response to detecting input byfingers multi-touch display device 600 corresponding rotates each of displayedobjects -
FIG. 6B illustrates a multi-touch display device performing transformations to an organizational tool and its attached object when the attached objects are attached to the organizational tool at only one point. As illustrated in FIG. 6B(a), objects 602, 604, 606, and 608 each are attached toorganizational tool 610 at one point. Therefore, when themulti-touch display device 600 rotatesorganizational tool 610 in response to detecting input byfingers multi-touch display device 600 corresponding translates each of displayedobjects organizational tool 610 impacts the point upon theorganizational tool 610 at which objects 602, 604, 606, and 608 are attached. - In some implementations, a multi-touch display device may provide controls for further organizing objects attached to an organizational tool provided by the multi-touch display device even after the objects have been attached to the organizational tool. For example, the multi-touch display device may attach an object to the organizational tool such that the object is attached to the organizational tool at the position occupied by the object on the organizational tool at the point in time when the multi-touch display device determined to attach the object to the organizational tool irrespective of how many other objects also are attached to the organizational tool at the same or similar positions. Furthermore, the multi-touch display device may preserve the angular orientation of the an object upon attaching the object to an organizational tool. This may lead to a cluttering of the organizational tool as more and more objects are attached to the organizational tool. Therefore, the multi-touch display device may provide controls for rearranging the objects attached to an organizational tool into a more organized fashion.
-
FIG. 7A is aflowchart 750 of an example of a process for organizing objects attached to a rectangular two-dimensional organizational tool. The process illustrated inflowchart 750 may be performed by, for example, themulti-touch display device 100 ofFIGS. 1A-1K , themulti-touch display device 200 ofFIGS. 2( a)-2(b), themulti-touch display device 300 ofFIGS. 3( a)-3(b), themulti-touch display device 300 ofFIGS. 4B-4E and 4G-4H, themulti-touch display device 510 ofFIGS. 5B-5F , or themulti-touch display device 600 ofFIGS. 6A-6B . - The multi-touch display device identifies a current orientation of a reference axis of the organizational tool (752). In some implementations, the multi-touch display device may determine the reference axis based on a vertical and/or horizontal axis of the organizational tool, stored and updated by the multi-touch display device. Alternatively, the multi-touch display device may determine the reference axis based on a comparison of multiple predetermined points of the organizational tool with regard to multiple predetermined points of the display surface of the multi-touch display device. Alternatively, the multi-touch display device may determine the reference axis by detecting an input from a user indicating the reference axis.
- Subsequently, the multi-touch display device identifies a current orientation of the reference axis of each object attached to the organizational tool (754). In order to determine the reference axis for a given object, the multi-touch display device may utilize similar methods as discussed above with regard to determining the reference axis of the organizational tool. Next, the multi-touch display device adjusts the orientation of each object attached to the organizational tool being organized, such that the reference axes of the attached objects align with the reference axis of the organizational tool (756). The multi-touch display device may align the reference axis of a given object with the reference axis of the organizational tool by, for example, rotating the object until the reference axis of the object and the reference axis of the organizational tool are parallel. Alternatively, the multi-touch display device may, for example, rotate the object until the reference axis of the object and the reference axis of the organizational tool are offset by a predefined angle.
- Referring to
FIG. 7B ,multi-touch display device 700 partitions organizational tool 701 into enough sections to accommodate the number of attached objects.Multi-touch display device 700 assigns each attached object a section. Asmulti-touch display device 700 attaches a new object,multi-touch display device 700 may automatically move the newly attached object to the closest free section, if one is available. Alternatively,multi-touch display device 700 may give the newly attached object priority to the closest section, free or not, and, if the closest section is not available, move the object attached in that section to the closest free section in order to accommodate the newly attached object. If, in either case, a free section is not available,multi-touch display device 700 may resize all of the objects attached to organizational tool 701 such that organizational tool 701 may be repartitioned in order to create a free section for all attached objects. - FIG. 7B(a) illustrates how
multi-touch display device 700 has partitioned organizational tool 701 into six sections to accommodateobjects multi-touch display device 700 attaches objects to organizational tool 701,multi-touch display device 700 maintains the angular orientation that the objects possessed upon being attached to organizational tool 701. Thus,multi-touch display device 700 attaches and displaysobjects -
Multi-touch display device 700 may also orient all of the objects attached to organizational tool 701 to correspond with the orientation of the organizational tool 701. Referring to FIG. 7B(b),multi-touch display device 700 may orient each ofobjects objects multi-touch display device 700 may orient the objects attached to organizational tool 701 by, for example, determining the vertical axes of organizational tool 701 andobjects objects Multi-touch display device 700 may also utilize a similar process, except using the horizontal axes of organizational tool 701 andobjects - In certain implementations,
multi-touch display device 700 may cause each object to be oriented in correspondence with the orientation of organizational tool 701 upon attaching the object to organizational tool 701. In other implementations,multi-touch display device 700 may maintain the angular orientation that each object possesses upon being attached to organizational tool 701, as shown in FIG. 7B(a), and then concurrently reorient all of attachedobjects - As illustrated in FIG. 7C(a), organizational tool 701 is operating in a free (“messy”) attachment mode such that the
multi-touch display device 700 preserves the angular orientation of objects upon attaching the objects to the organizational tool 701. Furthermore,multi-touch display device 700 attaches the objects to the organizational tool 701 at the positions occupied by the objects at the points in time when themulti-touch display device 700 determines to attach the objects to the organizational tool 701.Multi-touch display device 700 may display a button to indicate in which attachment mode organizational tool 701 is operating. Thus, in FIG. 7C(a),multi-touch display device 700 displays a “Messy” button in the top right-hand corner of organizational tool 701. In the event that the user eventually wishes to organize the objects that were attached in the free attachment mode, the organizational tool 701 provides a control to toggle the organizational tool 701 into an organized (“clean”) attachment mode. - FIG. 7C(b) illustrates a possible result of a transition from a free attachment mode to an organized attachment mode. Thus, upon detecting an input from a user indicating a request to implement organized attachment mode, organizational tool 701 is partitioned into six sections to accommodate
objects organizational tool 710 in organized mode,multi-touch display device 700 may display a “Clean” button in the top right-hand corner of organizational tool 701. -
FIG. 8A is aflowchart 800 of an example of a process for merging/composing two rectangular two-dimensional organizational tools. The process illustrated inflowchart 800 may be performed by, for example, themulti-touch display device 100 ofFIGS. 1A-1K , themulti-touch display device 200 ofFIGS. 2( a)-2(b), themulti-touch display device 300 ofFIGS. 3( a)-3(b), themulti-touch display device 300 ofFIGS. 4B-4E and 4G-4H, themulti-touch display device 510 ofFIGS. 5B-5F , themulti-touch display device 600 ofFIGS. 6A-6B , or themulti-touch display device 700 ofFIGS. 7A-7B . - The multi-touch display device displays multiple organizational tools and monitors the position of each organizational tool (802). When the multi-touch display device manipulates one or more organizational tools, the multi-touch display device determines whether the translated organizational tool overlaps any other organizational tool(s) (804). If the multi-touch display device determines that the translated organizational tool does not overlap any other organizational tool(s), the multi-touch display device continues to monitor the position of each organizational tool. In contrast, when the multi-touch display device determines the translated organizational tool does overlap an organizational tool, multi-touch display device merges or composes the translated organizational tool with the overlapped organizational tools(s) (806).
- In certain implementations, the multi-touch display device will compose the overlapping organizational tool with the overlapped organizational tool. To compose two or more overlapping organizational tools, the multi-touch display device, in effect, attaches the overlapping organizational tool to the overlapped organizational tool, similar to the way previously described multi-touch display devices attach an object to an organizational tool. When
multi-touch display device 810 composes two or more organizational tools,multi-touch display device 810 maintains the existence of each of the organizational tools, but creates a relationship between them. Thus, becausemulti-touch display device 810 maintains the existence of each of the organizational tools,multi-touch display device 810 may separate two or more composed organizational tools. -
FIG. 8B illustrates a multi-touch display device composing two overlapping organizational tools and detaching two composed organizational tools. Referring to FIG. 8B(a),multi-touch display device 810 displays object 812 as attached toorganizational tool 814, and objects 816, 818, and 820 as attached toorganizational tool 822. As illustrated in FIG. 8B(b), a user has engaged the surface ofmulti-touch display device 810 withfinger 826 at a point corresponding toorganizational tool 822 and dragsfinger 826 across the surface of the screen. Themulti-touch display device 810 detects the movement offinger 826, and translatesorganizational tool 822 to a point whereorganizational tool 822 overlapsorganizational tool 814 in response. In this example, determining thatorganizational tool 822 overlapsorganizational tool 814,multi-touch display device 810 composesorganizational tool 822 withorganizational tool 814. Thereafter, becauseorganizational tool 822 is composed withorganizational tool 814,multi-touch display device 810 translatesorganizational tool 822 along withorganizational tool 814. - FIGS. 8B(c)-8B(d) illustrate transformations being applied to
organizational tool 814. In FIG. 8B(c), a user has engaged the surface ofmulti-touch display device 810 withfinger 826 at a point corresponding toorganizational tool 814 and is dragging his finger along themulti-touch display device 810 in a rightward direction from a first point, as shown in FIG. 8B(c), to a second point, as shown in FIG. 8B(d).Multi-touch display device 810 detects this input byfinger 826, associates the input withorganizational tool 814, interprets the input as a request to translateorganizational tool 814 by a distance corresponding to the distance from the first point to the second point, and translatesorganizational tool 814 by the detected distance of some function of the detected distance. Furthermore, becauseorganizational tool 822 is composed withorganizational tool 814,organizational tool 822,multi-touch display device 810 translatesorganizational tool 822 along withorganizational tool 814, as shown in FIG. 8B(d). - FIGS. 8B(e)-8B(f) illustrate transition of an object between two composed organization tools. In FIG. 8B(e), a user has engaged the surface of
multi-touch display device 810 withfinger 826 at a first point corresponding to object 818.Multi-touch display device 810 detects the input offinger 826, associates the detected input withobject 818, and determines whether any future movement of the detected input should trigger the detachment ofobject 818 fromorganizational tool 822. In this example,multi-touch display device 810 determines that future movement of the detected input should trigger the detachment ofobject 818. - Thereafter, as shown in FIG. 8B(f), the user drags
finger 826 upward, from the first point to a second point, and disengages the surface ofmulti-touch display device 810. In response to detecting the movement offinger 826,multi-touch display device 810 translatesobject 818 by a distance corresponding to the distance from the first point to the second point, and detachesobject 818 fromorganizational tool 822. However, in response to detecting thatfinger 826 has disengaged the surface ofmulti-touch display device 810,multi-touch display device 810 determines whetherobject 818 is overlapping any of the displayed organizational tools. As a consequence of detecting thatobject 818 overlapsorganizational tool 814,multi-touch display device 810 then determines whetherobject 818 should be attached toorganizational tool 814. In this example,multi-touch display device 810 determines thatobject 818 should be attached toorganizational tool 814. Therefore,multi-touch display device 810 attachesobject 818 toorganizational tool 814. - Once stated differently,
multi-touch display device 810 has ceased the relationship betweenobject 818 andorganizational tool 822, but created a relationship betweenobject 818 andorganizational tool 814. Thus, as shown in FIGS. 8B(g)-8B(h), iforganizational tool 822 is transformed, apart fromorganizational tool 814,multi-touch display device 810 will not apply any corresponding transformations to object 818. In FIG. 8B(g), a user has engaged the surface ofmulti-touch display device 810 withfinger 826 at a first point corresponding toorganizational tool 822.Multi-touch display device 810 detects the input offinger 826, associates the detected input withorganizational tool 822, and determines whether any future movement of the detected input should trigger the detachment oforganizational tool 822 fromorganizational tool 814. In this example,multi-touch display device 810 determines that future movement of the detected input should trigger the detachment oforganizational tool 822. - Thereafter, as shown in FIG. 8B(h), the user drags
finger 826 leftward across the surface ofmulti-touch display device 810 from the first point to a second point. In response to detecting the movement offinger 826,multi-touch display device 810 translates organizational tool 822 a distance corresponding to the distance from the first point to the second point and detachesorganizational tool 822 fromorganizational tool 814. Because multi-touch display device has detachedobject 818 fromorganizational tool 822, and subsequently attachedobject 818 toorganizational tool 814, the translation oforganizational tool 822 may have no visual effect onobject 818. - In other implementations, the multi-touch display device will merge the overlapping organizational tool with the overlapped organizational tool. To merge two or more overlapping organizational tools, the multi-touch display device, in effect, dissolves the overlapping organizational tool into the overlapped organizational tool, such that the objects attached to the overlapping organizational tool are then attached to the overlapped organizational tool in a similar manner. When
multi-touch display device 810 merges two or more organizational tools,multi-touch display device 810 deletes the overlapping organizational tool(s). Thus, becausemulti-touch display device 810 deletes the overlapping organizational tool(s),multi-touch display device 810 cannot separate merged organizational tools. Instead,multi-touch display device 810 provides the capability of splitting the overlapped organizational tool into multiple organizational tools. -
FIG. 8C illustrates a multi-touch display device merging two overlapping organizational tools and splitting the single, merged organizational tool into multiple separate organizational tools. Referring to FIG. 8C(a),multi-touch display device 810object 812 attached toorganizational tool 814, and objects 816, 818, and 820 toorganizational tool 822. As illustrated in FIG. 8C(b), a user has engaged the surface ofmulti-touch display device 810 withfinger 826 at a point corresponding toorganizational tool 822 and dragsfinger 826 across the surface of the screen. Themulti-touch display device 810 detects the movement offinger 826, and translatesorganizational tool 822 to a point whereorganizational tool 822 overlapsorganizational tool 814 in response. In this example, determining thatorganizational tool 822 overlapsorganizational tool 814,multi-touch display device 810 mergesorganizational tool 822 withorganizational tool 814. As illustrated in FIG. 8C(c), becauseorganizational tool 822 is merged withorganizational tool 814,multi-touch display device 810 deletesorganizational tool 822, and attachesobjects organizational tool 814 in the same manner in which they were attached toorganizational tool 822. - In certain instances, a user may desire to split
organizational tool 814 into multiple separate organizational tools. In order to separateorganizational tool 814,multi-touch display device 810 may determine a group of objects to separate fromorganizational tool 814, and, in some instances, attach to a new, separate organizational tool. By separating a group of objects fromorganizational tool 814 concurrently,multi-touch display device 810 is providing a more efficient method than first creating a new organizational tool, followed by individually detaching each desired object fromorganizational tool 814 and attaching it to the new organizational tool. - FIG. 8C(d) illustrates one example of how
multi-touch display device 810 may determine which objects to include in the group to be separated fromorganizational tool 814. A user has engaged the surface ofmulti-touch display device 810 withfinger 826 at a point corresponding toorganizational tool 814.Multi-touch display device 810 detects the input offinger 826, associates the detected input withorganizational tool 814, and interprets the input as a request to separateorganizational tool 814. The user dragsfinger 826 in a circle around a portion oforganizational tool 814. In order to provide the user a visual indication of what portion oforganizational tool 814 the user has selected,multi-touch display device 810 displays anannotation 830 corresponding to where the input offinger 826 has been detected. - FIGS. 8C(e)-8C(i) illustrate three variations of separating
organizational tool 814 and/or objects attached toorganizational tool 814. In some implementations,multi-touch display device 810 may detect the input fromfinger 826, and select those objects attached toorganizational tool 814 that come in contact or are interior to the detected circular input. Stated differently,multi-touch display device 810 selects theobjects annotation 830. - With reference to FIG. 8C(e)-8C(g), in certain implementations,
finger 826 may disengage the surface ofmulti-touch display device 810 after completing the desired selection. Therefore, after an initial disengagement of the detected input,multi-touch display device 810 maintains the display ofannotation 830 and the selection ofobjects multi-touch display device 810 what to do with selectedobjects finger 826 engages the surface ofmulti-touch display device 810 at a first point internal to the visual boundaries ofannotation 830.Multi-touch display device 810 detects the input, and associates it with the selectedobjects annotation 830. - In FIG. 8C(f), as the user drags
finger 826 rightward across the surface oforganizational tool 810, from the first point to a second point,multi-touch display device 810 detects the movement offinger 826, and translates the selectedobjects objects annotation 830 do not overlaporganizational tool 814. Referring to FIG. 8C(g), oncefinger 826 disengages the surface,multi-touch display device 810 creates a neworganizational tool 832 of a proper size and shape and at a proper position to accommodate the selected objects. Onceorganizational tool 832 has been created,multi-touch display device 810 attaches selectedobjects organizational tool 832. Concurrently, upon detecting the disengagement offinger 826,multi-touch display device 810 deletesannotation 830. - Alternatively, in FIG. 8C(h), upon detecting an initial disengagement of
finger 826 afterobjects multi-touch display device 810 creates a neworganizational tool 834 of a proper size and shape to accommodate selectedobjects objects multi-touch display device 810 resizesorganizational tool 814 so as to only occupy the space necessary to display theobject 816, which was not selected. In other implementations,multi-touch display device 810 deletesorganizational tool 814 altogether and creates another new organizational tool of a proper size and shape to accommodatenon-selected object 816. - Alternatively, in FIG. 8C(i), upon detecting an initial disengagement of
finger 826 afterobjects multi-touch display device 810 detaches the selectedobjects organizational tool 814 and resizesorganizational tool 814 so as to only occupy the space necessary to display theobject 816, which was not selected. -
FIG. 9 illustrates an alternative implementation of the rectangular two-dimensional organizational tool. In this alternative implementation, the rectangular two-dimensional organizational tool may be implemented as a window through the canvas that provides a view into an organizational tool situated beneath (or behind) the canvas. In such an implementation, not only might the canvas be infinite workspace (or at least have an extent that is larger than the visual display of the multi-touch display device), but the organizational tool might provide a secondary, embedded infinite workspace (or at least have an extent that is larger than the visual display of the multi-touch display device). In effect, the multi-touch display device would operate as if there were two layers, wherein a lower layer further from the surface of the multi-touch display device may include the organizational tool and a higher layer closer to the surface of the multi-touch display device may include a canvass and a window through which to view the organizational tool. -
FIG. 9( a) illustrates amulti-touch display device 900 displayingobjects canvas 906.Multi-touch display device 900 also displaysorganizational tool 908 andobjects organizational tool 914.Canvas 906, and therefore objects 902 and 904, andwindow 908, is on top of, or visually closer to the surface ofmulti-touch display device 900 than,organizational tool 914.Canvas 906 can be manipulated similar to any of the other canvasses described previously. Thus,canvas 906 may be shifted or resized, and objects 902 and 904, being associated withcanvas 906, will be correspondingly shifted and resized. However, any manipulations ofcanvas 906, or the objects associated with it, may not affectorganizational tool 914, or the objects attached to it. Furthermore, manipulations ofcanvas 906 may or may not affect the size and position ofwindow 908. -
Window 908 effectively provides a transparent area through whichorganizational tool 914 may be viewed and manipulated.Organizational tool 914 can be manipulated similar to any of the other organizational tools described previously. Thus,organizational tool 914 may be shifted or resized, and objects 910, 912, 920, and 922, being attached toorganizational tool 914, will be correspondingly shifted and resized. However, only that area oforganizational tool 914 directly “under”window 908 will be displayed bymulti-touch display device 900. As a result, inFIG. 9( a), objects 920 and 922, are not displayed bymulti-touch display device 900, because they are belowcanvas 906, which may be effectively opaque. - If a user wishes to view another portion of
organizational tool 918, he may engageorganizational tool 918 throughwindow 908. For example, referring toFIG. 9( a), a user has engaged the surface ofmulti-touch display device 900 withfinger 926 at a first point corresponding towindow 908, and thusorganizational tool 918.Multi-touch display device 900 detects the input offinger 926, associates the input withorganizational tool 918. Referring toFIG. 9( b), as the user dragsfinger 926 across themulti-touch display device 900 in a leftward direction, from the first point to a second point,multi-touch display device 900 interprets the input as a request to translateorganizational tool 918 by a distance corresponding to a distance from the first point to the second point, and translatesorganizational tool 918 by the detected distance or a function of the detected distance. Again, sinceobjects organizational tool 918,multi-touch display device 900 translatesobjects organizational tool 918. - However,
multi-touch display device 900 only displays that area oforganizational tool 918 directly underneathwindow 908. Thus, asorganizational tool 918 andobjects multi-touch display device 900 begins displayingobjects object 910. Furthermore, sincecanvas 906 exists separately fromorganizational tool 918, these manipulations performed on theorganizational tool 918 will not affectcanvas 906 or the objects associated with it. Thus,canvas 906 andobjects FIG. 9( a) toFIG. 9( b). - Likewise, a user may wish to change the scale of
organizational tool 918. Referring toFIG. 9( c), a user has engaged the surface ofmulti-touch display device 900 withfinger 926 and thumb 928 at a points corresponding towindow 908, and thusorganizational tool 918.Multi-touch display device 900 concurrently detects the inputs offinger 926 and thumb 928, and associates the inputs withorganizational tool 918. Referring toFIG. 9( d), as the user dragsfinger 926 and thumb 928 away from each other across themulti-touch display device 900,multi-touch display device 900 interprets the input as a request to scaleorganizational tool 918, and scales the size oforganizational tool 918 as a function of the movement offinger 926 and thumb 928. Again, sinceobjects organizational tool 918,multi-touch display device 900scales objects organizational tool 918. - Again,
multi-touch display device 900 only displays that area oforganizational tool 918 directly underneathwindow 908. Thus, as organizational tool and objects 910, 912, 920, and 922 are scaled,multi-touch display device 900displays objects object 912. Furthermore, sincecanvas 906 exists separately fromorganizational tool 918, these manipulations performed on theorganizational tool 918 will not affectcanvas 906 or the objects associated with it. Thus,canvas 906 andobjects FIG. 9( c) toFIG. 9( d). - As described above in connection with
FIGS. 2( a)-2(b), a multi-touch display device may provide a one-dimensional linear organizational tool that enables manipulations similar to functionality provided by a clothes line or a short-order cook ticket line. -
FIG. 10 is a diagram of a multi-touch display device that provides a one-dimensional linear organizational tool that illustrates techniques for attaching objects to the one-dimensional linear organizational tool. In this example,multi-touch display device 1000 displays objects 1002, 1004, 1006, 1008, 1010, 1012, and 1014 andorganizational tool 1016.Organizational tool 1016 includes anattachment strip 1018 andboundary handles organizational tool 1016 andattachment strip 1018 may be greater than the visual display oforganizational tool 1016. Boundary handles 1020 and 1022 define the boundaries of the visual display oforganizational tool 1016, but the extent oforganizational tool 1016 extends beyond boundary handles 1020 and 1022, even though the visual display oforganizational tool 1016 is confined to the region between boundary handles 1020 and 1022. In effect, the greater extents oforganizational tool 1016 andattachment strip 1018 allow for “off-screen” storage, such thatmulti-touch display device 1000 may reduce the area used to displayorganizational tool 1016 while, at the same time, preserving easy access to the objects attached to the organizational tool. - As illustrated in
FIG. 10( a), objects 1004-1014 are attached toorganizational tool 1016 throughattachment strip 1018.Object 1002, meanwhile, remains unattached. - In
FIG. 10( a), a user is engaging the surface ofmulti-touch display device 1000 withfinger 1026 at a first point corresponding to object 1002.Multi-touch display device 1000 detects the input offinger 1026 and associates the detected input withobject 1002. - Referring to
FIG. 10( b), as the user dragsfinger 1026 across the surface from the first point to a second point,multi-touch display device 1000 detects the position of thefinger 1026 and continuously updates the display ofobject 1002 to correspond with the detected position offinger 1026. Eventually, as a result of the detected motion offinger 1026, themulti-touch display device 1000 translatesobject 1002 by a distance corresponding to the distance from the first point to the second point, a position beneathattachment stripe 1018. - Subsequently, the user disengages
finger 1026 from the surface of themulti-touch display device 1000, and the multi-touch display device detects thatuser 1024 has disengagedfinger 1026 from the surface ofmulti-touch display device 1000. In response, the multi-touch display device determines whetherobject 1002 visually overlapsattachment strip 1018. Sinceobject 1002 does visually overlapattachment strip 1018,multi-touch display device 1000 determines whether the disengagement ofobject 1002 triggers attachment ofobject 1002 toorganizational tool 1016. - In certain implementations, the determination of whether the disengagement should trigger an attachment is based on a stored rule or set of rules. For example, a rule may specify that a disengagement of the surface while
multi-touch display device 1000 concurrently detects a separate, continuous engagement ofattachment strip 1018 will trigger an attachment ofobject 1002 to theorganizational tool 1016. Alternatively or additionally, a rule may specify that a disengagement of the surface preceded by an engagement of the surface at a pressure greater that a predetermined threshold and at a point corresponding to whereobject 1002 overlaps theattachment strip 1018 will trigger an attachment ofobject 1002 to theorganizational tool 1016. In this example, becausefinger 1026 disengaged the surface ofmulti-touch display device 1000 at a point corresponding toattachment strip 1018,multi-touch display device 1000 determines thatobject 1002 should be attached toorganizational tool 1016. Therefore,multi-touch display device 1000 attachesobject 1002 toorganizational tool 1016. -
FIG. 11 is a diagram of a multi-touch display device that provides a one-dimensional linear organizational tool that illustrates techniques for detaching objects from the one-dimensional linear organizational tool. In this example,multi-touch display device 1100 displays objects 1102, 1104, 1106, 1108, 1110, and 1112 andorganizational tool 1116.Organizational tool 1116 includes anattachment strip 1118. - Referring to
FIG. 11( a), a user has engaged the surface ofmulti-touch display device 1100 withfinger 1126 at a first point corresponding to object 1102. Themulti-touch display device 1100 detects the input offinger 1126, and associates the input withobject 1102. - After detecting the engagement of
finger 1126,multi-touch display device 1100 determines whether any future movements of thefinger 1126 should trigger a detachment ofobject 1102. In this example,multi-touch display device 1100 determines thatobject 1102 should be detached fromorganizational tool 1116. Thereafter, in reference toFIG. 11( b), the user dragsfinger 1126 across the surface of themulti-touch display device 1100 in a downward motion from the first point to a second point. In response to detecting the movement offinger 1126,multi-touch display device 1100 translatesobject 1102 by a distance corresponding to the distance from the first point to the second point. In some implementations,multi-touch display device 1110 detachesobject 1102 fromorganizational tool 1116 whenmulti-touch display device 1100 detects thatfinger 1126 disengages the surface ofmulti-touch display device 1100 and determines thatobject 1102 no longer overlapsorganizational tool 1116, as shown inFIG. 11( c). - As described above, a multi-touch display device may automatically adjust the angular orientation of an object upon attaching the object to an organizational tool. For example, in one implementation, the multi-touch display device may automatically adjust the angular orientation of an object upon attaching the object to the multi-touch display device such that the angular orientation of the object is consistent with the angular orientation of objects already attached to the organizational tool, irrespective of the angular orientation of the organizational tool itself. Alternatively, the multi-touch display device may automatically adjust the angular orientation of an object upon attaching the object to the multi-touch display device such that the angular orientation of the object is consistent with the angular orientation of the organizational tool itself.
-
FIG. 12A is aflowchart 1250 of an example of a process for organizing objects attached to a linear one-dimensional organizational tool. The process illustrated inflowchart 1250 may be performed by, for example, themulti-touch display device 1000 ofFIGS. 10( a)-10(b) or themulti-touch display device 1100 ofFIGS. 11( a)-11(c). - The multi-touch display device displays an organizational tool and one or more objects (1252), and defines a reference axis for attaching objects to the organizational tool (1254). The multi-touch display device continuously determines whether the one or more objects being displayed should be attached to the organizational tool (1256). The multi-touch touch display device may make the attachment determination in a manner similar to those discussed above with regard to
FIGS. 4A-4G and 10(a)-10(b). - If the multi-touch display device determines that one or more of the displayed objects should be attached to the organizational tool, the multi-touch display device identifies the reference axis of each object to be attached to the organizational too (1268). In some implementations, the multi-touch display device may determine the reference axis based on a vertical and/or horizontal axis of the object, stored and updated by the multi-touch display device. Alternatively, the multi-touch display device may determine the reference axis based on a comparison of multiple predetermined points of the object with regard to multiple predetermined points of the display surface of the multi-touch display device. Alternatively, the multi-touch display device determine the reference axis by detecting an input from a user indicating the reference axis. The multi-touch display device identifies the defined reference axis for attaching objects to the organizational tool (1260).
- Next, the multi-touch display device adjusts the orientation of each object that is being attached to the organizational tool, such that the reference axes of the attached objects align with the defined reference axis for attaching objects to the organizational tool (1262). Once the multi-touch display device adjusts the object(s), the multi-touch display device attaches the object(s) to the organizational tool (1264).
-
FIGS. 12B and 12C are diagrams of a multi-touch display device that illustrate two different examples the multi-touch display device automatically adjusting the angular orientation of an object upon attaching the objects to a one-dimensional linear organizational tool. In these figures,multi-touch display device 1200 displays objects 1202, 1204, and 1206 andorganizational tool 1216.Organizational tool 1216 includes anattachment strip 1218 andboundary handles Objects organizational tool 1216 throughattachment strip 1218. - Referring to FIG. 12B(a),
object 1204 andobject 1206 are attached to the one-dimensional linear organizational tool such that their angular orientations are consistent with one another, but not necessarily with the angular orientation of the one-dimensional linear organizational tool. Stated differently, the vertical and horizontal axes of each ofobjects organizational tool 1216. - As illustrated in the sequence of FIGS. 12B(a)-12B(c), when the
multi-touch display device 1200 attachesobject 1202 to the one-dimensionalorganizational tool 1216, the multi-touch display device automatically adjusts the angular rotation ofobject 1202 such that it is consistent with the angular orientation ofobjects object 1202,multi-touch display device 1200 may, for example, determine the vertical and/or horizontal axes ofobject 1202, determine the vertical and/or horizontal axes ofobjects 1204 and/or 1206, and rotateobject 1202 such that its vertical and/or horizontal axes align with the vertical and/or horizontal axes ofobjects 1204 and/or 1206. - Referring to FIG. 12C(a),
object 1204 andobject 1206 are attached to the one-dimensional linear organizational tool such that their angular orientations are consistent with one another, and with the angular orientation of the one-dimensional linear organizational tool. Stated differently, the vertical and horizontal axes of each ofobjects organizational tool 1216. - As illustrated in the sequence of FIGS. 12C(a)-12C(c), when the
multi-touch display device 1200 attachesobject 1202 to the one-dimensionalorganizational tool 1216, the multi-touch display device automatically adjusts the angular rotation ofobject 1202 such that it is consistent with the angular orientation ofobjects object 1202,multi-touch display device 1200 may, for example, determine the vertical and/or horizontal axes ofobject 1202, determine the vertical and/or horizontal axes oforganizational tool 1216, and rotateobject 1202 such that its vertical and/or horizontal axes align with the vertical and/or horizontal axes oforganizational tool 1216. - As discussed above, one effect of an object being attached to an organizational tool is that when the multi-touch display device applies transformations to the organizational tool, the multi-touch display device also may apply transformations to the objects attached to the organizational tool as a consequence of their attachment. However, when the multi-touch display device applies a transformation to an organization tool, the transformations that the multi-touch display device applies to objects attached to the organizational tool may depend upon how the objects are attached to the organizational tool.
- In some cases, the multi-touch display device may attach an object to an organizational tool at only a single point. In such cases, transformations applied to the organizational tool only impact an attached object (i.e., cause a corresponding transformation to be applied to the attached object) if the transformations applied to the organizational tool impact the point upon the organizational tool at which the object is attached.
- The manner in which all objects are attached to a organizational tool may be set for a given organizational tool by the user or they may be automatically preset upon creation by the multi-touch display device. Alternatively, the manner in which objects are attached to a organizational tool may be detected separately for each object upon being attached to the organizational tool.
-
FIG. 13 illustrates a multi-touch display device performing transformations to a one-dimensional linear organizational tool and its attached objects when the attached objects are attached to the organizational tool at only one point. As illustrated inFIG. 13( a), objects 1302, 1304, and 1306 each are attached toorganizational tool 1316 throughattachment strip 1318 at one point. Therefore, when themulti-touch display device 1300 rotatesorganizational tool 1316 in response to detecting input byfingers FIG. 13( b), themulti-touch display device 1300 corresponding translates each of displayedobjects organizational tool 1316 impacts the point upon theattachment strip 1318 at which objects 1302, 1304, and 1306 are attached. Thus, inFIG. 13( c),multi-touch display device 1300 has translated the position ofobjects attachment strip 1318, while maintaining the angular orientation (illustrated by the consistent x and y axes next to each object) ofobjects - Alternative manners of attaching objects to
organizational tool 1316 also may be employed. For example, objects may be attached toorganizational tool 1316 at two or more points as opposed to a single point as illustrated inFIGS. 13( a)-13(c). - As discussed above, the extent of a one-dimensional linear organizational tool provided by a multi-touch display device may be greater than the visual display of the one-dimensional linear organizational tool itself. In such implementations, a preview screen may be provided by the multi-touch display device to provide a glimpse into the regions of the one-dimensional linear organizational tool that exist beyond the boundaries of the visual display of the one-dimensional linear organizational tool. Such a preview screen may serve to alert a user to (or remind the user of) the existence of objects attached to the one-dimensional linear organizational tool beyond the visual boundaries of the one-dimensional linear organizational tool. In addition, the preview screen also may portray to a user the spatial relationships between the objects attached to the one-dimensional linear organizational tool, whether or not the objects are within the boundaries of the visual display of the one-dimensional linear organizational tool, thereby providing the user with an indication of how far away certain objects attached to the one-dimensional linear organizational tool are from the region of the one-dimensional linear organizational tool currently within the boundaries of the visual display of the one-dimensional linear organizational tool.
-
FIG. 14A is illustrative. FIG. 14A(a) is a schematic diagram of a one-dimensional linearorganizational tool 1424. As illustrated in FIG. 14A(a), the one dimensional linearorganizational tool 1424 includes anattachment strip 1428 andobjects attachment strip 1428. Dashed box 1442 represents the region of the one-dimensional linearorganizational tool 1424 that is displayed on the multi-touch display device. As illustrated in FIG. 14A(a), the extent of the one-dimensional linearorganizational tool 1424 extends beyond the boundaries of the visual display of the one-dimensional linearorganizational tool 1424. In particular, the one-dimensional linearorganizational tool 1424 has a region to the left that extends beyond the boundaries of the visual display of the one-dimensional linearorganizational tool 1424 and a region to the right that extends beyond the boundaries of the visual display of the one-dimensional linear organizational tool. - Because
objects organizational tool 1424 within the boundaries of the visual display of the one-dimensional linearorganizational tool 1424, objects 1410, 1412, 1414, 1416, 1418, and 1420 will be displayed by the multi-touch display device as being attached to the one-dimensional linearorganizational tool 1424. In contrast, becauseobjects organizational tool 1424 outside of the boundaries of the visual display of the one-dimensional linearorganizational tool 1424, objects 1402, 1404, 1406, 1408, and 1422 will not be displayed by the multi-touch display device given the current boundaries of the visual display of the one-dimensional linearorganizational tool 1424. - FIG. 14A(b) is a diagram of a
multi-touch display device 1400 displaying the one-dimensional linearorganizational tool 1424 depicted in the schematic diagram of FIG. 14A(a). As illustrated in FIG. 14A(b), the boundaries of the visual display of the one-dimensional linearorganizational tool 1424 are defined byboundary handles linear grouping tool 1424 depicted by the dashed box 1442 in FIG. 14A(a). As a consequence of the presently defined boundaries of the visual display of the one-dimensional organizational tool,multi-touch display device 1400 displays objects 1410, 1412, 1414, 1416, 1418, and 1420, but not objects 1402, 1404, 1406, 1408, and 1422. - Furthermore, to provide users with a glimpse of the full extent of the one-dimensional linear
organizational tool 1424,multi-touch display device 1400 displayspreview screen 1426, which provides a visual representation of the entirety of the one-dimensional linearorganizational tool 1424 and the spatial relationships between objects attached to the one-dimensionalorganizational tool 1424. As illustrated in FIG. 14A(b),multi-touch display device 1400 provideswindow 1434 in connection with thepreview screen 1426 to identify the potion of one-dimensional linearorganizational tool 1424 and the objects attached thereto that themulti-touch display device 1400 currently is displaying between boundary handles 1430 and 1432. - As the
multi-touch display device 1400 manipulates one-dimensionalorganizational tool 1424 in response to user input, themulti-touch display device 1400 also updatespreview screen 1426 accordingly. - For example, referring to FIG. 14A(c), when the
multi-touch display device 1400 translatesattachment strip 1428 to the right, thereby bringingobjects organizational tool 1424 while expellingobjects organizational tool 1424,multi-touch display device 1400 also translateswindow 1434 to the left identify the portion of one-dimensional linear organizational tool now currently displayed betweenboundaries - In some implementations, as users interact with objects attached to a one-dimensional linear organizational tool provided by a multi-touch display device, the multi-touch display device causes a preview screen displayed in connection with the one-dimensional linear organizational tool to be updated to visually distinguish the representations of the objects with which the users are interacting with from the visual representations of the other objects attached to the one-dimensional organizational tool. For example, if a user has selected an object attached to the one-dimensional organizational tool, the multi-touch display device may highlight (or otherwise visually distinguish) the corresponding visual representation of the selected object in the preview screen. Additionally or alternatively, when an object is newly attached to the one-dimensional organizational tool, the multi-touch display device may highlight (or otherwise visually distinguish) the visual representation of the newly attached object in the preview screen. Similarly, when an object recently has been removed from the one-dimensional organizational tool, the multi-touch display device may preserve the corresponding visual representation of the removed object within the preview screen, but the multi-touch display device may highlight (or otherwise visually distinguish) the recently detached object to reflect that the object has been detached from the one-dimensional linear organizational tool.
-
FIG. 14B is a diagram of amulti-touch display device 1400 that provides a one-dimensional linearorganizational tool 1424 andcorresponding preview screen 1426 and that visually distinguishes the visual representations of objects displayed in thepreview screen 1426 in response to detecting user interaction with corresponding objects attached to the one-dimensional linearorganizational tool 1424. - In particular, as illustrated in
FIG. 14B , a user is engaging the surface ofmulti-touch display device 1400 withfinger 1438 at a point corresponding to object 1414.Multi-touch display device 1400 detects the input offinger 1438 and associates the detected input withobject 1414. Furthermore, in response to detecting this interaction withobject 1414,multi-touch display device 1400 highlights the correspondingvisual representation 1414′ ofobject 1414 withinpreview screen 1426 to reflect that the user presently is interacting withobject 1414. (In this example, the multi-touch display device visually distinguishes thevisual representation 1414′ ofobject 1414 within thepreview screen 1426 from the other visual representations of attached objects within the preview screen by providing a dark outline and a light fill to thevisual representation 1414′ ofobject 1414.) - In addition to providing controls for translating the attachment strip of a one-dimensional linear organizational tool, the multi-touch display device also may provide controls for increasing and/or decreasing the scale of the one-dimensional linear organizational tool and the objects attached to it. In such implementations, when the multi-touch display device increases or decreases the scale of the one-dimensional linear organizational tool and the objects attached to it, the multi-touch display device also may update a preview screen provided in connection with the one-dimensional linear organizational tool.
-
FIG. 14C is a sequence of diagrams of a multi-touch display device that provides controls for increasing or decreasing the scale of a one-dimensional linear organizational tool. As illustrated in the sequence of diagrams presented inFIG. 14C , in response to detecting that a user has engagedattachment strip 1428 with twofingers fingers multi-touch display device 1400 increases the scale ofattachment strip 1428 and the objects attached to theattachment strip 1428 as a function of the detected movement offingers attachment strip 1428 with two fingers and then moved the two fingers closer together, themulti-touch display device 1400 decreases the scale of theattachment strip 1428 and the objects attached to theattachment strip 1428 as a function of the detected movement of the fingers. - As illustrated in the sequence of diagrams presented in
FIG. 14C , the visual effect of increasing the scale ofattachment strip 1428 and the objects attached to the attachment strip is to zoom in on a shorter length ofattachment strip 1428. As a result,object 1412 is expelled from the visual display of the one-dimensionalorganizational tool 1424 and the sizes ofobjects window 1434 provided in connection withpreview window 1426 so as to identify the portion of the one-dimensionalorganizational tool 1424 now located between boundary handles 1430 and 1432. In particular, themulti-touch display device 1400 shortens the length ofwindow 1434 to reflect that a shorter length of the one-dimensional linearorganizational tool 1424 is located between boundary handles 1430 and 1432 as a consequence of the scaling operation. -
FIG. 14D is a sequence of diagrams of a multi-touch display device that provides controls for redefining the visual boundaries of a one-dimensional linear organizational tool. As illustrated in FIG. 14D(a), in response to detecting that a user has engagedboundary handle 1430 withfinger 1438 at a first point, and then has movedfinger 1438 in a leftward direction, from the first point to a second point, as shown in FIG. 14D(b), themulti-touch display device 1400 brings the visual boundaries oforganizational tool 1424 closer together by a distance corresponding to the distance from the first point to the second point. Similarly, though not illustrated, in response to detecting that a user has engagedboundary handle 1430 with a finger and then moving the finger in a rightward direction, themulti-touch display device 1400 spreads the visual boundaries oforganizational tool 1424 further apart as a function of the detected movement of the fingers. - As illustrated in the sequence of diagrams presented in
FIG. 14D , the visual effect of bringing the visual boundaries oforganizational tool 1424 closer together is to display a shorter length ofattachment strip 1428, whileattachment strip 1428 remains unaltered. As a result, objects 1414 and 1416 are expelled from the visual display of the one-dimensionalorganizational tool 1424, yet the sizes of 1404, 1408, 1410, and 1412 remain constant. Furthermore, in response to redefining the visual boundaries of a one-dimensional linear organizational tool, the multi-touch display device also modifieswindow 1434 provided in connection withpreview window 1426 so as to identify the portion of the one-dimensionalorganizational tool 1424 now located between boundary handles 1430 and 1432. In particular, themulti-touch display device 1400 shortens the length ofwindow 1434 to reflect that a shorter length of the one-dimensional linearorganizational tool 1424 is located between boundary handles 1430 and 1432 as a consequence of the redefining operation. -
FIG. 14E illustrates a rotation of the organizational tool and the effect that the rotation has on the preview screen and window. In this example,multi-touch display device 1400 currently displaysobjects organizational tool 1424, andpreview screen 1426.Organizational tool 1424 includes anattachment strip 1428 andboundaries Objects organizational tool 1424 throughattachment strip 1428.Preview screen 1426 indicates the current total utilized length ofattachment strip 1428, and includeswindow 1434, which outlines the potion ofattachment strip 1428 that is currently being displayed betweenboundaries - As
multi-touch display device 1400 rotatesorganizational tool 1424, it also rotatespreview screen 1426 andwindow 1434 in order to maintain a visual correspondence betweenorganizational tool 1424,preview screen 1426, andwindow 1434. Additionally, in this example objects 1402-1418 are attached toattachment strip 1428 in the two-point manner, such thatmulti-touch display device 1400 displays objects 1402-1418 so they will always visually maintain their absolute position with respect toorganizational tool 1424. In other words, the angular orientation ofobjects organizational tool 1424. - In some implementations, a multi-touch display device that provides a one-dimensional linear organizational tool may provide controls for increasing and/or decreasing the scale of one or more portions of the one-dimensional linear organizational tool and the objects attached to such portion(s), while preserving the pre-existing scale of the other portions and the objects attached to such other portions.
-
FIG. 15 is a sequence of diagrams of amulti-touch display device 1500 that provides a one-dimensional linearorganizational tool 1524 and controls for increasing and/or decreasing the scale of one or more portions of the one-dimensional linear organizational tool and the objects attached to such portion(s), while preserving the pre-existing scale of the other portions and the objects attached to such portions. - In
FIG. 15( a), a user is engaging the surface ofmulti-touch display device 1500 withfingers attachment strip 1528.Multi-touch display device 1500 detects the inputs offingers organizational tool 1524 andattachment strip 1528. As the user increases the distance betweenfingers FIG. 15( b),multi-touch display device 1500 detects the movement offingers organizational tool 1524 and the objects attached thereto between the points onattachment strip 1528 engaged byfingers fingers multi-touch display device 1500 maintains the points on the attachment strip engaged byfingers fingers attachment strip 1528 and the objects attached thereto in the region ofattachment strip 1528 betweenfingers fingers - In order to enlarge the display of one-dimensional
organizational tool 1524 between the two points engaged byfingers multi-touch display device 1500 essentially zooms in on the portion ofattachment strip 1528 between the two points contacted byfingers organizational tool 1524. In addition,multi-touch display device 1500 increases the size of objects attached between the two detected contact points (i.e. objects 1504, 1506, and 1508). Concurrently,multi-touch display device 1500 shifts the remainder of theattachment strip 1528 and its attached objects not between the two detected contact points (i.e. objects 1502, 1510, and 1512) in the direction of the motion of the detected inputs, so as to make room for the increased size of the zoomed portion. In other words, as the scale ofobjects multi-touch display device 1500 shifts object 1502 left andobjects -
Multi-touch display device 1500 may utilize this alternative method for scaling a one-dimensional linear organizational tool by default, or only when a special mode is set. Furthermore, when it detects the disengagement of the detected inputs,multi-touch display device 1500 may maintain the scale of theattachment strip 1528 or, alternatively,multi-touch display device 1500 may returnattachment strip 1528 to its previous scale, much like a rubber band snaps back to its previous state when the contact points on the rubber band are released. -
FIGS. 16A and 16B illustrate how different parts of a single attachment strip may be viewed through multiple organizational tools, possibly on more than one multi-touch display device. -
FIG. 16A illustrates two separate multi-touch display devices, or two portions of the same multi-touch display device, displaying a single attachment strip through two organizational tools. Alternatively, the multi-touch display device(s) may display a single attachment strip through separate instances of the same organizational tool, to the same effect. The two separate multi-touch display devices may be in the same room, or they may be on separate continents. In certain implementations, two separate multi-touch display devices may facilitate such a parallel view with a communication link between the two multi-touch display devices. For example, two multi-touch display devices, each connected to the Internet through a standard Internet Service Provider (ISP), may set up a communication link and share a single attachment strip. In response to any manipulation done to the attachment strip, the multi-touch display device(s) may concurrently reflect the manipulation through both organizational tools to two different users. - FIG. 16A(a) is a diagram of a
multi-touch display device 1600 displaying the one-dimensional linearorganizational tool 1616. The boundaries of the visual display of the one-dimensional linearorganizational tool 1616 are defined byboundary handles multi-touch display device 1600 displays objects 1602, 1604, 1606, and 1608. Likewise, FIG. 16A(b) is a diagram of amulti-touch display device 1624 displaying the one-dimensional linearorganizational tool 1626. The boundaries of the visual display of the one-dimensional linearorganizational tool 1626 are defined byboundary handles multi-touch display device 1624 displays objects 1608, 1610, 1612, and 1614. - In FIG. 16A(a), a user is engaging the surface of
multi-touch display device 1600 withfinger 1634 at a first point corresponding toattachment strip 1618.Multi-touch display device 1600 detects the input offinger 1634 and associates the detected input withattachment strip 1618. Referring to FIG. 16A(c), as the user dragsfinger 1634 across the surface, from the first point to a second point,multi-touch display device 1600 detects the position of thefinger 1634 and translatesattachment strip 1618 by a distance corresponding to the distance from the first point to the second point. Thus, themulti-touch display device 1600 translatesattachment strip 1628 to the right by the detected distance or a function of the detected distance, thereby expellingobject 1608 from the visual display of one-dimensionalorganizational tool 1600. Concurrently,multi-touch display device 1600 communicates the translation ofattachment strip 1628 tomulti-touch display device 1624. Upon receiving the translation information frommulti-touch display device 1600, as shown in FIG. 16A(d),multi-touch display device 1624 translatesattachment strip 1628 to the right by the detected distance or a function of the detected distance, thereby bringingobjects organizational tool 1626. -
FIG. 16B illustrates a similar concept asFIG. 16A , exceptorganizational tools multi-touch display device 1600 withfinger 1634 at a first point corresponding toattachment strip 1618.Multi-touch display device 1600 detects the input offinger 1634 and associates the detected input withattachment strip 1618. Referring to FIG. 16B(b), as the user dragsfinger 1634 across the surface, from the first point to a second point,multi-touch display device 1600 detects the position of thefinger 1634 and translatesattachment strip 1618 by a distance corresponding to the distance from the first point to the second point. Thus, themulti-touch display device 1600 translatesattachment strip 1628 to the right by the detected distance or a function of the detected distance, thereby expellingobject 1608 from the visual display of one-dimensionalorganizational tool 1600. Concurrently,multi-touch display device 1624 translatesattachment strip 1628 to the right by the detected distance or a function of the detected distance, thereby bringingobjects organizational tool 1626. - As described above in connection with
FIGS. 3( a)-3(b), a multi-touch display device may provide a two-dimensional rotary organizational tool that enables manipulations similar to functionality provided by a Lazy Susan. -
FIG. 17 is a diagram of a multi-touch display device that provides a two-dimensional rotary organizational tool that illustrates techniques for attaching objects to the two-dimensional rotary organizational tool. In this example,multi-touch display device 1700 displays objects 1702, 1704, 1706, 1708, 1710, and 1712 andorganizational tool 1716.Organizational tool 1716 includes ancontrol strip 1718. As described above, thecontrol strip 1718 provides a mechanism for manipulatingorganizational tool 1716. In some implementations,control strip 1718 will appear on top of objects attached toorganizational tool 1716 in order to provide a convenient way to manipulateorganizational tool 1716. Again,multi-touch display device 1700 may, in some implementations, interpret an input to controlstrip 1716 as a request to rotateorganizational tool 1716 about its center. As such, no matter how largemulti-touch display device 1700 displaysorganizational tool 1716, a user may engagecontrol strip 1716 to rotateorganizational tool 1716, and have convenient access to all objects attached toorganizational tool 1716 without having to change his position with relation tomulti-touch display device 1700 or the overall position oforganizational tool 1716 with relation to the displayed workspace. - As illustrated in
FIG. 17( a),object 1712 is attached toorganizational tool 1716.Objects - In
FIG. 17( a), a user is engaging the surface ofmulti-touch display device 1700 withfingers multi-touch display device 1700, each point corresponding to a point on the surface ofobjects Multi-touch display device 1700 detects the input offingers objects - Referring to
FIG. 17( b), as the user dragsfingers multi-touch display device 1700 detects the position of thefingers objects objects multi-touch display device 1700 withfinger 1730 at a third point corresponding to controlstrip 1718. Eventually, as a result of the detected motion offingers multi-touch display device 1700 translatesobjects organizational tool 1716 and beneathcontrol stripe 1718. - Subsequently, the user disengages
fingers multi-touch display device 1700, and the multi-touch display device detects that the user has disengagedfingers multi-touch display device 1700. In response, the multi-touch display device determines whetherobjects organizational tool 1716. Sinceobjects organizational tool 1716,multi-touch display device 1700 determines whether the disengagement ofobjects objects organizational tool 1716. - In certain implementations, the determination of whether the disengagement should trigger an attachment is based on a stored rule or set of rules. For example, a rule may specify that a disengagement of the surface while
multi-touch display device 1000 concurrently detects a separate, continuous engagement ofcontrol strip 1718 will trigger an attachment ofobjects organizational tool 1716. Alternatively or additionally, a rule may specify that a disengagement of the surface preceded by an engagement of the surface at a pressure greater than a predetermined threshold and at points corresponding to whereobjects organizational tool 1716 will trigger an attachment ofobjects organizational tool 1716. - In this example, because
finger 1730 was engagingorganizational tool 1700 at the time whenfingers objects fingers multi-touch display device 1700 determines thatobjects organizational tool 1716. Therefore,multi-touch display device 1700 attachesobjects organizational tool 1716. - Referring now to
FIG. 17( c), as the user dragsfinger 1730 in a clockwise fashion across the surface ofmulti-touch display device 1700, from the third point to a fourth point,multi-touch display device 1700 detects the movement offinger 1730, interprets the detected movement as a request to rotateorganizational tool 1716 by an angular distance corresponding to the distance from the first point to the second point, and rotatesorganizational tool 1716 by the detected distance or a function or the detected distance. In this example, objects 1702, 1704, and 1712 each are attached toorganizational tool 1716 at two or more points. Therefore, when themulti-touch display device 1700 rotatesorganizational tool 1716, themulti-touch display device 1700 corresponding rotates each of displayedobjects -
FIG. 18 is a diagram of a multi-touch display device that provides a two-dimensional rotary organizational tool that illustrates techniques for detaching objects from the two-rotary organizational tool. In this example,multi-touch display device 1800 displays objects 1802, 1804, and 1806 andorganizational tool 1816.Organizational tool 1816 includes acontrol strip 1818. - Referring to
FIG. 18( a), a user has engaged the surface ofmulti-touch display device 1800 withfinger 1826 at a first point corresponding to object 1802. Themulti-touch display device 1800 detects the input offinger 1826, and associates the input withobject 1802. - After detecting the engagement of
finger 1826,multi-touch display device 1800 determines whether any future movements of thefinger 1826 should trigger a detachment ofobject 1802. In this example,multi-touch display device 1800 determines thatobject 1802 should be detached fromorganizational tool 1816. Thereafter, in reference toFIG. 18( b), the user dragsfinger 1826 across the surface of themulti-touch display device 1800 in a rightward motion from the first point to a second point. In response to detecting the movement offinger 1826,multi-touch display device 1800 translatesobject 1802 by a distance corresponding to a distance from the first point to the second point. In some implementations, multi-touch display device 1810 detachesobject 1802 fromorganizational tool 1816 whenmulti-touch display device 1800 detects thatfinger 1826 disengages the surface ofmulti-touch display device 1800 and determines thatobject 1802 no longer overlapsorganizational tool 1816. -
FIGS. 19( a)-19(c) are diagrams of a multi-touch display device that illustrate an example of the multi-touch display device automatically adjusting the angular orientation of an object upon attaching the object to a two-dimensional rotary organizational tool. In these figures,multi-touch display device 1900 displays objects 1902, 1904, 1906, and 1908 andorganizational tool 1916.Organizational tool 1916 includes ancontrol strip 1918.Objects organizational tool 1916. - Referring to
FIG. 19( a), objects 1904, 1906, and 1908 are attached to the two-dimensional rotary organizational tool such that their angular orientations are consistent with radial axes of the two-dimensional rotary organizational tool. Stated differently, the vertical axes of each ofobjects organizational tool 1916, and the horizontal axes of each ofobjects organizational tool 1916. - As illustrated in the sequence of
FIGS. 19( a)-19(c), when themulti-touch display device 1900 attachesobject 1902 to the organizational tool 1616 (shown inFIG. 19( c)) the multi-touch display device automatically adjusts the angular rotation ofobject 1902 such that it is consistent with a corresponding radial axis oforganizational tool 1916. To adjust the angular rotation ofobject 1902,multi-touch display device 1900 may, for example, determine the vertical and/or horizontal axes ofobject 1902, determine the corresponding radial axis oforganizational tool 1616 at the point at which object 1902 is being attached, and rotateobject 1902 such that its vertical axes ofobject 1902 is parallel to the corresponding radial axis oforganizational tool 1616 and/or the horizontal axis ofobject 1902 is orthogonal to the corresponding radial axis oforganizational tool 1616. - The
multi-touch display device 1900 adjusting the angular rotation ofobject 1902 to be consistent with a corresponding radial axis oforganizational tool 1916 is important where, as shown inFIG. 19( d),multi-touch display device 1900 displaysorganizational tool 1916 at the top portion ofmulti-touch display device 1900 and such thatmulti-touch display device 1900 only displays a bottom portion oforganizational tool 1916. Whenmulti-touch display device 1900 makes the angular rotation of the objects attached toorganizational tool 1916 to be consistent with a corresponding radial axis oforganizational tool 1916,multi-touch display device 1900 will display the attached objects such that a user will have access to attached objects at an orientation appropriate for the user to properly view the object. Alternatively,multi-touch display device 1900 displaysorganizational tool 1916 at the bottom portion ofmulti-touch display device 1900 and such thatmulti-touch display device 1900 only displays a top portion oforganizational tool 1916. In such an example,multi-touch display device 1900 may rotate the angular orientation of each attached object one hundred and eighty degrees with respect to the corresponding radial axis oforganizational tool 1916, in order to produce a similar visual effect as shown inFIG. 19( d). - In certain implementations,
multi-touch display device 1900 may also determine a preferred orientation for objects that are detached fromorganizational tool 1916, such thatmulti-touch display device 1900 displays the detached object consistent with the orientation ofmulti-touch display device 1900 with respect to the user, and automatically rotate the detached object to the preferred orientation. As illustrated in the sequence ofFIGS. 19( e)-19(f),multi-touch display device 1900 automatically adjusting the angular orientation of an object upon detachingobject 1902 fromorganizational tool 1916. To adjust the angular rotation ofobject 1902,multi-touch display device 1900 may, for example, determine the vertical and/or horizontal axes ofmulti-touch display device 1900 with relation to the viewing angle of the viewer, determine the vertical and/or horizontal axes of theobject 1902, and rotateobject 1902 such that its vertical axes and/or horizontal axes corresponds to the vertical and/or horizontal axes ofmulti-touch display device 1900. - As discussed above, one effect of an object being attached to an organizational tool is that when the multi-touch display device applies transformations to the organizational tool, the multi-touch display device also may apply transformations to the objects attached to the organizational tool as a consequence of their attachment. However, when the multi-touch display device applies a transformation to an organization tool, the transformations that the multi-touch display device applies to objects attached to the organizational tool may depend upon how the objects are attached to the organizational tool.
- In some cases, the multi-touch display device may attach an object to an organizational tool at only a single point. In such cases, transformations applied to the organizational tool only impact an attached object (i.e., cause a corresponding transformation to be applied to the attached object) if the transformations applied to the organizational tool impact the point upon the organizational tool at which the object is attached.
- The manner in which all objects are attached to a organizational tool may be set for a given organizational tool by the user or they may be automatically preset upon creation by the multi-touch display device. Alternatively, the manner in which objects are attached to a organizational tool may be detected separately for each object upon being attached to the organizational tool.
-
FIG. 20 illustrates a multi-touch display device performing transformations to an organizational tool and its attached objects when the attached objects are attached to the organizational tool at only one point. As illustrated inFIG. 20( a), objects 2002, 2004, and 2006 each are attached toorganizational tool 2016 at one point. Therefore, when themulti-touch display device 2000 rotatesorganizational tool 2016 in response to detecting input byfinger 2026 engaging the organizational tool, as shown inFIG. 20( b), themulti-touch display device 2000 corresponding translates each of displayedobjects organizational tool 2016 impacts the point upon theorganizational tool 2016 at which objects 2002, 2004, and 2006 are attached. - In some implementations, a multi-touch display device may provide controls for further organizing objects attached to an organizational tool provided by the multi-touch display device even after the objects have been attached to the organizational tool. For example, the multi-touch display device may attach an object to the organizational tool such that the object is attached to the organizational tool at the position occupied by the object on the organizational tool at the point in time when the multi-touch display device determined to attach the object to the organizational tool irrespective of how many other objects also are attached to the organizational tool at the same or similar positions. Furthermore, the multi-touch display device may preserve the angular orientation of the an object upon attaching the object to an organizational tool. This may lead to a cluttering of the organizational tool as more and more objects are attached to the organizational tool. Therefore, the multi-touch display device may provide controls for rearranging the objects attached to an organizational tool into a more organized fashion.
-
FIG. 21 illustrates a multi-touch display device providing controls for organizing objects attached to an organizational tool. - As illustrated in
FIG. 21( a),organizational tool 2118 is operating in a free (“messy”) attachment mode such that themulti-touch display device 2100 preserves the angular orientation of objects upon attaching the objects to theorganizational tool 2118 and such that the objects are attached to theorganizational tool 2118 at the positions occupied by the objects at the points in time when themulti-touch display device 2100 determines to attach the objects to the organizational tool.Multi-touch display device 2100 may display a button to indicate in which attachment modeorganizational tool 2118 is operating. Thus, inFIG. 21( a),multi-touch display device 2100 displays a “Messy” button in the center oforganizational tool 2118. In the event that the user eventually wishes to organize the objects that were attached in the free attachment mode, theorganizational tool 2118 provides a control to toggle theorganizational tool 2118 into an organized (“clean”) attachment mode. -
FIG. 21( b) illustrates a possible result of a transition from a free attachment mode to an organized attachment mode. Thus, upon detecting an input from a user indicating a request to implement organized attachment mode,organizational tool 2118 is partitioned into sections (similar in this example to pie slices of organizational tool 2118) to accommodateobjects FIGS. 19( a)-19(c)). While operatingorganizational tool 2118 in organized mode,multi-touch display device 2100 may display a “Clean” button in the center oforganizational tool 2118. - Furthermore, in
FIG. 21( b), a user is engaging the surface ofmulti-touch display device 2100 withfingers objects fingers objects organizational tool 2118.Multi-touch display device 2100 detects the input offingers objects objects fingers objects multi-touch display device 2100 displays theobjects FIG. 21( c). -
Multi-touch display device 2100 detects the disengagement offingers objects organizational tool 2118. In order to attachobjects organizational tool 2118 in organized mode, however,multi-touch display device 2100 rearranges the already attachedobjects multi-touch display device 2100 first resizes all attachedobjects organizational tool 2118. Objects that are engaged by user 2124 may be given priority to the position at which they are moved, and all other objects are moved to the next nearest available position. Thus, inFIG. 21( d),multi-touch display device 2100 gives positional priority toobjects Multi-touch display device 2100 shiftsobjects organizational tool 2118 and were not engaged by the user, to the next closest position that is available around the circumference. Finally,multi-touch display device 2100 reorients each attached object such that each object is consistent with a corresponding radial axis oforganizational tool 2118. - The described systems, methods, and techniques may be implemented in digital electronic circuitry, computer hardware, firmware, software, or in combinations of these elements. Apparatuses embodying these techniques may include appropriate input and output devices, a computer processor, and a tangible computer-readable storage medium on which a computer program or other computer-readable instructions are stored for execution by one or more processing devices (e.g., a programmable processor).
- A process embodying these techniques may be performed by a programmable processor executing a program of instructions to perform desired functions by operating on input data and generating appropriate output. The techniques may be implemented in one or more computer programs that are executable on a programmable system including at least one programmable processor coupled to receive data and instructions from, and to transmit data and instructions to, a data storage system, at least one input device, and at least one output device. Each computer program may be implemented in a high-level procedural or object-oriented programming language, or in assembly or machine language if desired; and in any case, the language may be a compiled or interpreted language.
- Suitable processors include, by way of example, both general and special purpose microprocessors. Generally, a processor will receive instructions and data from a read-only memory and/or a random access memory. Storage devices suitable for storing computer program instructions and data include all forms of non-volatile memory, including by way of example semiconductor memory devices, such as Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), and flash memory devices; magnetic disks such as internal hard disks and removable disks; magneto-optical disks; and Compact Disc Read-Only Memory (CD-ROM). Any of the foregoing may be supplemented by, or incorporated in, specially-designed application-specific integrated circuits (ASICs).
- Multi-touch display devices encompass a wide variety of display devices and associated systems and components. Some multi-touch display devices require physical contact with a surface of the multi-touch display device in order to receive input. For example, such a multi-touch display device may receive input by detecting contact with a surface of the multi-touch display device by a finger, a stylus, some other mechanical, electro-mechanical, or magnetic input mechanism and/or any combination of multiple such input mechanisms at the same time. Furthermore, some such multi-touch display devices may be configured such that the surface that receives input may appear to be the same surface on which the multi-touch display device displays objects (whether or not the surface that receives input actually is the same surface as the surface on which the multi-touch display device displays objects). Alternatively, other such multi-touch display devices may receive input on a surface that is clearly remote and distinct from the surface on which the multi-touch display device displays objects. One example of such a multi-touch display system is a multi-point input capable standalone tablet that provides input to a remote and distinct display.
- Other multi-touch display devices do not require physical contact with the surface of the multi-touch display device in order to receive input. For example, such multi-touch display devices may receive input by detecting the presence of a finger, a stylus, some other mechanical, electro-mechanical, or magnetic input mechanism and/or any combination of multiple such input mechanisms in the vicinity of the surface of the multi-touch display device even when such input mechanisms are not in physical contact with the surface of the multi-touch display device.
- Furthermore, the various different transformations and annotations disclosed herein may be implemented by any other type of multi-point computing system configured to receive multiple inputs at the same, including, for example, systems configured to receive concurrent input from multiple pointing devices (e.g., multiple computer mice) and/or concurrent input from one or more pointing devices and another input device (e.g., a keyboard). Moreover, some of the various different transformations and annotations disclosed herein are not limited to implementation on a multi-touch device and thus may be implemented on a single-point device.
- Various modifications may be made. For example, useful results still may be achieved if steps of the disclosed techniques are performed in a different order. Moreover, useful results may be achieved by combining various steps or components of the various disclosed techniques in a different manner and/or if components of the disclosed systems are combined in a different manner and/or replaced or supplemented by other components.
Claims (7)
1. A computer-implemented method for merging two sets of objects displayed on a multi-input display device, the method comprising:
defining a first target element that enables objects displayed on a multi-input display device to be grouped together through interaction with the first target element;
visually displaying, on the multi-input display device, at least a portion of the first target element;
defining a second target element that enables objects displayed on the multi-input display device to be grouped together through interaction with the second target element;
visually displaying, on the multi-input display device, at least a portion of the second target element;
visually displaying, on the multi-input display device, a first set of objects, each constituent object of the first set of objects at least partially overlapping the first target element;
invoking processes that establish relationships between the constituent objects of the first set of objects and one or more corresponding positions on the first target element that are overlapped by the constituent objects of the first set of objects, the processes causing the relationships between the constituent objects of the first set of objects and the corresponding positions on the first target element to be maintained when transformations are applied to the first target element such that transformations applied to the first target element also are applied to the constituent objects of the first set of objects;
visually displaying, on the multi-input display device, a second set of objects, each constituent object of the second set of objects at least partially overlapping the second target element;
invoking processes that establish relationships between the constituent objects of the second set of objects and one or more corresponding positions on the second target element that are overlapped by the constituent objects of the second set of objects, the processes causing the relationships between the constituent objects of the second set of objects and the corresponding positions on the second target element to be maintained when transformations are applied to the second target element such that transformations applied to the second target element also are applied to the constituent objects of the second set of objects;
detecting that an input mechanism has engaged at least the displayed portion of the second target element;
in response to detecting that the input mechanism has engaged at least the displayed portion of the second target element, monitoring movement of the input mechanism while the input mechanism remains engaged with at least the displayed portion of the second target element;
translating the second target element and the constituent objects of the second set of objects as a function of the monitored movement of the input mechanism such that at least a portion of the displayed portion of the second target element is overlapping the first target element;
after translating the second target element and the constituent objects of the second set of objects, detecting that the input mechanism has disengaged the second target element;
determining, upon detecting that the input mechanism has disengaged the second target element, that at least the portion of the displayed portion of the second target element is overlapping at least a portion of the first target element as a consequence of the translation of the second target element; and
in response to determining that at least the portion of the displayed portion of the second target element is overlapping the portion of the first target element upon detecting that the input mechanism has disengaged the second target element, invoking processes that establish new relationships between the constituent objects of the second set of objects and one or more corresponding positions on the first target element, the processes causing the relationships between the constituent objects of the second set of objects and the corresponding positions on the first target element to be maintained when transformations are applied to the first target element such that transformations applied to the first target element also are applied to the constituent objects of the second set of objects.
2. The method of claim 1 , further comprising:
after invoking processes that establish new relationships between the constituent objects of the second set of objects and one or more corresponding positions on the first target element, the processes causing the relationships between the constituent objects of the second set of objects and the corresponding positions on the first target element to be maintained when transformations are applied to the first target element such that transformations applied to the first target element also are applied to the constituent objects of the second set of objects, detecting that an input mechanism has newly engaged the first target element;
in response to detecting that the first target element has newly engaged the input mechanism, monitoring movement of the input mechanism while the input mechanism remains engaged with the first target element;
applying a transformation to the first target element as a function of the monitored movement of the input mechanism;
applying a corresponding transformation to the first set of objects as a consequence of applying the transformation to the first target element and the established relationships between the first set of objects and the corresponding positions on the first target element; and
applying the corresponding transformation to the second set of objects as a consequence of applying the transformation to the first target element and the established relationships between the second set of objects and the corresponding positions on the first target element.
3. The method of claim 1 , wherein:
translating the second target element and the constituent objects of the second set of objects as a function of the monitored movement of the input mechanism such that at least a portion of the displayed portion of the second target element is overlapping the first target element includes visually displaying the second target element as overlapping at least a portion of the first target element and any constituent objects of the first set of objects for which relationships have been established with positions on the first target element that correspond to positions at which the second target element overlaps the first target element; and
invoking processes that establish new relationships between the constituent objects of the second set of objects and one or more corresponding positions on the first target element that are overlapped by the constituent objects of the second set of objects, the processes causing the relationships between the constituent objects of the second set of objects and the corresponding positions on the first target element to be maintained when transformations are applied to the first target element such that transformations applied to the first target element also are applied to the constituent objects of the second set of objects includes invoking a process that establishes a relationship between the second target element and one or more corresponding positions on the first target element that causes the relationship between the second target element and the corresponding positions on the first target element to be maintained when transformations are applied to the first target element such that transformations applied to the first target element also are applied to the second target element.
4. A computer-implemented method for merging two sets of objects displayed on a multi-input display device, the method comprising:
defining a first target element that enables objects displayed on a multi-input display device to be grouped together through interaction with the first target element;
visually displaying, on the multi-input display device, at least a portion of the first target element;
defining a second target element that enables objects displayed on the multi-input display device to be grouped together through interaction with the second target element;
visually displaying, on the multi-input display device, at least a portion of the second target element;
visually displaying, on the multi-input display device, a first set of objects, each constituent object of the first set of objects at least partially overlapping the first target element;
invoking processes that establish relationships between the constituent objects of the first set of objects and one or more corresponding positions on the first target element that are overlapped by the constituent objects of the first set of objects, the processes causing the relationships between the constituent objects of the first set of objects and the corresponding positions on the first target element to be maintained when transformations are applied to the first target element such that transformations applied to the first target element also are applied to the constituent objects of the first set of objects;
visually displaying, on the multi-input display device, a second set of objects, each constituent object of the second set of objects at least partially overlapping the second target element;
invoking processes that establish relationships between the constituent objects of the second set of objects and one or more corresponding positions on the second target element that are overlapped by the constituent objects of the second set of objects, the processes causing the relationships between the constituent objects of the second set of objects and the corresponding positions on the second target element to be maintained when transformations are applied to the second target element such that transformations applied to the second target element also are applied to the constituent objects of the second set of objects;
detecting that an input mechanism has engaged the second target element;
in response to detecting that the input mechanism has engaged the second target element, monitoring movement of the input mechanism while the input mechanism remains engaged with the second target element;
translating the second target element and the constituent objects of the second set of objects as a function of the monitored movement of the input mechanism such that at least a portion of the second target element is overlapping the first target element;
after translating the second target element and the constituent objects of the second set of objects, detecting that the input mechanism has disengaged the second target element;
determining, upon detecting that the input mechanism has disengaged the second target element, that the second target element is overlapping at least a portion of the first target element as a consequence of the translation of the second target element; and
in response to determining that the second target element is overlapping the portion of the first target element upon detecting that the input mechanism has disengaged the second target element, invoking processes that establish new relationships between the constituent objects of the second set of objects and one or more corresponding positions on the first target element, the processes causing the relationships between the constituent objects of the second set of objects and the corresponding positions on the first target element to be maintained when transformations are applied to the first target element such that transformations applied to the first target element also are applied to the constituent objects of the second set of objects, wherein:
translating the second target element and the constituent objects of the second set of objects as a function of the monitored movement of the input mechanism includes translating the second target element and the constituent objects of the second set of objects such that the entirety of the second target element is overlapping the first target element;
determining that the second target element is overlapping at least a portion of the first target element as a consequence of the translation of the second target element includes determining that the entirety of the second target element is overlapping the first target element;
invoking processes that establish new relationships between the constituent objects of the second set of objects and one or more corresponding positions on the first target element that are overlapped by the constituent objects of the second set of objects, the processes causing the relationships between the constituent objects of the second set of objects and the corresponding positions on the first target element to be maintained when transformations are applied to the first target element in response to determining that the second target element is overlapping the first target element, includes, in response to determining that the entirety of the second target element is overlapping the first target element:
terminating the relationships between the constituent objects of the second set of objects and the one or more corresponding positions on the second target element; and
eliminating the second target element from display on the multi-input display device.
5. The method of claim 4 , further comprising:
after eliminating the second target element from display on the multi-input display device, detecting that an input mechanism has newly engaged the first target element;
in response to detecting that the first target element has newly engaged the input mechanism, monitoring movement of the input mechanism while the input mechanism remains engaged with the first target element;
applying a transformation to the first target element as a function of the monitored movement of the input mechanism;
applying a corresponding transformation to the first set of objects as a consequence of applying the transformation to the first target element and the established relationships between the first set of objects and the corresponding positions on the first target element; and
applying the corresponding transformation to the second set of objects as a consequence of applying the transformation to the first target element and the established relationships between the second set of objects and the corresponding positions on the first target element.
6. The method of claim 4 , wherein invoking processes that establish new relationships between the constituent objects of the second set of objects and one or more corresponding positions on the first target element that are overlapped by the constituent objects of the second set of objects, the processes causing the relationships between the constituent objects of the second set of objects and the corresponding positions on the first target element to be maintained when transformations are applied to the first target element such that transformations applied to the first target element also are applied to the constituent objects of the second set of objects includes:
determining that, as a consequence of having translated the second target element and the constituent objects of the second set of objects, a particular object of the second set of objects is overlapping at least a portion of another object of the first set of objects; and
as a consequence of determining that the particular object of the second set of objects is overlapping at least a portion of the other object of the first set of objects, transforming the particular object such that it does not overlap any object of the first set of objects but continues to overlap at least a portion of the first target element.
7. A computer-implemented method for merging two sets of objects displayed on a multi-input display device, the method comprising:
defining a first target element that enables objects displayed on a multi-input display device to be grouped together through interaction with the first target element;
visually displaying, on the multi-input display device, at least a portion of the first target element;
defining a second target element that enables objects displayed on the multi-input display device to be grouped together through interaction with the second target element;
visually displaying, on the multi-input display device, at least a portion of the second target element;
visually displaying, on the multi-input display device, a first set of objects, each constituent object of the first set of objects at least partially overlapping the first target element;
invoking processes that establish relationships between the constituent objects of the first set of objects and one or more corresponding positions on the first target element that are overlapped by the constituent objects of the first set of objects, the processes causing the relationships between the constituent objects of the first set of objects and the corresponding positions on the first target element to be maintained when transformations are applied to the first target element such that transformations applied to the first target element also are applied to the constituent objects of the first set of objects;
visually displaying, on the multi-input display device, a second set of objects, each constituent object of the second set of objects at least partially overlapping the second target element;
invoking processes that establish relationships between the constituent objects of the second set of objects and one or more corresponding positions on the second target element that are overlapped by the constituent objects of the second set of objects, the processes causing the relationships between the constituent objects of the second set of objects and the corresponding positions on the second target element to be maintained when transformations are applied to the second target element such that transformations applied to the second target element also are applied to the constituent objects of the second set of objects;
detecting that an input mechanism has engaged at least the displayed portion of the second target element;
in response to detecting that the input mechanism has engaged at least the displayed portion of the second target element, monitoring movement of the input mechanism while the input mechanism remains engaged with at least the displayed portion of the second target element;
translating the second target element and the constituent objects of the second set of objects as a function of the monitored movement of the input mechanism such that at least a portion of the displayed portion of the second target element is overlapping the first target element;
after translating the second target element and the constituent objects of the second set of objects, detecting that the input mechanism has disengaged the second target element;
determining, upon detecting that the input mechanism has disengaged the second target element, that at least the portion of the displayed portion of the second target element is overlapping at least a portion of the first target element as a consequence of the translation of the second target element;
in response to determining that at least the portion of the displayed portion of the second target element is overlapping the portion of the first target element upon detecting that the input mechanism has disengaged the second target element, invoking processes that establish new relationships between the constituent objects of the second set of objects and one or more corresponding positions on the first target element, the processes causing the relationships between the constituent objects of the second set of objects and the corresponding positions on the first target element to be maintained when transformations are applied to the first target element such that transformations applied to the first target element also are applied to the constituent objects of the second set of objects;
in response to determining that at least the portion of the displayed portion of the second target element is overlapping the portion of the first target element upon detecting that the input mechanism has disengaged the second target element, invoking a process that establishes a relationship between the second target element and one or more corresponding positions on the first target element, the process causing the relationship between the second target element and the corresponding positions on the first target element to be maintained when transformations are applied to the first target element such that transformations applied to the first target element also are applied to the second target element;
detecting that the input mechanism has engaged at least the displayed portion of the second target element;
in response to detecting that the input mechanism has engaged at least the displayed portion of the second target element, invoking a process to determine if future movements by the input mechanism while the input mechanism remains engaged with at least the displayed portion of the second target element are to terminate the relationship between the second set of objects and the corresponding positions on the first target element and the relationship between the second target element and the corresponding positions on the first target element;
based on results of the process to determine if future movements by the input mechanism while the input mechanism remains engaged with at least the displayed portion of the second target element are to terminate the relationship between the second set of objects and the corresponding positions on the first target element and the relationship between the second target element and the corresponding positions on the first target element, determining that future movements by the input mechanism are to terminate the relationship between the second set of objects and the corresponding positions on the first target element and the relationship between the second target element and the corresponding positions on the first target element;
after determining that future movements by the input mechanism are to terminate the relationship between the second set of objects and the corresponding positions on the first target element and the relationship between the second target element and the corresponding positions on the first target element, detecting movement of the input mechanism while the input mechanism remains engaged with at least the displayed portion of the second target element; and
as a consequence of detecting the movement of the input mechanism while the input mechanism remains engaged with at least the displayed portion of the second target element and having determined that future movements by the input mechanism are to terminate the relationship between the second set of objects and the corresponding positions on the first target element and the relationship between the second target element and the corresponding positions on the first target element, terminating the relationship between the second set of objects and corresponding positions on the first target element and the relationship between the second target element and the corresponding positions on the first target element such that the relationships between the second set of objects and the corresponding positions on the first target element and the relationship between the second target element and the corresponding positions on the first target element are no longer maintained when transformations are applied to the first target element.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/404,439 US20130069860A1 (en) | 2009-05-21 | 2012-02-24 | Organizational Tools on a Multi-touch Display Device |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US18040809P | 2009-05-21 | 2009-05-21 | |
US12/785,134 US8473862B1 (en) | 2009-05-21 | 2010-05-21 | Organizational tools on a multi-touch display device |
US13/404,439 US20130069860A1 (en) | 2009-05-21 | 2012-02-24 | Organizational Tools on a Multi-touch Display Device |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/785,134 Continuation US8473862B1 (en) | 2009-05-21 | 2010-05-21 | Organizational tools on a multi-touch display device |
Publications (1)
Publication Number | Publication Date |
---|---|
US20130069860A1 true US20130069860A1 (en) | 2013-03-21 |
Family
ID=47880192
Family Applications (9)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/785,134 Active 2031-05-22 US8473862B1 (en) | 2009-05-21 | 2010-05-21 | Organizational tools on a multi-touch display device |
US13/404,473 Active 2030-10-24 US10031608B2 (en) | 2009-05-21 | 2012-02-24 | Organizational tools on a multi-touch display device |
US13/404,718 Abandoned US20130093792A1 (en) | 2009-05-21 | 2012-02-24 | Organizational Tools on a Multi-touch Display Device |
US13/404,899 Abandoned US20130093694A1 (en) | 2009-05-21 | 2012-02-24 | Organizational Tools on a Multi-touch Display Device |
US13/404,583 Expired - Fee Related US8429567B2 (en) | 2009-05-21 | 2012-02-24 | Organizational tools on a multi-touch display device |
US13/404,439 Abandoned US20130069860A1 (en) | 2009-05-21 | 2012-02-24 | Organizational Tools on a Multi-touch Display Device |
US13/404,926 Active 2033-04-26 US9671890B2 (en) | 2009-05-21 | 2012-02-24 | Organizational tools on a multi-touch display device |
US13/404,494 Expired - Fee Related US8499255B2 (en) | 2009-05-21 | 2012-02-24 | Organizational tools on a multi-touch display device |
US13/953,680 Active 2032-01-01 US9626034B2 (en) | 2009-05-21 | 2013-07-29 | Organizational tools on a multi-touch display device |
Family Applications Before (5)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/785,134 Active 2031-05-22 US8473862B1 (en) | 2009-05-21 | 2010-05-21 | Organizational tools on a multi-touch display device |
US13/404,473 Active 2030-10-24 US10031608B2 (en) | 2009-05-21 | 2012-02-24 | Organizational tools on a multi-touch display device |
US13/404,718 Abandoned US20130093792A1 (en) | 2009-05-21 | 2012-02-24 | Organizational Tools on a Multi-touch Display Device |
US13/404,899 Abandoned US20130093694A1 (en) | 2009-05-21 | 2012-02-24 | Organizational Tools on a Multi-touch Display Device |
US13/404,583 Expired - Fee Related US8429567B2 (en) | 2009-05-21 | 2012-02-24 | Organizational tools on a multi-touch display device |
Family Applications After (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/404,926 Active 2033-04-26 US9671890B2 (en) | 2009-05-21 | 2012-02-24 | Organizational tools on a multi-touch display device |
US13/404,494 Expired - Fee Related US8499255B2 (en) | 2009-05-21 | 2012-02-24 | Organizational tools on a multi-touch display device |
US13/953,680 Active 2032-01-01 US9626034B2 (en) | 2009-05-21 | 2013-07-29 | Organizational tools on a multi-touch display device |
Country Status (1)
Country | Link |
---|---|
US (9) | US8473862B1 (en) |
Cited By (34)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20120182224A1 (en) * | 2011-01-13 | 2012-07-19 | Sony Computer Entertainment America Llc | Handing control of an object from one touch input to another touch input |
US20130346894A1 (en) * | 2012-06-04 | 2013-12-26 | Htc Corporation | Method, apparatus and computer-readable medium for adjusting size of screen object |
US20140118273A1 (en) * | 2012-11-01 | 2014-05-01 | Samsung Electronics Co., Ltd. | Method and apparatus for controlling virtual screen |
US20140164967A1 (en) * | 2012-12-06 | 2014-06-12 | Konica Minolta, Inc. | Object operation apparatus and non-transitory computer-readable storage medium |
US20140168714A1 (en) * | 2012-12-19 | 2014-06-19 | Konica Minolta, Inc. | Image processing terminal, image processing system, and computer-readable storage medium storing control program of image processing terminal |
US20150149954A1 (en) * | 2013-11-28 | 2015-05-28 | Acer Incorporated | Method for operating user interface and electronic device thereof |
EP2887195A1 (en) * | 2013-12-20 | 2015-06-24 | Dassault Systèmes | A computer-implemented method for designing a three-dimensional modeled object |
US20150278238A1 (en) * | 2014-03-31 | 2015-10-01 | Fuji Xerox Co., Ltd. | Information processing apparatus and non-transitory computer readable medium |
WO2015160132A1 (en) * | 2014-04-14 | 2015-10-22 | Samsung Electronics Co., Ltd. | Multi-display system, electronic device, and content output method |
US20160054908A1 (en) * | 2014-08-22 | 2016-02-25 | Zoho Corporation Private Limited | Multimedia applications and user interfaces |
USD760295S1 (en) * | 2014-09-24 | 2016-06-28 | Lexmark International, Inc. | Portion of a display screen with icon |
CN105814511A (en) * | 2013-10-14 | 2016-07-27 | 诺基亚技术有限公司 | Deviational plane wrist input |
GB2536090A (en) * | 2015-03-06 | 2016-09-07 | Collaboration Platform Services Pte Ltd | Multi-user information sharing system |
AU2015246821B2 (en) * | 2014-04-14 | 2017-09-14 | Samsung Electronics Co., Ltd. | Multi-display system, electronic device, and content output method |
WO2017172457A1 (en) * | 2016-03-28 | 2017-10-05 | Microsoft Technology Licensing, Llc | Applications for multi-touch input detection |
US20170336923A1 (en) * | 2016-05-19 | 2017-11-23 | Microsoft Technology Licensing, Llc | Gesture-controlled piling of displayed data |
US10019151B2 (en) * | 2013-02-08 | 2018-07-10 | Motorola Solutions, Inc. | Method and apparatus for managing user interface elements on a touch-screen device |
US20190155623A1 (en) * | 2017-11-23 | 2019-05-23 | Research & Business Foundation Sungkyunkwan University | Method for user based application grouping under multi-user environment and table top display apparatus for performing the same |
US20190286245A1 (en) * | 2016-11-25 | 2019-09-19 | Sony Corporation | Display control device, display control method, and computer program |
EP3662357A4 (en) * | 2017-08-31 | 2020-07-29 | Samsung Electronics Co., Ltd. | Display apparatus for providing preview ui and method of controlling display apparatus |
US10790046B2 (en) * | 2012-02-24 | 2020-09-29 | Perkinelmer Informatics, Inc. | Systems, methods, and apparatus for drawing and editing chemical structures on a user interface via user gestures |
US20210357098A1 (en) * | 2020-05-15 | 2021-11-18 | Fujifilm Business Innovation Corp. | Information presentation system and non-transitory computer readable medium |
US11250208B2 (en) * | 2019-04-08 | 2022-02-15 | Microsoft Technology Licensing, Llc | Dynamic whiteboard templates |
US11249627B2 (en) | 2019-04-08 | 2022-02-15 | Microsoft Technology Licensing, Llc | Dynamic whiteboard regions |
US11409405B1 (en) * | 2020-12-22 | 2022-08-09 | Facebook Technologies, Llc | Augment orchestration in an artificial reality environment |
US11592979B2 (en) | 2020-01-08 | 2023-02-28 | Microsoft Technology Licensing, Llc | Dynamic data relationships in whiteboard regions |
US11636655B2 (en) | 2020-11-17 | 2023-04-25 | Meta Platforms Technologies, Llc | Artificial reality environment with glints displayed by an extra reality device |
US11651573B2 (en) | 2020-08-31 | 2023-05-16 | Meta Platforms Technologies, Llc | Artificial realty augments and surfaces |
US11714543B2 (en) * | 2018-10-01 | 2023-08-01 | T1V, Inc. | Simultaneous gesture and touch control on a display |
US11748944B2 (en) | 2021-10-27 | 2023-09-05 | Meta Platforms Technologies, Llc | Virtual object structures and interrelationships |
US11762952B2 (en) | 2021-06-28 | 2023-09-19 | Meta Platforms Technologies, Llc | Artificial reality application lifecycle |
US11769304B2 (en) | 2020-08-31 | 2023-09-26 | Meta Platforms Technologies, Llc | Artificial reality augments and surfaces |
US11798247B2 (en) | 2021-10-27 | 2023-10-24 | Meta Platforms Technologies, Llc | Virtual object structures and interrelationships |
US11947862B1 (en) | 2022-12-30 | 2024-04-02 | Meta Platforms Technologies, Llc | Streaming native application content to artificial reality devices |
Families Citing this family (46)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9772751B2 (en) | 2007-06-29 | 2017-09-26 | Apple Inc. | Using gestures to slide between user interfaces |
JP5596914B2 (en) * | 2008-09-16 | 2014-09-24 | 富士通株式会社 | Terminal device, display control method, and program having display function |
US9081492B2 (en) * | 2009-06-15 | 2015-07-14 | Nokia Technologies Oy | Apparatus, method, computer program and user interface |
US20110246875A1 (en) * | 2010-04-02 | 2011-10-06 | Symantec Corporation | Digital whiteboard implementation |
US9244606B2 (en) | 2010-12-20 | 2016-01-26 | Apple Inc. | Device, method, and graphical user interface for navigation of concurrently open software applications |
JP5664249B2 (en) * | 2011-01-06 | 2015-02-04 | ソニー株式会社 | Information processing apparatus, information processing method, and information processing program |
US9239672B2 (en) * | 2011-04-20 | 2016-01-19 | Mellmo Inc. | User interface for data comparison |
US8819586B2 (en) | 2011-05-27 | 2014-08-26 | Microsoft Corporation | File access with different file hosts |
EP2718797A4 (en) * | 2011-06-08 | 2015-02-18 | Sony Corp | Information processing device, information processing method and computer program product |
KR101948645B1 (en) * | 2011-07-11 | 2019-02-18 | 삼성전자 주식회사 | Method and apparatus for controlling contents using graphic object |
KR101873734B1 (en) * | 2011-07-19 | 2018-07-03 | 엘지전자 주식회사 | Mobile terminal and Method for controlling display thereof |
EP2756379A1 (en) * | 2011-09-13 | 2014-07-23 | Newman Infinite, Inc. | Apparatus, method and computer-readable storage medium for manipulating a user interface element |
KR20130063196A (en) * | 2011-12-06 | 2013-06-14 | 현대자동차주식회사 | A divided screen interlocking control method and apparatus thereof using dynamic touch interaction |
WO2013104054A1 (en) * | 2012-01-10 | 2013-07-18 | Smart Technologies Ulc | Method for manipulating a graphical object and an interactive input system employing the same |
JP5832339B2 (en) * | 2012-03-04 | 2015-12-16 | アルパイン株式会社 | Scale display method and apparatus for scaling operation |
CN109298789B (en) | 2012-05-09 | 2021-12-31 | 苹果公司 | Device, method and graphical user interface for providing feedback on activation status |
CN105260049B (en) * | 2012-05-09 | 2018-10-23 | 苹果公司 | For contacting the equipment for carrying out display additional information, method and graphic user interface in response to user |
JP6082458B2 (en) | 2012-05-09 | 2017-02-15 | アップル インコーポレイテッド | Device, method, and graphical user interface for providing tactile feedback of actions performed within a user interface |
JP5502943B2 (en) * | 2012-06-29 | 2014-05-28 | 楽天株式会社 | Information processing apparatus, authentication apparatus, information processing method, and information processing program |
US9046943B1 (en) * | 2012-10-26 | 2015-06-02 | Google Inc. | Virtual control for touch-sensitive devices |
US9298970B2 (en) | 2012-11-27 | 2016-03-29 | Nokia Technologies Oy | Method and apparatus for facilitating interaction with an object viewable via a display |
US11861561B2 (en) | 2013-02-04 | 2024-01-02 | Haworth, Inc. | Collaboration system including a spatial event map |
US10304037B2 (en) | 2013-02-04 | 2019-05-28 | Haworth, Inc. | Collaboration system including a spatial event map |
KR102289203B1 (en) * | 2013-02-07 | 2021-08-12 | 디즈모 아게 | System for organizing and displaying information on a display device |
EP3076276A4 (en) * | 2013-11-28 | 2017-08-09 | Sony Corporation | Information processing device, information processing method, and program |
WO2016029934A1 (en) * | 2014-08-26 | 2016-03-03 | Quizista GmbH | Dynamic boxing of graphical objects, in particular for knowledge quantification |
JP2016053769A (en) * | 2014-09-02 | 2016-04-14 | ソニー株式会社 | Information processing device, information processing method, and program |
US9632664B2 (en) | 2015-03-08 | 2017-04-25 | Apple Inc. | Devices, methods, and graphical user interfaces for manipulating user interface objects with visual and/or haptic feedback |
US9639184B2 (en) | 2015-03-19 | 2017-05-02 | Apple Inc. | Touch input cursor manipulation |
US9921811B2 (en) * | 2015-03-26 | 2018-03-20 | Wal-Mart Stores, Inc. | Multiple display environment for software development projects |
US20160328098A1 (en) | 2015-05-06 | 2016-11-10 | Haworth, Inc. | Virtual workspace viewport location markers in collaboration systems |
US9860451B2 (en) | 2015-06-07 | 2018-01-02 | Apple Inc. | Devices and methods for capturing and interacting with enhanced digital images |
US9830048B2 (en) | 2015-06-07 | 2017-11-28 | Apple Inc. | Devices and methods for processing touch inputs with instructions in a web page |
US20160378291A1 (en) * | 2015-06-26 | 2016-12-29 | Haworth, Inc. | Object group processing and selection gestures for grouping objects in a collaboration system |
US9880735B2 (en) | 2015-08-10 | 2018-01-30 | Apple Inc. | Devices, methods, and graphical user interfaces for manipulating user interface objects with visual and/or haptic feedback |
US10713304B2 (en) * | 2016-01-26 | 2020-07-14 | International Business Machines Corporation | Entity arrangement by shape input |
US10545658B2 (en) * | 2017-04-25 | 2020-01-28 | Haworth, Inc. | Object processing and selection gestures for forming relationships among objects in a collaboration system |
US11934637B2 (en) | 2017-10-23 | 2024-03-19 | Haworth, Inc. | Collaboration system including markers identifying multiple canvases in multiple shared virtual workspaces |
US10817472B2 (en) * | 2017-10-23 | 2020-10-27 | Dropbox, Inc. | Storage organization system with associated storage utilization values |
US10942632B1 (en) | 2017-11-06 | 2021-03-09 | Zocomotion Ltd. | Self-clustering rearrangeable stacks of visible information and active in situ link portals in an infinitely zoomable space on a graphical computing device |
CN109948791B (en) * | 2017-12-21 | 2022-03-29 | 河北科技大学 | Method for optimizing BP neural network by using genetic algorithm and application thereof in positioning |
CN109407960A (en) * | 2018-11-05 | 2019-03-01 | 北京中海创达科技有限公司 | A kind of Intellisense desktop assembly, cognitive method and sensory perceptual system |
WO2020176517A1 (en) * | 2019-02-25 | 2020-09-03 | Haworth, Inc. | Gesture based workflows in a collaboration system |
US11016643B2 (en) * | 2019-04-15 | 2021-05-25 | Apple Inc. | Movement of user interface object with user-specified content |
US11750672B2 (en) | 2020-05-07 | 2023-09-05 | Haworth, Inc. | Digital workspace sharing over one or more display clients in proximity of a main client |
CN113870439A (en) * | 2021-09-29 | 2021-12-31 | 北京百度网讯科技有限公司 | Method, apparatus, device and storage medium for processing image |
Citations (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5835094A (en) * | 1996-12-31 | 1998-11-10 | Compaq Computer Corporation | Three-dimensional computer environment |
US5847707A (en) * | 1994-08-29 | 1998-12-08 | Casio Computer Co., Ltd. | Icon menu display devices and methods |
US6088032A (en) * | 1996-10-04 | 2000-07-11 | Xerox Corporation | Computer controlled display system for displaying a three-dimensional document workspace having a means for prefetching linked documents |
US20020080180A1 (en) * | 1992-04-30 | 2002-06-27 | Richard Mander | Method and apparatus for organizing information in a computer system |
US20040066414A1 (en) * | 2002-10-08 | 2004-04-08 | Microsoft Corporation | System and method for managing software applications in a graphical user interface |
US20050138572A1 (en) * | 2003-12-19 | 2005-06-23 | Palo Alto Research Center, Incorported | Methods and systems for enhancing recognizability of objects in a workspace |
US20050234981A1 (en) * | 2004-04-14 | 2005-10-20 | Manousos Nicholas H | Method and apparatus for creating, assembling, and organizing compound media objects |
US20050240880A1 (en) * | 2004-04-23 | 2005-10-27 | Microsoft Corporation | System and method for displaying stack icons |
US20060004692A1 (en) * | 2003-03-27 | 2006-01-05 | Microsoft Corporation | System and method for filtering and organizing items based on common elements |
US20070192727A1 (en) * | 2006-01-26 | 2007-08-16 | Finley William D | Three dimensional graphical user interface representative of a physical work space |
US20070226652A1 (en) * | 2006-03-23 | 2007-09-27 | Sony Corporation | Information processing apparatus, information processing method, and program thereof |
US20080126979A1 (en) * | 2006-11-29 | 2008-05-29 | Sony Corporation | Content viewing method, content viewing apparatus, and storage medium in which a content viewing program is stored |
US20080307335A1 (en) * | 2007-06-08 | 2008-12-11 | Apple Inc. | Object stack |
US20080307359A1 (en) * | 2007-06-08 | 2008-12-11 | Apple Inc. | Grouping Graphical Representations of Objects in a User Interface |
US20090119590A1 (en) * | 2007-11-05 | 2009-05-07 | Verizon Data Services Inc. | Interactive group content systems and methods |
US20090307623A1 (en) * | 2006-04-21 | 2009-12-10 | Anand Agarawala | System for organizing and visualizing display objects |
US20100083111A1 (en) * | 2008-10-01 | 2010-04-01 | Microsoft Corporation | Manipulation of objects on multi-touch user interface |
US20100095248A1 (en) * | 2008-10-14 | 2010-04-15 | International Business Machines Corporation | Desktop icon management and grouping using desktop containers |
US20100162151A1 (en) * | 2008-12-19 | 2010-06-24 | Microsoft Corporation | Techniques for organizing information on a computing device using movable objects |
US20100229129A1 (en) * | 2009-03-04 | 2010-09-09 | Microsoft Corporation | Creating organizational containers on a graphical user interface |
US20100289753A1 (en) * | 2009-05-14 | 2010-11-18 | Microsoft Corporation | Adjusting organization of media content on display |
US20150205775A1 (en) * | 2008-05-01 | 2015-07-23 | Eric Berdahl | Managing Documents and Document Workspaces |
Family Cites Families (30)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5838317A (en) * | 1995-06-30 | 1998-11-17 | Microsoft Corporation | Method and apparatus for arranging displayed graphical representations on a computer interface |
US5861886A (en) * | 1996-06-26 | 1999-01-19 | Xerox Corporation | Method and apparatus for grouping graphic objects on a computer based system having a graphical user interface |
JP3904671B2 (en) * | 1997-05-28 | 2007-04-11 | 富士通株式会社 | Virtual clay system and simulation method thereof |
US8479122B2 (en) * | 2004-07-30 | 2013-07-02 | Apple Inc. | Gestures for touch sensitive input devices |
US7434177B1 (en) | 1999-12-20 | 2008-10-07 | Apple Inc. | User interface for providing consolidation and access |
US7138983B2 (en) | 2000-01-31 | 2006-11-21 | Canon Kabushiki Kaisha | Method and apparatus for detecting and interpreting path of designated position |
US7299424B2 (en) * | 2002-05-14 | 2007-11-20 | Microsoft Corporation | Lasso select |
US7752572B1 (en) | 2002-12-16 | 2010-07-06 | Sap Ag | Method of grouping objects in a graphical user interface |
US7493573B2 (en) | 2003-02-07 | 2009-02-17 | Sun Microsystems, Inc. | Scrolling vertical column mechanism for cellular telephone |
US7334195B2 (en) * | 2003-10-14 | 2008-02-19 | Microsoft Corporation | System and process for presenting search results in a histogram/cluster format |
US20050108620A1 (en) * | 2003-11-19 | 2005-05-19 | Microsoft Corporation | Method and system for selecting and manipulating multiple objects |
US7743348B2 (en) * | 2004-06-30 | 2010-06-22 | Microsoft Corporation | Using physical objects to adjust attributes of an interactive display application |
US7719523B2 (en) * | 2004-08-06 | 2010-05-18 | Touchtable, Inc. | Bounding box gesture recognition on a touch detecting interactive display |
US20060074928A1 (en) * | 2004-09-28 | 2006-04-06 | Microsoft Corporation | Selection based container listing |
US7454717B2 (en) * | 2004-10-20 | 2008-11-18 | Microsoft Corporation | Delimiters for selection-action pen gesture phrases |
US20080115081A1 (en) | 2006-11-09 | 2008-05-15 | Microsoft Corporation | Enhanced windows management feature |
US7770136B2 (en) * | 2007-01-24 | 2010-08-03 | Microsoft Corporation | Gesture recognition interactive feedback |
JP4888165B2 (en) * | 2007-03-12 | 2012-02-29 | 富士ゼロックス株式会社 | Image processing apparatus and program |
JP4863908B2 (en) * | 2007-03-16 | 2012-01-25 | 株式会社ソニー・コンピュータエンタテインメント | Data processing apparatus, data processing method, and data processing program |
JP4775332B2 (en) | 2007-06-14 | 2011-09-21 | ブラザー工業株式会社 | Image selection apparatus and image selection method |
US8068121B2 (en) | 2007-06-29 | 2011-11-29 | Microsoft Corporation | Manipulation of graphical objects on a display or a proxy device |
US20090058820A1 (en) * | 2007-09-04 | 2009-03-05 | Microsoft Corporation | Flick-based in situ search from ink, text, or an empty selection region |
US8209628B1 (en) * | 2008-04-11 | 2012-06-26 | Perceptive Pixel, Inc. | Pressure-sensitive manipulation of displayed objects |
JP5448370B2 (en) * | 2008-05-20 | 2014-03-19 | キヤノン株式会社 | Information processing apparatus, control method therefor, program, and storage medium |
JP5500855B2 (en) * | 2008-07-11 | 2014-05-21 | キヤノン株式会社 | Information processing apparatus and control method thereof |
US20100031202A1 (en) * | 2008-08-04 | 2010-02-04 | Microsoft Corporation | User-defined gesture set for surface computing |
US20100185965A1 (en) | 2009-01-21 | 2010-07-22 | Frederick Collin Davidson | Artistic file manager |
JP5377143B2 (en) * | 2009-07-29 | 2013-12-25 | 京セラ株式会社 | Portable electronic devices |
US20120026100A1 (en) * | 2010-07-30 | 2012-02-02 | Migos Charles J | Device, Method, and Graphical User Interface for Aligning and Distributing Objects |
US9886188B2 (en) | 2011-08-25 | 2018-02-06 | International Business Machines Corporation | Manipulating multiple objects in a graphic user interface |
-
2010
- 2010-05-21 US US12/785,134 patent/US8473862B1/en active Active
-
2012
- 2012-02-24 US US13/404,473 patent/US10031608B2/en active Active
- 2012-02-24 US US13/404,718 patent/US20130093792A1/en not_active Abandoned
- 2012-02-24 US US13/404,899 patent/US20130093694A1/en not_active Abandoned
- 2012-02-24 US US13/404,583 patent/US8429567B2/en not_active Expired - Fee Related
- 2012-02-24 US US13/404,439 patent/US20130069860A1/en not_active Abandoned
- 2012-02-24 US US13/404,926 patent/US9671890B2/en active Active
- 2012-02-24 US US13/404,494 patent/US8499255B2/en not_active Expired - Fee Related
-
2013
- 2013-07-29 US US13/953,680 patent/US9626034B2/en active Active
Patent Citations (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20020080180A1 (en) * | 1992-04-30 | 2002-06-27 | Richard Mander | Method and apparatus for organizing information in a computer system |
US5847707A (en) * | 1994-08-29 | 1998-12-08 | Casio Computer Co., Ltd. | Icon menu display devices and methods |
US6088032A (en) * | 1996-10-04 | 2000-07-11 | Xerox Corporation | Computer controlled display system for displaying a three-dimensional document workspace having a means for prefetching linked documents |
US5835094A (en) * | 1996-12-31 | 1998-11-10 | Compaq Computer Corporation | Three-dimensional computer environment |
US20040066414A1 (en) * | 2002-10-08 | 2004-04-08 | Microsoft Corporation | System and method for managing software applications in a graphical user interface |
US20060004692A1 (en) * | 2003-03-27 | 2006-01-05 | Microsoft Corporation | System and method for filtering and organizing items based on common elements |
US20050138572A1 (en) * | 2003-12-19 | 2005-06-23 | Palo Alto Research Center, Incorported | Methods and systems for enhancing recognizability of objects in a workspace |
US20050234981A1 (en) * | 2004-04-14 | 2005-10-20 | Manousos Nicholas H | Method and apparatus for creating, assembling, and organizing compound media objects |
US20050240880A1 (en) * | 2004-04-23 | 2005-10-27 | Microsoft Corporation | System and method for displaying stack icons |
US20070192727A1 (en) * | 2006-01-26 | 2007-08-16 | Finley William D | Three dimensional graphical user interface representative of a physical work space |
US20070226652A1 (en) * | 2006-03-23 | 2007-09-27 | Sony Corporation | Information processing apparatus, information processing method, and program thereof |
US20090307623A1 (en) * | 2006-04-21 | 2009-12-10 | Anand Agarawala | System for organizing and visualizing display objects |
US20080126979A1 (en) * | 2006-11-29 | 2008-05-29 | Sony Corporation | Content viewing method, content viewing apparatus, and storage medium in which a content viewing program is stored |
US20080307335A1 (en) * | 2007-06-08 | 2008-12-11 | Apple Inc. | Object stack |
US20080307359A1 (en) * | 2007-06-08 | 2008-12-11 | Apple Inc. | Grouping Graphical Representations of Objects in a User Interface |
US20090119590A1 (en) * | 2007-11-05 | 2009-05-07 | Verizon Data Services Inc. | Interactive group content systems and methods |
US20150205775A1 (en) * | 2008-05-01 | 2015-07-23 | Eric Berdahl | Managing Documents and Document Workspaces |
US20100083111A1 (en) * | 2008-10-01 | 2010-04-01 | Microsoft Corporation | Manipulation of objects on multi-touch user interface |
US20100095248A1 (en) * | 2008-10-14 | 2010-04-15 | International Business Machines Corporation | Desktop icon management and grouping using desktop containers |
US20100162151A1 (en) * | 2008-12-19 | 2010-06-24 | Microsoft Corporation | Techniques for organizing information on a computing device using movable objects |
US20100229129A1 (en) * | 2009-03-04 | 2010-09-09 | Microsoft Corporation | Creating organizational containers on a graphical user interface |
US20100289753A1 (en) * | 2009-05-14 | 2010-11-18 | Microsoft Corporation | Adjusting organization of media content on display |
Cited By (51)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8907903B2 (en) * | 2011-01-13 | 2014-12-09 | Sony Computer Entertainment America Llc | Handing control of an object from one touch input to another touch input |
US20120182224A1 (en) * | 2011-01-13 | 2012-07-19 | Sony Computer Entertainment America Llc | Handing control of an object from one touch input to another touch input |
US10790046B2 (en) * | 2012-02-24 | 2020-09-29 | Perkinelmer Informatics, Inc. | Systems, methods, and apparatus for drawing and editing chemical structures on a user interface via user gestures |
US20130346894A1 (en) * | 2012-06-04 | 2013-12-26 | Htc Corporation | Method, apparatus and computer-readable medium for adjusting size of screen object |
US9851876B2 (en) * | 2012-06-04 | 2017-12-26 | Htc Corporation | Method, apparatus and computer-readable medium for adjusting size of screen object |
US20140118273A1 (en) * | 2012-11-01 | 2014-05-01 | Samsung Electronics Co., Ltd. | Method and apparatus for controlling virtual screen |
US10019148B2 (en) * | 2012-11-01 | 2018-07-10 | Samsung Electronics Co., Ltd. | Method and apparatus for controlling virtual screen |
US20140164967A1 (en) * | 2012-12-06 | 2014-06-12 | Konica Minolta, Inc. | Object operation apparatus and non-transitory computer-readable storage medium |
US20140168714A1 (en) * | 2012-12-19 | 2014-06-19 | Konica Minolta, Inc. | Image processing terminal, image processing system, and computer-readable storage medium storing control program of image processing terminal |
US10019151B2 (en) * | 2013-02-08 | 2018-07-10 | Motorola Solutions, Inc. | Method and apparatus for managing user interface elements on a touch-screen device |
CN105814511A (en) * | 2013-10-14 | 2016-07-27 | 诺基亚技术有限公司 | Deviational plane wrist input |
US20150149954A1 (en) * | 2013-11-28 | 2015-05-28 | Acer Incorporated | Method for operating user interface and electronic device thereof |
US9632690B2 (en) * | 2013-11-28 | 2017-04-25 | Acer Incorporated | Method for operating user interface and electronic device thereof |
EP2887195A1 (en) * | 2013-12-20 | 2015-06-24 | Dassault Systèmes | A computer-implemented method for designing a three-dimensional modeled object |
US9472024B2 (en) | 2013-12-20 | 2016-10-18 | Dassault Systemes | Computer-implemented method for designing a three-dimensional modeled object |
US9946714B2 (en) * | 2014-03-31 | 2018-04-17 | Fuji Xerox Co., Ltd. | Information processing apparatus and non-transitory computer readable medium for associating multiple targets |
US20150278238A1 (en) * | 2014-03-31 | 2015-10-01 | Fuji Xerox Co., Ltd. | Information processing apparatus and non-transitory computer readable medium |
RU2660634C2 (en) * | 2014-04-14 | 2018-07-06 | Самсунг Электроникс Ко., Лтд. | Multi-display system, electronic device and content output method |
WO2015160132A1 (en) * | 2014-04-14 | 2015-10-22 | Samsung Electronics Co., Ltd. | Multi-display system, electronic device, and content output method |
AU2015246821B2 (en) * | 2014-04-14 | 2017-09-14 | Samsung Electronics Co., Ltd. | Multi-display system, electronic device, and content output method |
US10592071B2 (en) | 2014-04-14 | 2020-03-17 | Samsung Electronics Co., Ltd. | Multi-display system, electronic device, and content output method |
US10795567B2 (en) * | 2014-08-22 | 2020-10-06 | Zoho Corporation Private Limited | Multimedia applications and user interfaces |
US20160054908A1 (en) * | 2014-08-22 | 2016-02-25 | Zoho Corporation Private Limited | Multimedia applications and user interfaces |
USD760295S1 (en) * | 2014-09-24 | 2016-06-28 | Lexmark International, Inc. | Portion of a display screen with icon |
GB2536090A (en) * | 2015-03-06 | 2016-09-07 | Collaboration Platform Services Pte Ltd | Multi-user information sharing system |
WO2017172457A1 (en) * | 2016-03-28 | 2017-10-05 | Microsoft Technology Licensing, Llc | Applications for multi-touch input detection |
US10579216B2 (en) | 2016-03-28 | 2020-03-03 | Microsoft Technology Licensing, Llc | Applications for multi-touch input detection |
US10345997B2 (en) * | 2016-05-19 | 2019-07-09 | Microsoft Technology Licensing, Llc | Gesture-controlled piling of displayed data |
US20170336923A1 (en) * | 2016-05-19 | 2017-11-23 | Microsoft Technology Licensing, Llc | Gesture-controlled piling of displayed data |
US20190286245A1 (en) * | 2016-11-25 | 2019-09-19 | Sony Corporation | Display control device, display control method, and computer program |
US11023050B2 (en) * | 2016-11-25 | 2021-06-01 | Sony Corporation | Display control device, display control method, and computer program |
EP3662357A4 (en) * | 2017-08-31 | 2020-07-29 | Samsung Electronics Co., Ltd. | Display apparatus for providing preview ui and method of controlling display apparatus |
US10871898B2 (en) | 2017-08-31 | 2020-12-22 | Samsung Electronics Co., Ltd. | Display apparatus for providing preview UI and method of controlling display apparatus |
US20190155623A1 (en) * | 2017-11-23 | 2019-05-23 | Research & Business Foundation Sungkyunkwan University | Method for user based application grouping under multi-user environment and table top display apparatus for performing the same |
US11048529B2 (en) * | 2017-11-23 | 2021-06-29 | Research & Business Foundation Sungkyunkwan University | Method for user based application grouping under multi-user environment and table top display apparatus for performing the same |
US11714543B2 (en) * | 2018-10-01 | 2023-08-01 | T1V, Inc. | Simultaneous gesture and touch control on a display |
US11250208B2 (en) * | 2019-04-08 | 2022-02-15 | Microsoft Technology Licensing, Llc | Dynamic whiteboard templates |
US11249627B2 (en) | 2019-04-08 | 2022-02-15 | Microsoft Technology Licensing, Llc | Dynamic whiteboard regions |
US11592979B2 (en) | 2020-01-08 | 2023-02-28 | Microsoft Technology Licensing, Llc | Dynamic data relationships in whiteboard regions |
US20210357098A1 (en) * | 2020-05-15 | 2021-11-18 | Fujifilm Business Innovation Corp. | Information presentation system and non-transitory computer readable medium |
US11769304B2 (en) | 2020-08-31 | 2023-09-26 | Meta Platforms Technologies, Llc | Artificial reality augments and surfaces |
US11651573B2 (en) | 2020-08-31 | 2023-05-16 | Meta Platforms Technologies, Llc | Artificial realty augments and surfaces |
US11847753B2 (en) | 2020-08-31 | 2023-12-19 | Meta Platforms Technologies, Llc | Artificial reality augments and surfaces |
US11636655B2 (en) | 2020-11-17 | 2023-04-25 | Meta Platforms Technologies, Llc | Artificial reality environment with glints displayed by an extra reality device |
US11409405B1 (en) * | 2020-12-22 | 2022-08-09 | Facebook Technologies, Llc | Augment orchestration in an artificial reality environment |
US11928308B2 (en) | 2020-12-22 | 2024-03-12 | Meta Platforms Technologies, Llc | Augment orchestration in an artificial reality environment |
US11762952B2 (en) | 2021-06-28 | 2023-09-19 | Meta Platforms Technologies, Llc | Artificial reality application lifecycle |
US11798247B2 (en) | 2021-10-27 | 2023-10-24 | Meta Platforms Technologies, Llc | Virtual object structures and interrelationships |
US11748944B2 (en) | 2021-10-27 | 2023-09-05 | Meta Platforms Technologies, Llc | Virtual object structures and interrelationships |
US11935208B2 (en) | 2021-10-27 | 2024-03-19 | Meta Platforms Technologies, Llc | Virtual object structures and interrelationships |
US11947862B1 (en) | 2022-12-30 | 2024-04-02 | Meta Platforms Technologies, Llc | Streaming native application content to artificial reality devices |
Also Published As
Publication number | Publication date |
---|---|
US8499255B2 (en) | 2013-07-30 |
US9671890B2 (en) | 2017-06-06 |
US20130093694A1 (en) | 2013-04-18 |
US20130069991A1 (en) | 2013-03-21 |
US8473862B1 (en) | 2013-06-25 |
US20130093792A1 (en) | 2013-04-18 |
US8429567B2 (en) | 2013-04-23 |
US20130093693A1 (en) | 2013-04-18 |
US10031608B2 (en) | 2018-07-24 |
US9626034B2 (en) | 2017-04-18 |
US20130093695A1 (en) | 2013-04-18 |
US20130314353A1 (en) | 2013-11-28 |
US20130069885A1 (en) | 2013-03-21 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8429567B2 (en) | Organizational tools on a multi-touch display device | |
KR100832355B1 (en) | 3d pointing method, 3d display control method, 3d pointing device, 3d display control device, 3d pointing program, and 3d display control program | |
JP5738495B2 (en) | Information display device and display information operation method | |
JP5580694B2 (en) | Information processing apparatus, control method therefor, program, and storage medium | |
JP5738494B2 (en) | Information display device and display information operation method | |
US20170228138A1 (en) | System and method for spatial interaction for viewing and manipulating off-screen content | |
US20160110053A1 (en) | Drawing Support Tool | |
US20130082928A1 (en) | Keyboard-based multi-touch input system using a displayed representation of a users hand | |
US20130307765A1 (en) | Contactless Gesture-Based Control Method and Apparatus | |
JP2017527882A (en) | Auxiliary display of application window | |
JP2017526057A (en) | Application window area-based size adjustment and placement | |
US20130257734A1 (en) | Use of a sensor to enable touch and type modes for hands of a user via a keyboard | |
WO2016018801A1 (en) | Dynamic joint dividers for application windows | |
WO2013028424A1 (en) | Multi-function affine tool for computer-aided design | |
JP2006228215A (en) | Method for manipulating touch-sensitive graphical user interface and touch-sensitive graphical user interface | |
TWM341271U (en) | Handheld mobile communication device | |
CN104285195A (en) | Overscan display device and method of using the same | |
US20150268736A1 (en) | Information processing method and electronic device | |
EP2791773B1 (en) | Remote display area including input lenses each depicting a region of a graphical user interface | |
US20180210642A1 (en) | Navigation System | |
Uddin | Improving Multi-Touch Interactions Using Hands as Landmarks | |
Everitt et al. | Modal spaces: spatial multiplexing to mediate direct-touch input on large displays | |
JP6327834B2 (en) | Operation display device, operation display method and program | |
US10915240B2 (en) | Method of selection and manipulation of graphical objects | |
Adachi et al. | Forearm menu: Using forearm as menu widget on tabletop system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: PERCEPTIVE PIXEL INC., NEW YORK Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:DAVIDSON, PHILIP L.;REEL/FRAME:027907/0112 Effective date: 20100806 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |