WO1992005519A1

WO1992005519A1 - Manipulator interaction simulation system

Info

Publication number: WO1992005519A1
Application number: PCT/GB1991/001637
Authority: WO
Inventors: Robert John Stone; James R. Hennequin
Original assignee: Advanced Robotics Research Limited
Priority date: 1990-09-25
Filing date: 1991-09-24
Publication date: 1992-04-02
Also published as: AU8657391A; GB9020848D0

Abstract

A glove and associated system for simulating interactions between a manipulator and an object. A glove is provided with inflatable enclosures the pressure within which is to be controlled to simulate interaction between the manipulator, for example a human hand, and the object, for example an object gripped by a device the movement of which tracks movement of the human hand.

Description

MANIPULATOR INTERACTION SIMULATION SYSTEM

The present invention relates to a manipulator interaction simulation system; and a glove for use in a such a system.

In the general field of robots, man-machine interfaces have been designed to input data to machines and to indicate to users the response of machines to such input data. The literature shows that there is a need for interfacing systems which enable system users to directly interact with machines they control so that complex data entry procedures can be avoided. Various data entry "gloves" have been devised which enable a user to directly input data generated as a result of movements of a hand wearing the glove.

One example described in US Patent Specification No. A-4 414 537 provides a glove with flex sensors which enable the relative angular positions of parts of the glove to generate data describing the position and configuration of the users hand. The glove can be used by a deaf person to input data to a receiving device by forming his or her hand into the character positions defined by the Single Hand Manual Alphabet. Other examples are widely used in game applications, for example the Data Glove described in US Patent No. 4,988,981 available from VPL Research, and the Power Glove available from Mattel.

The known gloves are useful in providing data related to hand position, and can be used to drive mechanical manipulators (real manipulators) or images of simulated manipulators (virtual manipulators). The known gloves do not however provide tactile feedback, and it is tactile feedback that makes it possible for the human hand to perform the complex and delicate manipulations of which it is capable. The human hand is positioned in a generally appropriate position by reference to visual cues, but once in contact with an object tactile feedback is used for example to control the pressure applied to an object, and to control accelerations to an object gripped by the hand. Without tactile feedback accurate simulations of hand movements are very difficult to achieve.

It is an object of the present invention to provide a system which can provide the benefits of tactile feedback in man-machine interfaces, and a glove for use in such a system.

According to the present invention, there is provided a system for simulating interactions between a manipulator and an object, comprising a glove adapted for mounting on the manipulator, the glove including at least one deformable enclosure, and means for controlling the supply of a pressurised fluid to the or each enclosure to apply a pressure between the manipulator and the object appropriate to the simulated interaction.

The present invention also provides a glove for use in the system defined above.

The manipulator may be a human hand, and the glove may be dimensioned to fit the hand such that the or each enclosure is located at a predetermined location on the hand, whereby tactile stimulus or stimuli can be applied to the hand appropriate to the simulation.

The system may comprise means for monitoring the position and configuration of the hand, means for manoeuvring a simulation of at least elements of the hand in accordance with the monitored position and configuration, means for detecting contact between the simulated hand and an object, and means for controlling the pressure of fluid within the or each enclosure to simulate the detected contact. The simulated hand and object may be virtual objects displayed to the wearer of the glove, or real objects visible to the wearer of the glove.

The simulated hand may be provided with at least one pressure sensor the position of which on the simulated hand corresponds to the position of the or a respective enclosure on the glove, the output of the pressure sensor controlling the pressure applied to the hand of the wearer of the glove by the or a respective enclosure.

In an alternative arrangement, the manipulator may be a simulation of at least elements of a human hand, and a further glove may be provided which is worn on a human hand. Means are provided for monitoring the position and configuration of the hand, and the further glove comprises at least one pressure sensor the position of which on the further glove corresponds to the position of the or a respective enclosure on the manipulator. Further means are provided for manoeuvring the manipulator in accordance with the monitored position and configuration, and for controlling the pressure applied by the manipulator to an object it contacts by controlling the pressure in the or each enclosure to correspond with the pressure applied to the or a respective pressure sensor.

Means may be provided for storing data relating to the pressure applied to a human hand when that hand is in contact with an object in each of a plurality of predetermined positions relative to the object, and for identifying the said predetermined position corresponding to the detected contact between the simulated hand and a simulation of the object, the pressure controlling means controlling the pressure within the or each enclosure to apply pressure to a wearer of the glove consistent with the pressure stored in respect of the said predetermined position. Preferably, a further glove is provided which may be worn on a human hand, the further glove comprising at least one pressure sensor the position of which on the further glove corresponds to the position of the or a respective enclosure on the said glove, and means being provided for storing the output of the or each pressure sensor when the further glove is positioned in each of said plurality of predetermined positions in contact with the object.

A member formed from a material which is resilient to adhesion to the sheets may be located within the enclosure. A plurality of enclosures can be formed between the sheets of resilient material, each enclosure being defined by the space formed around a respective member which is resilient to adhesion to the sheets.

Each enclosure may be connected to a fluid pressure control system by a flexible tube supported in the glove. Embodiments of the present invention will now be described, by way of example with reference to the accompanying drawings, in which:

Figure 1 is a schematic illustration of a glove in accordance with the present invention;

Figure 2 is a sectional illustration of a fingertip portion of the glove of Figure 1;

Figure 3 illustrates a sectional view of an alternative enclosure for incorporation in a glove in accordance with the present invention;

Figure 4 is a plan view of the enclosure of Figure 3; Figure 5 is a schematic illustration of a system in accordance with the present invention;

Figure 6 illustrates a further alternative enclosure, Figures 7 and 8 illustrate an enclosure incorporating four independently controlled enclosures, Figure 8 being a section on lines 8-8 of the structure parts of which are illustrated in Figure 7;

Figure 9 illustrates a further glove in accordance with the present invention;

Figures 10 and 11 illustrates an enclosure assembly for use with the glove of Figure 9, Figure 11 being a section through the assembly; and

Figure 12 schematically illustrates a pressure control system for pressuring one glove enclosure.

Referring to Figures 1 and 2, the illustrated glove comprises an outer Neoprene glove 1 and an inner glove 2. It will be appreciated that materials other than Neoprene e.g. any suitable elastomer, could be used. The outer glove 1 may incorporate flex sensors or other equivalent hand position monitoring devices. An enclosure in the form of an air chamber 3 is defined between the inner and outer glove and a capillary tube 4 communicates with the air chamber 3 to enable control of the pressure within the air chamber. It will be appreciated that increasing the pressure within the air chamber 3 increases the pressure applied to the fingertip adjacent which the air chamber 3 is positioned. Figure 1 shows the distribution of twenty chambers or pressure pads of the type illustrated in Figure 2. Essentially the pad positions correspond to the positions on the hand which come in to contact with an object that is grasped lightly by the hand. Thus by appropriate modulation of the pressure applied to each of the twenty air chambers a tactile stimulus can be given to the hand corresponding to the stimulus which would be felt if that hand grasped an object of any particular shape, configuration or position.

The arrangement shown in Figure 2 is only one of various alternative configurations. For example, the glove could be fabricated of Lycra material to be used either independently of or together with a glove equipped with flex sensors or the like. A glove equipped with flex sensors could be simply pulled over a glove incorporating air chambers.

Figures 3 and 4 illustrate an alternative air chamber structure which could be mounted on or incorporated in any suitable glove design. In the case illustrated in Figures 3 and 4, a cavity is defined between Neoprene sheets 5 and 6, a capillary tube 7 communicating with the air chamber to enable the pressure within the air chamber to be controlled as desired.

Referring now to Figure 5, this schematically illustrates the system used for controlling the pressure within one of the air chambers. Pressurized air from a compressed air supply source 8 is fed via an input solenoid valve 9 to the air chamber which is represented by the box labelled feed back pressure pad 10. Air is exhausted from the pressure pad via an output solenoid valve 11 communicating with an exhaust 12. Pressure within the pad 10 is monitored by a transducer 13. The solenoid valves and transducer are connected to a pressure control board 14 which controls the two solenoid valves in accordance with instructions derived from a computer 15, the transducer 13 providing a feedback loop to ensure that the pad pressure specified by the output of the computer 15 are achieved.

Pneumatic pressure control systems of the general type described with regard to Figure 5 have been developed for use in particular with devices as described for example in US Patent No. 4,944,755. Details may be obtained from Airmuscle Limited of 12 Orchard Close, Cranfield, Bedfordshire, MK43 OHX, England.

As an alternative to providing controlling inputs to the controller 15 from a computer, those inputs may be derived directly from an input glove incorporating pressure sensors positioned on the glove in the same locations as are respective ones of the pressure pads. The outputs of the glove incorporating pressure sensors could be applied directly to the pressure control board or could be used to build up a look-up table from which data could be drawn by the computer to apply the appropriate pressure to the pressure pads. Thus there are many alternative system configurations which can be devised in accordance with the present invention.

Referring now to Figure 6, this illustrates a further alternative pressure pad structure. The pressure pad is formed by adhering two sheets of Neoprene 16, 17 around a PTFE disc 18. The PTFE disc 18 does not adhere to the Neoprene sheets and therefore ensures that an inflatable chamber is defined between it and the adjacent sheets even if adhesive leaks between the PTFE and these sheets. The sheet 16 is secured to an inner Lycra glove 19 and the pressure pad is covered by an outer Lycra glove 20. A PTFE tube 22 having an internal diameter of for example 0.5 mm and an external diameter of for example 1.0 mm is routed to the pressure pad between the Lycra gloves 20 and 21. The PTFE tube is threaded through an opening (not shown) in the outer glove 21 and connected to a strain relief support (not shown) supported on a wrist section of the glove. The tube length is sufficient to ensure that even when the glove is fully stretched in use the tube remains slack.

As described below, more than one pressure pad may be provided on a single finger tip. Figures 7 and 8 show how a four-pad single location unit can be fabricated. In the illustrated case, two discs of Neoprene 23 and 24 have four sector-shaped PTFE members 25 disposed therebetween. PTFE tubes 26 are positioned on the lower disc 23 with the members 25 and the upper disc 24 is then glued to the disc 23 both around its periphery and between the adjacent pairs of PTFE members 25. Four independently controlled pressure pads are thus provided.

Figure 9 illustrates an alternative pressure pad distribution to that shown in Fig. 1. In Fig. 9, the tip of the thumb, first finger and third finger each support three independently controlled pressure pads 27, the thumb, first finger and second finger support two further pads 28, and the second and fourth finger each support two further pads 29. Lozenge shaped pads 30 are provided between the thumb and first finger, between the first and second finger, on the side of the hand remote from the thumb, and on the outer side of the thumb. A further relatively large pad 31 is provided on the back of the hand as indicated by dotted lines, and pads (not shown) are distributed on the palm of the glove as shown in Figure 1 or as described below with reference to Fig. 10. Each pad is connected to a respective single PTFE supply tube. Of course, different pad distribution may be selected, for example by providing three or four pads on each finger tip.

Referring now to Figs. 10 and 11, an alternative distribution of pads on the palm is illustrated. These pads are independently mounted as an assembly on Velcro strips to enable the distribution to be changed by replacing one assembly with another. The illustrated assembly comprises six pear-shaped pads 32 superimposed on a bellows-like structure formed by two independent large pads 33. The pads 33 are mounted on a carrier 34 that is secured to the palm of the glove 35 by Velcro 36. The pads 32 adjacent the fingers may be interconnected by PTFE tubes 37 to finger pads 38 indicated by broken lines 39. The PTFE tubes are secured together as a bundle 40. THe palm of the hand is relatively insensitive in comparison with the finger but the assembly of Fig. 10 should enable meaningful responses to be achieved.

Large structures as shown in Figure 10 also enable the pads to expand sufficiently to provide a relatively large area about which the hand can close. It may be appropriate to place the pads 32 beneath rather than on top of the pads 33.

Referring now to Fig. 11, the structure of a proportional controller for controlling the pressure in any one pad is schematically illustrated. A pressure pad 41 is connected by a PTFE tube 42 to a transducer 43, a pressurised air supply valve 44, and a valve 45 which can be arranged when open to vent the tube 42 to atmosphere or as shown to connect the tube to a vacuum system. The use of a vacuum system accelerates the response time by deflating the pad 41 rapidly.

A pressure comparator circuit 46 receives an input 47 from a control computer which determines the pressure which it is desired to feed to the pad 41. Assuming the pad is initially deflated, when the input 47 indicates a requirement for a particular pressure to be applied to the pad 41, the comparator closes valve 45 and opens valve 44. When the transducer 43 indicates that the desired pressure has been achieved, the comparator 46 closes the valve 44. Further adjustment to the pressure in pad 41 can then be achieved by appropriate control of the valves 44 and 45.

Each of the pads may be independently pressurised. Alternatively, in some circumstances the pressure applied to adjacent pads may be interrelated. For example, where a number of pads are located close together, as illustrated in Figs 7 and 8, the pressure supplied to adjacent pads can be oscillated out of phase to give a tactile sense of slip to the finger tip. This maintains the sense of contact but also gives the sense that the point of contact is moving on the finger tip. Of course, even if only one pad is provided on a finger tip, the pressure may be oscillated to give the feel of slip. The frequency of oscillation may be controlled to vary the effect, higher frequencies equating to smooth surfaces and lower frequencies to rough surfaces. Rapid detection of slip is of course important to enable the user to react sufficiently quickly to prevent a gripped object from being dropped. Some alternative possible system configurations are outlined below. To simplify terminology, the following terms will be used hereinafter:

The term "air glove" will be used to indicate a glove incorporating pressure pads of the type described above.

The term "position glove" will be used to indicate a glove incorporating for example flex sensors to enable the position and attitude of the glove to be determined.

The term "pressure glove" will be used to indicate a glove incorporating pressure sensors, the location of the pressure sensors on the pressure glove corresponding to the location of the pressure pads on the air glove.

Thus examples of possible configurations are as follows: 1. An air glove and a position glove are worn together on a human hand. The position monitored by the position glove determines the position of a real manipulator which copies movements of the position glove. The manipulator may be used for example to move real objects, the position of which at any one time is known (for example a lever the position of which can be monitored) . By comparison between the known position of the object and the known position of the manipulator, contact therebetween is assessed by computation, there being no direct feedback corresponding to detection of actual physical contact between the manipulator and the object. Once the occurrence of contact has been computed, appropriate pressures are applied to the air glove. The appropriate pressures may be determined by computation in real time or read out from a look-up table. The contents of the look-up table could be generated computationally or by using a pressure glove to perform predetermined operations intended to replicate operations which will be performed in the future by the manipulator.

2. An air glove and a position glove are worn together on a human hand. A real manipulator copies movement of the hand as monitored by the position glove. The manipulator carries sensors corresponding in position on the manipulator to the position of the pressure pads in the air glove. For example, the pressure pads could be provided in the tips of the thumb and first finger of the air glove and corresponding sensors could be provided on tips of a pair of simple manipulator gripper fingers. The sensor outputs control pressure in the pads to tell the user how hard the object is being gripped. Thus this system configuration corresponds to that identified in paragraph 1 above with the addition of direct sensing of the pressure applied by the manipulator, the sensed pressure being used to control the pressure applied to the human hand by the air glove.

3. The system of paragraph 2 above could be effectively reversed so that a pressure glove is worn on the human hand and an air glove is "worn" by the manipulator. The human hand also wears a position glove. The user manipulates an object using the pressure glove and the manipulator follows those movements and applies pressure to a second identical object. Outputs of the pressure glove control the pressure of the pads in the air glove on the manipulator to thereby take up any small discrepancies between the intended and actual manipulator positions so that the pressure applied to the second object truly corresponds to the pressure applied to the first object.

4. The position glove and an air glove are worn on a human hand. The movement of the human hand as detected by the position glove are followed by a simulated hand which is displayed to the user on the screen. The screen also displays simulated objects into contact with which the simulated hand may be moved. In the event of contact between the simulated hand and the simulated object the pressures applied to the pressure pad in the air glove are controlled to simulate that context. Again the pressures could be obtained from a look¬ up table or calculated in real time.

Thus with the system configuration in accordance with paragraph 1 above, pressure control data can be derived from a look up table, that data being pertinent to known objects and the force and tactile sensations resulting from manipulation of such objects. The data may be generated in advance by a human operator using a pressure glove to manipulate real objects. It is important to note that the manipulator which follows movement of the human hand need not be of a dextrous form. The manipulator could be of dextrous form but would also be effective if in the form of a conventional two or three finger gripper. One would expect that a combination of a priori knowledge about the location, size and mass of a particular object, plus some confirmation of this knowledge using the output of visual and non-visual sensors mounted on or around the manipulator (rangefinder, TV, ultrasonic) would drive the selection of appropriate data from a look-up table.

It will be appreciated that the present system may be taken beyond the position of simply applying tactile stimulus corresponding to contact with an object to the human hand. Force information may also be fed back to the human hand so as to indicate accelerations applied to an object by a hand or to a hand by an object. Force information could be supplemented by using for example a fore and upper arm pneumatic assembly. The system could of course be developed to a whole-body suit capable of applying pressure to any part of the human body.

It will be appreciated that the enclosures or air chambers defined in the gloves used in accordance with the present invention will generally be elastic and that generally pressurized fluid will be supplied to these enclosures in a proportional manner.

It will be appreciated also that the distribution of pads illustrated in Figs. 1, 9 and 10 could be adjusted to meet specific system requirements. For example the relatively small pair of pads on the palm of the hand shown in Fig. 1 could be extended in area or replaced by groupings of pads connected together in parallel. The pads may have any convenient shape, for example, circular, rectangular or triangular. Less pressure pads could of course be provided, for example if the application only really required fingertip manipulation pads.

It will be appreciated that gloves in accordance with the invention can be used in a variety of fields, e.g. the control of robot manipulators, research into perceptual motor skills rehabilitation and neurological treatment, and any other man- machine interface, human factors or ergonomics application.

Claims

1. A system for simulating interactions between a manipulator and an object, comprising a glove adapted for mounting on the manipulator, the glove including at least one deformable enclosure, and means for controlling the supply of a pressurised fluid to the or each enclosure to apply a pressure between the manipulator and the object appropriate to the simulated interaction.

2. A system according to Claim 1, wherein the manipulator is a human hand, and the glove is dimensioned to fit the hand such that the or each enclosure is located at a predetermined location on the hand, whereby tactile stimulus or stimuli can be applied to the hand appropriate to the simulation.

3. A system according to Claim 2, comprising means for monitoring the position and configuration of the hand, means for manoeuvring a simulation of at least elements of the hand in accordance with the monitored position and configuration, means for detecting contact between the simulated hand and an object, and means for controlling the pressure of fluid within the or each enclosure to simulate the detected contact.

4. A system according to Claim 3, wherein the simulated hand and object are virtual objects displayed to the wearer of the glove.

5. A system according to Claim 3, wherein the simulated hand and object are real objects.

6. A system according to Claim 5, wherein the simulated hand is provided with at least one pressure sensor the position of which on the simulated hand corresponds to the position of the or a respective enclosure on the glove, the output of the pressure sensor controlling the pressure applied to the hand of the wearer of the glove by the or a respective enclosure.

7. A system according to Claim 1, wherein the manipulator is a simulation of at least elements of a human hand, a further glove is provided which is worn on a human hand, means are provided for monitoring the position and configuration of the hand, the further glove comprises at least one pressure sensor the position of which on the further glove corresponds to the position of the or a respective enclosure on the manipulator, means are provided for manoeuvring the manipulator in accordance with the monitored position and configuration, and means are provided for controlling the pressure applied by the manipulator to an object it contacts by controlling the pressure in the or each enclosure to correspond with the pressure applied to the or a respective pressure sensor.

8. A system according to any one of the claims 3 to 6, comprising means for storing data relating to the pressure applied to a human hand when that hand is in contact with an object in each of a plurality of predetermined positions relative to the object, and means for identifying the said predetermined position corresponding to the detected contact between the simulated hand and a simulation of the object, the pressure controlling means controlling the pressure of the hand within the or each enclosure to apply pressure to a wearer of the glove consistent with the pressure stored in respect of the said predetermined position.

9. A system according to claim 8, comprising a further glove which may be worn on a human hand, the further glove comprising at least one pressure sensor the position of which on the further glove corresponds to the position of the or a respective enclosure on the said glove, and means for storing the output of the or each pressure sensor when the further glove is positioned in each of said plurality of predetermined positions in contact with the object.

10. A system according to any preceding claim, wherein said at least one deformable enclosure is defined between a pair of sheets of resilient material.

11. A system according to claim 10, wherein a member formed from a material which is resistant to adhesion to the sheets is located within the enclosure.

12. A system according to claim 11, wherein a plurality of enclosures are formed between the sheets of resilient material, each enclosure being defined by the space formed around a respective member which is resilient to adhesion to the sheets.

13. A system according to any preceding claim, wherein the or each enclosure is connected to a fluid pressure control system by a flexible tube supported in the glove.

14. A system according to any preceding claim, comprising means for oscillating the pressure applied to the enclosure to simulate slip.

15. A system for simulating interaction between a manipulator and an object substantially as hereinbefore described.

16. A glove for use in a system according to any preceding claim.

17.. A glove substantially as hereinbefore described with reference to the accompanying drawings.