Human Factors in Haptic Interfaces

by Christopher M. Smith

Introduction to Haptics

Haptics is the study of how to couple the human sense of touch with a computer-generated world. One problem with current virtual reality systems is the lack of stimulus for the sense of touch. For example, if a user tries to grab a virtual cup there isn't a non-visual way to let the user know that the cup is in contact with the user's virtual hand. Also, there isn't a mechanism to keep the virtual hand from passing through the cup [8]. Haptic research attempts to solve these problems and can be subdivided into two sub-fields, force (kinesthetic) feedback and tactile feedback.

Force feedback is the area of haptics that deals with devices that interact with the muscles and tendons that give the human a sensation of a force being applied. These devices mainly consist of robotic manipulators that push back against a user with the forces that correspond to the environment that the virtual effector is in.

Tactile feedback deals with the devices that interact with the nerve endings in the skin which indicate heat, pressure, and texture. These devices typically have been used to indicate whether or not the user is in contact with a virtual object. Other tactile feedback devices have been used to simulate the texture of a virtual object.

Are Haptics Useful?

Ivan Sutherland, a founding father of virtual reality, suggested that the ``human kinesthetic sense is as yet another independent channel to the brain, a channel whose information is assimilated quite subconsciously [2].'' This and other statements led researchers to develop haptic interfaces. By adding an independent input channel, the amount of information that is processed by the brain is increased. The increase in information reduces the error and time taken to complete a task. It also reduces the energy consumption and the magnitudes of contact forces used in a teleoperation situation [12, 13].

Humans use their hands in exploring environments that have poor or no visibility. For instance, divers in murky water use their haptic senses in substitution for their visual senses with little loss in performance. Humans are very good at identifying three-dimensional objects placed in their hands, but are not as able to identify two-dimensional objects [5]. Although not as adept at searching across a two-dimensional space, humans have particular ways of exploring such spaces. In two-dimensional exploration, such as exploring raised surfaces on a plane, humans use a set of exploratory procedures as observed by Lederman, Klatzky and Balakrishnan [11]. Their research describes how humans gather information about a two-dimensional surface. This usually happens by first identifying an edge and then following a contour [11].

Haptic displays alone are nearly useless, but when they are used in conjunction with a visual display, they can become more useful than a stereoscopic display or a display with multiple viewpoints [2]. Batter and Brooks [1] did an experiment to prove that the haptic interfaces actually affected how well a user could learn from the system. They tested several sections of a physics class that were learning electrostatic fields. The experimental part of the class was allowed to use a force feedback device in a laboratory exercise, and the control group was not. The experimental group did better than the control groups because of their access to a haptic display in their lab work [1].

Anatomy and Physiology

In order to correctly design a haptic interface for a human, the anatomy and physiology of the body must be taken into consideration. In force feedback, the proportions and strengths of average joints must be considered. Since the hands are most often used for haptic interfaces, the properties of the hand should be considered when designing a new interface. In tactile feedback, the interface must track several variables of the human sense of touch. The fingers are one of the most sensitive parts of the surface of the skin, with up to 135 sensors per square centimeter at the finger tip [3]. Also, the finger is sensitive to up to 10,000 Hz vibrations when sensing textures, and is most sensitive at approximately 230 Hz. The fingers also can not distinguish between two force signals above 320 Hz; they are just sensed as vibrations. Forces on individual fingers should be less than 30-50 N total. For the ``average user'', the index finger can exert 7 N, middle finger 6 N, and ring fingers 4.5 N without experiencing discomfort or fatigue [3].

Humans are very adept at determining if a force is real or simulated. In an experiment conducted by Edin et al. [7], a device was used to determine how humans reacted when they sensed that an object they were holding began to slip. The device consisted of a solenoid attached to a metal plate which was allowed to slide when the solenoid was turned off. None of the subjects were 'tricked' into believing that the object was actually slipping. They all noted that ``something was wrong with the object'', but none commented that the object behaved as if it were slippery.

Studies show that there is a strong link between the sensations felt by a human hand, such as an object slipping, and the motions the hand was going through to acquire that knowledge, such as holding an experimental apparatus [14]. The human haptic system is made up of two sub-systems, the motor sub-system and the sensory sub-system. There is a strong link between the two systems. Unlike the visual system, it is not only important what the sensory system detects, but what motions were used to gain that information.

Humans use two different forms of haptic exploration: active and passive. Active haptic exploration is when the user controls his own actions. Passive haptic exploration is when the hand or finger of the user is guided by another person. When the user is in control they often makes mistakes. In the case of two-dimensional exploration the most common mistake is that of wandering off of a contour and the user must spend a large amount of effort to stay on the contour. However, when the subject is being guided, her entire attention can be devoted to identifying the object represented.

Many features can be identified more readily with passive haptic exploration. Experiments comparing the accuracy of active versus passive tactile stimulations show that passive haptics are more accurate at identifying features as a whole. When a subject's finger was guided around a two dimensional object, such as the profile of a swan, they were more likely to be able identify the object. Some studies point out that active observers make more distracting 'errors', and may have difficulty differentiating between the erroneous paths and the correct paths of exploration [10].

When faced with a multi-dimensional task, such as moving an object in three dimensional space, studies at the University of North Carolina at Chapel Hill have shown that users usually break the task into a series of one or two-dimensional problems [2]. They would move an object in the x-y plane before moving it into its final position by moving in the z direction. This dimensional decomposition could be due to the particular experiment, or it could hold a clue as to how people think about multi-dimensional tasks.

Another important factor in virtual reality systems is the situation when a visual cue and a haptic cue are in contradiction. The visual cue typically over-powers the haptic cue. This fact could help solve `the stiff wall problem', which is as follows. It is very difficult to create a machine that will correctly simulate the meeting of a virtual object with a hard immovable object. If the user is presented with a visual cue that the virtual effector has reached a hard surface, even though the haptic interface does not give the force of a hard, stiff surface, but rather a linear Hooke's law approximation, the user can be fooled into thinking the virtual wall is rigid.

Safety Issues

In attempting to portray physical forces, robotic systems which are much stronger than the finger joints must be designed to account for the flexion and extension strengths and flexibility of the human joints. If an interface for the 'average user' is being considered, it should be remembered that the size of the hand affects the extension of fingers and flexion range. A hand that is larger than the 'average user' will not be able to flex its finger as designed and could be injured by the interface [12, 13]. Also, the user must be able to overpower the system. This is because the user must feel like she is in control and can not be injured by the device if it or its control system should fail [6]. The user must not be confined to the haptic device until they are rescued by some other person when the power fails [3].

Computers for the Handicapped and the Blind

Haptic research can directly benefit the blind community. By giving the blind an additional input channel greatly increases the amount of information that can be acquired. Much of the navigational ability of the sighted and the blind are similar. Many tests done with blindfolded sighted subjects and blind subjects reveal that there is not a significant difference in the ability of the two groups to navigate. For example, the blindfolded and blind groups were walked along two legs of an isosceles triangle and told to return to the origin. Both groups were able to walk the correct distance, but often in the wrong direction [10].

Other than the similarities in navigation without landmarks, the blind and sighted fared similarly when faced with converting descriptions of routes into spatial comprehension by drawing out the route described. This suggests that blind individuals use some form of mental imagery. The use of mental imagery is also demonstrated when blind children are able to recognize objects to which they have had no previous exposure. For example, blind children are able to recognize models, or raised drawings of objects such as the moon, a rocket, or a jet airplane [10].

However, blind and sighted people process information in different ways. Most sighted people make visual representations of the information they acquire. Although the blind may use some form of mental imagery, many do not have direct experience with visual representation. Since much of their world is mapped through their haptic senses they benefit when using instruments, such as a cane, to navigate. They are able to infer information about their surroundings without having to recall from memory the length of the cane to determine size characteristics of their environment. Through the vibrations in the cane, and wrist orientation, blind people are able to map their environment so well that they can throw a stone and strike an object several feet away [10].

An advantage that the blind have over sighted people is that they do not have a problem blending mirror images of objects that are 180 degrees out of orientation. The blind tend to use an absolute orientation for objects. It is easier for blind individuals to image both the front and back of a spatial array at the same time. They are able to mentally superimpose two images of a map of the same area with different information on each, even if they are in different orientations. For example, they are able to superimpose a map of the public transit routes onto a mental street map of the same area. Even if the two maps presented are of different scale and orientation, they do not have a problem with the superimposition [10].

The deaf-blind portion of the population gather almost all of their information about their world through their haptic senses. They must rely on their haptic senses for all forms of communication. They have developed a method of communicating with speaking individuals through a method called Tadoma. The Tadoma user feels the vibrations of the throat and face and jaw positions of the speaker as she speaks. Unfortunately, this requires years of training and practice, and can be slow. Although highly-skilled Tadoma users can comprehend speech at near listening rates, most Tadoma users are much slower and the added restriction of the user having to be in contact with the speaker adds to the problems associated with the Tadoma method.

Most deaf-blind individuals communicate by signing to a sighted interpreter, and the interpreter finger spells to the deaf-blind individual. It is a slow process to finger spell long sentences, and it is also very tiring for the interpreter and the deaf-blind individual. The fastest way of communicating with a deaf-blind individual seems to be through Braille interface. The deaf-blind individual reads what is typed by the person with whom she is trying to communicate on the Braille display, and replies by typing a response which can be displayed on a video monitor.

Haptic interfaces which allow deaf-blind individuals to ``listen'' to the world are few in number and most rely only on vibrotactile feedback. Although a few combine the kinesthetic and tactile feedback, all such devices still require many hours of training in order to be used effectively [6]. Devices that utilize both kinesthetic and tactile stimulation work on the principle of being able to produce a wide spectrum of frequency and amplitude variations.

The Tactuator, designed by Hong Tan and William Rabinowitz, combines large finger movements and small vibrational movements. By varying the rate and amplitude of vibration, as well as vibrating three fingers separately, several patterns can be recognized at a rate of 2-3 patterns per second for maximal comprehension [16].

Further work is being planned to map phonic sounds to vibration patterns which can be displayed on the Tactuator. By using differing patterns mapped onto the haptic senses, a method of feeling spoken speech can be established. This could lead to a device that will enable the hearing impaired and deaf-blind individuals to communicate more easily with the hearing community without the need for a human translator.

Possibilities for the Future

Haptic interfaces will continue to be centered on the hands because people gather information from their surroundings in a haptic manner with their hands more than any other body part. The search for an inexpensive, portable and useful haptic display will be long and difficult, but it will continue for many years to come. Many researchers look for a 'natural' interface, but since there is a physical barrier between the human sensory-motor capabilities and the electronic world of the computer, there will not be a natural system until they can use direct neural stimulation of the brain. Instead, some suggest the search should be for an intuitive system. Robert Stone emphasized this point by stating:

An intuitive interface between man and machine is one which requires little training and proffers a working style most like that used by the human being to interact with environments and objects in his day-to-day life. In other words, the human interacts with elements of his task by looking, holding, manipulating, speaking, listening, and moving, using as many of his natural skills as are appropriate, or can reasonably be expected to be applied to a task [15].

When creating a haptic interface, it is important to keep in mind what the device is going to be used for. If it is not going to be used in a way that is intuitive to an operator, it may cause problems even though the user is trained in its use. In times of stress, excitement, or fatigue, people forget much of their training and do what comes intuitively. So if a haptic interface is not being used in an intuitive manner the operator may misuse the interface by doing something that is natural to them.

The study of haptics holds a key to unlocking interface problems with the computer. Haptics enable a fairly intuitive way for the human user to get information into the computer, and for the computer to display information from a virtual world. Research in this area can help enable those who have been unable to use a computer to its fullest extent overcome a physical limitation, and it can enable users to explore objects and places that have been inaccessible under normal circumstances.

References

1: Batter, J. J. and Brooks, F. P., Jr. 1971. Grope-1: A computer display to the sense of feel. Proceedings of IFIP 1972, 759-763.
2: Brooks, F. P., Jr. 1977. The computer ``scientist'' as toolsmith--studies in interactive computer graphics. Proceedings of IFIP 1977, 625-634.
3: Burdea, G. and Coiffet, P. 1994. Virtual Reality Technology. New York: John Wiley & Sons, Inc.
4: De Rossi, D. 1991. Artificial tactile sensing and haptic perception. Meas. Sci. Technol, (2), pp 1003-16.
5: Driels, M. and Bereri, P., 1994. A finite memory model for Haptic Teleoperation. IEEE transactions on Systems, Man, and Cybernetics, 24 (4) 690-698.
6: Eberhardt, S. P., Bernstein, L. E., Coulter, D. C., and Hunckler, L. A. 1993. OMAR - A haptic display for speech perception by deaf and deaf-blind individuals. IEEE Annual Virtual Reality International Symposium, pp. 195-201.
7: Edin, B. B., Howe, R., Westling, G., and Cutkowsky, M. 1993. A physiological method for relaying frictional information to a human teleoperator. Transactions on Systems, Man, and Cybernetics 23 (2), 427-432.
8: Gigante, M. A., 1993. Virtual Reality Systems. Earnshaw, R.A., Gigante, M. A., and Jones, H. (editors). London, Academic Press.
9: Ishii, M. and Makoto, S. 1993. A 3d interface device with force feedback: A virtual work space for pick-and-place tasks. 1993 IEEE Annual Virtual Reality International Symposium, pp. 331-335.
10: Kennedy, J. M., Gabias, P., and Heller, M. A. 1992. Space, haptics and the blind. Geoforum, 23 (2), pp. 175-189.
11: Lederman, S. J., Klatzky, R. L., and Balakrishnan, J. D., 1991. Task-driven extraction of object contour by human haptics: Part 2. Robica, 9, pp. 179-188.
12: Shimoga, K. B. 1993. A survey of perceptual feedback issues in dexterous telemanipulation: Part I. Finger force feedback. 1993 IEEE Annual Virtual Reality International Symposium, pp. 263-270.
13: Shimoga, K. B. 1993. A survey of perceptual feedback issues in dexterous telemanipulation: Part II. Finger touch feedback. 1993 IEEE Annual Virtual Reality International Symposium, pp. 271-279.
14: Stansfield, S. A. 1992. Haptic perception with an articulate, sensate robot hand. Robotica, 10, pp. 497-508.
15: Stone, R. J. 1993. Virtual Reality Systems. Earnshaw, R.A., Gigante, M. A., and Jones, H. (editors).. London, Academic Press.
16: Tan, H. Z. , and Rabinowitz, W. M. 1996. A new multi-finger tactual display. In Proceedings of the ASME Dynamics Systems and Control Division, ASME 1996, DSC-Vol. 58, pp. 515-522 .

Crossroads

The ACM Student Magazine