Using Multimodal Cues to Direct the User's Attention

Navigating a 3D haptic environment without vision is extremely difficult. As part of the Directed Attention project we have been investigating the feasibility of using haptic and audio cues to guide the user's hand around a 3D environment, to facilitate data recognition at both the micro (individual elements) and macro (overall structure) level.

Individual elements (either cubes or spheres) are arranged in a particular pattern within the environment. A haptic spring directs the stylus to the leftmost element, and a constraint keeps the stylus attached to the surface of the object, whilst the user feels its shape. The user can move through the elements in sequence by pressing the space bar: the spring moves the stylus from the first element to the second, the second to the third, the third to the fourth and from the fourth back to the first again. The user can repeat this process until he or she is confident of the spatial layout of the data in the environment. When the stylus is in touch with the object, the user can also request an audio cue, which indicates whether the element is a cube or a sphere.

A recent evaluation showed the technique to be extremely effective. Without any cues, participants struggled to locate a single element, let alone identify their spatial structure. Haptic cues enabled participants to move efficiently between objects, and identify the structure quickly and accurately. The addition of audio cues improved participants' ability to identify individual elements, and as such significantly speeded up their recognition of the overall structure.

Directed attention home

Tracking eye movements on the BBC News Website

Comparing saliency maps and eye movement data

The directed attention project is funded by the EPSRC under grant EP/D036518/1 (Sept 2005 - July 2006).