x

While graphical user interfaces have gained much popularity in recent years, there are situations when the need to use existing applications in a nonvisual modality is clear. Examples of such situations include the use of applications on hand-held devices with limited screen space (or even no screen space, as in the case of telephones), or users with visual impairments.

We have developed an architecture capable of transforming the graphical interfaces of existing applications into powerful intuitive nonvisual interfaces. Our system, called Mercator, provides new input and output techniques for working in the nonvisual domain. Navigation is accomplished by traversing a hierarchical tree representation of the interface structure. Output is primarily auditory, although other output modalities (such as tactile) can be used as well. The mouse, an inherently visually-oriented device, is replaced by keyboard and voice interaction.

Our system is currently in its third major revision. We have gained insight into both the nonvisual interfaces presented by our system and the architecture necessary to construct such interfaces. This architecture uses several novel techniques to efficiently and flexibly map graphical interfaces into new modalities.

MediaMosaic is an editing environment developed to provide several features that are either unavailable or not adequately addressed in current editing systems. First, it is a multimedia editor of an open architecture. General media are inserted in documents by embedded virtual screens. Second, it allows users to do markup editing in context. The marked comments are overlapped and attached to the commented areas. Third, it provides a mechanism to allow users to bring data from more than one source to a single document. The views of the included data can be tailored. Fourth, users can work on an included medium through its embedded view or through another complete and duplicated view. It isolates and simplifies the interface design of individual media editors.