display

We present a novel display metaphor which extends traditional tabletop projections in collaborative environments by introducing freeform, environment-aware display representations and a matching set of interaction schemes. For that purpose, we map personalized widgets or ordinary computer applications that have been designed for a conventional, rectangular layout into space-efficient bubbles whose warping is performed with a potential-based physics approach. With a set of interaction operators based on laser pointer tracking, these freeform displays can be transformed and elastically deformed using focus and context visualization techniques. We also provide operations for intuitive instantiation of bubbles, cloning, cut & pasting, deletion and grouping in an interactive way, and we allow for user-drawn annotations and text entry using a projected keyboard. Additionally, an optional environment-aware adaptivity of the displays is achieved by imperceptible, realtime scanning of the projection geometry. Subsequently, collision-responses of the bubbles with non-optimal surface parts are computed in a rigid body simulation. The extraction of the projection surface properties runs concurrently with the main application of the system. Our approach is entirely based on off the-shelf, low-cost hardware including DLP-projectors and FireWire cameras.

Most current interface designs require that the user focus their attention on them in order to be of value. However, as the price of computation falls, and computational capabilities make their way into many everyday objects, the demand for attention from many different directions may begin to seriously reduce the usefulness of these computational objects. Ambient information displays are intended to fit in a part of the interface design space that does not have this property. They are designed to convey background or context information that the user may or may not wish to attend to at any given time. Ambient Displays are designed to work primarily in the periphery of a user's awareness, moving to the center of attention only when appropriate and desirable. This paper describes a new ambient information display that is designed to give a rich medium of expression placed within an aesthetically pleasing decorative object. This display --- the Information Percolator --- is formed by air bubbles rising up tubes of water. By properly controlling the release of air, a set of pixels which scroll up the display is created. This allows a rendition of any (small, black and white) image to be displayed. The detailed design and construction of this display device will be considered, along with several applications.

We develop design principles and an interaction framework for sharable, interactive public ambient displays that support the transition from implicit to explicit interaction with both public and personal information. A prototype system implementation that embodies these design principles is described. We use novel display and interaction techniques such as simple hand gestures and touch screen input for explicit interaction and contextual body orientation and position cues for implicit interaction. Techniques are presented for subtle notification, self-revealing help, privacy controls, and shared use by multiple people each in their own context. Initial user feedback is also presented, and future directions discussed.

Normally, the primary purpose of an information display is to convey information. If information displays can be aesthetically interesting, that might be an added bonus. This paper considers an experiment in reversing this imperative. It describes the Kandinsky system which is designed to create displays which are first aesthetically interesting, and then as an added bonus, able to convey information. The Kandinsky system works on the basis of aesthetic properties specified by an artist (in a visual form). It then explores a space of collages composed from information bearing images, using an optimization technique to find compositions which best maintain the properties of the artist's aesthetic expression.

To enable common mobile terminals to interact with contents shown on large screens, we propose "C-Blink", a new light signal marker method that uses the color liquid-crystal display of a mobile terminal as a visible light source. We overcome the performance limitations of such displays by developing a hue-difference-blink technique. In combination with a screen-side sensor, we describe a system that detects and receives light signal markers sent by cell phone displays. Evaluations of a prototype system confirm that C-Blink performs well under common indoor lighting. The C-Blink program can be installed in any mobile terminal that has a color display, and the installation costs are small. C-Blink is a very useful way of enabling ubiquitous large screens to become interfaces for mobile terminals.

Computer users working with large visual documents, such as large layouts, blueprints, or maps perform tasks that require them to simultaneously access overview information while working on details. To avoid the need for zooming, users currently have to choose between using a sufficiently large screen or applying appropriate visualization techniques. Currently available hi-res "wall-size" screens, however, are cost-intensive, space-intensive, or both. Visualization techniques allow the user to more efficiently use the given screen space, but in exchange they either require the user to switch between multiple views or they introduce distortion.In this paper, we present a novel approach to simultaneously display focus and context information. Focus plus context screens consist of a hi-res display and a larger low-res display. Image content is displayed such that the scaling of the display content is preserved, while its resolution may vary according to which display region it is displayed in. Focus plus context screens are applicable to practically all tasks that currently use overviews or fisheye views, but unlike these visualization techniques, focus plus context screens provide a single, non-distorted view. We present a prototype that seamlessly integrates an LCD with a projection screen and demonstrate four applications that we have adapted so far.

Light Emitting Diodes (LEDs) offer long life, low cost, efficiency, brightness, and a full range of colors. Because of these properties, they are widely used for simple displays in electronic devices. A previously characterized, but little known property of LEDs allows them to be used as photo sensors. In this paper, we show how this capability can be used to turn unmodified, off the shelf, LED arrays into touch sensitive input devices (while still remaining capable of producing output). The technique is simple and requires little or no extra hardware - in some cases operating with the same micro-controller based circuitry normally used to produce output, requiring only software changes. We will describe a simple hybrid input/output device prototype implemented with this technique, and discuss the design opportunities that this type of device opens up.

Normally, the primary purpose of an information display is to convey information. If information displays can be aesthetically interesting, that might be an added bonus. This paper considers an experiment in reversing this imperative. It describes the Kandinsky system which is designed to create displays which are first aesthetically interesting, and then as an added bonus, able to convey information. The Kandinsky system works on the basis of aesthetic properties specified by an artist (in a visual form). It then explores a space of collages composed from information bearing images, using an optimization technique to find compositions which best maintain the properties of the artist's aesthetic expression.

Recent work is beginning to reveal the potential of numerical optimization as an approach to generating interfaces and displays. Optimization-based approaches can often allow a mix of independent goals and constraints to be blended in ways that would be difficult to describe algorithmically. While optimization-based techniques appear to offer several potential advantages, further research in this area is hampered by the lack of appropriate tools. This paper presents GADGET, an experimental toolkit to support optimization for interface and display generation. GADGET provides convenient abstractions of many optimization concepts. GADGET also provides mechanisms to help programmers quickly create optimizations, including an efficient lazy evaluation framework, a powerful and configurable optimization structure, and a library of reusable components. Together these facilities provide an appropriate tool to enable exploration of a new class of interface and display generation techniques.

We present a new focus+context (fisheye) scheme for visualizing and manipulating large hierarchies. The essence of our approach is to lay out the hierarchy uniformly on the hyperbolic plane and map this plane onto a circular display region. The projection onto the disk provides a natural mechanism for assigning more space to a portion of the hierarchy while still embedding it in a much larger context. Change of focus is accomplished by translating the structure on the hyperbolic plane, which allows a smooth transition without compromising the presentation of the context.

We have previously developed a collaborative infrastructure called SCAPE - an acronym for Stereoscopic Collaboration in Augmented and Projective Environments - that integrates the traditionally separate paradigms of virtual and augmented reality. In this paper, we extend SCAPE by formalizing its underlying mathematical framework and detailing three augmented Widgets constructed via this framework: CoCylinder, Magnifier, and CoCube. These devices promote intuitive ways of selecting, examining, and sharing synthetic objects, and retrieving associated documentary text. Finally we present a testbed application to showcase SCAPE's capabilities for interaction in large, augmented virtual environments.

We have previously developed a collaborative infrastructure called SCAPE - an acronym for Stereoscopic Collaboration in Augmented and Projective Environments - that integrates the traditionally separate paradigms of virtual and augmented reality. In this paper, we extend SCAPE by formalizing its underlying mathematical framework and detailing three augmented Widgets constructed via this framework: CoCylinder, Magnifier, and CoCube. These devices promote intuitive ways of selecting, examining, and sharing synthetic objects, and retrieving associated documentary text. Finally we present a testbed application to showcase SCAPE's capabilities for interaction in large, augmented virtual environments.

We present a new focus+context (fisheye) scheme for visualizing and manipulating large hierarchies. The essence of our approach is to lay out the hierarchy uniformly on the hyperbolic plane and map this plane onto a circular display region. The projection onto the disk provides a natural mechanism for assigning more space to a portion of the hierarchy while still embedding it in a much larger context. Change of focus is accomplished by translating the structure on the hyperbolic plane, which allows a smooth transition without compromising the presentation of the context.

We develop design principles and an interaction framework for sharable, interactive public ambient displays that support the transition from implicit to explicit interaction with both public and personal information. A prototype system implementation that embodies these design principles is described. We use novel display and interaction techniques such as simple hand gestures and touch screen input for explicit interaction and contextual body orientation and position cues for implicit interaction. Techniques are presented for subtle notification, self-revealing help, privacy controls, and shared use by multiple people each in their own context. Initial user feedback is also presented, and future directions discussed.

This paper describes new interaction techniques for direct pen-based interaction on the Interactive Mural, a large (6′x3.5′) high resolution (64 dpi) display. They have been tested in a digital brainstorming tool that has been used by groups of professional product designers. Our "interactive wall" metaphor for interaction has been guided by several goals: to support both free-hand sketching and high-resolution materials, such as images, 3D models and GUI application windows; to present a visual appearance that does not clutter the content with control devices; and to support fluid interaction, which minimizes the amount of attention demanded and interruption due to the mechanics of the interface. We have adapted and extended techniques that were developed for electronic whiteboards and generalized the use of the FlowMenu to execute a wide variety of actions in a single pen stroke, While these techniques were designed for a brainstorming tool, they are very general and can be used in a wide variety of application domains using interactive surfaces.

A passive wand tracked in 3D using computer vision techniques is explored as a new input mechanism for interacting with large displays. We demonstrate a variety of interaction techniques that exploit the affordances of the wand, resulting in an effective interface for large scale interaction. The lack of any buttons or other electronics on the wand presents a challenge that we address by developing a set of postures and gestures to track state and enable command input. We also describe the use of multiple wands, and posit designs for more complex wands in the future.

We present radial scroll, an interface widget to support scrolling particularly on either small or large scale touch displays. Instead of dragging a elevator in a scroll bar, or using repetitive key presses to page up or down, users gesture anywhere on the document surface such that clockwise gestures advance the document; counter clockwise gestures reverse the document. We describe our prototype implementation and discuss the results of an initial user study.

This paper presents tangible interaction techniques for fine-tuning one-to-one scale NURBS curves on a large display for automotive design. We developed a new graspable handle with a transparent groove that allows designers to manipulate virtual curves on a display screen directly. The use of the proposed handle leads naturally to a rich vocabulary of terms describing interaction techniques that reflect existing shape styling methods. A user test raised various issues related to the graspable user interface, two-handed input, and large-display interaction.

We describe a new widget and interaction technique, known as a "Frisbee," for interacting with areas of a large display that are difficult or impossible to access directly. A frisbee is simply a portal to another part of the display. It consists of a local "telescope" and a remote "target". The remote data surrounded by the target is drawn in the telescope and interactions performed within it are applied on the remote data. In this paper we define the behavior of frisbees, show unique affordances of the widget, and discuss design characteristics. We have implemented a test application and report on an experiment that shows the benefit of using the frisbee on a large display. Our results suggest that the frisbee is preferred over walking back and forth to the local and remote spaces at a distance of 4.5 feet.

We introduce Shadow Reaching, an interaction technique that makes use of a perspective projection applied to a shadow representation of a user. The technique was designed to facilitate manipulation over large distances and enhance understanding in collaborative settings. We describe three prototype implementations that illustrate the technique, examining the advantages of using shadows as an interaction metaphor to support single users and groups of collaborating users. Using these prototypes as a design probe, we discuss how the three components of the technique (sensing, modeling, and rendering) can be accomplished with real (physical) or computed (virtual) shadows, and the benefits and drawbacks of each approach.

Classroom BRIDGE supports activity awareness by facilitating planning and goal revision in collaborative, project-based middle school science. It integrates large-screen and desktop views of project times to support incidental creation of awareness information through routine document transactions, integrated presentation of awareness information as part of workspace views, and public access to subgroup activity. It demonstrates and develops an object replication approach to integrating synchronous and asynchronous distributed work for a platform incorporating both desktop and large-screen devices. This paper describes an implementation of these concepts with preliminary evaluation data, using timeline-based user interfaces.

Tape drawing is the art of creating sketches on large scale upright surfaces using black photographic tape. Typically used in the automotive industry, it is an important part of the automotive design process that is currently not computerized. We analyze and describe the unique aspects of tape drawing, and use this knowledge to design and implement a digital tape drawing system. Our system retains the fundamental interaction and visual affordances of the traditional media while leveraging the power of the digital media. Aside from the practical aspect of our work, the interaction techniques developed have interesting implications for current theories of human bimanual interaction.

This paper presents a new GUI architecture for creating advanced interfaces. This model is based on a limited set of general principles that improve flexibility and provide capabilities for implementing information visualization techniques such as magic lenses, transparent tools or semantic zooming. This architecture also makes it possible to create multiple views and application-sharing systems (by sharing views on multiple computer screens) in a simple and uniform way and to handle bimanual interaction and multiple pointers. An experimental toolkit called Ubit was implemented to test the feasibility of this approach. It is based on a pseudo-declarative C++ API that tries to simplify GUI programming by providing a higher level of abstraction.

We present HybridPointing, a technique that lets users easily switch between absolute and relative pointing with a direct input device such as a pen. Our design includes a new graphical element, the Trailing Widget, which remains "close at hand" but does not interfere with normal cursor operation. The use of visual feedback to aid the user's understanding of input state is discussed, and several novel visual aids are presented. An experiment conducted on a large, wall-sized display validates the benefits of HybridPointing under certain conditions. We also discuss other situations in which HybridPointing may be useful. Finally, we present an extension to our technique that allows for switching between absolute and relative input in the middle of a single drag-operation.

Recent work is beginning to reveal the potential of numerical optimization as an approach to generating interfaces and displays. Optimization-based approaches can often allow a mix of independent goals and constraints to be blended in ways that would be difficult to describe algorithmically. While optimization-based techniques appear to offer several potential advantages, further research in this area is hampered by the lack of appropriate tools. This paper presents GADGET, an experimental toolkit to support optimization for interface and display generation. GADGET provides convenient abstractions of many optimization concepts. GADGET also provides mechanisms to help programmers quickly create optimizations, including an efficient lazy evaluation framework, a powerful and configurable optimization structure, and a library of reusable components. Together these facilities provide an appropriate tool to enable exploration of a new class of interface and display generation techniques.

Traditionally, computer interfaces have been confined to conventional displays and focused activities. However, as displays become embedded throughout our environment and daily lives, increasing numbers of them must operate on the periphery of our attention. Peripheral displays can allow a person to be aware of information while she is attending to some other primary task or activity. We present the Peripheral Displays Toolkit (PTK), a toolkit that provides structured support for managing user attention in the development of peripheral displays. Our goal is to enable designers to explore different approaches to managing user attention. The PTK supports three issues specific to conveying information on the periphery of human attention. These issues are abstraction of raw input, rules for assigning notification levels to input, and transitions for updating a display when input arrives. Our contribution is the investigation of issues specific to attention in peripheral display design and a toolkit that encapsulates support for these issues. We describe our toolkit architecture and present five sample peripheral displays demonstrating our toolkit's capabilities.

In interspecies households, it is common for the non homo sapien members to be isolated and ignored for many hours each day when humans are out of the house or working. For pack animals, such as canines, information about a pack member's extended pack interactions (outside of the nuclear household) could help to mitigate this social isolation. We have developed a Pack Activity Watch System: Allowing Broad Interspecies Love In Telecommunication with Internet-Enabled Sociability (PAWSABILITIES) for helping to support remote awareness of social activities. Our work focuses on canine companions, and includes, pawticipatory design, labradory tests, and canid camera monitoring.

Video projectors have typically been used to display images on surfaces whose geometric relationship to the projector remains constant, such as walls or pre-calibrated surfaces. In this paper, we present a technique for projecting content onto moveable surfaces that adapts to the motion and location of the surface to simulate an active display. This is accomplished using a projector based location tracking techinque. We use light sensors embedded into the moveable surface and project low-perceptibility Gray-coded patterns to first discover the sensor locations, and then incrementally track them at interactive rates. We describe how to reduce the perceptibility of tracking patterns, achieve interactive tracking rates, use motion modeling to improve tracking performance, and respond to sensor occlusions. A group of tracked sensors can define quadrangles for simulating moveable displays while single sensors can be used as control inputs. By unifying the tracking and display technology into a single mechanism, we can substantially reduce the cost and complexity of implementing applications that combine motion tracking and projected imagery.

A number of projects within the computer graphics, computer vision, and human-computer interaction communities have recognized the value of using projected structured light patterns for the purposes of doing range finding, location dependent data delivery, projector adaptation, or object discovery and tracking. However, most of the work exploring these concepts has relied on visible structured light patterns resulting in a caustic visual experience. In this work, we present the first design and implementation of a high-resolution, scalable, general purpose invisible near-infrared projector that can be manufactured in a practical manner. This approach is compatible with simultaneous visible light projection and integrates well with future Digital Light Processing (DLP) projector designs -- the most common type of projectors today. By unifying both the visible and non-visible pattern projection into a single device, we can greatly simply the implementation and execution of interactive projection systems. Additionally, we can inherently provide location discovery and tracking capabilities that are unattainable using other approaches.

A technique for creating a touch-sensitive input device is proposed which allows multiple, simultaneous users to interact in an intuitive fashion. Touch location information is determined independently for each user, allowing each touch on a common surface to be associated with a particular user. The surface generates location dependent, modulated electric fields which are capacitively coupled through the users to receivers installed in the work environment. We describe the design of these systems and their applications. Finally, we present results we have obtained with a small prototype device.

Many computer operating systems provide seamless support for multiple display screens, but there are few cross-platform tools for collaborative use of multiple computers in a shared display environment. Mighty Mouse is a novel groupware tool built on the public domain VNC protocol. It is tailored specifically for face-to-face collaboration where multiple heterogeneous computers (usually laptops) are viewed simultaneously (usually via projectors) by people working together on a variety of applications under various operating systems. Mighty Mouse uses only the remote input capability of VNC, but enhances this with various features to support flexible movement between the various platforms, "floor control" to facilitate smooth collaboration, and customization features to accommodate different user, platform, and application preferences in a relatively seamless manner. The design rationale arises from specific observations about how people collaborate in meetings, which allows certain simplifying assumptions to be made in the implementation.

In this paper, we describe the YeTi information sharing system that has been designed to foster community building through informal digital content sharing. The YeTi system is a general information parsing, hosting and distribution infrastructure, with interfaces designed for individual and public content reading. In this paper we describe the YeTi public display interface, with a particular focus on tools we have designed to provide lightweight awareness of others' interactions with posted content. Our tools augment content with metadata that reflect people's reading of content - captured video clips of who's reading and interacting with content, tools to allow people to leave explicit freehand annotations about content, and a visualization of the content access history to show when content is interacted with. Results from an initial evaluation are presented and discussed.

The proliferation of wireless handheld devices is placing the World Wide Web in the palms of users, but this convenience comes at a high interactive cost. The Web that came of age on the desktop is ill-suited for use on the small displays of handhelds. Today, handheld browsing often feels like browsing on a PC with a shrunken desktop. Overreliance on scrolling is a big problem in current handheld browsing. Users confined to viewing a small portion of each page often lack a sense of the overall context --- they may feel lost in a large page and be forced to remember the locations of items as those items scroll out of view. In this paper, we present a synthesis of interaction techniques to address these problems. We implemented these techniques in a prototype, WebThumb, that can browse the live Web.

Conventional scrolling methods for small sized display in PDAs or mobile phones are difficult to use when frequent switching of scrolling and editing operations are required, for example, browsing and operating large sized WWW pages.In this paper, we propose a new user-interface method to provide seamless switching between scrolling and other operations such as editing, based on "Paperweight Metaphor". A sheet of paper that has been placed on a slippery table is difficult to draw on. Therefore, in order to write or draw something on the sheet of paper, a person must secure the paper with his/her palm to avoid the paper from moving. This will be a good metaphor to design switching operation of scroll and editing modes.We have made prototype systems by placing a touch sensor under each PDA display where user's palm will be hit. Three application programs - map browser, WWW browser, and photograph browser - that switch between scrolling and other operation modes depending on sensor output have been developed. We have carried out user tests on this mode switching method and have received favorable feedback on the same.

We introduce the Boom Chameleon, a novel input/output device consisting of a flat-panel display mounted on a tracked mechanical boom. The display acts as a physical window into 3D virtual environments, through which a one-to-one mapping between real and virtual space is preserved. The Boom Chameleon is further augmented with a touch-screen and a microphone/speaker combination. We present a 3D annotation application that exploits this unique configuration in order to simultaneously capture viewpoint, voice and gesture information. Design issues are discussed and results of an informal user study on the device and annotation software are presented. The results show that the Boom Chameleon annotation facilities have the potential to be an effective, easy to learn and operate 3D design review system.

ThinSight is a novel optical sensing system, fully integrated into a thin form factor display, capable of detecting multi-ple fingers placed on or near the display surface. We describe this new hardware in detail, and demonstrate how it can be embedded behind a regular LCD, allowing sensing without degradation of display capability. With our approach, fingertips and hands are clearly identifiable through the display. The approach of optical sensing also opens up the exciting possibility for detecting other physical objects and visual markers through the display, and some initial experiments are described. We also discuss other novel capabilities of our system: interaction at a distance using IR pointing devices, and IR-based communication with other electronic devices through the display. A major advantage of ThinSight over existing camera and projector based optical systems is its compact, thin form-factor making such systems even more deployable. We therefore envisage using ThinSight to capture rich sensor data through the display which can be processed using computer vision techniques to enable both multi-touch and tangible interaction.

We explore the design space of freehand pointing and clicking interaction with very large high resolution displays from a distance. Three techniques for gestural pointing and two for clicking are developed and evaluated. In addition, we present subtle auditory and visual feedback techniques to compensate for the lack of kinesthetic feedback in freehand interaction, and to promote learning and use of appropriate postures.

Volumetric displays provide interesting opportunities and challenges for 3D interaction and visualization, particularly when used in a highly interactive manner. We explore this area through the design and implementation of techniques for interactive direct manipulation of objects with a 3D volumetric display. Motion tracking of the user's fingers provides for direct gestural interaction with the virtual objects, through manipulations on and around the display's hemispheric enclosure. Our techniques leverage the unique features of volumetric displays, including a 360° viewing volume that enables manipulation from any viewpoint around the display, as well as natural and accurate perception of true depth information in the displayed 3D scene. We demonstrate our techniques within a prototype 3D geometric model building application.

Volumetric displays, which display imagery in true 3D space, are a promising platform for the display and manipulation of 3D data. To fully leverage their capabilities, appropriate user interfaces and interaction techniques must be designed. In this paper, we explore 3D selection techniques for volumetric displays. In a first experiment, we find a ray cursor to be superior to a 3D point cursor in a single target environment. To address the difficulties associated with dense target environments we design four new ray cursor techniques which provide disambiguation mechanisms for multiple intersected targets. Our techniques showed varied success in a second, dense target experiment. One of the new techniques, the depth ray, performed particularly well, significantly reducing movement time, error rate, and input device footprint in comparison to the 3D point cursor.

We present HybridPointing, a technique that lets users easily switch between absolute and relative pointing with a direct input device such as a pen. Our design includes a new graphical element, the Trailing Widget, which remains "close at hand" but does not interfere with normal cursor operation. The use of visual feedback to aid the user's understanding of input state is discussed, and several novel visual aids are presented. An experiment conducted on a large, wall-sized display validates the benefits of HybridPointing under certain conditions. We also discuss other situations in which HybridPointing may be useful. Finally, we present an extension to our technique that allows for switching between absolute and relative input in the middle of a single drag-operation.