reality

We introduce an inexpensive position input device called the FieldMouse, with which a computer can tell the position of the device on paper or any flat surface without using special input tablets or position detection devices. A FieldMouse is a combination of an ID recognizer like a barcode reader and a mouse which detects relative movement of the device. Using a FieldMouse, a user first detects an ID on paper by using the barcode reader, and then drags it from the ID using the mouse. If the location of the ID is known, the location of the dragged FieldMouse can also be calculated by adding the amount of movement from the ID to the position of the FieldMouse. Using a FieldMouse in this way, any flat surface can work as a pointing device that supports absolute position input, just by putting an ID tag somewhere on the surface. A FieldMouse can also be used for enabling a graphical user interface (GUI) on paper or on any flat surface by analyzing the direction and the amount of mouse movement after detecting an ID. In this paper, we introduce how a FieldMouse can be used in various situations to enable computing in real-world environments.

It is well known that paper is a very fluid, natural, and easy to use medium for manipulating some kinds of information. It is familiar, portable, flexible, inexpensive, and offers good readability properties. Paper also has well known limitations when compared with electronic media. Work in hybrid paper electronic interfaces seeks to bring electronic capabilities to real paper in order to obtain the best properties of each. This paper describes a hybrid paper electronic system --- the Paper PDA --- which is designed to allow electronic capabilities to be employed within a conventional paper notebook, calendar, or organizer. The Paper PDA is based on a simple observation: a paper notebook can be synchronized with a body of electronic information much like an electronic PDA can be synchronized with information hosted on a personal computer. This can be accomplished by scanning, recognizing and processing its contents, then printing a new copy. This paper introduces the Paper PDA concept and considers interaction techniques and applications designed to work within the Paper PDA. The StickerLink technique supports on-paper hyperlinking using removable paper stickers. Two applications are also considered which look at aspects of electronic communications via the Paper PDA.

We describe a view-management component for interactive 3D user interfaces. By view management, we mean maintaining visual constraints on the projections of objects on the view plane, such as locating related objects near each other, or preventing objects from occluding each other. Our view-management component accomplishes this by modifying selected object properties, including position, size, and transparency, which are tagged to indicate their constraints. For example, some objects may have geometric properties that are determined entirely by a physical simulation and which cannot be modified, while other objects may be annotations whose position and size are flexible.We introduce algorithms that use upright rectangular extents to represent on the view plane a dynamic and efficient approximation of the occupied space containing the projections of visible portions of 3D objects, as well as the unoccupied space in which objects can be placed to avoid occlusion. Layout decisions from previous frames are taken into account to reduce visual discontinuities. We present augmented reality and virtual reality examples to which we have applied our approach, including a dynamically labeled and annotated environment.

Ordinary paper offers properties of readability, fluidity, flexibility, cost, and portability that current electronic devices are often hard pressed to match. In fact, a lofty goal for many interactive systems is to be "as easy to use as pencil and paper". However, the static nature of paper does not support a number of capabilities, such as search and hyperlinking that an electronic device can provide. The Paper PDA project explores ways in which hybrid paper electronic interfaces can bring some of the capabilities of the electronic medium to interactions occurring on real paper. Key to this effort is the invention of on-paper interaction techniques which retain the flexibility and fluidity of normal pen and paper, but which are structured enough to allow robust interpretation and processing in the digital world. This paper considers the design of a class of simple printed templates that allow users to make common marks in a fluid fashion, and allow additional gestures to be invented by the users to meet their needs, but at the same time encourages marks that are quite easy to recognize.

Using a participatory design process, we created three prototype augmented laboratory notebooks that provide the missing link between paper, physical artifacts and on-line data. The final a-book combines a graphics tablet and a PDA. The tablet captures writing on the paper notebook and the PDA acts as an "interaction lens" or window between physical and electronic documents. Our approach is document-centered, with a software architecture based on layers of physical and electronic information.

We present a situation-awareness aid for augmented reality systems based on an annotated "world in miniature." Our aid is designed to provide users with an overview of their environment that allows them to select and inquire about the objects that it contains. Two key capabilities are discussed that are intended to address the needs of mobile users. The aid's position, scale, and orientation are controlled by a novel approach that allows the user to inspect the aid without the need for manual interaction. As the user alternates their attention between the physical world and virtual aid, popup annotations associated with selected objects can move freely between the objects' representations in the two models.

In this paper, we describe The Designer's Augmented Reality Toolkit (DART). DART is built on top of Macromedia Director, a widely used multimedia development environment. We summarize the most significant problems faced by designers working with AR in the real world, and discuss how DART addresses them. Most of DART is implemented in an interpreted scripting language, and can be modified by designers to suit their needs. Our work focuses on supporting early design activities, especially a rapid transition from story-boards to working experience, so that the experiential part of a design can be tested early and often. DART allows designers to specify complex relationships between the physical and virtual worlds, and supports 3D animatic actors (informal, sketch-based content) in addition to more polished content. Designers can capture and replay synchronized video and sensor data, allowing them to work off-site and to test specific parts of their experience more effectively.

Video projectors have typically been used to display images on surfaces whose geometric relationship to the projector remains constant, such as walls or pre-calibrated surfaces. In this paper, we present a technique for projecting content onto moveable surfaces that adapts to the motion and location of the surface to simulate an active display. This is accomplished using a projector based location tracking techinque. We use light sensors embedded into the moveable surface and project low-perceptibility Gray-coded patterns to first discover the sensor locations, and then incrementally track them at interactive rates. We describe how to reduce the perceptibility of tracking patterns, achieve interactive tracking rates, use motion modeling to improve tracking performance, and respond to sensor occlusions. A group of tracked sensors can define quadrangles for simulating moveable displays while single sensors can be used as control inputs. By unifying the tracking and display technology into a single mechanism, we can substantially reduce the cost and complexity of implementing applications that combine motion tracking and projected imagery.

A significant problem encountered when building Augmented Reality (AR) systems is that all spatial knowledge about the world has uncertainty associated with it. This uncertainty manifests itself as registration errors between the graphics and the physical world, and ambiguity in user interaction. In this paper, we show how estimates of the registration error can be leveraged to support predictable selection in the presence of uncertain 3D knowledge. These ideas are demonstrated in osgAR, an extension to OpenSceneGraph with explicit support for uncertainty in the 3D transformations. The osgAR runtime propagates this uncertainty throughout the scene graph to compute robust estimates of the probable location of all entities in the system from the user's viewpoint, in real-time. We discuss the implementation of selection in osgAR, and the issues that must be addressed when creating interaction techniques in such a system.

A number of projects within the computer graphics, computer vision, and human-computer interaction communities have recognized the value of using projected structured light patterns for the purposes of doing range finding, location dependent data delivery, projector adaptation, or object discovery and tracking. However, most of the work exploring these concepts has relied on visible structured light patterns resulting in a caustic visual experience. In this work, we present the first design and implementation of a high-resolution, scalable, general purpose invisible near-infrared projector that can be manufactured in a practical manner. This approach is compatible with simultaneous visible light projection and integrates well with future Digital Light Processing (DLP) projector designs -- the most common type of projectors today. By unifying both the visible and non-visible pattern projection into a single device, we can greatly simply the implementation and execution of interactive projection systems. Additionally, we can inherently provide location discovery and tracking capabilities that are unattainable using other approaches.

Touch is a compelling input modality for interactive devices; however, touch input on the small screen of a mobile device is problematic because a user's fingers occlude the graphical elements he wishes to work with. In this paper, we present LucidTouch, a mobile device that addresses this limitation by allowing the user to control the application by touching the back of the device. The key to making this usable is what we call pseudo-transparency: by overlaying an image of the user's hands onto the screen, we create the illusion of the mobile device itself being semi-transparent. This pseudo-transparency allows users to accurately acquire targets while not occluding the screen with their fingers and hand. Lucid Touch also supports multi-touch input, allowing users to operate the device simultaneously with all 10 fingers. We present initial study results that indicate that many users found touching on the back to be preferable to touching on the front, due to reduced occlusion, higher precision, and the ability to make multi-finger input.

We have previously developed a collaborative infrastructure called SCAPE - an acronym for Stereoscopic Collaboration in Augmented and Projective Environments - that integrates the traditionally separate paradigms of virtual and augmented reality. In this paper, we extend SCAPE by formalizing its underlying mathematical framework and detailing three augmented Widgets constructed via this framework: CoCylinder, Magnifier, and CoCube. These devices promote intuitive ways of selecting, examining, and sharing synthetic objects, and retrieving associated documentary text. Finally we present a testbed application to showcase SCAPE's capabilities for interaction in large, augmented virtual environments.

In this paper, we describe The Designer's Augmented Reality Toolkit (DART). DART is built on top of Macromedia Director, a widely used multimedia development environment. We summarize the most significant problems faced by designers working with AR in the real world, and discuss how DART addresses them. Most of DART is implemented in an interpreted scripting language, and can be modified by designers to suit their needs. Our work focuses on supporting early design activities, especially a rapid transition from story-boards to working experience, so that the experiential part of a design can be tested early and often. DART allows designers to specify complex relationships between the physical and virtual worlds, and supports 3D animatic actors (informal, sketch-based content) in addition to more polished content. Designers can capture and replay synchronized video and sensor data, allowing them to work off-site and to test specific parts of their experience more effectively.

This paper describes a Computer Aided Design system for sketching free-form polygonal surfaces such as terrains and other natural objects. The user manipulates two 3D position and orientation trackers with three buttons, one for each hand. Each hand has a distinct role to play, with the dominant hand being responsible for picking and manipulation, and the less-dominant hand being responsible for context setting of various kinds. The less-dominant hand holds the workpiece, sets which refinement level that can be picked by the dominant hand, and generally acts as a counterpoint to the dominant hand. In this paper, the architecture of the system is outlined, and a simple surface is shown.

We present a survey of design issues for developing effective free-space three-dimensional (3D) user interfaces. Our survey is based upon previous work in 3D interaction, our experience in developing free-space interfaces, and our informal observations of test users. We illustrate our design issues using examples drawn from instances of 3D interfaces.

For example, our first issue suggests that users have difficulty understanding three-dimensional space. We offer a set of strategies which may help users to better perceive a 3D virtual environment, including the use of spatial references, relative gesture, two-handed interaction, multisensory feedback, physical constraints, and head tracking. We describe interfaces which employ these strategies.

Our major contribution is the synthesis of many scattered results, observations, and examples into a common framework. This framework should serve as a guide to researchers or systems builders who may not be familiar with design issues in spatial input. Where appropriate, we also try to identify areas in free-space 3D interaction which we see as likely candidates for additional research.

An extended and annotated version of the references list for this paper is available on-line through mosaic at address http://uvacs.cs.virginia.edu/~kph2q/.

We present the Haptic Shading Framework (HSF), a framework for procedurally defining haptic texture. HSF haptic texture shaders are short procedures allowing an application-programmer to easily define interesting haptic surface interaction and the parameters that control the surface properties. These shaders provide the illusion of surface characteristics by altering previously calculated forces from object collision in the haptic pipeline.HSF can be used in an existing haptic application with few modifications. The framework consists of user-programmable modules that are dynamically loaded. This framework and all user-defined procedures are written in C++, with a provided library of useful math and geometry functions. These functions are meant to mimic RenderMan functionality, creating a familiar shading environment. As we demonstrate, many procedural shading methods and algorithms can be directly adopted for haptic shading.

We have previously developed a collaborative infrastructure called SCAPE - an acronym for Stereoscopic Collaboration in Augmented and Projective Environments - that integrates the traditionally separate paradigms of virtual and augmented reality. In this paper, we extend SCAPE by formalizing its underlying mathematical framework and detailing three augmented Widgets constructed via this framework: CoCylinder, Magnifier, and CoCube. These devices promote intuitive ways of selecting, examining, and sharing synthetic objects, and retrieving associated documentary text. Finally we present a testbed application to showcase SCAPE's capabilities for interaction in large, augmented virtual environments.