Personal Interfaces in Ubiquitous Environments

Position paper for the CHI'2001 Workshop on
Building the Ubiquitous Computing User Experience

Jeffrey Nichols
Brad A. Myers
Rob Miller

Human Computer Interaction Institute
School of Computer Science
Carnegie Mellon University
5000 Forbes Avenue
Pittsburgh, PA 15213-3891
(412) 268-8827
FAX: (412) 268-1266
jeffreyn+@cs.cmu.edu
http://www.cs.cmu.edu/~jeffreyn/

The Pebbles research project (http://www.pebbles.cs.cmu.edu/) [Myers 2000] has been studying the use of hand-held devices at the same time as other computing devices.  As people move about the world, they will be entering and leaving spaces where there are embedded or desktop computers, such as offices, conference rooms, classrooms, and even "smart homes." We are exploring the many issues surrounding how to have the user interface, functionality, and information spread across multiple devices that are used simultaneously. For example, there are many ways that a personal digital assistant (PDA), such as a Palm Pilot or PocketPC device, can serve as a useful adjunct to a personal computer to enhance the interaction with existing desktop applications for individuals or groups. New applications may distribute their user interfaces across multiple devices so the user can choose the appropriate device for each part of the interaction.  A key focus of our research is that the hand-held computers are used both as output devices and as input devices to control the activities on the other computers.

A recent direction for the Pebbles project has been to explore the use of hand-helds as a "Personal Universal Controller." PDAs like the Palm Pilot are becoming increasingly ubiquitous, and technology such as Bluetooth and IEEE 802.11 will bring PDAs into close interactive communication with other types of devices. Furthermore, cell-phones and pagers, which are primarily used for communication, are increasingly becoming programmable. We are investigating how these kinds of hand-held devices can be used with all kinds of home, office, and factory equipment. The concept is that people can use their own hand-held to control the lights, a photocopier in an office, a machine tool in a factory, a VCR at home, a piece of test equipment in the field, or almost any other kind of device. The device will send to the hand-held a description of its control parameters, and the hand-held uses this information to automatically create an appropriate control panel, taking into account the properties of the controls that are needed, the properties of the hand-held (the display type and input techniques available), and the properties of the user (what language is preferred, whether left or right handed, how big the buttons should be based on whether the user prefers using a finger or a stylus). The user can then control the device using the hand-held.

We believe that controller can solve many of the difficult problems inherent in ubiquitous technologies. Here are some examples:

• As a user moves through an environment, they may not be aware of the services available to them or the devices that they may be able to control. A personal controller gives the user a means to discover what is available around them. The service discovery layer of the Bluetooth protocol [Bluetooth 1999] or a similar standard could be used to locate the controllable services and devices within range. These could be presented to the user in a list on the controller. A map of relative device and service locations could also be generated, where appropriate, if a location service [Bahl 2000] was available to the controller unit. This enables users to, for example, quickly locate the nearest public video phone when they need to make a video conference call. They could also quickly see what electronic amenities are available in an unfamiliar hotel room without thumbing through pages of hotel documentation.
• While each user will have a space of devices that they are familiar with, it is certain that they will encounter unfamiliar devices when they enter a new environment. This could be in a hotel room or at a customer's office. Users may encounter difficulties because many of the devices that are found in these new spaces will offer the same functionality as familiar devices but have user interfaces that are inconsistent with the users' previous experience. For example, many people have difficulties using the alarm clocks in hotel rooms. Everyone knows what the device is for, but there is such variation across manufacturers and models that most people call the front desk for a wake-up call without even trying to set the alarm clock. Similar examples could be made for copy machines, home stereos, and office telephones. A personal controller unit solves this problem by providing a manufacturer-independent interface to all devices. The same interface that is used to set the alarm clock at home could be used on the clock in the hotel room. The controller can exploit previously generated interfaces to maintain consistency and improve usability.
• No user is the same. A ubiquitous interface must be customizable enough to work for a wide range of users. The controller unit handles this by allowing the user to influence how interfaces are generated for unfamiliar devices. This might involve moving controls to accomodate a left-handed user or using larger labels for older users with poor eyesight. A direct manipulation interface would also be provided for those users that want control of where every interface element is placed. Users could drag elements out of a current interface and layout them out manually on the controller screen. Users could even share their forms as users of the Philips Pronto Interactive Remote Control currently do [Philips 2001, RemoteCentral 2001].
• No device or service exists in a world of its own. Users may find that they commonly deal with several particular devices at one time and would benefit from combining some functionality from each device into a single "custom" interface. The controller unit provides a convenient location for such an interface to be created and used. The direct manipulation method mentioned above does not need to be confined to creating new screens for a single device. Functions from multiple devices in the environment could be combined to create control panels that grouped commonly used functions. The user might put controls for their overhead lights on the same panel as the television power button, for example. The user would also be allowed to create macros, sets of commands that might control several devices in a single stream. Macros could be assigned to controls on a custom interface screen. The user might create several "mood" buttons, each one setting the lights at different levels and playing a different selection on the CD player.
• If users are listening to their stereo and are then interrupted by the telephone, they might like to turn down the stereo volume before they begin speaking on the phone. They may not always want the volume turned down however, and could be annoyed by an automatic change. The controller unit allows the user to stay in control of their environment. When the phone rings, the user picks up their controller which is showing a phone interface with a volume control and mute button for the stereo overlaid in one corner. The user hits the mute button before pressing the control that picks up the phone. These kinds of interactions are easily enabled because the controller unit can establish a two-way connection with all the devices and services in the current environment. It can perform a query to discover what is in use and present appropriate controls to a user when the state of the environment changes.

There are many significant research issues involved in bringing this vision to fruition, many of which we intend to investigate. Our initial focus will be on interface generation and the specification language that supports it. We have begun our research by creating hand-designed interfaces for an AIWA shelf stereo system and an AT&T office telephone/answering machine, targeted at the Palm hand-held computer. We also plan to create hand-designed interfaces for several other devices, a copier and all-in-one printer, and several other target hand-held units, a WAP phone and a custom device built by a local corporation. After we complete our hand-designed interfaces, the next step is to analyze the interfaces in order to understand what information from the controlled device is needed to create a high-quality interface. The specification language and the interface generator will be created on the basis of this analysis.

For our method to work, we need to validate that our hand-designed interfaces are of high quality. In a preliminary experiment, we showed that users made about 1/5 the errors and took about 1/2 the time to perform complex tasks using the Palm-based interface compared to the real manufacturer's interfaces for the stereo and telephone [Nichols 2001].

Jeffrey Nichols, a PhD student working on the personal universal controller project, would like to participate in this workshop as a representative of our interest in the usability issues inherent in creating a ubiquitous computing environment. We believe that hand-held computers will be important as portals to investigating and using the devices and services of a ubiquitous environment. Although one might hope that the interfaces will disappear into the environment, we believe that there will always be devices that users will bring with them, and that these devices could augment the interface to whatever environment the user is in.

References

[Bahl 2000] P. Bahl and V. Padmanabhan, "RADAR: An in-building RF-based user location and tracking system," In Proceedings of the IEEE Infocom 2000, Tel-Aviv, Israel, vol. 2, Mar. 2000, pp. 775-784. Adobe Acrobat (pdf)

[Bluetooth 1999] Bluetooth Consortium, Specification of the Bluetooth System: Core, Vol 1B, December 1999, http://www.bluetooth.com Adobe Acrobat (pdf)

[Myers 2000] Brad A. Myers, Robert C. Miller, Benjamin Bostwick, and Carl Evankovich, "Extending the Windows Desktop Interface With Connected Handheld Computers," 4th USENIX Windows Systems Symposium, August 3-4, 2000, Seattle, WA. pp. 79-88. Adobe Acrobat (pdf)

[Nichols 2001] Jeffrey W. Nichols. "Using Handhelds as Controls for Everyday Appliances: A Paper Prototype Study." ACM CHI'2001 Student Posters. Seattle, WA. March 31-April 5, 2001. To appear. Adobe Acrobat (pdf)

[Philips 2001] Royal Philips Corp. http://www.pronto.philips.com/, January 2001.

[RemoteCentral 2001] RemoteCentral.com, http://www.remotecentral.com/pronto/, January 2001.

Last modified: Fri Jan 26 16:16:43 EST 2001