Project Summary

With the coming wireless technologies, such as BlueTooth and IEEE 802.11, connecting hand-held computers and conventional computers together will no longer be an occasional event for synchronization. Instead, the devices will frequently be in close, interactive communication. Many environments, such as offices, meeting rooms and classrooms, already contain computers, and the smart homes of the future will have ubiquitous embedded computation. When the user enters one of these environments carrying a hand-held or wearable computer, how will that computer interact with the environment? The Pebbles project is exploring the many ways that small handheld Personal Digital Assistants (PDAs) such as 3Com Palm Pilots or Windows CE devices can serve as a useful adjunct to the "fixed" computers in those situations. One set of applications supports meetings where the participants are co-located. All participants' PDAs are in continuous two-way communication with each other, and with the main computer which is often projected on a screen to serve as the focal point of the discussion. For classrooms, we are investigating how the student's hand-helds can enhance testing and notetaking when they are connected to the instructor's PC. Another set of applications supports a single user using the PDA as an extra input and output device. In the future, we will also explore the use of the PDA as a customizable, intelligent front end to other applications and devices.

Funding

• Partially funded by a grant from NSF.
  2003-2006. "Using Handhelds to Help People with Motor Impairments."
  This research is funded in part by the National Science Foundation under Grant No. UA-0308065. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect those of the National Science Foundation. 
• Partially funded by a grant from NSF.
  2001-2004. "Making it Easier to Interact with Technology Through Handheld Personal Universal Controllers."
  This research is funded in part by the National Science Foundation under Grant No. IIS-0117658.  Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect those of the National Science Foundation.
• Partially funded by grants from MICROSOFT.
  This research is also partially funded by five generous grants from Microsoft (in 1998, 1999, 2000, 2001 and 2002), sponsored by Microsoft Research, and by the Windows CE development group.
• Partially funded by two  grants from General Motors, as part of the CMU General Motors Collaborative Laboratory, for Summer, 2002, and calendar year 2003.
• Partially funded by a grant from NEC Foundation of America. "Handhelds as Assistive Technologies for People with Muscular Disabilities." 2003.  See the NEC Foundation page for this project.

Former Funding Grants:

• Partially funded by a grant from DARPA.
  1998-2002. This research was performed in part in connection with Contract number DAAD17-99-C-0061 with the U.S. Army Research Laboratory. The views and conclusions contained in this document are those of the authors and should not be interpreted as presenting the official policies or position, either expressed or implied, of the U.S. Army Research Laboratory or the U.S. Government unless so designated by other authorized documents. Citation of manufacturer's or trade names does not constitute an official endorsement or approval of the use thereof. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation hereon.
• Partially funded by a grant from the Pittsburgh Digital Greenhouse.
  2000-2002.  This grant, in collaboration with Maya Design focused on the "Personal Universal Controller".

Equipment Grants:

• Thanks for a generous equipment grant from Symbol Technologies of one PPT 2700 (ruggedized Palm-size PC running Windows CE with a laser scanner and radio), two SPT 1750s (ruggedized Palm with laser scanner and radio), a Spectrum24 Wireless LAN AP 4111 Access Point, and a LA 3020 Wireless LAN PC Card.
• Thanks for a generous equipment grant from Hewlett Packard, under the University Grants Program of 10 Jornada 430se, 120 Jornada 680 and about 100 Jornada 720.
• Thanks for a generous equipment grant from Lucent Technologies of about 300 Wavelan wireless network PCMCIA cards for the HP Jornada
• Thanks for a generous equipment grant from Palm Computing, Inc. of 10 Palm IIIx Connected Organizers and 12 Palm Vx Handhelds and 4 m100 handhelds.
• Thanks for a generous equipment grant from IBM of ten IBM Workpads 8602-30X
• Thanks to SMART Technologies, Inc. for the donation of a SMART Board 580 and stand and USB cable.
• Thanks to Synergy Solutions, a division of BarPoint.com, which donated 10 ten-foot Palm cables, that are normally distributed free with a purchase of SlideShow Commander.
• Thanks to TDK Systems Europe, LTD for six BlueTooth modules for Palms and PCs: 4 Blue5 sleds for Palms, and 2 USB Adaptors.
• Thanks to Mitsubishi Electric Research Laboratory for donating a Mitsubishi WS-65909 65" Integrated HDTV Diamond Series television and Mitsubishi HS-HD2000U High Definition Digital VCR that support the HAVi protocol. Also for funding our purchase of the VividLogic HAVi SDK.
• Thanks to Lutron, Inc. for the donation of a Radio RA "dollhouse" demonstration unit and the RS232 interface for it.
• Thanks to IBM Canada, Ltd. for one IBM Thinkpad A31 laptop.
• Thanks to Vivid Logic for a discount price on their HAVi SDK.
• Thanks to Lantronix, Inc. for the donation of the Premise Home Control Software - Dealer Edition V2.0, and Lantronix UDS-10-01 Port Device Server
• Thanks to Microsoft Research for multiple PocketPCs, Smartphones, and MSDN
• Thanks to AT Sciences, LLC for the donation of one Everest & Jennings Hand Control, model 90487500 joystick controller for a wheelchair
• Thanks to the Nokia Research Center for the donation of 2 Nokia NGage and 8 Nokia 3650 phones
• Thanks to Synaptics, Inc. for the donation of  3 Touchpads.

We welcome funding from other sources. Please contact Brad Myers for more information.

Papers and Talks about Pebbles

The first  one is the best current reference to use for Pebbles:

1. Brad A. Myers. "Using Hand-Held Devices and PCs Together," Communications of the ACM. Volume 44, Issue 11. November, 2001. pp. 34 - 41. Adobe Acrobat (pdf)

   Talks and Videos:

2. "Mobile Devices for Control", talk given as part of: CS547: Stanford University Human-Computer Interaction Seminar on People, Computers, and Design, November 22, 2002, Palo Alto, CA. Available as streaming video (86min)  (warning: streaming video link crashes Netscape 4.7!)
3. "Using Handheld Computers and PCs Together" (an overview of the Pebbles project). 90 minute talk given to the HCI Seminar Series, Carnegie Mellon University, 11-17-1999. Click here to see a video of this talk

   These are in forward chronological order

4. Brad A. Myers, Herb Stiel, and Robert Gargiulo. "Collaboration Using Multiple PDAs Connected to a PC." Proceedings CSCW'98: ACM Conference on Computer-Supported Cooperative Work, November 14-18, 1998, Seattle, WA. pp. 285-294.
   Postscript (about 4.4 megabytes) or Adobe Acrobat (pdf) (about 400K)
5. Brad A. Myers. An Implementation Architecture to Support Single-Display Groupware. Carnegie Mellon University School of Computer Science Technical Report, no. CMU-CS-99-139 and Human Computer Interaction Institute Technical Report CMU-HCII-99-101. May, 1999. Available in postscript or pdf formats.
6. Robert C. Miller and Brad A. Myers, "Synchronizing Clipboards of Multiple Computers," CHI Letters: ACM Symposium on User Interface Software and Technology, UIST'99, vol. 1, no. 1. November 7-10, 1999. Asheville, NC. pp. 65-66.
7. Brad Myers, Kin Pou Lie and Bo-Chieh ("Jerry") Yang, "Two-Handed Input Using a PDA and a Mouse", Proceedings CHI'2000: Human Factors in Computing Systems. April 1-6, 2000. The Hague, The Netherlands. pp. 41-48. Postscript or Acrobat (pdf).
8. Karen Cross and Adrienne Warmack. "Contextual Inquiry: Quantification and Use in Videotaped Analysis". (Student Poster) Adjunct Proceedings CHI'2000: Human Factors in Computing Systems. April 1-6, 2000. The Hague, The Netherlands. pp. 317-318. pdf (Acrobat).
9. Brad Myers, "The Pebbles Project: Using PCs and Hand-held Computers Together; Demonstration Extended Abstract." Adjunct Proceedings CHI'2000: Human Factors in Computing Systems. April 1-6, 2000. The Hague, The Netherlands. pp. 14-15.
10. Franklin Chen, Brad Myers and David Yaron, Using Handheld Devices for Tests in Classes. Carnegie Mellon University School of Computer Science Technical Report, no. CMU-CS-00-152 and Human Computer Interaction Institute Technical Report CMU-HCII-00-101. July, 2000. pdf or postscript
11. Brad A. Myers, Robert C. Miller, Benjamin Bostwick, and Carl Evankovich, "Extending the Windows Desktop Interface With Connected Handheld Computers," 4th USENIX Windows Systems Symposium, August 3-4, 2000, Seattle, WA. pp. 79-88. postscript or Adobe Acrobat (pdf).
12. Brad A. Myers. "Collaboration Using Multiple PDAs Connected to a PC," Workshop on Shared Environments to Support Face-to-Face Collaboration at CSCW'2000, Philadelphia, PA. html
13. Brad A. Myers. "Using Multiple Devices Simultaneously for Display and Control." IEEE Personal Communications special issue on "Networking the Physical World." vol. 7, no. 5, Oct. 2000. pp. 62-65.
14. Brad A. Myers, Jeff Nichols, Rob Miller. "User Interfaces that Span Hand-Held and Fixed Devices" Workshop on Distributed and Disappearing User Interfaces in Ubiquitous Computing at CHI'2001, Seattle, WA. Albrecht Schmidt, Peter Ljundgstrand, and Anind Dey, editors. University of Karlsruhe Faculty of Information Technical Report 2001-6. ISSN 1432-7864. html
15. Jeffrey Nichols, Brad A. Myers, Rob Miller. "Personal Interfaces in Ubiquitous Environments". Workshop on Building the Ubiquitous Computing User Experience at CHI'2001, Seattle, WA. html
16. Choon Hong Peck, "Useful Parameters for the Design of Laser Pointer Interaction Techniques." ACM CHI'2001 Student Posters. Seattle, WA. March 31-April 5, 2001. pp. 461-462. Adobe Acrobat (pdf)
17. Jeffrey W. Nichols. "Using Handhelds as Controls for Everyday Appliances: A Paper Prototype Study." ACM CHI'2001 Student Posters. Seattle, WA. March 31-April 5, 2001. pp. 443-444. Adobe Acrobat (pdf)
18. Brad A. Myers, Choon Hong Peck, Jeffrey Nichols, Dave Kong, and Robert Miller, "Interacting At a Distance Using Semantic Snarfing," ACM UbiComp'2001, Sept 30 - Oct 2, 2001, Atlanta, Georgia. pp. 305-314. Adobe Acrobat (pdf)
19. Brad A. Myers and Jeffrey Nichols, "Communication Ubiquity Enables Ubiquitous Control." 'Boaster' for Human-Computer Interaction Consortium (HCIC'2002). Winter Park, CO, Feb, 2002.  html
20. Brad A. Myers, Rishi Bhatnagar, Jeffrey Nichols, Choon Hong Peck, Dave Kong, Robert Miller, and A. Chris Long. "Interacting At a Distance: Measuring the Performance of Laser Pointers and Other Devices." Proceedings CHI'2002: Human Factors in Computing Systems. Minneapolis, Minnesota, April 20-25, 2002. pp. 33-40. Adobe Acrobat (pdf)
21. Jeffrey Nichols. "Informing Automatic Generation of Remote Control Interfaces with Human Designs" CHI'2002 Student Posters. Minneapolis, Minnesota, April 20-25, 2002. pp. 864-865. Adobe Acrobat (pdf)
22. Brad A. Myers, Jacob O. Wobbrock, Sunny Yang, Brian Yeung, Jeffrey Nichols, and Robert Miller. "Using Handhelds to Help People with Motor Impairments," Fifth International ACM SIGCAPH Conference on Assistive Technologies; ASSETS 2002. July 8-10, 2002. Edinburgh, Scotland. pp. 89-96. Adobe Acrobat (pdf) or html
23. Brad A. Myers. "Mobile Devices for Control," The Fourth Symposium on Human-Computer Interaction for Mobile Devices, Mobile HCI'02. (Keynote speech), September 18-20, 2002, Pisa, Italy. pp. 1-8. pdf
24. Jeffrey Nichols, Brad Myers, Thomas K. Harris, Roni Rosenfeld, Stefanie Shriver, Michael Higgins, Joseph Hughes. "Requirements for Automatically Generating Multi-Modal Interfaces for Complex Appliances," IEEE Fourth International Conference on Multimodal Interfaces, ICMI'02. Pittsburgh, PA. October 14-16, 2002. pp. 377-382. pdf
25. Brad Myers, Robert Malkin, Michael Bett, Alex Waibel, Ben Bostwick, Robert C. Miller, Jie Yang, Matthias Denecke, Edgar Seemann, Jie Zhu, Choon Hong Peck, Dave Kong, Jeffrey Nichols, Bill Scherlis. "Flexi-modal and Multi-Machine User Interfaces," IEEE Fourth International Conference on Multimodal Interfaces, ICMI'02, Pittsburgh, PA. October 14-16, 2002. pp. 343-348. pdf
26. Jeffrey Nichols, Brad A. Myers, Michael Higgins, Joe Hughes, Thomas K. Harris, Roni Rosenfeld, Mathilde Pignol. "Generating Remote Control Interfaces for Complex Appliances." CHI Letters: ACM Symposium on User Interface Software and Technology, UIST'02, 27-30 Oct. 2002, Paris, France. pp. 161-170. pdf
27. Jacob O. Wobbrock, Jodi Forlizzi, Scott E. Hudson, Brad A. Myers, "WebThumb: Interaction Techniques for Small-Screen Browsers," CHI Letters: ACM Symposium on User Interface Software and Technology, UIST'02, 27-30 Oct. 2002, Paris, France. pp. 205-208. pdf
28. Brad A. Myers, Jeffrey Nichols, Jacob O. Wobbrock, Kevin Litwack, Michael Higgins, Joe Hughes, Thomas K. Harris, Roni Rosenfeld, Mathilde Pignol. "Handheld Devices for Control". Human-Computer Interaction Consortium (HCIC'2003), Winter Park, CO, Feb 5-9, 2003. pdf
29. Jeffrey Nichols, Brad A. Myers, Michael Higgins, Joseph Hughes, Thomas K. Harris, Roni Rosenfeld, Kevin Litwack. "Personal Universal Controllers: Controlling Complex Appliances With GUIs and Speech," Extended Abstract CHI'2003: Human Factors in Computing Systems. (Demonstration Abstract). Fort Lauderdale, Florida, April 5-10, 2003. pp. 624-625. pdf
30. Jacob O. Wobbrock. "The Benefits of Physical Edges in Gesture-Making: Empirical Support for an Edge-Based Unistroke Alphabet." Extended Abstracts of the ACM Conference on Human Factors in Computing Systems (CHI '03). Ft. Lauderdale, FL, April 2003. pp. 942-943. pdf or ACM
31. Jeffrey Nichols and Brad A. Myers. "Studying The Use of Handhelds to Control Smart Appliances". International Workshop on Smart Appliances and Wearable Computing. IWSAWC 2003. In the Proceedings of the 23rd IEEE Conference on Distributed Computing Systems Workshops (ICDCS'03). May 19-22, 2003, Providence, Rhode Island. pp. 274-279. pdf
32. Jacob O. Wobbrock, Brad A. Myers, and Scott E. Hudson. "Exploring Edge-based Input Techniques for Handheld Text Entry". International Workshop on Smart Appliances and Wearable Computing. IWSAWC 2003. In the Proceedings of the 23rd IEEE Conference on Distributed Computing Systems Workshops (ICDCS'03). May 19-22, 2003, Providence, Rhode Island. pp. 280-282. pdf
33. Jeffrey Nichols and Brad A. Myers, "Automatically Generating Interfaces for Multi-Device Environments" Ubicomp 2003 Workshop on Multi-Device Interfaces for Ubiquitous Peripheral Interaction. October 12, 2003. Seattle, WA. html.
34. Jacob O. Wobbrock, Brad A. Myers, and John Kembel. "EdgeWrite: A High-Accuracy Stylus Text Entry Method," CHI Letters: ACM Symposium on User Interface Software and Technology, UIST'03, Nov. 2-5, 2003, Vancouver, British Columbia, Canada. pp. 61-70. pdf
35. Jeffrey Nichols, Brad A. Myers and Kevin Litwack. "Improving Automatic Interface Generation with Smart Templates," IUI'04, Jan. 13-16, 2004, Madeira, Funchal, Portugal. pp. 286-288. pdf
36. Jacob O. Wobbrock, Brad A. Myers, and Htet Htet Aung. "Joystick Text Entry Using Date Stamp, Selection Keyboard, and EdgeWrite." Extended Abstract CHI'2004: Human Factors in Computing Systems. (Poster Abstract). Vienna, Austria, April 24-29, 2004. To Appear. pdf
37. Jacob O. Wobbrock, Brad A. Myers, and Htet Htet Aung. "Writing with a Joystick: A Comparison of Date Stamp, Selection Keyboard, and EdgeWrite," Graphics Interface GI'2004. Canadian Human-Computer Communications Society. May 17-19, 2004, London, Ontario, Canada. pp. 1-8. pdf
38. Jeffrey Nichols, Brad A. Myers, Kevin Litwack, Michael Higgins, Joseph Hughes, Thomas K. Harris. "Describing Appliance User Interfaces Abstractly with XML," in Workshop on Developing User Interfaces with XML: Advances on User Interface Description Languages, 25 May, 2004, Gallipoli, Italy. To appear. pdf
39. Jacob O. Wobbrock, Brad A. Myers, Htet Htet Aung, and Edmund F. LoPresti. "Text Entry from Power Wheelchairs: EdgeWrite for Joysticks and Touchpads." The Sixth International ACM SIGCAPH Conference on Assistive Technologies: ASSETS 2004. October 18-20, 2004. Atlanta, GA. To appear. pdf

    Submitted for Publication

40. Brad A. Myers. "Handheld Devices for Control of PCs and Appliances." pdf
41. Jacob O. Wobbrock and Brad A. Myers. "Exploiting the Input Stream in Character-level Error Analyses for Text Entry Evaluation." pdf

    Unpublished

42. Karen Cross, Adrienne Warmack, and Brad Myers. "Lessons Learned: Using Contextual Inquiry Analysis to Improve PDA Control of Presentations". postscript or pdf.
43. Brad A. Myers, Yu Shan A. Chuang, Marsha Tjandra, Mon-chu Chen, and Chun-Kwok Lee. "Floor Control in a Highly Collaborative Co-Located Task." pdf or Postscript.

Articles about the Pebbles project by others (oldest to newest)

1. Paul Beebe, "Software Marketed to trade under CMU brand name," Pittsburgh Tribune-Review, vol. 112, no. 285, Nov. 14, 2000.
2. Jennie Borodko Stack, "Palm Pilot Connects Girl with Classroom," QUEST, Volume 8, Number 1, February 2001. html
3. Catherine Zandonella, "How to Snarf with the Geeks," New Scientist, London, October 6, 2001, p. 24.
4. John Zyskowski, "Handhands in a new world order," Federal Computer Week, March 18, 2002. html
5. Mike Crissey, "Designers Work on All-in-One Remote," Associated Press, August 27, 2002. html
6. WPXI, Channel 11 news at 11:00pm on October 15, 2002. (Includes a brief shot of Jeff and a PUC interface). mpg
7. Michael Yeomans, "CMU Scientists Improving Computers' People Skills," Pittsburgh Tribune-Review, Tuesday, October 22, 2002. pages B7, B10. html
8. Ann Light, "Pebbles Project connects PDAs up Smartly," UsabilityNews.com, 17 December 2002. html
9. Walter McKenzie, "PDAs in the Classroom," Innovative Teaching Newsletter, Vol. 5, no. 15, January 6, 2003, html
10. Kimberly Patch, "Handhelds Gain Space," Technology Research News, February 26/March 5, 2003, p. 4. html
11. Dan Gillmore, "Dan Gillmor: Designing new handhelds to improve human-computer interaction," SiliconValley.Com; The San Jose Mercury News, April 9, 2003. html

Future Work

If you have ideas or requests, please email them to us at bam at cs.cmu.edu.

Please also consider supporting the Pebbles project by providing a research grant. This will help us continue this research. The companies that are "affiliates" of the Pebbles projects will be given advance notice of future directions, and will be asked for advice on future directions. They will also have access to source code. We would also be interested in gifts of equipment, especially additional PDAs or wireless technologies for connecting them.

Related Projects at CMU

• CMU's Command Post of the Future Project.
• Amulet Project
• Handheld Andrew project

Maintained by: Brad A. Myers
Page design by: Joonhwan Lee