CMU Command Post of The Future:
MultiModal Command and Control
Overview
As part of DARPA's
Command Post of the Future
program, the CMU group worked on components for a Command Post
Information Environment to provide new ways to collaborate
with each other and to interact with supporting information assets and
sources.This project ran from 1998 through 2002, and was credited with
influencing some of the technology currently in use by the military.
Components included:
- Use PDAs to access information and control devices,
- Recognize speech from free microphones,
- Interact in shared 3-D space,
- Recognize gestures from PDAs and hands in the air,
- Provide active links between information products, analyses, sources
-- enabling rapid drill-down, update,
- Tailorable information awareness.
See our article that summarized the CMU part of the work:
- Brad Myers, Robert Malkin, Michael Bett, Alex Waibel, Ben
Bostwick, Robert C. Miller, Jie Yang, Matthias Denecke, Edgar Seemann,
Jie Zhu, Choon Hong Peck, Dave Kong, Jeffrey Nichols, Bill
Scherlis. "Flexi-modal and Multi-Machine User Interfaces". IEEE Fourth International Conference on Multimodal Interfaces,
Pittsburgh, PA. October 14-16, 2002. pp. 343-348.
pdf
- See the
web page of one of our collaborators: MayaViz.
See our longer
overview of the CMU work written in June 1998.
See some images of multi-modal technology in
action. April 2000. (Warning: large PowerPoint file.)
Principal Investigators
Human Computer Interaction Institute
School of Computer Science
Carnegie Mellon University
Constituent Technologies
This list of documents describes some of the research that leads up to
the components CMU will be contributing to the CPOF effort.
- Speech and multimodal input:
This will leverage off of the substantial progress of the
Interactive Systems Laboratory (see the technology descriptions
of the
Multimodal interface and the
Multimodal architecture).
Some papers:
- "From Gaze To Focus Of Attention,"
Rainer Stiefelhagen, Michael Finke, Jie Yang, Alex Waibel
Proceedings of the Workshop on Perceptual User Interfaces (PUI98),
San Francisco.
[compressed postscript]
- "Meeting Browser: Tracking And Summarising Meetings,"
Alex Waibel, Michael Bett, Michael Finke
Proceedings of the DARPA Broadcast News Workshop 1998.
[compressed postscript]
- "Visual Tracking For Multimodal Human Computer Interaction,"
Jie Yang, Rainer Stiefelhagen, Uwe Meier, and Alex Waibel
Proceedings of CHI98, pp. 140-147.
[compressed postscript]
- "Interactive Translation Of Conversational Speech,"
Alex Waibel, Computer, 29(7), July 1996
[compressed postscript]
- "Model-based and
Empirical Evaluation of Multimodal Interactive Error Correction."
Bernhard Suhm, Brad Myers and Alex Waibel,
Proceedings CHI'99: Human Factors in
Computing Systems. Pittsburgh, PA, May 15-20, 1999. To appear.
[postscript].
- Support for Collaboration Information Management:
This will leverage off of the substantial progress of the
CSPACE Project
(see technology description). Some papers:
- "Coordination, Overload, and Team Performance: Effects of Team
Communication Strategies." Susan R Fussell, Robert E Kraut, F Javier Lerch,
William L Scherlis, Matthew M McNally, Jonathan J Cadiz,
Proceedings CSCW'98: ACM Conference
on Computer-Supported Cooperative Work, November 14-18, 1998, Seattle,
WA. pp. 275-284. [MS
Word].
- "An Awareness Tool for Asynchronous Distributed Workgroups"
Jonathan J Cadiz, Robert E Kraut, F Javier Lerch,
Susan R Fussell, Matthew M McNally, William L Scherlis.
CSCW'98 Demonstration, Novermber 14-18, 1998, Seattle, WA.
[MS
Word].
- Use of PDAs:
This will leverage off of the substantial progress of the
Pebbles Project
(see technology description).
A paper:
- "Collaboration Using
Multiple PDAs Connected to a PC,"
Brad A. Myers, Herb Stiel, and Robert Gargiulo.
Proceedings CSCW'98: ACM Conference
on Computer-Supported Cooperative Work, November 14-18, 1998, Seattle,
WA. pp. 285-294.
[postscript] or
[pdf].
- Interaction in Three Dimensions:
This will leverage off of the substantial progress of the
Alice Project
(see technology description). Some papers:
- "Alice: Rapid Prototyping System for Virtual Reality",
Randy Pausch (head), Tommy Burnette, A.C. Capeheart, Matthew
Conway, Dennis Cosgrove, Rob DeLine, Jim Durbin, Rich Gossweiler,
Shuichi Koga, Jeff White, IEEE Computer Graphics and
Applications, May 1995. [html]
[postscript]
- "Designing a Successful HMD-Based Experience," Jeffrey
Pierce, Randy Pausch, Christopher Sturgill, and Kevin
Christiansen [postscript]
- "Two-Handed Manipulation of Voodoo Dolls in Virtual
Environments," Jeffrey Pierce, Brian Stearns, and Randy Pausch,
[postscript]
- "Quantifying Immersion in Virtual Reality," Randy Pausch, Dennis
Proffitt, George Williams, ACM SIGGRAPH 97.
[html]
- "Image Plane Interaction Techniques in 3D Immersive Environments,"
Jeffrey Pierce, Andrew Forsberg, Matthew Conway, Seung Hong, and
Robert Zeleznik, 1997 Symposium on Interactive 3D Graphics.
[html]
or [postscript]
Papers
These are papers produced as part of the CMU CPOF effort.
Main Paper
- Brad Myers, Robert Malkin, Michael Bett, Alex Waibel, Ben
Bostwick, Robert C. Miller, Jie Yang, Matthias Denecke, Edgar Seemann,
Jie Zhu, Choon Hong Peck, Dave Kong, Jeffrey Nichols, Bill
Scherlis. "Flexi-modal and Multi-Machine User Interfaces". IEEE Fourth International Conference on Multimodal Interfaces,
Pittsburgh, PA. October 14-16, 2002. pp. 343-348.
pdf
Others
Funding
This research was performed in part in connection with Contract number
DAAD17-99-C-0061 with the U.S. Army Research Laboratory. The views and
conclusions contained in this document are those of the authors and
should not be interpreted as presenting the official policies or
position, either expressed or implied, of the U.S. Army Research
Laboratory or the U.S. Government unless so designated by other
authorized documents. Citation of manufacturer's or trade names does
not constitute an official endorsement or approval of the use
thereof. The U.S. Government is authorized to reproduce and distribute
reprints for Government purposes notwithstanding any copyright
notation hereon.
Links
CMU CPOF Contact