In 1986, the most impressive computer on the market was Compaq’s 44-pound Deskpro 386, with its 32-bit microprocessor and four kilobytes of memory.

But at Carnegie Mellon University, Allen Newell was already picturing a future where computing power would keep increasing to nearly unimaginable levels. If Moore’s Law, which states that the capabilities of computers will double every two years, continued to prove true, there would (in a few decades) come computers powerful enough to perform nearly any function asked of them.

What would be next for computer science then? So Newell, an artificial-intelligence pioneer who had been one of the leaders of the university’s computer science programs since the 1960s, was focusing on the study of man’s relationship to machine. In 1983, along with CMU alumni Stuart K. Card (TPR’70, DC’78) and Thomas P. Moran (CS’74) of Xerox’s Palo Alto Research Center, Newell co-authored the book The Psychology of Human Computer Interaction, which popularized the phrase, “human-computer interaction.”

Newell, who died in 1992, would be proud and perhaps amazed by what’s grown from the seeds he planted. For the past 20 years, CMU’s Human-Computer Interaction Institute has been the leader in what—as Newell predicted—has become one of the most vital subsets of computer science. And the researchers who founded the HCII, many of them Newell’s colleagues, “have become the all-stars of this field,” says Anind Dey, who was named the Charles M. Geschke Director of the Human-Computer Interaction Institute in June.

More than 200 alumni and past and present faculty members gathered in Pittsburgh Nov. 14, 15 and 16 to mark the HCII’s anniversary with a grand conference (and a party) that included workshops, demonstrations of current projects and tutorials. Sponsors included Visa, Google, Bloomberg, The Walt Disney Co. and Microsoft.

Founded in 1994, HCII was the first academic program to offer a Ph.D. in human-computer interaction. Today, the department’s 40 faculty members and 200 students—comprised of psychologists, cognitive scientists, graphic and industrial designers and computer scientists—together annually contribute about 10 percent of all of the papers submitted to the Association for Computing Machinery’s CHI Conference on Human Factors in Computing.

‘More central’ than computer science itself

“Many departments can point to a few particular research projects that are important, but our strength is in our nimbleness and breadth of the impact of our research in the HCI area in general,” says Brad Myers, CMU professor of human-computer interaction, who helped develop the concept of user-interface toolkits and led CMU’s User Interface Software Group, which created the Garnet and Amulet systems for rapid development of graphical user interfaces.

In educational scope, too, HCII stands out, with more than 500 alumni, many of whom now hold leadership roles at other universities or in corporate research and development departments. HCII today offers undergraduate, masters and doctorate degrees, and organizations such as Apple, NASA and the RAND Corporation regularly sponsor HCII’s research and even outsource problems in interface design to the department.

HCI “started out as a field where we tried to understand how people used computers,” says Dey, a member of the HCII faculty since 2005. Now, he says, “we’re at a place where we’re focusing not just on how to understand how people are using today’s technology, but using that information to guide the design of new technology.”

More than a generation after Newell predicted the importance of human-computer interaction as a research area, the rest of the world appreciates the usefulness of HCI, in part because computers are mediating nearly every aspect of our everyday lives—from shopping to job searching to looking for a spouse.

 “(HCI) is becoming more central to computing than computer science itself,” says Jim Morris (S’63), former dean of the School of Computer Science and currently a professor of computer science and human-computer interaction. “Computer science has done such an incredible job of advancing the engineering and mathematics behind computing, that integration (of technology) into our world is now the bigger problem.”

There was a void of research into human-computer interaction in the 1980s and early 1990s—there were even doubts that HCI was a field worthy of study. But no one thinks that way today.

The void was filled, in large part, by CMU’s Human-Computer Interaction Institute—though the process of getting there wasn’t as neat and orderly as you might expect from an endeavor started and planned by computer scientists.

Roots in wartime studies of ‘pilot error’

The field of HCI traces its roots back to World War II, when the government funded university research into what was then called “human factors and ergonomics” in the design of airplane cockpits in order to decrease pilot error. But although industrial designers were studying the physical layout of controls for tools, vehicles and appliances, few people were applying the same principles to computer interfaces. Computer controls often consisted of little more than cryptically labeled knobs, lights, wires and buttons—the Altair 8800, one of the first popular microcomputers marketed to home users, was programmed from its front panel by using 24 nearly identical metal toggle switches.

In the mid-1970s, Xerox’s PARC research lab created the Alto—the first personal computer to combine use of a mouse with a graphical user interface that organized files on a “desktop.” Newell was a big fan of Alto. So was Apple co-founder Steve Jobs. While Xerox donated Altos to CMU, Massachusetts Institute of Technology and other universities, there were few research papers written about the system. Even Xerox failed to understand Alto’s importance, and stopped development in the early 1980s.

The importance of human-computer interaction just wasn’t obvious, either in industry or academia. “There were conferences on it here and there,” recalls Bonnie John (DC’84,’88), currently an adjunct professor in the Human-Computer Interaction Institute, and then a CMU graduate student, “and there were some places that were good at it—the University of Colorado at Boulder, Georgia Tech—but there really weren’t many places focusing on it, certainly not any when we (in the ’80s) were coming up.”

Newell was CMU’s most passionate evangelist for bringing academic rigor and discipline to the study of HCI. “In the mid ’80s, he called a meeting of the (Department of Computer Science) and said we should do HCI,” recalls Morris, then the director of CMU’s Information Technology Center, which developed Andrew, the university’s revolutionary computer network that combined advanced applications with a GUI desktop environment. “About 40 people showed up and we were all enthusiastic,” Morris says. “As with many things, nothing happened because no one had a stake in it.”

To be sure, there was interest at CMU from people besides Newell. Before Duane Adams left his position as deputy director of the federal Defense Advanced Research Projects Agency, he sent email to all of his soon-to-be-former colleagues, imploring them to step up their research into HCI and software development. After leaving DARPA, Adams joined the CMU faculty, where he continued to push for more study of HCI. Nothing concrete happened until several months after Newell’s untimely death.

In 1989, along with cognitive psychologist Peter Lucas and industrial designer Joseph Ballay, Morris launched MAYA Design, a consulting firm that helped clients solve problems in human-computer interaction. But Morris stepped down from day-to-day leadership of MAYA to return to CMU and head the Computer Science Department. Now, along with Myers and Bonnie John, both then junior faculty members, Morris pressed ahead with plans for an HCI institute, at least partially to fulfill Newell’s wishes. One person they wanted to recruit was Robert Kraut.

Kraut was a Yale-educated psychologist who had faculty stints at Cornell and the University of Pennsylvania. He had a keen interest in group dynamics and how they’re impacted by technology. Kraut was not in academia at the time: He had moved into a job at Bellcore, one of the research and development groups that was spun out of the breakup of the Bell System. “I wanted back into academia,” Kraut says. “For the work I was doing, there were only two schools to consider—MIT and CMU—so I put some feelers out.”

Despite Kraut’s expertise in user interactions with technology, and despite his many supporters in the recently created School of Computer Science, hiring Kraut was seen as a no-go.

“The reaction of several faculty was that ‘Bob Kraut is a brilliant researcher, but he’s not a computer scientist … we’re not qualified to evaluate him,’” Morris says. “We ran into a lot of resistance inside CS.”

“One of the major reasons we wanted to found an institute (of HCI) was that we wanted to hire Bob Kraut,” agrees Myers, a graduate of MIT and the University of Toronto, who joined the CMU faculty as a research computer scientist in 1987.

As created in 1989 (see “Institutional Memories,” The Link, Summer 2014), the School of Computer Science had no departments—the faculty comprised one collective group, drawn together from the former Computer Science Department, the Center for Machine Translation, the Information Technology Center and the Robotics Institute. If there were a separate department within SCS that was dedicated to the social implications of computer use—instead of, for instance, software, algorithms and programming languages—it could have its own criteria for hiring. The idea of an HCI institute became a frequent topic at faculty meetings.

“The School of Computer Science was just getting too big,” Myers says. “It was monolithic. There were formal theory people and AI people, plus a few systems people. It made sense (for SCS) to branch out.”   

There was another reason for creation of an HCI institute: gender balance. The School of Computer Science was almost entirely male, Morris says. “There’s just something about that culture of nerds talking to other nerds that excludes women, even women academics,” he says. He thought an institute with one foot in psychology—a field with better gender balance—would draw more female academics into the SCS.

Morris and other faculty members put the idea into a memo to Raj Reddy, then dean of SCS. Other contributors included John; Myers; CSD research scientist Roger Dannenberg (CS’81,’83); Steve Shafer of the Robotics Institute; and members of CMU’s Psychology Department, including professor John Anderson, research associate Al Corbett, and then-postdoc Ken Koedinger (DC’90).

“Computer science has not studied (HCI) issues adequately to date, but they are critical to the broadening issue of computers in society,” they wrote. “Accordingly, there are new opportunities for funding in this area; and CMU is poised to pursue them. However, we have been lacking an adequate framework within which to crystallize these studies and come to grips with its inherently interdisciplinary nature … we have concluded that the appropriate structure would be a new HCI Institute within the SCS.”

‘We had 10 pigs, and five chickens’

Reddy—founding director of the Robotics Institute—understood the advantages and flexibility that semi-autonomous institutes had within a school. CMU’s Human-Computer Interaction Institute was green-lit in 1994, with Morris as its first director. Kraut got his faculty appointment in 1993—a joint appointment between the School of Computer Science, the Department of Social and Decision Sciences in H&SS (now CMU’s Dietrich College), and GSIA (now the Tepper School of Business).

For its first year, there wasn’t much to the HCII—just a semi-regular meeting of interested faculty. “We had a joke about level of commitment,” Morris says. “We classified people as pigs or chickens: In a ham and egg breakfast, the chicken contributes, but the pig is fully committed. When we started there were about 10 pigs, and five chickens.”

The institute did begin attracting personnel from other disciplines besides computer science. “We had open meetings, so anyone from any department could attend,” Morris says. Sara Kiesler, a member of the faculty of SDS who co-authored one of the first books on the impact of email on workplaces, came over to SCS to join the HCII. Dan Boyarski, a professor of design, also became active in the HCII.

This confluence of disciplines birthed what Morris calls a “three-legged stool” approach: Computer science formed one leg of the stool, along with psychology (to understand how people could and would use computers) and design (to best present and acclimate technology to users).

The first bits of research done under the umbrella of the HCII were promising. NYNEX, the local telephone carrier for much of New England and New York state, was considering the implementation of new workstations for its operators at a cost of $160 million. It planned to test them beforehand, but it wanted a guarantee that the new workstations would shave at least three seconds from the average time it took to place a call. In their book, Newell, Card and Moran had proposed a method called “Goals, Operators, Methods and Selection” as an alternative to costly, time-consuming field tests. GOMS was a kind of specialized model for the way that humans process information. It used measurements of motor skills to predict how well a skilled person could use a new program or device. With GOMS, “you could test machines before they were even built,” John says.

The NYNEX project was the perfect opportunity to prove the effectiveness of the GOMS method. The research project was dubbed “Project Ernestine,” after Lily Tomlin’s telephone operator character on “Laugh-In.”

Using video of operators in action, John and co-researchers were able to dissect every keystroke that went into taking a call and predict how long operators would take on the new stations, crunching the numbers through a GOMS analysis. The analysis found that the new workstations were turkeys. They were actually slower than the existing workstations; it would take the operator 0.63 seconds longer to handle the average call on them. Some baffling design decisions meant that operators needed more time to read the screen and make the actual keystrokes. The researchers then compared the theoretical model to one of NYNEX’s own field tests to see if they got the same results. They did. More importantly, John’s GOMS analysis was able to predict results in six weeks; NYNEX’s field test took six months.

Early research into the Internet surprises the experts

In the meantime, Kraut was conducting some of the earliest research into how people would use the Internet in their leisure time. Back in 1995, only 17 percent of U.S. adults were using the Internet on a regular basis, according to the Pew Research Center, and there was little data on what they were doing online. The National Science Foundation, Apple, Hewlett-Packard, Bell Atlantic and other firms agreed to sponsor “Homenet,” a project to study “residential Internet usage.” (Another sponsor was the U.S. Postal Service, which wanted some actionable insight into its suspicion that email would diminish its load of letters and paper documents.)

Kraut and his co-researchers provided 157 families in Pittsburgh, from a wide range of racial and economic backgrounds, with a computer and net access. They also held three-hour classes in which they explained to the families how to use the Internet.

They learned “the Internet was being used for socializing,” Kraut says. Thirty-five percent of respondents used it to communicate to people from afar, and 24 percent to communicate with those who lived nearby. Twenty-nine percent of users reported using the Internet for schoolwork, and 17 percent for reading news. Their discoveries wouldn’t surprise anyone in a post-Facebook world, but it was a shock, Kraut says, at a time when Vice President Al Gore was describing the Internet as an “information superhighway,” and most people were thinking of it as a place where you went to look up facts.

Yet the socializing didn’t make users feel more connected; in fact, Kraut says, those who reported socializing online felt more isolated. “They were investing social time with people they didn’t really have a connection with,” he says.

All of this research was important, but Reddy had another message for the faculty of the new institute. They needed to provide education as well. Morris is characteristically blunt: “Raj told us … if we were going to make any money, we would need a master’s program.” The HCII began offering its first master’s degree in 1995. The undergraduate program, offering a second major, began in 1997, and the doctoral program began in 2000. Last year, the HCII also added an HCI minor. “One day soon, I think we’d like to consider having an independent major and recruit students directly into our HCII program,” Dey says.

Compared to other universities, HCII masters’ degrees compress about two years of work into a 12-month span. In addition to the core master’s in human-computer interaction that launched the institute, HCII last year added a master’s in education technology and learning science, or METALS, which started with seven students and now has 20. (See “Training teaching’s technologists,” The Link, Summer 2014.)

At the heart of the HCII master’s programs is a unique Capstone Project assigned to teams of students each year. In order to graduate, masters’ students must complete this real-life project sponsored by an outside corporation or agency, such as Google. Those agencies pay the HCII for the work produced by students; more importantly, they provide real feedback on real problems.

“It is the hardest project you have ever completed,” says Jack Beaton (CS’07), who earned his master’s degree in human-computer interaction and went on to work for Nokia and Accenture. His team’s project was creating a handheld prototype of a device for logging technical issues and operational errors for NASA. It’s still the most difficult problem he’s ever worked on, Beaton says: “Everything I have done since has seemed manageable.” In the fall of 2007, the HCII began offering a dual-degree master’s program in partnership with Portugal’s University of Madeira. During 2013, the most recent year for which complete figures are available, HCII granted 66 master’s degrees and two doctorates. The department had 91 master’s students and 39 doctoral students; as Morris had hoped, the ratio of men to women is almost even, according to figures supplied by the university.

Adding an entrepreneurship leg to HCI education

One measure of the importance and effectiveness of HCII’s educational programs is the number of Ph.D. alumni who hold leadership positions at other institutions around the world, Myers says. “We can now trace influences down about five generations,” he says.

HCII education still has its three traditional legs—design, psychology and technology—but a fourth leg is becoming more important, according to Dey. “I think it’s time that we add a business and entrepreneurship component to that,” he says. More and more alumni are landing jobs with boutique firms and startups, Dey says. “In terms of educational programs, our goal isn’t just to educate students, but also to make sure that there are jobs for them to get when they’re finished,” he says. “We need to start thinking about how we bring business acumen and business development into our program, particularly for our students who are going to go off and be practitioners of HCI in the field.”

The notable research by faculty members and grad students continues to add up. John Anderson worked on ACT-R, an ambitious program to reconstruct human cognition in programming language, perhaps the ultimate expression of the HCII’s study of the bridge between man and machine. Myers headed up Pebbles, a multi-project research trove that integrated smart phones and other handheld technology into the use of regular household appliances and personal computers. Jennifer Mankoff led StepGreen, an umbrella project dealing with limiting energy consumption.

Among research areas that continue to resonate, there is the work of professor of human-computer interaction and design Jodi Forlizzi, as well as that of Kraut and Kiesler. Forlizzi’s research into interaction design and social behavior has helped shape technologies ranging from on-screen displays to assistive robots that can help people with physical or mental challenges to live independently. Kraut and Kiesler’s continuing research into the social and interpersonal aspects of computer networks has both predicted applications such as Facebook and Twitter, and helped to shape them. “It might be hard sometimes to trace that lineage, but it’s there,” Myers says.

Kiesler, for instance, has done extensive studies into group behavior and decision-making among people collaborating on large, distributed projects via the Internet. She and Kraut, along with Paul Resnick of the University of Michigan and others, conducted a five-year research project to study the growth of online communities. The resulting book, 2012’s “Building Successful Online Communities: Evidence-Based Social Design,” examined both quantitative and qualitative data to find out why some communities (such as Wikipedia) seem to keep evolving, while others are torn apart by disruptive participants or wither and die from lack of interest.

The success of the Human-Computer Interaction Institute has been emulated in the academic world; John says she knew the HCII model was successful when she started to see other departments, at other universities, set up along the same lines that CMU had used. But as Dey points out, that also means that HCII now faces competition for students and funding that didn’t exist 20 years ago.

“I think there are some unique opportunities that we have at CMU that make it hard for others to compete with,” Dey says. “Our ability to have this really interesting combination of faculty from multiple disciplines all in the same place, that we’re all together, and we all meet often, means that we’re able to come up with novel ideas on a fairly regular basis.”

Those “novel ideas” by HCII faculty members continue to win the respect of their peers around the world. In 2001, ACM’s Special Interest Group on Computer-Human Interaction, or SIGCHI, created the “CHI Academy,” an honorary society for those researchers who have made the most important contributions to the field of human-computer interaction. Of 93 honorees, six are current HCII faculty members—Kiesler, Kraut, Myers, John, Scott Hudson and Forlizzi. Past CMU faculty members in the CHI Academy include Newell (who was elected posthumously) and the late Randy Pausch (CS’88), who was inducted a few months before his death in 2008.

Other members of the CHI Academy with CMU connections include Card and Moran (who were part of the first group of CHI Academy inductees, in 2001) and the University of Washington’s James Landay (CS’93,’96). Card, Moran and Kiesler also are recipients of SIGCHI’s Lifetime Achievement Awards (in 2000, 2004 and 2009, respectively).

Human-computer interaction on-the-go

One change in the work of the HCII has been both shaping and reacting to the shift from desktop computing to mobile computing on smart phones, tablets and other devices. “Our smart phones provide a wealth of data about how we really think and act,” says Jason Hong, an associate professor who joined the HCII in 2004. “From the time I get up to the time I go to sleep, there is this little device recording data about everything I do. Psychologists used to only have access to the attitudes that were observable in the lab and that people would freely give them. That’s no longer the case.”

Chris Harrison (CS’13), a recently recruited assistant professor, says he picked human-computer interaction, and the HCII, because he felt other branches of computer science had fewer new avenues to explore. Computer reliability and processing speed, for instance, isn’t the issue it once was. “The PC I had in high school crashed all the time,” Harrison says, “but my laptop hasn’t failed in months. In that short period of time, computer performance has improved so that it’s no longer an issue for most people.”

If researchers want to advance the field of computer science, Harrison says, they need to be looking beyond increases in computing power and instead increase the range of things people can do with that computing power. For fun, Harrison and a colleague recently crunched some numbers to compare the power of an early 1990s supercomputer with devices that can be purchased at Best Buy. They concluded that if a time-traveler from 2014 went back to 1994 (the year the HCII was founded) with an iPhone 6 in her pocket, she would possess the third-most powerful computer on the planet.

“We’ve figured out how to put these very powerful computers, by historic standards, in tiny devices, but so far we don’t do all that much on them,” Harrison says. “We send emails and texts and take Instagram photos, when they’re capable of so much more.”

Harrison is one of the young faculty members who are “expanding our worldview of human-computer interaction,” Dey says. “We have individuals who are combining human-computer interaction with games and behavior change,” he says. “Those are areas we’ve explored a little bit in the past, but with these new hires, we’re being able to explore them quite a bit more.”

Powered by the Internet and the mobile phone revolution, it seems that we’re all living in the future that Allen Newell predicted. “Thirty years ago, we couldn’t have predicted either of these technologies,” Dey says, adding that 30 years from now, “I expect there will be similar kinds of revolutionary technologies that will occur, that will cause our field to shift.

“It’s a little bit of a strange line to walk on,” he says. “In some ways, we have to be very reactive to what happens in the world, because we study the world and the people in it. In other ways, we’re trying to invent the future.”

Nick Keppler is a Pittsburgh-based freelance writer and a regular contributor to The Link as well as The Village Voice and other publications. Link Editor Jason Togyer (DC’96) contributed to this story.