Interaction technique course logo

Interaction Techniques –
History, Design and Evaluation

CHI’2024 Course #C04 (registration instructions below)

Abstract

Interaction techniques (IxTs) are the low-level reusable building blocks out of which user interfaces are constructed. Examples include physical buttons, menus, scrollbars, touchscreen gestures such as flicking, text entry on computers and touchscreens, input for virtual reality, interactions with conversational agents, etc. UX professionals and researchers will often need to decide which IxTs to use, or even to invent new ones. This course will discuss the history of IxTs, and complexities and appropriate evaluations when designing new ones. The content of this course will be based on Brad Myers’s university courses and book on this topic.

CCS CONCEPTS

Human-centered computing - Human computer interaction (HCI) – Interaction Techniques • Interaction paradigms • Social and professional topics - Professional topics - History of computing

Additional Keywords and Phrases

Interaction Techniques (IxT), Widgets, Controls, History of Computing

Time and Location

This short course will be offered during the ACM SIGCHI Conference, in Honolulu, Hawaiʻi, on Tuesday, May 14, 2024, during the two morning sessions, from 9:00am-12:20pm HST, in room 323B of the Hawaiʻi Convention Center, which holds about 100 people, so register now! See below for registration instructions.

See the full list of accepted courses. This is course #C04.

Benefits

This course provides a study of the many ways to interact with computers and computerized devices. Participants will learn what are interaction techniques (IxTs), and why they are important.

Next, a brief history and design variations for various interaction techniques will be discussed, including the considerations and constraints that drive the designs. Participants will learn about the complexities of interaction techniques.

"IxT" stands for "Interaction Technique" in the same way that "IxD" stands for "Interaction Design" – see, for example, https://www.interaction-design.org/literature/topics/interaction-design.

For example, consider the lowly checkbox in Figure 1. How many states can it be in? An obvious answer is 2: for checked and not checked. But the correct answer is more like 48 different states! As shown in Figure 1b, a checkbox can show it is neither checked nor not checked, for example if text is selected which is partially crossed out. So that is 3 states instead of 2. But it turns out that checkboxes show other things as well. For example, if the mouse is placed over a checkbox, then the outline turns blue to show the mouse is hovering and something will happen if you click (Figure 1c). This blue glow happens no matter the checked status, so that means there are 2 (hover and not) times 3 (check states) = 6 possibilities so far. But if you press down the mouse button over the checkbox and hold it, then the box is filled with blue to show that an interaction is in progress (Figure 1d), and there is also a keyboard focus (Figure 1e) because checkboxes in Windows can be controlled entirely using the keyboard without using the mouse. Finally, checkboxes can be grayed out when they are inactive and cannot be selected. Again, these are mostly independent of the checked status and from each other, making about 2^4*3 = 48 possible states!

(a) The checkbox widget, which is a square with a checkmark, with the label "Strikethrough" next to it. (b) The same checkbox widget as in (a), but this time with a filled-in square instead of a checkmark in it. (c) A vertical list of checkboxes with the first one checked, but the mouse is over the second one, so the outline changes from black to blue. (d) Pressing the mouse button down and holding it causes the box to be filled with light blue instead of white. (e) When the keyboard focus is moved to a checkbox item, the label is indicated with an outline rectangle.

Figure 1: (a) A checkbox that is “checked” in Windows. (b) This checkbox shows a square to indicate it is neither checked nor unchecked, since both strikethrough and not strikethrough text are selected. (c) Checkboxes showing the blue mouse hover, (d) the mouse pressed down, and (e) text focus outline.

Other interaction techniques, like text entry, gestures, menus, scrolling, and so on, all have similar complexities and history, which makes it quite tricky to design new ones and variations without understanding what has been tried before and what the tradeoffs are. And new interaction techniques are invented all the time, and more are needed. For example, pull-to-refresh on smartphones was invented in 2010 and now has almost completely replaced having a refresh icon on apps. There is no standardization of interactions for Virtual and Augmented Reality (VR/AR) systems, and many of these systems are hard to use. And text entry for smartphones and smart watches remains tedious and difficult. This course will explain this context and the appropriate design and evaluations methods for interaction techniques.

Another small benefit is that attendees in this course will get a 30% discount on the purchase of my new book about Interaction Techniques.

Intended Audience(s)

This course will be of interest to multiple audiences:

  • For historians of technology and journalists: This course will provide value to anyone interested in the history of the interaction techniques of computers and other technology. This history has, somewhat surprisingly, not been previously captured.
  • For designers of user interfaces: Virtually all user interfaces (UIs) are made out of interaction techniques, so everyone designing or implementing a user interface of any kind will be using the interaction techniques discussed in this course.
  • For designers of new interaction techniques: New interaction techniques are being invented all the time, and this course motivates where and why new ones are needed. It presents the constraints and tradeoffs relevant to new designs in many different areas, and the industry-standard ways to evaluate the quality of the interaction techniques. New techniques can have a high economic value, and this course also touches on legal issues around interaction techniques such as patents and copyrights.
  • For students and faculty of UI courses: This course provides an overview and introduction to the designs and tradeoffs for UI elements, and appropriate methods for designing and evaluating them. This has been the topic of HCI courses.
  • For users of interaction techniques (which is everyone using technology): Few people are aware of the details about how the interaction techniques on their technology works. In the discussions of individual interaction techniques, the course will provide lots of neat tricks that make interacting with devices faster, less error prone and more efficient.

Prerequisites

This course will provide value to anyone interested in the history of user interfaces of computers and other technology, since all UIs are made up of IxTs, and their history has not been previously captured. It will also be of interest to UX designers since they will always have to design with IxTs and sometimes will need to create new ones.

No prerequisite knowledge is required. This course will mention conventional UX methods, like think-aloud usability studies and Fitts’ law tests, but these will be briefly explained for anyone who is not familiar with them.

Content

This course will introduce the story of the interaction techniques that make up how we use all electronic devices, from regular computers, to “dumb” and smart phones, to consumer electronic devices. We will touch on why these are the way that they are, which will be of interest to anyone who cares about the history of technology. It also explains how some interaction techniques operate now and will in the future, which will be of interest to people who use devices (that is, almost everyone), and people who design new systems and need to know the issues and constraints that previous designers considered. We will also introduce how to evaluate interaction techniques, so participants will know when their devices could be doing things a better way. This will also help designers know if their new designs are better than existing approaches.

This course is based on a university course that Brad Myers has taught four times at Carnegie Mellon University (CMU). See: https://www.cs.cmu.edu/~bam/uicourse/05440inter/. The CHI course will use content and in-class exercises from the university course, shortened down to select the most appropriate parts for the specified time provided. All of the content from the university course is available for free from the course website. The CHI course is also related to Myers’s new book, to be printed by ACM Press Books in 2024, called Pick, Click, Flick! The Story of Interaction Techniques. See www.ixtbook.com for more about the book, including nice reviews from many luminaries in the field.

Detailed Topics

The specific topics I am current planning to cover include:

  • Introduction
    • What are "interaction techniques"
    • Why are they important
    • A quick history of IxTs
    • How are they evaluated, in general (e.g., Fitts' law)
  • Interaction Techniques for Navigation in Applications
    • Scrolling - different tasks that require scrolling, historical and current options, how evaluated
    • Other navigation, like in maps or Augmented Reality / Virtual Reality / Extended Reality (AR/VR/XR)
    • Advanced topics: cancel and undo in navigation
  • Interaction Techniques for Text Entry in Applications
    • Text entry on desktops, smartphones, tablets, watches, smart speakers, etc.
    • Autocomplete, autocorrect, autofill, manual corrections, and undo
    • Dictation (speaking) for text entry
    • Measuring speed and accuracy for text entry
    • Advanced topics: text entry in the future
  • Future Interaction Techiques

If possible, I will poll the people registered for this course before the conference to see which of the many possible topics they are most interested in, and adjust the above list accordingly. See the list of topics in my class, or the list of chapters of my new book for a list of potential topics.

Practical Work

If the Internet in the classroom is adequate, the course will include exercises using whatever devices people have brought with them (such as a laptop, tablet and/or smartphone) to perform some evaluations on, based on the homeworks in my CMU course, which use web-based evaluations. For example, we may do a Fitts’ law test of pointing on the various devices. We can also use our custom scrolling test to compare different ways to move content, and off-the-shelf typing tests for text entry.

Instructor Background

Brad A. Myers has been designing, implementing, documenting and teaching interaction techniques for over 40 years. He is the Charles M. Geschke (SCS 1973) Director of the Human-Computer Interaction Institute (HCII) and Professor in the School of Computer Science at Carnegie Mellon University, with an affiliated faculty appointment in the Software and Societal Systems Department (S3D). He was chosen to receive the ACM SIGCHI Lifetime Achievement Award in Research in 2017, for outstanding fundamental and influential research contributions to the study of human-computer interaction, and was awarded the 2022 Alan J. Perlis Award for Imagination in Computer Science "for pioneering human-centered methods to democratize programming", from the School of Computer Science, Carnegie Mellon University. He is an IEEE Fellow, ACM Fellow, member of the CHI Academy, and winner of 18 Best Paper type awards and 6 Most Influential Paper Awards. He is the author or editor of over 550 publications, including the book "Pick, Click, Flick! The Story of Interaction Techniques", and he has been on the editorial board of six journals. He has been a consultant on user interface design and implementation to over 90 companies, and regularly teaches courses on user interface design and software. Myers received a PhD in computer science at the University of Toronto where he developed the Peridot user interface tool. He received the MS and BSc degrees from the Massachusetts Institute of Technology during which time he was a research intern at Xerox PARC. From 1980 until 1983, he worked at PERQ Systems Corporation. His research interests include user interfaces, programming environments, programming language design, end-user software engineering (EUSE), API usability, developer experience (DevX or DX), interaction techniques, programming by example, mobile computing, and visual programming. He belongs to ACM, SIGCHI, IEEE, and the IEEE Computer Society.

Resources

The slides for this short course are available as a pdf.

The scrolling test we used during the course is available as https://tinyurl.com/c04scrolltest2024 which links here. To set your own scrolling parameters, start here: https://www.cs.cmu.edu/~bam/ixtbook/scrolling/.

All of the course materials from the university course from which this material was adapted are available: https://www.cs.cmu.edu/~bam/uicourse/05440inter/.

The book related to this course, Pick, Click, Flick! The Story of Interaction Techniques, from ACM Press Books, will be of interest to participants in this course. See www.ixtbook.com. That book has more than 570 citations, which will be helpful for people looking for further information. Attendees in this course will get a 30% discount on the purchase of the book.

Accessibility

Participants with physical disabilities would add significant value to the course, since they can provide their perspective on how interaction techniques can be best adapted to their abilities. It would also be interesting to have their participation in the in-class exercises to show how their adaptations for pointing, scrolling, and other interaction techniques work. Participants with low or no vision can be accommodated by textual descriptions of the images in the slides, since all the images in the book have alt-text. We would be open to adaptations to make the course more accessible for others as well.

Acknowledgments

The university course has been improved over the years by input from the teaching assistants and participants, and I thank them for their input. The book contains a large number of acknowledgements for all the people over the last 9 years who have helped with the book’s content, which will be reflected in the course materials. For help with putting together this course, I thank Jonathan Grudin and Jasmin Niess, and also Sean Pidgeon and Brent Beckley from ACM Press.

How to Register

Course Participants – Registration Instructions:

If you are not yet registered for CHI, CLICK THIS LINK: https://cvent.me/ZnP1Lr

  1. Scroll to the bottom of the page and select the appropriate category button (Member or Non-Member) to begin your registration. 
  2. Register - fill in all the required information needed for registration to attend CHI 2024.
  3. Proceed to the Optional Items page – select the course (#C04) from the list and click next at the bottom of the page.
  4. Continue through registration to complete by making a payment and clicking “Submit”. A confirmation email is automatically generated.

 If you are adding a course to your existing registration, CLICK THIS LINK: https://cvent.me/ZnP1Lr

  1. Scroll towards the bottom and click on the “Already Registered” option. Do not begin a new registration if you are already registered for CHI 2024.
  2. Enter your email address and confirmation code – found in your confirmation email.
  3. Click “Login” to proceed, then select “Modify Registration” towards the bottom on the next page.
  4. Proceed through your registration (“Next” button until the Optional Items page) to add the desired course.
  5. Select this course (#C04) from the list and click “Next” at the bottom.
  6. Follow through to complete the addition by making a payment and clicking “Submit”.

scroll bars picture