Homework 1: Evaluate Pointing Devices

Assigned: Thursday, Jan. 20, 2022; Due Thursday, Feb. 3, 2022 at 1:25pm ET

Be sure to see the policies for grading, late turn-in, cheating etc, on the homework policy page.

Overview

In this assignment, each student will evaluate two (2) different pointing devices using three (3) different people, and write up a report on the results. Students will enter the devices they will evaluate into a GoogleDoc, and run the Fitts Law Test using our web app.

Pointing Devices:

Each student will choose two (2) different pointing devices. We hope that we can get good coverage of all of the kinds of pointing devices that are currently in use, with approximately the same number of people doing each one. If you have an "interesting" way of doing pointing, please use that as one of your devices. The assignment of people to devices will be in this GoogleDoc. (Note that you need to be using your CMU Andrew Google account to access this.)

For example, here are some pointing devices to pick from:

• Mouse
  • Various "gain ratios" on mouse settings
• Laptop touchpad from various laptops:
  • Macintosh
  • Thinkpad
  • others
• IBM Pointing Stick on Lenovo Thinkpad laptops
• Finger touch:
  • iPhone (of various sizes) - touch with finger
  • Android phone (of various sizes) - touch with finger
  • iPad (of various sizes) - touch with finger
  • Android tablet (of various sizes) - touch with finger
  • Windows Surface Laptop
  • Large Windows Surface wall-size display
• Touch with Stylus (pointer)
  • Tablet (not a screen), like Wacom or Summagraphics
  • Smartphone / Tablet - touchscreen with stylus, like the Samsung Galaxy Note
  • iPad Pencil
• Trackballs
• Wii controller pointer in the air, pointing at a web page on a "smart TV"
• Microsoft Kinect using your hand to point at a web page on a "smart TV"
• Game controller connected to a PC to control the cursor
• Contour's "RollerMouse Red plus"
• Leap Motion pointing in the air
• etc. -- what other pointing devices can you get access to?

Extra points on this assignment for doing a third or more devices! Also extra points if you are doing an "interesting" device.

Everyone enter the devices you will do here.

Participants

Every student should test three (3) people:

• Yourself -- you should use the two pointing devices and time yourself.
• Someone who is like you - for example, another student, but not anyone in the class.
• Someone who is not like you - for example, someone your parent's age, a child, someone with little or no computer skills, etc.

Preference and Demographic Questionnaire

You should create a simple (paper if in person or Google Forms) questionnaire to give each of your subjects (including yourself!) to be filled out at the end after doing all the tasks. It should ask basic demographic information (see report below), experience with computers and experience with the devices you are using. Then, you should have question(s) about their preferences between the devices they used and why. You need to include a blank copy of the questionnaire with your report so we can see what you asked.

Test Software

The TA for this year (Chaoran Chen) and last time (Asit Parida), along with the professor, have created a version of the standard pointing test software for the web. You will be able to run the software from any web-enabled device that can run a browser that supports Javascript (probably not a watch!). The software will display the targets to tap on and record the times. You will perform the test with one practice trial, followed by 16 real trials of 15 taps each with each device. For your three people, use both orders (have some people use one device first, and the other people use the other device first). (If you do extra devices, then still have everyone do all the devices, with different orders for the different people.)

Be sure that participants only use one (1) finger for touch devices. That is, the participants must move their one finger back and forth between the dots. It is not valid to use two fingers, tapping with different fingers on different dots.

The instructions for the test software are here:
        http://www.cs.cmu.edu/~bam/uicourse/05440inter2022/fitts-law-test/
which includes the link to the web page with the web app.

Report

The deliverable for this homework is a written report. We expect it to be maybe about 4-5 pages (not including figures or the copy of the questionnaire), single-spaced in TimesRoman 12 point font. Create a pdf of this report, and upload it to Canvas before class on the due date.

Be sure your name is on the first page of your report and in the file name! Your pdf file should be named: lastname_firstname_ixt22_hw1.pdf. For example, mine would be: myers_brad_ixt22_hw1.pdf.

The report should have the following sections. (Note that you should use this as a template using these section names):

• Description: A detailed description of your two pointing devices. This should be sufficient to allow the reader to reproduce your test exactly. For example, if doing a mouse, what brand and model number? What mouse settings did you use on the computer? What browser did you use to run the test? If doing a Smartphone, what exact screen resolution and screen size (diagonal)? (For example, you can often find these by searching Google for "screen resolution" -- e.g., Apple says the "Display" on the iPhone 13 pro max is: "6.7-inch (diagonal)" and has "2778-by-1284-pixel resolution at 458 ppi"). Also include the measured width and height that you entered into the on-line program. It would also be great to include photos of your actual devices in your report.
• Users: A detailed description of your users. (Note: not their names!) Describe all the relevant demographic characteristics, like their experience with computers, their experience with devices you used, whether they are left-handed or right-handed (and if left-handed, did they use their left hands -- many left-handed people still use computers right-handed), etc. This section of the report should include anything of interest from your questionaire.
• Environment: A detailed description of the environment in which you did the test. In particular, were the subjects seated at a desk, standing, at a table? If a phone or tablet, was it on the table or in their hands? (You could even test both if you want and see if there is a difference.) Also include the order of the devices used for each user. If your participants are remote, then ask them to show you their setup and describe the environment so you can write these into the report.
• Results: Then present your results: what is the measured speed and accuracy of your devices for your users? Present the actual measured results and the averages for each device across the users, etc. Key questions to address:
  • Which device was fastest?
  • Which was the most accurate?
  • How did the speed and accuracy change with respect to distance and target size?
  • Which did the subjects prefer and why?
  • Was there any effect of the order in which people used the devices (did it seem like there was a learning effect happening)?
  • Did the device do better or worse on horizontal or vertical or diagonal movements?
  • Optional (up to +5 extra credit): Perform a Fitts evaluation like was done in the IxT Textbook, in section 4.14.1, and as explained in lecture 2 (You can also see the Card paper mentioned below).
• Discussion: Next in your report, have a discussion section that shows that you have command of the following topics from the class syllabus, with respect to the pointing devices you studied:
  • Comparative Performance: How well did your devices perform compared to each other, and compared to other devices in the literature?
    • In particular, compare and contrast your results with the results reported for various devices in the IxT Textbook, in section 4.14.1. For extra credit (up to +3), you can also compare to the following two references for other pointing devices:
      • Brad A. Myers, Rishi Bhatnagar, Jeffrey Nichols, Choon Hong Peck, Dave Kong, Robert Miller, and A. Chris Long. "Interacting At a Distance: Measuring the Performance of Laser Pointers and Other Devices." Proceedings CHI'2002: Human Factors in Computing Systems. Minneapolis, Minnesota, April 20-25, 2002. pp. 33-40. pdf.
      • Card, S.K., English, W.K., & Burr, B.J. Evaluation of mouse, rate-controlled isometric joystick, step keys, and text keys for text selection on a CRT. Ergonomics, 21(8), pp. 601-613. 1978. pdf
    • If you did a Fitts' evaluation: What is the index of difficulty of your device? How does it compare to other devices, in the literature and the other device you did?
  • Design Issues: The design issues regarding the different pointing devices, especially the trade-offs among them.
  • Appropriate Evaluation: How well did the evaluation of the pointing devices work? Were you using the appropriate tests for performance. Were the tests you used the right ones? What did the tests miss (not measure)?
• Future Work: Finally, you should have section of future work, which talks about new kinds of studies, or improved pointing devices or pointing techniques or how to measure pointing devices in 2D.
• Appendix: Questionnaire: Include a blank copy of your questionnaire (just like it was given to the participants) in an appendix. This should be included in the same pdf file as the report.

Grading

The various parts of this homework will be calculated as follows, so you know how much each part is worth:

• 9 points: Overall following instructions
• 15 points: Description, Users, Environment sections of report
• 30 points: Results
• 40 points: Discussion
• 3 point: Future Work
• 3 points: Appendix
• Extra Credit (up to these values)
  • +3 points: if test an extra person or people (total, no matter how many extra people)
  • +3 points: if test an extra device or devices (total, no matter how many extra devices)
  • +2 points: if test a really interesting device, like a VR controller, game controller, etc. (total no matter if multiple devices were interesting)
  • +5 points: If perform a Fitts evaluation
  • +3 points: Compare to Myers and/or Card papers