Homework 1: Evaluate Pointing Devices
Assigned Wednesday, Jan. 16, 2019, due: Wednesday, Feb. 6, 2019 (extended!)
Be sure to see the policies for grading, late turn-in, cheating etc, on the homework policy page.
NOTE: TURN IN THIS ASSIGNMENT IN HARDCOPY IN CLASS on Monday, Feb. 4, 2019. Please print out your entire report on paper, staple it together, and bring it to class. (If you have to miss class for any reason, you can turn in your hardcopy in advance in my office, NSH 3517, or if necessary, upload your assignment to Canvas to demonstrate that you have it done on time.) Be sure your name is on your report!
In this assignment, each student will evaluate two (2) different pointing devices using three (3) different people, and write up a report on the results.
Pointing Devices:
Each student will choose two (2) different pointing devices. We hope that we can get good coverage of all of the kinds of pointing devices that are currently in use, with approximately the same number of people doing each one. If you have an "interesting" way of doing pointing, please use that as one of your devices. The assignment of people to devices will be in this GoogleDoc.
For example, here are some pointing devices to pick from:
- Mouse
- Various "gain ratios" on mouse settings
- Laptop touchpad from various laptops:
- Macintosh
- Thinkpad
- others
- IBM Pointing Stick on Thinkpad laptops
- Finger touch:
- iPhone (of various sizes) - touch with finger
- Android phone (of various sizes) - touch with finger
- iPad (of various sizes) - touch with finger
- Android tablet (of various sizes) - touch with finger
- Windows Surface Laptop
- Large Windows Surface wall-size display
- Touch with Stylus (pointer)
- Smartphone / Tablet - touch with stylus, like the Samsung Galaxy Note
- iPad Pencil
- Wii controller pointer in the air, pointing at a web page on a "smart TV"
- Microsoft Kinect using your hand to point at a web page on a "smart TV"
- Game controller connected to a PC to control the cursor
- Contour's "RollerMouse Red plus"
- Trackball
- etc. -- what other pointing devices can you get access to?
Extra points on this assignment for doing a third or more devices! Also extra points if you are doing an "interesting" device.
Everyone enter the devices you will do here.
Participants
Every student should test three (3) people:
- Yourself -- you should use the two pointing devices and time yourself.
- Someone who is like you - for example, another student, but not anyone in the class.
- Someone who is not like you - for example, someone your parent's age, a child, someone with little or no computer skills, etc.
Extra points on this assignment for testing a fourth or more people!
Preference and Demographic Questionnaire
You should create a simple (paper) questionnaire to give each of your subjects (including yourself!) to be filled out at the end after doing all the tasks. It should ask basic demographic information (see report below), experience with computers and experience with the devices you are using. Then, you should have question(s) about their preferences between the devices they used and why. You need to include a blank copy of the questionnaire with your report so we can see what you asked.
Test Software
The professor and TA have created a version of the standard pointing test software for the web. You will be able to run the software from any web-enabled device that can run a browser that supports Javascript (probably not a watch!). The software will display the targets to tap on and record the times. You will perform the test with one practice trial, followed by 16 real trials of 15 taps each with each device. For your three people, use both orders (have some people use one device first, and the other people use the other device first). (If you do extra devices, then still have everyone do all the devices, with different orders for the different people.)
Be sure that participants only use one (1) finger for touch devices. That is, the participants must move their one finger back and forth between the dots. It is not valid to use two fingers, tapping with different fingers on different dots.
The instructions for the test software are here:
http://www.cs.cmu.edu/~bam/uicourse/05440inter2019/fitts-law-test/fittstestinstructions.html
which includes the link to the web page with the web app.
Report
The deliverable for this homework is a written report. We expect it to be maybe about 4-5 pages (not including figures or the copy of the questionnaire), single-spaced in TimesRoman 12 point font. Print this report and turn it in before class on the due date. Be sure your name is on your report! The report should have the following sections. (Note that you should use this as a template using these section names):
- Description: A detailed description of your two
pointing devices. This should be sufficient to allow the reader to reproduce your test exactly. For example, if doing a mouse, what brand and model number? What mouse settings did you use on the computer? What browser did you use to run the test? If doing a Smartphone, what exact screen resolution and screen size (diagonal)? (For example, you can often find these by searching Google for "screen resolution" -- e.g., Apple says the "Display" on the iPhone Xr is: "6.1-inch (diagonal) Retina display /
1792-by-828-pixel resolution at 326 ppi"). Also include the measured width and height that you entered into the on-line program. It would also be great to include photos of your actual devices in your report.
- Users: A detailed description of your users. (Note: not their names!) Describe all the relevant demographic characteristics, like their experience with computers, their experience with devices you used, whether they are left-handed or right-handed (and if left-handed, did they use their left hands -- many left-handed people still use computers right-handed), etc. This section of the report should include anything of interest from your questionaire.
- Environment: A detailed description of the environment in which you did the test. In particular, were the subjects seated at a desk, standing, at a table? If a phone or tablet, was it on the table or in their hands? (You could even test both if you want and see if there is a difference.) Also include the order of the devices used for each user.
- Results: Then present your results: what is the measured speed and accuracy of your devices for your users? Present the actual measured results and the averages for each device across the users, etc. Which device was fastest? Which was the most accurate? How did the speed and accuracy change with respect to distance and target size? Which did the subjects prefer and why? Was there any effect of the order in which people used the devices (did it seem like there was a learning effect happening)? Did the device do better or worse on horizontal or vertical or diagonal movements?
- Required for PhD students; Optional for others (extra credit): Perform a Fitts evaluation like was done in the Card paper mentioned below, and as explained in lecture 3.
- Discussion: Next in your report, have a discussion section that shows that you have command of the following topics from the class syllabus, with respect to the pointing devices you studied:
- Comparative Performance: How well did your devices perform compared to each other, and compared to other devices in the literature?
- In particular, compare and contrast your results with the results reported in the following two references for other pointing devices:
- Brad A. Myers, Rishi Bhatnagar, Jeffrey Nichols, Choon Hong Peck, Dave Kong, Robert Miller, and A. Chris Long. "Interacting At a Distance: Measuring the Performance of Laser Pointers and Other Devices." Proceedings CHI'2002: Human Factors in Computing Systems. Minneapolis, Minnesota, April 20-25, 2002. pp. 33-40. pdf.
- Card, S.K., English, W.K., & Burr, B.J.
Evaluation of mouse, rate-controlled isometric joystick,
step keys, and text keys for text selection on
a CRT. Ergonomics, 21(8), pp. 601-613. 1978. pdf
- If you did a Fitts' evaluation: What is the index of difficulty of your device? How does it compare to other devices, in the literature and the other device you did?
- Design Issues: The design issues regarding the different pointing devices, especially the trade-offs among them.
- Appropriate Evaluation: How well did the evaluation of the pointing devices work? Were you using the appropriate tests for performance. Were the tests you used the right ones? What did the tests miss (not measure)?
- Future Work: Finally, you can optionally (for extra credit) have section of future work, for example, if you have any ideas for new studies, or improved pointing devices or pointing techniques.
- Appendix: Questionnaire: Include a blank copy of your questionnaire (just like it was given to the participants) in an appendix.