Homework 4: Evaluate Designs for Text Entry

Assigned: Tuesday, Mar 1, 2022; Due: Tuesday, Mar. 22, 2022 at 1:25pm ET

Overview

Text Entry Techniques:

Each student will choose three (3) different text entry techniques. We hope that we can get good coverage of all of the kinds of text entry techniques that are currently in use, with approximately the same number of people doing each one. If you have an "interesting" way of doing text entry, please use that as one of your choices. The assignment of people to techniques will be in this GoogleDoc.

For example, here are some text entry techniques we thought of that you can pick from:

• Typing on a regular keyboard, either on a laptop or a desktop computer.
• Typing on an "ergonomic" keyboard attached to a desktop computer (such as this one).
• Typing on an iPhone or Android on-screen keyboard with one hand with auto-correct on like usual (be sure to note the "pose" or position of holding it in your report!)
• Typing on an iPhone or Android on-screen keyboard with two thumbs with auto-correct on like usual.
• Typing on those phones or tablets with autocorrect turned off. (For example, on the iPhone, you can turn it off from: Settings>General>Keyboard>Auto-correction Toggle switch and Predictive Toggle switch both to be Off. For Android: Tap on any text field, such as a search box, e-mail body, or a browser address bar to summon the keyboard, then tap the gear in the bottom left of the keyboard, beside the Sym button, then tap Predictive text to turn it off and on.)
• Using the iPhone or Android on-screen keyboard in landscape mode (instead of portrait mode) -- there might be a difference. Be sure to indicate whether autocorrect was on or off.
• Using the built-in Swype on Android phones or iPhone ("slide to type") to enter words by swiping across the keyboard
• Using a small physical keyboard that is part of an Android or Blackberry phone (examples: HTC Touch Pro2, or Droid 3).
• Typing on the on-screen keyboard for a tablet with the tablet sitting on a table or in a holder or a lap (so 2 hands can be used) -- note that all users should use the same pose, whatever you pick (and again, be sure to note the pose in your report).
• Using an attached physical keyboard on a tablet, such as the small keyboards for a Microsoft Surface, or the Logitech keyboard for an iPad (e.g., one of these). It would be best if these keyboards are more different from conventional laptop keyboards.
• Using Siri or Google voice to dictate the words.
• etc. -- what other text entry techniques can you get access to?

Extra points on this assignment for doing a fourth or more techniques! Also, extra credit (up to +2) if you test a really interesting text entry technique, like entering text in VR/AR, on a game console using a game controller, on a consumer electronics device (like a smart TV) using a remote, etc.

Everyone enter the techniques you will do here.

Participants

Every student should test three (3) people (it is OK to use the same people you used for previous homeworks):

• Yourself—you should use the three scrolling techniques yourself..
• Someone who is like you - for example, another student, but not anyone in the class.
• Someone who is not like you—for example, someone your parent’s age, a child, someone with little or no computer skills, etc. (Note that a person who uses a PC if you use a Macintosh is not sufficiently different!) Be sure to note in your report which person you are counting is not-like-you, and why.

Extra points (up to +3) on this assignment for testing a fourth or more people!

Test Software

We were not able to create special software for this homework, but luckily, there are lots of typing tests on-line. The best one I found is: https://10fastfingers.com/typing-test/english, because it reports both uncorrected errors ("Wrong words") and "Keystrokes" so one can get an estimate of backspaces, and it works on both a desktop and a phone. What I don't like is that you cannot back up to fix previous words, and it uses random words, rather than real phrases. Also, I would have preferred a test that used Wobbrock's analyses, but couldn't find one. Let me know if you find a better typing test than the one above!

Jacob O. Wobbrock and Brad A. Myers. 2006. Analyzing the input stream for character- level errors in unconstrained text entry evaluations. ACM Trans. Comput.-Hum. Interact. 13, 4 (December 2006), pp. 458-489. ACM DL or local pdf. Note: You can skip section 4 -- you do not need to read about the algorithms.

Use the default 1:00 minute test: http://10fastfingers.com/typing-test/english. Each participant should do the test three (3) times, with breaks in between, and you should note if there is any learning effects. Note that you will need to write down the appropriate numbers since the software doesn't save them. You will want to capture for each run the following:

• Words per minute (WPM) - which is "gross WPM", shown as the big green number, not counting the errors.
• Error keystrokes - which is the red number in parenthesis after Keystrokes - this will approximate the corrected errors
• Wrong Words - which are the uncorrected errors.

You should be able to run the software from any web-enabled device. The software displays the words to type, and then measures how fast and accurately they are entered. Be sure to tell your participants: "Please enter the phrases as quickly and accurately as possible." For your three people, use different orders. (If you do extra techniques or extra people, then still have everyone do all the techniques, with different orders for the different people.) Note that we have designed this homework to take somewhat longer per participant -- about 15 minutes per technique, because we want to measure any learning effects.

Run the software from here: http://10fastfingers.com/typing-test/english
(Please let the Professor know if you find a better test to use!)

Please enter the three times into the class's GoogleDoc form, and also use the results in your own analysis for your report.

Paste your results into this form on GoogleDoc: https://forms.gle/XBDCw2sSXNdFEboC8.

Preference and Demographic Questionnaire

You should create a simple questionnaire to give each of your subjects (including yourself!) to be filled out at the end after doing all the tasks. It should ask basic demographic information (see below), experience with computers and experience with the devices and text entry techniques you are using. Then, you should have question(s) about their preferences and why. You need to include a blank copy of the questionnaire with your report so we can see what you asked.

Report

The deliverable for this homework is a written report. We expect it to be maybe about 4-5 pages (not including figures or the copy of the questionnaire), single-spaced in TimesRoman 12 point font. Create a pdf of this report, and upload it to Canvas before class on the due date.

Be sure your name is on the first page of your report and in the file name! Your pdf file should be named: lastname_firstname_ixt22_hw4.pdf. For example, mine would be: myers_brad_ixt22_hw4.pdf.

The report should have the following sections. (Note that you should use this as a template using these section names). Please put your information into the appropriate section as described below, so it is easier to grade.

• Description: A detailed description of your three text entry techniques. This should be sufficient to allow the reader to reproduce your test exactly. For example, if doing a keyboard on a PC, what is the exact model of computer or keyboard were you using? If a laptop, then what model? If a desktop computer, what kind of keyboard was attached? If doing a Smartphone, what model? It would also be great to include screenshots or photos of your text entry techniques (e.g., a picture of the keyboard).
• Users: A detailed description of your users. (Note: not their names!) Describe all the relevant demographic characteristics, like their experience with computers, their experience with text entry techniques you used, etc.
• Environment: A detailed description of the environment in which you did the test. In particular, were the subjects seated at a desk, standing, at a table? If a phone or tablet, was it on the table or in their hands? Did you use the phone in portrait or landscape mode? (You could even test both if you want and see if there is a difference.) What browser did you use to run the test? Also include the order of the techniques used for each user. You can include anything else of interest from your questionnaire and your observations about each user. If your participants are remote, then ask them to show you their setup and describe the environment so you can write these into the report
• Results: Then present your results:
  • Present the actual measured results for the wpm and uncorrected error numbers for each of your text entry techniques for your users and the averages for each technique across the users, etc.
  • Discuss the error keystrokes: did people make a lot of errors? Was the error correction method (usually the backspace key) itself hard to enter (did people make mistakes backspacing - note this is not measured by the tool so you will have to observe it or ask the user)?
  • Which techniques was the most accurate? Were there particular letters that seemed harder to enter than others? You don't need to report the corrected error rate, but what is your impression of how many errors the participant made and corrected?
  • Report on any learning effects. Possibly plot the WPM over the course of the entire study per device. Did people get faster?
  • Which text entry method did the subjects prefer and why?
  • Was there any effect of the order in which people used the techniques?
• Discussion: Next in your report, have a discussion section that shows that you have command of the following topics from the class syllabus, with respect to the scrolling techniques you studied:
  • Comparative Performance: How well did your devices perform compared to each other, and compared to other devices in the literature? In particular, compare and contrast your results with the relevant rows of Table 8-1 of the textbook.
  • Design Issues: The design issues regarding the different text entry techniques, especially the trade-offs among them. (e.g. Does a text entry system perform better, but take longer to learn? Is one fast, but at the cost of having many errors? Would one text entry technique benefit from some of the advances in another one? etc.)
  • Appropriate Evaluation: How well did the evaluation of the text entry methods work? Were you using the appropriate tests for performance? Were the tests you used the right ones? What did the tests miss (not measure)?
• Future Work: Finally, you should have section of future work, for example, if you have any ideas for new studies, or improved ways of doing text entry.
• Appendix: Questionnaire: Include a blank copy of your questionnaire (just like it was given to the participants) in an appendix.

Grading

The various parts of this homework will be calculated as follows, so you know how much each part is worth:

• 9 points: Overall following instructions
• 15 points: Description, Users, Environment sections of report
• 50 points: Results
• 20 points: Discussion
• 3 points: Future Work
• 3 points: Appendix
• Extra Credit (up to these values)
• +3 points: if test an extra person or people (total, no matter how many extra people)
• +3 points: if test an extra text entry technique (total, no matter how many extra devices).
  
• +2 points: if test a really interesting device, interesting text entry technique, like entering text in VR/AR, on a game console using a game controller, on a consumer electronics device (like a smart TV) using a remote. (Total no matter if multiple techniques were interesting).