This is an OLD version of the course. Please go to the overview page to find the current version!

Homework 4: Evaluate Designs for Text Entry

For 05-899A/05-499A: Interaction Techniques, Spring, 2014

Assigned Monday, March 3, 2014, due: Monday, March 17, 2014

Be sure to see the policies for grading, late turn-in, cheating etc, on the homework policy page.

NOTE: TURN IN THIS ASSIGNMENT IN HARDCOPY IN CLASS on Monday, March 17, 2014. Please print out your entire report on paper, staple it together, and bring it to class. (If you have to miss class for any reason, you can turn in your hardcopy in advance in my office, NSH 3517, or if necessary, upload your assignment to Blackboard to demonstrate that you have it done on time.) Be sure your name is on your report!

In this assignment, each student will evaluate two (2) different techniques for text entry using two (2) different people, and write up a report on the results. Note that this assignment is quite similar to Homeworks 1 & 3, but we have decreased the number of technologies and kept the number of people you need to do at two, because it will take a little longer per participant.

Text Entry Techniques:

Each student will choose two (2) different text entry techniques. We hope that we can get good coverage of all of the kinds of scrolling techniques that are currently in use, with approximately the same number of people doing each one. If you have an "interesting" way of doing text entry, please use that as one of your choices. The assignment of people to techniques will be in this GoogleDoc.

For example, here are some text entry techniques we thought of that you can pick from:

• Typing on a regular keyboard, either on a laptop or a desktop computer.
• Typing on an "ergonomic" keyboard attached to a desktop computer.
• Typing on an iPhone or Android on-screen keyboard with one hand
• Typing on an iPhone or Android on-screen keyboard with two thumbs
• Using the iPhone or Android on-screen keyboard in landscape mode (instead of portrait mode) -- there might be a difference
• Using a small physical keyboard the is part of an Android or Blackberry phone
• Typing on the on-screen keyboard for a tablet with the tablet sitting on a table or in a holder or a lap (so 2 hands can be used) -- note that both users should use the same pose, whatever you pick.
• Using Swype on Android phones to enter words by swiping across the keyboard
• Using Siri or Google voice to dictate the words.
• etc. -- what other text entry techniques can you get access to?

Extra points on this assignment for doing a third or more techniques!

Everyone enter the techniques you will do here.

Participants

Every student should test two (2) people:

• Yourself -- you should use the two text entry techniques and measure yourself.
• Someone who is not like you - for example, someone your parent's age, a child, someone with little or no computer skills, etc. It is OK to use the same person you used for "someone not like you" for Homework 1 and/or 3.

Extra points on this assignment for testing a third or more people!

Test Software

We were not able to create special software for this homework, but luckily, there are lots of typing tests on-line. The best one I found is: http://typing-speed-test.aoeu.eu/?lang=en, but you can feel free to use a better one if you find it. Be sure to note in your report what test program you use.

That particular software tests how many words can be typed in one (1) minute. Each participant should do the test five (5) times, with breaks in between, and you should note if there is any learning effects. Note that you will need to write down the speed (WPM) and uncorrected errors since the software doesn't save them. Also you should try to notice corrected errors, as discussed in the following required reading:

Jacob O. Wobbrock and Brad A. Myers. 2006. Analyzing the input stream for character- level errors in unconstrained text entry evaluations. ACM Trans. Comput.-Hum. Interact. 13, 4 (December 2006), pp. 458-489. ACM DL or local pdf. Note: You can skip section 4 -- you do not need to read about the algorithms.

Note that the software does not tell you about corrected errros, so you will have to observe the participant's typing and try to see if you can detect any. It is not necessary to get an accurate count of corrected errors, but you should try to get a sense if the text entry technique is accurate or not.

You should be able to run the software from any web-enabled device. The software displays the words to type, and then measures how fast and accurately they are entered. Be sure to tell your participants: "Please enter the phrases as quickly and accurately as possible." For your two people, use different orders. (If you do extra techniques or extra people, then still have everyone do all the techniques, with different orders for the different people.) Note that we have designed this homework to take substantially longer per participant -- about 15 minutes per technique, because we want to measure any learning effects.

Run the software from here: http://typing-speed-test.aoeu.eu/?lang=en
(Please let the Professor know if you find a better test to use!)

Please enter the five times into the class's GoogleDoc form, and also use the results in your own analysis for your report.

Paste your results into this form on GoogleDoc.

Preference and Demographic Questionnaire

You should create a simple (paper) questionnaire to give each of your subjects (including yourself!) to be filled out at the end after doing all the tasks. It should ask basic demographic information (see below), experience with computers and experience with the devices and text entry techniques you are using. Then, you should have question(s) about their preferences and why. You need to include a blank copy of the questionnaire with your report so we can see what you asked.

Report

The deliverable for this homework is a written report. We expect it to be maybe about 2-3 pages (not including figures or the copy of the questionnaire), single-spaced in TimesRoman 12 point font. Print this report and turn it in before class on the due data. Be sure your name is on your report! The report should have the following sections:

• A detailed description of your two text entry techniques. This should be sufficient to allow the reader to reproduce your test exactly. For example, if doing a keyboard on a PC, what is the exact model of computer or keyboard were you using? If a laptop, then what model? If a desktop computer, what kind of keyboard was attached? If doing a Smartphone, what model? It would also be great to include screenshots or photos of your text entry techniques (e.g., a picture of the keyboard).
• A detailed description of your users. (Note: not their names!) Describe all the relevant demographic characteristics, like their experience with computers, their experience with text entry techniques you used, etc.
• A detailed description of the environment in which you did the test. In particular, were the subjects seated at a desk, standing, at a table? If a phone or tablet, was it on the table or in their hands? Did you use the phone in portrait or landscape mode? (You could even test both if you want and see if there is a difference.) What browser did you use to run the test? What test program did you use? Also include the order of the techniques used for each user.
• Then present your results:
  • Present the actual measured results for the wpm and uncorrected error rate for each of your text entry techniques for your users and the averages for each technique across the users, etc.
  • Report on any learning effects. Possibly plot the WPM over the course of the entire study per device. Did people get faster?
  • Which techniques was the most accurate? Were there particular letters that seemed harder to enter than others? You don't need to report the corrected error rate, but what is your impression of how many errors the participant made and corrected?
  • Which did the subjects prefer and why?
  • Was there any effect of the order in which people used the techniques?
• Next in your report, have a discussion section that shows that you have command of the following topics from the class syllabus, with respect to the scrolling techniques you studied:
  • The design issues regarding the different text entry techniques, especially the trade-offs among them. (e.g. Does a text entry system perform better, but take longer to learn? Is one fast, but at the cost of having many errors? Would one text entry technique benefit from some of the advances in another one? etc.)
  • What did your tests show about the performance? Were the tests you used the right ones? What did the tests miss (not measure)?
  • Describe the historical progression of text entry techniques and the factors that impacted their evolution and eventual widespread adoption. In particular, compare and contrast your results with the results reported in the Wobbrock and Myers required reading paper.
• Finally, you can optionally have section of future work, for example, if you have any ideas for new studies, or improved ways of doing text entry.
• Include a blank copy of your questionnaire (just like it was given to the participants) in an appendix.