Homework 3: Two Usability Evaluations of Paper Prototypes
Assigned: Monday, June 2, 2014; Due: Monday, June 9, 2014
Using the paper prototype you created in Homework 2, you are now going to run a usability evaluation of that prototype with one or more real people (users), revise the paper prototype to try to fix any problems found, and then run one or more additional people on the revised version.
PART A: The first part of this assignment is to perform a Usability Evaluation of the prototype you created for homework 2. You should put your paper prototype in front of at least one person. This should be a different user than you used for your CI in homework 1, and this person should also be a novice at your appliance. As with the CI, you might want to videotape your sessions. You might also use the camera and microphone built into many laptops to record the session. Another option if you are doing something on a computer (e.g., testing a web page) is to use screen recording and audio from the computer's microphone. Alternatively, almost everyone now has a phone that will record audio (on the iPhone, there are "voice memos" for example) or even video.
- The deliverable for PART A is the usability evaluation template filled out for at least one user. See the template here: http://www.cs.cmu.edu/~bam/uicourse/UsabilityEvalReport_template.doc (see also the partial example in doc format). This includes:
- A description of your user (note: not their name, but instead their experience and demographic properties),
- The script that you use to direct the users on what to do. If it is the same as previous assignments, please mention this, but go ahead and include it again, so we have everything in one place. Note that this must include exactly what you say to the user.
- A transcript or summary of what the user did and said, and what you did and said. If at some points you have to help the users, because they cannot figure out what to do, that must be included in your transcript. It is not necessary to write down every word that the user says, just what is interesting and useful. Be sure to write down all actions on the device, whether correct or wrong. Note: we do not want you to turn in your videotape or audiotape, just the transcript.
- At least five (5) rows describing problems that the user had. NOTE: If the user does not have at least 5 problems that you can report, then you can go back and revise your tasks to make them harder, or to include more parts of the user interface for your device, so you can find more complex tasks, and then do another user. Alternatively, you can do a second user for PART A, and it will be OK if you find at least 5 different things to describe across both users (but you still need to get additional users for PART C below).
- For the "Prototype Screen" column, you can either scan in a copy of the relevant screen and circle the relevant part, or maybe you can number the screens on the paper prototype you turn in (like put numbers on each screen), and then you can refer to that number in that column. Just make sure it is clear to the graders which part of which screen is being referenced for each issue.
- Possible extra credit for doing extra users and finding more than 5 problems. Use one template form per user.
PART B: Next, you will modify your paper prototype from homework 2 to try to fix the problems you observed in your user study, as described in the last column of your Usability Evaluation Feedback Analysis section of your report for PART A.
- The deliverables for PART B is a copy of your new paper prototype. Again, make a scan of this version as you did for homework 1.
- Also turn in a summary of what you changed. This should be a short description. You should reference back to the row number for each problem you fix from PART B, so it is clear which fix goes with which problem.
PART C: Next, you will do another usability evaluation template filled out for at least one different user using your revised prototype (from Part B). This user should also be a novice with your device, and, if possible, might be a different kind of person (different demographics) than your user for PART A. Use another copy of the same template as for part A: http://www.cs.cmu.edu/~bam/uicourse/UsabilityEvalReport_template.doc. Again, this is a different person, also a novice (so you have at least 3 people you have used so far -- at least 1 for HW#1, plus at least 2 more for this homework). You should videotape this session too.
- The deliverable for PART C is another copy of the usability evaluation template filled out. This includes:
- Hopefully, your script has not changed from PART A, but if you need to change the script, then fill in the new script. If the script is the same, then just say "same script as PART A" in this part of the usability-evaluation-report-template document.
- Fill in the remaining parts of the template as for PART A. Since you have a different user for PART C, all of the rest of the information should be different as well.
- For this user, you only need to fill in at least two (2) rows describing problems that the user had. Again, if you can't find 2 problems, then try a harder task, or yet another user.
- Note that this time you do not need to change your interface from Part B to reflect the "Way(s) to rectify" column for the results of the Part C. Instead, you will be making an implementation of your revised design as part of Homework 4.
- Possible extra credit for doing extra users. Use one template form per user.
In summary, you will be turning in two (2) copies of the filled out UsabilityEvalReport_template.doc (one for PART A and one for PART C), a revised paper prototype (PART B), and a short discussion of what you changed in the new version of the prototype. Please turn in this assignment a single pdf or zip file of all your deliverables, and upload the one file to Blackboard. See the submitting instructions.
Back to Homework Page