Homework 1: Contextual Inquiry and Analysis

Assigned: Monday, May 18, 2015; Due: Monday, May 25, 2015, at 10:30am EDT

(See how to submit homeworks -- in summary, you will submit on blackboard. See also the grading policies.)

The device choices are on the GoogleDocs page (see Blackboard for a link to it)

Normally, for contextual inquiry, you would just go and watch users performing their tasks in their normal course of business or home life. However, that is not feasible for the devices we selected. Therefore, we have to use as close a substitute as possible:

PART A:

The first part of this assignment is to describe one or more representative tasks that users would perform with your device, that will exercise the parts of the device you want to create. You should come up with enough tasks that an expert doing all of them will take about 5 minutes to perform, and a novice would take about 15 minutes (so they need to be somewhat complex). You must describe the tasks by the function and results and not by the steps to perform them. Therefore, the task descriptions should work no matter what the user interface looks like. For example, you might say "the clock should end up displaying the current time", rather than "push the HR and MIN buttons to set the time". You should also design a preliminary questionaire that asks about why, when and how the user does these kinds of tasks now, and any influenceson their decisions. You can ask these questions before, during or after the actual test activities.

• The deliverables for PART A are the description of your device (what kind of device and what parts of the interface to it), and
• The script of exactly what you will say to instruct the users on which tasks they should achieve (remember, the instructions should tell them what to do, not how). This includes the questions you will ask initially to find out the context of why, when and how the user does these kinds of tasks now. Note that this script will have two parts - the preliminary interview questions, and then the specific instructions to perform the tasks.

PART B:

Next, you will perform a Contextual Inquiry on one (1) user trying to perform your task(s) on an existing real device. Ideally, you would test multiple people with a variety of skill levels, but since we do not have time for that, you should pick one user who is not experienced with the device. Preferably, this should be a person who is not at all familiar with this type of device, and will have a lot of trouble using it. If you cannot find anyone who has trouble with the tasks, then you need to (1) find a user who is less skilled, (2) switch devices to something more complicated, and/or (3) invent harder tasks to perform and revise PART A. In general, elderly people tend to be less skilled with modern devices, so finding someone over 50 with little technical experience might be appropriate. It would not be surprising if your inexperienced user takes a long time to perform your tasks. If it looks like it will take more than an hour, then you probably want to provide some help.

You might want to videotape your session. Probably, most people can use the video recording on their phone, laptop or tablet. Another option if you are doing something on a computer (e.g., testing a web page) is to use screen recording and audio from the computer's microphone. Alternatively, almost everyone now has a phone that will record at least audio (on the iPhone, there are "voice memos" for example).

In performing the Contextual Inquiry, you should read your script from PART A, and record all the user's actions, preferably on video or audio.

• The deliverables for PART B will be a brief description of the user (note: not their name, but instead their experience and demographic properties), and
• A transcript of what the user did and said, and what you did and said. If at some points you have to help the users, because they cannot figure out what to do, that must be included in your transcript. It is not necessary to write down every word that the user says, just what is interesting and useful. Be sure to write down all actions on the device, whether correct or wrong. Note: we do not want you to turn in your videotape or audiotape, just the written transcript. The transcript should have line numbers or time codes so you can refer to particular points in the transcript in part C below. (Note that Microsoft Word can do line numbering for you, which makes this pretty easy.)

PART C:

Now construct Contextual Analysis diagrams from the transcript. This must include an artifact model, which might be an annotated photograph or photographs of the actual device (or just screenshots if it is a computerized application) showing the various buttons and screens that the user used. Annotate this with any breakdowns caused by the interface itself (e.g., if the user could not understand a label). You must also make a flow model showing the flow of information between the user and various parts of the interface. Include any breakdowns relevant to the information flow. Finally, create a social model (also called a "cultural model") showing any influences on why and how the user chose to use the device. Another kind of model is a physical model, which you are welcome to make if it seems useful. (We are not asking you to create a sequence model.) Each label on each model should reference the appropriate point in the transcript using a line number or timecode from your transcript for where each element on the models is found. See the lecture slides and textbook for examples.

• The deliverables for PART C are the three (or four) models, with their annotations. You can draw the models using any drawing tool you like, such as Visio, Illustrator or even PowerPoint. Or, if you are very neat, you can even just draw them by hand on paper and then scan them into the computer (however, we will deduct points if it is not legible).
• Finally, make a list of "do's" and "don't's" for the user interface of your device, that you learned as a result of this study, that will guide your alternative design for the device. All of the recommendations for "don'ts" should clearly refer back to breakdowns you observed. All the "do's" should refer back to things the user did successfully that you want to retain in your design.

Create an electronic version of your homework, preferably as a single PDF file, and upload it to Blackboard.

Homework 2: Paper Prototype of an Interface

Assigned: Monday, May 25, 2015; Due: Monday, June 1, 2015, at 10:30am EDT

Using the information that you collected in Homework 1 for what is good and bad about the real appliance's user interface for the tasks you designed, now you are going to make Sketches and a Paper Prototype of a (hopefully) better design in homework 2. Following the redesign, in the next homework (homework 3), you will run a usability evaluation of the prototype and revise your design based on the feedback you get.

Turn in all parts of this homework on blackboard.

PART A:

Sketch at least three different user interfaces for the functions you are targeting for your device (three different design directions). The three interfaces should be dramatically different; for example, with different organizations of what functions are on each screen, using different widgets for various functions, and using a different layout of the widgets on the screens. You might also vary which buttons are physical buttons and which are on-screen buttons. You should sketch the main screen or screens -- not necessarily the whole interface, but enough that we can see how it works. Note, the goal is to have at least 3 good interfaces that are different in interesting ways -- you shouldn't just create a good interface and then two more bad versions of it. Be creative! By "sketch," we mean informally draw the general idea, and label a few of the widgets so we can tell your intent. It is not necessary for the sketches to have every function or every screen, but there should be enough to show off your idea. Sketches should be done on paper with pens and/or pencils (not using a computer). The sketch for each design will typically include a few screens, which you can put on multiple pieces of paper. Please hand-print neatly for the textual labels and any annotations on the sketches. Next, you will select one of the designs for Parts B and C. (Alternatively, if you decide you want to use a collection of ideas from your three designs, then please make a fourth sketch which shows your final idea, and discuss why you used that one.)

• The first deliverable for PART A is a scan of your 3 (or more) sketches. Be sure to label the different versions so you can differentiate them in your discussion. Each sketch may be one or more pages. If you draw your sketch in color, please use color scanning. If you cannot find a way to scan in your design, please let us know -- there is a color scanner in the mail room near the Professor's office.
• The second deliverable is a discussion (e.g., a paragraph) talking about the differences between the versions. This should be done on a computer, for example in a word processor document.
• The third deliverable is a discussion (e.g., a sentence or two) about which design you picked for Parts B and C and why you selected that one. This should be put into the same document.
• The fourth deliverable is a discussion (e.g., a paragraph) talking about what you learned from the contextual inquiry of homework 1, and how that is reflected in your design. That is, what in your design is a result of what you learn? This is called "design rationale". This should also be put into the document.

PART B:

You should now decide what are the assumptions for the hardware on which your new design will be implemented. If you are doing a webpage or regular computer application, this might be limited to what screen size you are assuming. If you are doing a mobile phone, then describe the screen size and resolution you are assuming (note that Android phones, for example, can come with a variety of screen sizes). If you are doing a physical device (like a camera), you should describe assumptions such as whether the interface will be on a touch screen panel on your device (so you will need big, finger-sized buttons), or the interface will only use physical buttons around a display-only screen (not a touch screen).

Note that even if you assume you will have a touchscreen on your design, you will still be programming and doing the usability evaluation of your interface (in Homework 4) using a mouse on a regular computer. Conversely, if you assume your real user interface will be operated with a mouse, when you do the usability evaluation of your paper prototype (in Homework 3), you will tell the users to pretend their finger is a mouse. You should also write down any other assumptions about your hardware, like whether the screen is color or black and white, whether it is a touch panel or not, whether there are any additional hardware (physical) buttons, etc. (You will decide on what specific buttons and kinds of buttons later.)

Next, you should write down the assumptions about your users. What level of experience do you expect? How familiar with computers will they be? What ages? Are there any other factors about the users that will influence your design or evaluation?

• The deliverables for PART B are descriptions of the assumptions about (a) your hardware and (b) your users, put into the same document.

PART C:

Create a complete paper prototype of the version of your device that you selected in Part A. You can draw your paper prototype on paper or using simple drawing tools like Visio, Illustrator or even PowerPoint. (It appears that CMU has a free download of Visio, or you can download a free 60-day trial of Visio from Microsoft or a free 30-day trial of Illustrator from Adobe.) Unlike the sketches of Part A, your paper prototype must be complete, which means it should have at least a place-holder for every part of the entire user interface of the appliance. For example, if you were doing a microwave oven, there should be a square on the front panel for every button that would be on the front panel of the final implementation. For all the tasks you will ask the user to do (which should include at least those you did in Homework 1), you should have paper prototypes for all the screens, menus, etc. that the user would see. For parts of the interface that you do not intend to test, you can just make place holders (for example, screens with just titles), or buttons without labels. Note that the level of complexity we are expecting is as described in Homework 0: It should have at least 30 different controls that do different things; for web pages, it should have at least 10 pages, and for desktop applications, it should have at least 5 different screens or dialog boxes.

• The first deliverable for PART C is a pdf or scan of your new paper prototype. If you use color in your prototype, then you should scan in color. If you have little pieces of paper to represent pop-up menus and so forth, be sure to include each of those in your scan, and label them so we can tell what they are. If you use a computer tool to create your prototype, then create a pdf file of all the elements (pages) of your interface. (It is fine if this is a lot of pages, since we won't be printing it; but it is also OK to combine multiple elements on one page -- for example, to put multiple pop-up menus on the same page. Just make sure it is clear what each element is.)
• The second deliverable for PART C is a short description of what tasks you will have users do with the device.
• The third deliverable is to write down the specific sequence of actions that the user should take with your prototype to achieve those tasks.
• The fourth deliverable for PART C is the actual instructions you will give users to tell them to do those tasks. We want to see what you are planning to say to the users in your usability evaluation. This should go into the document. Note that it is OK if the second, third and fourth deliverables for PART C are the same as they were for Homework #1 -- but you should still repeat them here so we can have everything in one place.

Make a pdf or zip file of all your deliverables, and upload the one file to Blackboard.

Homework 3: Two Usability Evaluations of Paper Prototypes

Assigned: Monday, June 1, 2015; Due: Monday, June 8, 2015, at 10:30am EDT

Using the paper prototype you created in Homework 2, you are now going to run a usability evaluation of that prototype with one or more real people (users), revise the paper prototype to try to fix any problems found, and then run one or more additional people on the revised version. Note that for this assignment, you have to actually use paper, even if you made your prototype with Balsamiq.

Part A:

The first part of this assignment is to perform a Usability Evaluation of the prototype you created for homework 2. You should print out your prototype, and then put your paper prototype in front of at least one person. This should be a different user than you used for your CI in homework 1, and this person should also be a novice at your appliance. As with the CI, you might want to videotape your sessions. You might also use the camera and microphone built into many laptops to record the session. Alternatively, almost everyone now has a phone that will record audio (on the iPhone, there are "voice memos" for example) or even video.

The deliverable for PART A is the usability evaluation template filled out for at least one user. See the template here: http://www.cs.cmu.edu/~bam/uicourse/UsabilityEvalReport_template.docx (see also the partial example in doc format). This includes:

• A description of your user (note: not their name, but instead their experience and demographic properties),
• The script that you use to direct the users on what to do. If it is the same as previous assignments, please mention this, but go ahead and include it again, so we have everything in one place. Note that this must include exactly what you plan to say to the user.
• A transcript or summary of what the user did and said, and what you did and said. If at some points you have to help the users, because they cannot figure out what to do, that must be included in your transcript. It is not necessary to write down every word that the user says, just what is interesting and useful. Be sure to write down all actions on the device, whether correct or wrong. Note: we do not want you to turn in your videotape or audiotape, just the transcript.
• At least five (5) rows describing problems that the user had. NOTE: If the user does not have at least 5 problems that you can report, then you can go back and revise your tasks to make them harder, or to include more parts of the user interface for your device, so you can find more complex tasks, and then do another user. Alternatively, you can do a second user for PART A, and it will be OK if you find at least 5 different things to describe across both users (but you still need to get additional users for PART C below).
• For the "Prototype Screen" column, you can either scan in a copy of the relevant screen and circle the relevant part, or maybe you can number the screens on the paper prototype you turn in (like put numbers on each screen), and then you can refer to that number in that column. Just make sure it is clear to the graders which part of which screen is being referenced for each issue.
• Possible extra credit for doing extra users and finding more than 5 problems. Use one template form per user.

Part B:

Next, you will modify your paper prototype from homework 2 to try to fix the problems you observed in your user study, as described in the last column of your Usability Evaluation Feedback Analysis section of your report for PART A.

• The deliverables for PART B is a copy of your new paper prototype. Again, make a scan of this version as you did for homework 1.
• Also turn in a summary of what you changed. This should be a short description. You should reference back to the row number for each problem you fix from PART B, so it is clear which fix goes with which problem.

Part C:

Next, you will do another usability evaluation template filled out for at least one different user using your revised prototype (from Part B). This user should also be a novice with your device, and, if possible, might be a different kind of person (different demographics) than your user for PART A. Use another copy of the same template as for part A: http://www.cs.cmu.edu/~bam/uicourse/UsabilityEvalReport_template.docx. Again, this is a different person, also a novice (so you have at least 3 people you have used so far -- at least 1 for HW#1, plus at least 2 more for this homework). You should videotape this session too.

The deliverable for PART C is another copy of the usability evaluation template filled out. This includes:

• Hopefully, your script has not changed from PART A, but if you need to change the script, then fill in the new script. If the script is the same, then just say "same script as PART A" in this part of the usability-evaluation-report-template document.
• Fill in the remaining parts of the template as for PART A. Since you have a different user for PART C, all of the rest of the information should be different as well.
• For this user, you only need to fill in at least two (2) rows describing problems that the user had. Again, if you can't find 2 problems, then try a harder task, or yet another user.
• Note that this time you do not need to change your interface from Part B to reflect the "Way(s) to rectify" column for the results of the Part C. Instead, you will be making an implementation of your revised design as part of Homework 4.
• Possible extra credit for doing extra users. Use one template form per user.

In summary, you will be turning in two (2) copies of the filled out UsabilityEvalReport_template.doc (one for PART A and one for PART C), a revised paper prototype (PART B), and a short discussion of what you changed in the new version of the prototype. Please turn in this assignment a single pdf or zip file of all your deliverables, and upload the one file to Blackboard.

Homework 4: Implementation of the Prototype

Assigned: Monday, June 8, 2015; Due: Monday, June 15, 2015, at 10:30am EDT (note: no late turn-ins for HW4!)

Using the design that resulted from all your testing and revisions from Homework 3, you are now going to implement the interface. You can use any way to implement your prototype such as those that were discussed in class (see Lecture 7 which includes links to various places to download versions of the implementation tools).

Part A:

There is only one part to this assignment: to implement your interface. You should implement at least enough that all your user tasks can be completed, with sufficient feedback so the users can see what the system's responses are. You should simulate actions of your device. For example, if you were doing a copier, when the user clicks "Start" you might display a screen or dialog boxes that says everything about the current operation, for example: "now copying, 2-sided, no staples, using paper tray #1". If there are parts of your user interface you do not implement, there should still be controls (buttons, sliders, etc.) that would lead to those parts of the interface, but instead lead to screens or dialog boxes that say "not implemented yet" or something. That is, all of the screens you show should look complete, even if all the operations do not necessarily function. As another example, if you have a "search" button, it is fine if the search-results screen always shows the same thing, no matter what is searched for. As we have said a number of times, the complexity of the interface should be: "At least 30 "controls" (widgets: buttons, text fields). About 10 different screens/pages/windows/modes".

Also create a short "readme.txt" file that describes what one can do with the prototype. This will guide the graders on what is supposed to work. For example, tell what paths through your interface are working. If you pretended that the user searches for "aspirin", mention that kind of thing in the file. The "readme.txt" file should also contain your task test script that you will use for the user test, so the graders and others will know what you will tell users to do. Put the readme.txt file in the top level folder where the implementation is so it will be at the top level of the .zip file.

1. The deliverable for PART A is the complete implementation. If you have a web-based implementation, like for Balsamiq online, include the URL in the readme.txt file, and just turn in the readme file, but change the name to be HCI-HW4-lastname-firstname.txt. If you have a regular implementation, just zip up your entire directory structure including project files, the readme.txt file, etc. Name your file: HCI-HW4-lastname-firstname.zip, for example: HCI-HW4-myers-brad.zip. Put this file on blackboard in the Assignment 4 area. NOTE: Make sure it will be easy for us to run your system, and to see the source code. Make sure any required images are included in the zip file, and include the "readme.txt" file explaining how to run your application.

Important Notes:

• It must be easy for us to run your implementations. Note that it should be runnable on either a PC or a Mac. Points will be taken off if we have trouble making it run. Points will also be taken off if you forget to include your source code. It would be a good idea for you to test your zip file or URL on another computer to make sure you remembered everything.
• Make sure all the pathnames (filenames) of any pictures or sound files you use are relative pathnames, so the project will run when it is moved to the TA's computer. For example, the filenames should not start with C: or file://c:/
• We really need everyone to turn in this assignment on time, so we can give your final implementation to your classmates.

Deliverables Check List:

• All of the screens you show should look complete.
• At least 30 "controls" (widgets: buttons, text fields)
• About 10 different screens/pages/windows/modes
• Prototype should run. Teaching staff should not have to send it back to students for missing parts
  • Make sure any required images are included in the zip file, and include the "readme.txt" file explaining how to run your application.
• No dead links
• "readme.txt" file:
  • describes what one can do with the prototype.
  • should contain your task test script that you will use for the user test
• Implementation should run on PC or Mac
• Include source code, if any

Homework 5: Heuristic Evaluation of Each Other's Implementations

Assigned: Monday, June 15, 2015; Due: Monday, June 22, 2015, at 10:30am EDT(note: no late turn-ins for HW5!)

Each person will evaluate the user interfaces of two other people in the class. After you turn in your system for Homework 4, the TA will e-mail you the two systems you are to evaluate. You can also see the assignment of systems to people on Blackboard.

You will be getting two (2) systems. You should write a heuristic evaluation of the site using Neilsen's Ten Usability Heuristics. (There are lots of different lists of heuristics, but for this assignment, you must use only the ones that Nielsen lists). Although two people are assigned to evaluate each project. you should do the evaluation individually.

Perform a heuristic evaluation of the systems you have been assigned, and write a report using the HE Report template. (see also the partial example in doc format).

• You should include pictures of the system that illustrate each of your points (either directly in the "Prototype Screen" column as in the example, or below the table with a hyperlink reference, as in the template). You can use screen captures to do this (on a PC, the print screen key puts the screen on the clipboard; on a Mac, command-shift-4 let's you pick a selection on the screen to send to a file, command-control-shift-4 puts it on the clipboard).
• Each point should be annotated with the particular Name of Heuristic that is being violated or being followed (put this in the "Name of Heuristic" slot). If violated, you might suggest a fix in the Way(s) to rectify column.
• For each system you are assigned, you must list at least 8 (eight) points that illustrate at least 4 (four) different guidelines. I am looking for insight, subtlety and depth. Thus, saying "the system doesn't have good color choices because it doesn't use color" is pretty trivial and won't count for much. Also, since this is a prototype, saying that functions aren't implemented also doesn't count. For example, prototypes generally won't have "documentation and help" so it is not useful to point that out.
• Be sure each row lists which heuristic is violated or upheld, and why, and what you recommend be done about it. Be sure it is clear in your description whether you are saying the interface violates the heuristic or does it well.

You will be graded for this homework on how well you evaluate the prototype, the quality of the feedback you are providing, how many problems you found, and the extent to which you assign them to the correct heuristics.

In homework 6, you will be trying to fix the problems that other people identified with your system, and your grade will be based in part on your response to your classmate's Heuristic Analysis of your prototype (what you did based on what they found), so it is important to you to make sure your classmate gets a chance to evaluate your prototype.

• The deliverable for this assignment are two HE Report templates filled out, one for each of the systems you evaluated. This will be at least 2 (systems) x 8 (items) = 16 items.
• Please keep the documents as two files, so that we can email them to the people whose projects you evaluated. Please make sure the files are in .docx (Microsoft Word) format, like the tempkates, so your classmates can edit them. Please name your two files as follows: HE-Report-of-otherPersonsName-from-yourname.docx. For example, if one of the systems I was evaluating was by Julie Eckstrom, my file about that evaluation would be named: HE-Report-of-Eckstrom-from-Myers.docx.
• Collect all your files together into one zip file, and upload the one zip file named HCI-HW5-lastname-firstname.zip to Blackboard, as described in the submitting instructions.

Homework 6: Revised Interfaces Based on HE

Assigned: Monday, June 22, 2015; Due: Thursday, Jun 25, 2015
(Small Extension: This homework is due on Thursday at midnight EDT. The penalty of 10 points will apply after midnight. An additional 10 points will be applied if it is not turned in by Monday, June 29 at 10:30am EDT. After that, the penalty will be an additional 10 points every day at 10:30am.)

The TAs will send to you the two evaluations from your classmates. In this assignment, you should revise your interface based on that feedback. It is very important that you respond to each criticism, and avoid changing each good thing mentioned. In general, you should make the recommended changes. However, if you have a good reason for not making a fix, you can make that argument in your report. Your grade on this assignment will depend primarily on the quality of your final, resulting interface, and your responses to your classmates' comments. Your grade will be lowered if we do not agree with your reason for not fixing problems that were brought up in your previous user studies, and by your classmates in their heuristic analyses.

The deliverables for this assignment are:

• First, the final revised prototype, resulting from your responses to all the user studies and heuristic analyses.
• Second, you should write a report, discussing how you responded to each item mentioned by your classmates. First, fill in the final (yellow) column of the reports you received. You must turn in the two HE forms given to you, with your column filled in based on your responses. In addition to the responses on the tables, you should write extra information as follows. You must have screenshots to show what you changed, so it is easier to see -- circle your changes and reference them in the report. If you made any other changes to your system since Homework 4, for example in response to the TA's comments, or problems users had in your user studies which you had not previously addressed, then please make a note of that in your report too, and mention the source of the problem. You will get a better grade based on all the improvements you make from your previous (Assignment 4) version, whether the recommendation for the change results from a previous user study, the TA or an HE.
• Zip up your report and your implementation files as usual. Name this file: HCI-HW6-lastname-firstname.zip and upload it to Blackboard.
• You should also fill out the two (2) questionnaires for the class. We take all the comments seriously, and try to improve the class each year:
  • The official CMU course evaluation: http://cmu.onlinecourseevaluations.com
  • The class-specific questionnaire: https://www.surveymonkey.com/s/95763Z5summer2015

Final Exam Instructions

This course has a 3-hour exam. You will arrange to schedule the exam with a Proctor anytime between June 25 and June 29, 2015. (Note that this means the entry of your final grade for this course will be delayed until about July 6.)

A form describing how the proctor works will be distributed near the end of the course, but you can see a sample from previous years here: MSIT_Exam_Proctor_Form.pdf.

The final exam will cover material from all the lectures and all the homeworks. You can be sure that there will be questions on the exam from each lecture 1 through 12 and all homeworks. However, material that is in the readings but not covered in lecture will not be on the final. The exam is closed-book, so you can not use your computers, books or notes during the exam. Please bring pens or pencils to write with. Many of the questions will be short-answer, but you should also be able to do what you did in the homeworks.

There will be questions about the Heuristic Analysis, but you do not need to memorize the 10 heuristics -- I will give you the list, but you need to be able to recognize what they are. Similarly, I will give you the names of the kinds of Contextual Inquiry Diagrams, but you need to know what each is for and how to make them. If I ask about the various methods discussed in this course (such as Contextual Inquiry, Heuristic Analysis, etc., and any of the methods in Lecture 13), I will give you the name of the method, but you should know what it is and what it is good for, and how it differs from the other methods.

You will have 3 hours, but last year, most people finished in about two hours.