05-830, User Interface Software, Spring, 2001
Homework 3,   Issued: February 7, 2001
Due: Wed, February 21

Homework 3

Implement a Benchmark Task
Using a Prototyping Tool

Choose a Benchmark

If you decide to implement your own benchmark, be sure to get me a description in HTML format.  Please fix up the descriptions based on my comments first.  If there are pictures in your description, please put them in a place where I can get them (e.g., on /afs) and send me the full path name (or else give me a floppy).  All the benchmarks will be made available off the course's main page. The current list is available as: 

        http://www.cs.cmu.edu/~bam/uicourse/2001spring/benchmarks.html

Choose a Prototyping Tool to Implement the Benchmark

As discussed in class, a "prototyping tool" is one that is designed to quickly create applications without programming in a conventional language.  Prototyping tools might have scripting languages that are interpreted (e.g., Lingo for Director and Visual Basic), but they do not produce "real" C or C++ code. Examples of prototyping tools I am familiar with are:

• Visual Basic
• Delphi
• Director
• HyperCard

All of these are available on the various cluster machines.  If you would like to use a different tool, feel free to discuss it with the professor.

Implement the Benchmark Task using the Tool

There are many things you will need to keep track of while you are learning the tool and while implementing the benchmark. Be sure to read over the questions listed below that you will need to answer in your report to see what you need to pay attention to!  In particular, be sure to keep track of:

• Time spent reading the documentation for the tool.
• What specific references you used to learn about the tool (names of books or manuals, on-line documentation, etc.).  Be sure to include complete references.
• Exact amount of time spent doing the implementation.
  • Please try to be quite accurate in timing how long this takes.
• During the implementation, keep track of how much time is spent:
  • Thinking about the design.
  • Actually typing in the code.
  • Using interactive layout tools to draw parts of the interface.
  • Going back to the documentation to look up things and figure out how to do things.
  • Debugging code to make it work.
• The amount of code written to make this benchmark work:
  • For interactive tools, please count the number of lines of code in the scripting language you had to type in.
  • For example, the number of lines of Visual Basic code, or the number of lines of Lingo.
  • It might be helpful to print out the project and count them on the hardcopy.
• It would also be useful to have a count of the number of objects created interactively.
  • For example, how many objects were drawn with the editor?

Note that it may be impossible or very difficult to implement the entire benchmark in the prototyping tool.  In this case, don't.  It is acceptable if the implementation only gives a flavor for how the benchmark will work, which is what prototyping is usually for, after all.  Be sure to discuss this in your report.  However, if it is reasonable to implement the entire benchmark with the prototyping tool, then please do so.

Write a report about the implementation experience

Your report should preferably be in html of if not, in plaintext, because we will make all the information available to the whole class.  You must include some screen shots of your program, preferably in gif format.  Please turn in hardcopy (printout) of all the following:

• A written report discussing all the points below.
• A code listing or printout of the code for your system.
• Screen dumps of the screens produced by your implementation.

You should also place all of these on /afs/  or on the WWW and tell me the address, or else turn in a floppy containing all of these and put on the floppy your name and whether it is in PC or Mac format.

Be sure your report includes the information listed on the benchmark questions page, reproduced below:

The questions are:

• Which Benchmark are you implementing?
• What is the date that you performed this experiment?
• What is your name, phone, e-mail address, and physical mail address?
• What hardware did you run on?
• What tool and operating system, etc. did you use? Which versions? (e.g., "I used Visual Basic Version 4.0 on Microsoft Windows 95 using a PC with a 150mhz Pentium").
• What kinds of programs do you think the tool you used are best suited for? (I.e., are they best for graphical editors, text editors, form-filling programs, multi-media games, etc.?)
• If the specification of the benchmark leaves anything unspecified, what is your final design? (Please specify fully.)
• Did you implement anything not in the specification? (presumably, because your tools gave you this capability for free -- don't spend time adding extras that are not specified.)
• Were there any features that you could not implement because they were too difficult? Which ones and why?
• How familiar were you with the tool before starting the implementation?  (Expert, familiar with it, never used it before).  If experienced, how much have you used the tool before?
• What books or documentation did you use to learn the system?  (Give complete references if books.)
• What was the total time spent doing this benchmark?
• Then, please break the time down into:
  • Learning time before the implemention began?
  • Time to do the implementation?
    If possible, please further break the implementation time down into:
    • How long did it take to design the implementation (thinking time)?
    • How long did it take to use interactive layout tools to draw parts of the interface?
    • How much time did you spend going back to the documentation to look up things and figure out how to do things?
    • How long did it take to type in the code of the program?
    • How long did it take to debug the program?
• How many lines of code is the resulting program (excluding comments)?
  • For interactive tools, please count the number of lines of code in the scripting language you had to type in.
  • For example, the number of lines of Visual Basic code, or the number of lines of Lingo.
  • It might be helpful to print out the project and count them on the hardcopy.
• How many objects were created interactively?
  • For example, how many objects were drawn with the editor?
• How many bytes long is the source code, i.e., what is the size of the source file in bytes on the disk? If the source is in multiple files, please add them all together.
• How many bytes is the resulting binary (compiled) code?
• Please rate and discuss the quality of the tool you used along the following dimensions:
  • Ease of learning how to use the tool.
  • Appropriateness of the tool for creating applications like the benchmark.
  • Ease of creating the benchmark.
  • Quality of the implementation resulting from using the tool.
  • Discuss the good and bad points and features of the tool in general.
• How would you rate the run-time performance of the resulting implementation? (Does it run fast enough to be usable?)
• How would you rate the quality of the implementation created while using the tool ? (e.g., is it well-structured, would it be easy to modify and maintain, etc.)
• What features of the benchmark were easiest to create using the tool?
• What features of the benchmark were most difficult to create using the tool?
• What changes to the tool would have made it easier to handle the difficult parts?
• What was the most significant source of bugs in the code? (What were the hardest things to debug?)
• Please evaluate the benchmark task itself along the following dimensions:
  • Is the benchmark sufficiently specified that you knew what to implement?
  • What is too vague or unclear in the benchmark specification?
  • Is the benchmark a good test of this tool?
• Any other comments on the quality of the toolkit you were using or on this benchmark task?

Prepare an Oral Presentation to give in class on Wed, February 21

You should prepare a 5 minute presentation discussing the benchmark and the tool you used. We will try to fit everyone's presentation into one lecture.

This homework is worth 14% of your grade in the course.  It is due Wed, February 21. 

You must be ready to present your work in class. Turning your written material in by 5pm on the due date will be acceptable. Late penalty is one point out of 10 per lecture.

Back to 05-830 main page