15-410 Project 4 Hurdles

Policy

To proceed to Project 4, you need to have a solid Project 3. This means that you can pass 15 out of 19 of the basic tests and 12 out of 15 of the solidity tests.

Furthermore, you should evaluate what you know of your project which is not directly observable by the tests. For example, did you get your kernel to "work" by some non-sustainable short-cut? If you couldn't figure out how to safely free kernel stacks, leaking them instead might make things appear to work, but would not really address the problem. Likewise, disabling interrupts for essentially the entire body of every system call is papering over a serious issue. If you "trick" us into letting you do P4, you will be cheating yourself.

Also, if a particular test, when run on a freshly booted kernel, causes a crashes sometimes but not other times, this indicates an instability which you should probably count as a failure. At least, there is a good chance we will observe the crash and count it that way!

Once you've run the tests, please send us a report (success or failure) via the Project 4 Registration Page. Also, be sure to keep a record of which tests you believe you passed and failed in case we disagree later.

Tests

Note that some tests have multiple "phases". You should pass all phases of such tests to count the test as a pass.

Also note that it is possible for one or more tests to have a bug. If you think there is a bug in a test, or you believe a particular test relies on a specification ambiguity, submit a careful and detailed bug report to us via e-mail. If failing this test would not disqualify you from P4, go ahead and submit the registration page. Otherwise, indicate in your mail that the test is blocking you.

Observe that some tests, such as rodata_wr_test, "pass" if your kernel kills the process. That is, they should not run to completion.

The tests will appear in your 410_tests directory. As was true of earlier tests, you may wish to copy them to your user_tests directory to run them.

Good luck!

Basic Tests

brk_basic
deschedule_hang
echo
exec_basic (uses exec_basic_helper)
fork_test1
fork_wait
getpid_test1
halt_test
loader_test1
mem_eat_test
onestack
print_basic
readline_basic
stack_test1
stack_test2
wait_getpid
wild_test1
yield_desc_mkrun
sleep_test1

Solidity Tests

shell (should run ok, should run commands ok)
fork_bomb
fork_exit_bomb
fork_wait_bomb
loader_test2
register_test
kernel_wr_test
rodata_wr_test
text_wr_test
desch_hole
exec_nonexist
mkrun_getpid
brk_test2
print_test3
cho (mixes wait_getpid, wild_test1, exec_basic, print_basic, sleep_test1)

Agonizing

Some of you are almost passing by the criteria above and trying to figure out what to do, whether to appeal, etc. Here is some advice.

First, figure out where you stand with P3. Here are some kinds of problems, ranked from least important to most important.

1. Calling deschedule() on the hole kills the process instead of returning an error code. This isn't exactly right, but it isn't indefensible either. Early Unix kernels had this behavior, though modern ones do not. You probably understand the issues and don't need to spend time fixing this.

2. Calling deschedule() on the hole crashes youur kernel, but calling readline() on the hole doesn't. Crashing is bad (we don't want to see that), but since you solved that problem for readline() you presumably understand it. If your kernel is riddled with things like this you might want to solidify it, but a case could be made for moving on.

3. You have one or more serious structural issues. For example, readline() or wait() involves a yield() loop. Or you spin-wait somewhere(offhand I can't think of a valid reason for doing that). Or you can't support more than four user processes because every process gets a fully-populated set of page tables even though it never maps more than a tiny fraction of that address space. In this case you should probably talk to somebody until you understand the right way of solving the problem and then solve it. If you have a large number of these problems, you should focus on solving the worst one in a solid way.

4. You have a serious debugging problem, the kind of thing which is described by

   For some reason, when we swap the order of two tests our kernel crashes

   or maybe

   For some reason, suddenly all processes appear to exit and there is nothing to run but the idle process.

   The issue here is that part of what you should get from this class is strong investigation and debugging skills. If there's "something broken somewhere", this is an excellent opportunity for you to get more practice. Remember that, when you leave the academic environment, you will need to be good at debugging hard problems (the easy ones are found quickly), regardless of whether they're in your code, somebody else's code, or somebody else's code that you have inherited.

5. Also, if there is some part of your code which you're hoping desperately that we don't test because you know it will crash, this might be a good time to work on that. If you have several such areas, you should probably stay with P3.

[Last modified Saturday January 10, 2004]