15-410 Project 4 Hurdles

Policy

Since the kernel project is the core experience of the class, it is important to complete it in a solid fashion. Furthermore, Project 4 involves extending your Project 3 kernel, which requires a solid, debugged base from which to proceed.

Thus, when you reach the Project 3 deadline, you will gauge your degree of completion. Groups with solid kernels will proceed to Project 4, and groups needing extra time to genuinely complete their kernels will receive a 0% grade for Project 4 and will continue working on Project 3 until the "p3extra" deadline of Friday, December 2nd at 23:59.

Leaping the "hurdle" means that your kernel can pass the "shell test", 14 out of 17 of the basic tests, 8 out of 11 of the solidity tests and 1 out of 2 of the stability tests. Furthermore, your kernel's preemptibility and concurrency control must be reasonable--it is possible to get your kernel to pass more tests by structurally or artificially limiting preemption, but concurrency management will be a significant part of your Project 3 grade, so skipping past that and going to Project 4 will almost certainly be cheating yourselves... Also consider other issues which can't be directly observed by tests. For example, did you get your kernel to "work" by some non-sustainable short-cut? If you couldn't figure out how to safely free kernel stacks, leaking them instead might make things appear to work, but would not really address the problem.

If a particular test, when run on a freshly booted kernel, causes a crash sometimes but not other times then this indicates an instability which you should probably count as a failure. At least, there is a good chance we will observe the crash and count it that way!

Once you've run the tests, please send us a report (success or failure) via the Project 4 Registration Page. Also, be sure to keep a record of which tests you believe you passed and failed in case we disagree later.

Tests

Note that it is possible for one or more tests to have a bug. If you think there is a bug in a test, or you believe a particular test relies on a specification ambiguity, submit a careful and detailed bug report to us via e-mail. If failing this test would not disqualify you from P4, go ahead and submit the registration page. Otherwise, indicate in your mail that the test is blocking you.

Observe that some tests, such as remove_pages_test2, pass if your kernel kills the process. That is, they should not run to completion. These tests will announce themselves by emitting the string START__TYPE_ABORT when they start up. Tests announcing START__TYPE_FOREVER should "hang" (really: run forever) instead of completing.

The tests have been released into your 410_user/progs directory.

Good luck!

Required Tests

shell (should run ok, should run commands ok--lots of them!)

Basic Tests

deschedule_hang
exec_basic (uses exec_basic_helper)
fork_test1
fork_wait
getpid_test1
halt_test
loader_test1
ls (a build-in shell command)
mem_eat_test
print_basic
readline_basic
remove_pages_test1
sleep_test1
stack_test1
wait_getpid
wild_test1
yield_desc_mkrun

Solidity Tests

exec_nonexist
fork_bomb
fork_exit_bomb
fork_wait_bomb
loader_test2
make_crash (uses make_crash_helper)
mem_permissions
minclone_mem
new_pages
register_test
remove_pages_test2

Stability Tests

cho (mixes wait_getpid, wild_test1, exec_basic, print_basic, sleep_test1)
cho2 (mixes yield_desc_mkrun, loader_test1, remove_pages_test1, getpid_test1, fork_wait) 

Agonizing

Some groups may almost pass by the criteria above and will be trying to figure out what to do, whether to appeal, etc. Here is some advice.

First, figure out where you stand with P3. Here are some kinds of problems, ranked from least important to most important.

1. You fail one test too many but the failures are for known, simple reasons. For example, when the tests fail the kernel doesn't crash, but instead trips an internal assert(). Clearly there are issues, but there is some reason to believe they could be fixed quickly and are glitches rather than huge issues. Groups in this situation may wish to appeal for permission to proceed to Project 4.

2. You have one or more serious structural issues. For example, readline() or wait() involves a yield() loop. Or you spin-wait somewhere (offhand we can't think of a valid reason for doing that). Or you can't support more than four user processes because every process gets a fully-populated set of page tables even though it never maps more than a tiny fraction of that address space. In this case you should probably talk to somebody until you understand the right way of solving the problem and then solve it. If you have a large number of these problems, you should focus on solving the worst one in a solid way.

3. You have several tests which fail for reasons you don't understand, though they don't crash the kernel or cause subsequent tests to fail. This is potentially bad, since even minor bad symptoms may conceal potentially bottomless debugging sessions.

4. You have a serious debugging problem, the kind of thing which is described by

   For some reason, when we swap the order of two tests our kernel crashes

   or maybe

   For some reason, suddenly all processes vanish and there is nothing to run but the idle process.

   The issue here is that part of what you should get from this class is strong investigation and debugging skills. If there's "something broken somewhere", this is an excellent opportunity for you to get more practice. Remember that, when you leave the academic environment, you will need to be good at debugging hard problems (the easy ones are found quickly), regardless of whether they're in your code, somebody else's code, or somebody else's code that you have inherited.

5. Also, if there is some part of your code which you're hoping desperately that we don't test because you know it will crash, this might be a good time to work on that. If you have multiple such areas, you should probably stay with P3.

"Nearly" Passing

Please note that passing all of the basic tests and none of the stability tests is not "almost" passing--it's a clean miss. In the course of getting the stability tests to run you will discover qualitatively different bugs, and fixing them will teach you qualitatively different things. Skipping the stability tests is not an option.

Appeals

Any appeals must be presented in person to Professor Eckhardt and must include a printed list of which tests pass, which tests fail, and why.

[Last modified Monday February 27, 2006]