Hunting for Sources of I/O

So you've tried all the other suggestions, and there's still something keeping the disk from its peaceful slumber. How do you figure out what it is? I don't know any way of getting this information from a stock kernel. There is, however, a small patch I wrote, that will give you some basic information about what is going on.

The patch works by modifying one of the accounting functions that is called whenever a disk I/O request is submitted. [1] It simply adds a line that prints the name of the process that submitted the request, and the type of I/O requested (read or write). Unfortunately, this function has no information about what file the I/O is associated with. So we can't say what is being read or written. We can only say who is doing the reading or writing.

The messages are written using printk(). Depending on how logging is configured on your system, they may end up in /var/log/messages /var/log/kern.log , or some other file. Check your /etc/syslog.conf for "kern.*" or a similar entry. The entries look like this:

Aug 7 18:46:28 take2 kernel: write by kupdated
Aug 7 18:51:20 take2 kernel: read by rc
      

Note when examining the log that most writes will be from either kupdate or bdflush. This is because asynchronous writes, which are more common than synchronous writes, are normally flushed to disk by either kupdate or bdflush. You will only see a write by another process if it has files open for synchronous I/O, or if it calls on the synchronization functions (sync(), fsync(), fdatasync()).

To use the patch, you first apply the patch to the kernel source tree, build the kernel, and install it. See the Kernel-HOWTO if you need help.

After you have rebooted with your patched kernel, you need to enable the I/O tracing messages. I have borrowed one of the obsolete fields in /proc/sys/vm/bdflush for this purpose. You want to change the value of the 8th field to "1".

bash# cat /proc/sys/vm/bdflush | awk '{$8=1; print $0}' > /proc/sys/vm/bdflush

Note that this will generate a lot of messages. To disable the trace messages, change the value of the 8th field back to "0".

bash# cat /proc/sys/vm/bdflush | awk '{$8=0; print $0}' > /proc/sys/vm/bdflush

Hopefully, these messages will give you enough information to figure out what is causing your disk to spin up.