Braam, Callahan, and Schwan

The user level file server and cache manager will be the central topic of this paper: they were written in Perl.

Cache managers and file servers are particularly complex servers. A cache manager, for example, is a server for the kernel to supply cache data, and a client for the file servers to fetch data not yet cached. Advanced features such as write back caching and reintegration make file servers both servers to and clients of cache managers. In short the distinction between clients and servers is blurring, and advanced RPC mechanisms are needed.

While this request processing is in progress, very much state is shared, and must be carefully protected. Traditionally, threads have been used and this leads to complicated locking schemes. Asynchronous request processing has shown to be a very attractive alternative in settings like the ACE toolkit, ERLANG and in Windows NT and VMS kernel code. For InterMezzo our servers and cache managers, named collectively "lento", use the latter approach.

Lento is an event driven finite state machine, using the POE package and various extensions. Inside lento many sessions exists. Sessions react to events: our events are triggered by arrival of data and timers. When data has been delivered, it is either handled by an existing session, or it instantiates a new session. For example, if the kernel asks to refresh a directory, a session is set up for this, unless the directory is already involved in a session, in which case and event is posted to that session.

The data arrives over the network and from the kernel and is delivered by so called Wheels which are sessions handling asynchronous I/O. Wheels have Drivers for reading and writing and are supplemented by filters that assemble high level structures out of raw binary data. Some filters come with POE, such as a FreezeThaw filter. Others were constructed by us to deal with kernel user communication. In the near future we will also need Wheels to do asynchronous file I/O, possibly involving transactions for tied databases. Other Wheels accept incoming TCP connections, and set up client sessions for these. The Wheels form non-blocking portals to the core of the cache manager and file server.

The core sessions encode the semantics of file sharing. For example, the file server session may get a write lock request on a volume and will notify the readers that the data is about to be modified, before granting the write lock. Note that this in itself involves sending out multiple requests, collating the answers and dealing with readers that have timed out - something well beyond elementary RPC processing.

The performance of InterMezzo features somewhat high latency due to giving the clients great autonomy. After this latency read performance is that of local disk file systems, and tree modifications might be up to 5 times faster than NFS.

We hope to have a first release ready before the Perl Conference and our estimate is that the total amount of code (including kernel code for Linux and Windows NT, file servers and cache managers) will be around 12,000 lines - 8,000 of which is Perl.