FCL is implemented in three layers. The lowest level layer is a C interface between the persistent storage manager and Tcl. The second layer is some Tcl code that implements a simple object system. The final layer is a set of abstract classes for design objects, which is implemented in the object system. This document will describe the lower two layers, and will also show where the layers interact. The toplevel layer is described in the user's manual. The following diagram shows the major components of the FCL system.
Figure 1. Architectural Diagram for FCL.
In Exodus, all data are stored in volumes, which are managed by one or more servers. A client must use the client library to mount volumes before they can access any data. Mounting a volume will also result in opening a connection to the server that is managing that volume. All operations on data in a volume must be performed within the scope of a transaction. Transactions encapsulate a set operations on a volume which must either succeed or fail atomically. If a transaction succeeds, then the results of the operations in the transaction become visible on the volume. Otherwise, the volume is left in the state that it was in before the transaction began. Transactions also manage concurrent access to the database through a set of locking protocols that make concurrent updates appear to be happening in some consistent, sequential order.(1)
Objects in an Exodus volume are logically clustered into files, which also provide efficient operations for scanning and bulk loading the objects that they contain. Within a file, the library provides primitives for creating, destroying and modifying data objects on the volume. EAch objects comes with its own OID, identifying the volume, file, and file address of the object on disk. FCL provides mechanisms for dealing with OIDs in a fairly transparent, though right now very inefficient, manor. Finally, the library provides indexes that allow applications to build efficient data structures for searching and scanning related objects in a volume.
FCL provides an explicit interface to transactions and objects only. There is a lower level interface to volumes and files that is not seen at the user level. In addition, and interface to indexes is implemented but it is not used in the current system.
struct fcl_RefStruct {
OID myOID;
USERDESC *myHandle;
};
The myOID field of this record stores the unique object identifier used by Exodus to locate the object
in the database. The myHandle field of this record stores a pointer into the client libraries buffer pool.
Whenever an object is fetched from disk, the client library fixes the page or pages that the object occupies
in the buffer pool and returns a handle to the FCL runtime system. We can then use this handle to manipulate
the object in memory. The runtime library uses the following calls for these manipulations:
sm_ReadObject
sm_WriteObject
sm_AppendToObject
sm_DeleteFromObject
sm_CreateObject
sm_ReleaseObject
The higher levels of the FAM library use this primitive namespace to maintain many "logical" design files within a single "physical" Exodus file. This mechanism is described in the Users Manual.
sm_ReadConfigFile
sm_ParseCommandLine
sm_Initialize
sm_OpenBufferGroup
sm_GetRootEntry
sm_SetRootEntry
sm_CreateFile
sm_{Begin,Commit,Abort}Transaction
Exodus objects are referenced in FCL through "handles", which are based on the OID of the object. For the sake of clarity, throughout this document, we will call the physical OID an object reference and its user-level form an object handle.
Object handles are actually Tcl commands. The name of each command at the interpreter level depends on the OID of the object in the database. Each commands calls the same dispatch routine which is written in C. The dispatch routine handles a couple of low level built-in methods and passes control off to the main method dispatcher which is written in Tcl. Note that this structure implies that before an object can be used to call methods, its handle must be registered as a command with the interpreter.
The C layer defines the following new commands:
persist <command>
trans, abort
<objectHandle> <method> <args>
<tableHandle> <command>
An object value is the data that is actually stored to disk in a transaction. It contains information about the object's size, reference count, and the actual data in the object. Values are stored as variable sized records with a fixed header and a variable sized data field. In C, this is implemented this way:
struct varRecord {
HEADER header;
char data[0];
}
Here, the data field is just a dummy to mark the beginning of the data block. When we create a value, we allocate
enough space to contain both the header and the object's data and then use the dereference through the data field
to fetch the object's data string.(3) We store objects like this to avoid using
an extra level of indirection to get to the object data. This could be expensive, since it would be at the database level.
This design allows us to store an arbitrary simple Tcl value in the database as a string. Therefore, all encodings of persistent application data structures in Tcl code must be based on strings. This is generally the case anyway, but having hooks to Tcl tables would be nice.
Note: It should be possible to set up a transparent hook between Tcl arrays and an Exodus index file using Tcl calls that allow us to watch for all side effects to a variable. Index files and arrays go together naturally.
set foo [ persist new "some value" ].The result of persist new is the object handle, which we put into a variable for later reference. To access this handle across sessions, it must be stored in the database somewhere. A side effect of this operation is that the object handle is registered as a command. In addition, a reference record containing the new object's OID is registered with this new command in the clientData portion of the Tcl command record.(4) Tcl specifies that whenever the new command is called, this data will be passed to the object dispatch routine so it knows what object to operate on.
The C layer of the FCL system provides four low level object management commands that are used by the upper layers of the system:
<objHandle> getvalue
<objHandle> setvalue <value>
<objHandle> destroy
<objHandle> release
Note: This mechanism is largely transparent to the higher layers of the system except for the fact that anything built on top of the C library must be careful to use persist deref to register object handles as commands before the objects themselves are referenced. In the current implementation, this shows up in the Tcl code implementing objects and classes. This should be the responsibility of the C portion of the library, probably.
If a non-primitive method is dispatched to an object, the C library calls a Tcl routine called "__fcl__methodDispatch" to resolve the call. This is the main low-level dependency between the C code and the Tcl code. The default dispatch is implemented in Tcl to make it easier to modify and prototype. Eventually, it will be moved to C.
trans { <some Tcl commands > }.
The trans command acts just like a call to eval, but if any error is generated it implicitly aborts the current transaction.
In addition, if all the commands succeed, it does an implicit commit. The code can also do an explicit abort using the abort
command. This will abort the current transaction and signal an error.
Each instance of FCL can be running one top level transaction at a time. Calls to trans can be nested, in which case only the top-level call actually commits, but any abort aborts all the way to the top level.
Finally, the persist command has hooks for creating transaction blocks that span multiple Tcl commands, but they should not be used.
In general, applications should build a higher level convention on top of this command. In our system, the basic database class library defines procedures for storing all designs in one physical file, and keeps its own directory of the logical files available in this space. This directory is stored as a Tcl list in the first page of the physical file.
Note: We desperately need a real namespace for design files.
All of the other FCL commands check for initialization and the existence of transaction before attempting database operations, so if you don't call persist init, you can't do anything.
class <class-name> [ <super-name> ].If the optional super-name argument is present, then the new class is a subclass of super-name.
In addition to encapsulating a set of common characteristics, classes create objects that are instances of themselves. An instance contains all of the newly defined characteristics of its class, plus everything defined in the superclass of its class and so on. The characteristics that a class can define are an object's slots and its methods. These are described in the following sections.
In the implementation, classes are represented as records stored in Tcl strings. Each record contains the name of the class, the name of its superclass (or "root" if it is a root class) a list of slots that should be in each instance, a list of default values for those slots, and a list of methods defined for each instance. This list structure is rather inefficient, and will eventually be replaced with something reasonable.
Note: Our classes are used in a way that is very close to prototypes in systems like SELF. But, systems like SELF have no classes at all, and use special objects to perform the duties that we have placed in our classes.
In FCL, each object is a member of some class. The primitive methods isa, myClass, in addition to the super procedure provide information about the type of an object at runtime. These are described in the user's manual.
Sub-classes can redefine the slots of their super-classes. This has the effect of overriding the default value defined by the super-class. You cannot, however, define two slots by the same name in a particular class.
When an object is created, it is represented as a list with three components. The first is the name of its class, the second is a list of the names of its slots. The third is a list of the values of those slots. The getslot and setslot builtin methods provide access to these values. A slot is fetched by looking it up in the names list and then indexing into the values list based on that result. If an unknown slot name is given, an error occurs.
At creation time, an object is given all the slots defined for its class and recursively for all of the super-classes of its class. This implies that if new slots are later defined for any of these classes, they will not be available for objects created before the new definitions were added.
method <className> <methodName> <methodArgs> <body>This defines a new method called methodName for the class className which takes the arguments methodArgs and runs the code in body. In addition, every method has an implicit argument called self that refers to the object that the method was dispatched from.
The method can be dispatched from any object that is of the right class. Syntactically, method dispatch is done this way:
<objectHandle> <methodName> argsThe method dispatcher looks at the class of the object referred to by the object handle and sees if the method methodName has a definition in that class. If not, the super-class is searched and so on up to the root of the inheritance tree.
Sub-classes can redefine methods that were defined in their super-classes. The effect is to override whatever was there before, including the argument specification. This should probably be checked in future versions. of the system.
Note: Since methods names are not stored directly in an object, re-definitions of methods are available to all new and existing objects. This is inconsistent with the behavior for slots, and should probably be fixed.