MPI-Video Surveillance and Monitoring:
Implementation and Results

We implemented the MPI-Video EM using C++. The implementation consists of:

• an EM layer base class from which we build specific types of EM layers,
• code to assemble a set of layers into an EM based on a description of the EM, and
• code to run the EM, i.e., initialize the layers and then iteratively invoke the extrapolate and update methods.

How to Build an MPI-Video Environment Model

To build an MPI-video system, one must first decide what layers are needed and then implement them. This is done by creating subclasses of the EM layer base class. The subclass must contain state variables and methods to override the base class initialization, extrapolation and update methods. Most details of the iterative system are hidden from the implementation of a single layer type so these methods tend to be easy to write, however, a working knowledge of C++ is required. A layer need not always take advantage of the dynamic capabilities of the system. When there is no reasonable dynamic model of the state of a system, e.g., image data and static environment information, we simply use an identity operation for the extrapolation. A layer must provide a means for other layers to access its state. When a layer does its update, it is provided with a list of pointers to layers from which it obtains input. The update method invokes the state access methods for the other layers to get their data, and then performs an update using that data. Once the subclass for a layer type is defined, it is added to the library of all layer types.

With a sufficient library of layer types in hand, it is then necessary to configure the EM. We do this with a configuration file (plain text) that describes the layers in the EM and how they should interact. The file consists of a list of layers in the EM. Each layer has a layer type, a name, a group name, some initialization parameters, and a list of other layers from which it receives input. The group name is required for keeping track of groups of layers that work together. For example, when a three-dimensional tracking layer combines data from multiple cameras, it also needs camera calibration data from multiple layers and it must match each set of calibration data with the correct camera. Group names resolve the ambiguity. An example configuration file follows.

# EMV1.0
#
# Spec file to show EM for algorithm evaluation. Segmentation routine
# running w/ different parameters on same input sequence.
#
#

# column contents
#
# type | name | group |input connections | init info;
#

# Same input stream.
EmMpiv0In   vidIn0   group0            {"twocam0.mpiv"};
EmSegment   seg0     group0 vidIn0     {"initAlgorEval1.seg"};
EmSegment   seg1     group0 vidIn0     {"initAlgorEval2.seg"};
EmViz       vizlayer group0 seg0 seg1  {"init2.viz"};

The last step in building an MPI-Video system is to create a main program that invokes a method to build the EM from the configuration file, and then invokes a method to run the EM. To date, we have only used a single main program and alter the EM by changing the code for layers and by editing configuration files. Once a sufficiently powerful library of layers is developed over time only the configuration files will need to be altered to create new EMs.

At the time of writing, we have not implemented access methods to facilitate server/client interaction. Therefore, in order to visualize the state of the EM and verify its operation, we implemented visualization layers. These layers have no state and do no extrapolation. They simply display the image data that they get from other layers during update. In the long term, visualization will be done by an EM client, but visualization layers allow for easy debugging of a system without the need for a client.

A Running Environment Model

The figure below shows a screen snap shot taken during the operation of a simple MPI-video system.

• Implementation of methods for new layer types is easily done. This is because the EM manages most of the overhead involved with the dynamic system, leaving the author of the layer to focus solely on the extrapolate and update methods.
• There must be agreement among the layers about data types in the states. The current implementation requires that authors of different layers cooperate when deciding how to represent states. This gives ample flexibility to individual authors, while creating potential for incompatibility amongst independently designed layers. At the time of writing we are establishing an EM state base class that we expect will maintain some flexibility while providing a more uniform basis for creating state types.
• Once we had a few layer types available, it was a simple matter to arrange layers in different configurations to perform different tasks.
• Although we did not implement the client/server interaction with the EM, visualization using specialized layers worked well and simplified EM development.

Source Code Availability

Web page design by Jeffrey E. Boyd