Technical Area:
Video Surveillance and Monitoring (VSAM)

Texas Instruments Corporate Research Laboratories

Technical Objectives

The goal of this project is to develop and demonstrate new video understanding technology for battlefield information systems in rural and urban environments. New technology to be developed in this program will focus on video-specific problems that have not been solved reliably in previous image understanding research, with two key technical objectives:

Develop and demonstrate algorithms for robust moving object detection: Robust moving object detection and tracking is needed to detect significant change (e.g. motion of humans or vehicles) in IR and EO video sequences . This effort will produce the technology needed to reliably detect and track people or vehicles in regions under surveillance, in the presence of changes in temperature and lighting, intermittent occlusion, motion of animals and vegetation, and other sources of confusion. The output of the algorithms will specify position in 3D space and attributes such as facing direction (for humans), direction of movement, and appearance attributes for matching and description.

Develop and demonstrate algorithms for video event description and recognition: This work will develop methods of classifying motions and interactions of objects into specific categories constituting events that are important in the current mission context. TI will develop methods of representing and detecting such things as a person parking a vehicle in a sensitive location and then driving away in a second vehicle.

Approach

New capabilities produced in this research will be integrated into TI existing video surveillance architecture. The architecture will provide a graphical interface that supports display of detected objects and events on a map of the region under surveillance.

Robust Moving Object Detection and Tracking

Moving object detection algorithms generally work by comparing the incoming video image to a model, detecting and analyzing deviations from the model, and attributing the difference either to the presence of a moving object or to noise. Our approach to robust detection and tracking is to enhance the models to exploit:

• Context: We will use context to predict the expected appearance and location in the image of moving objects, to predict occlusion, and to select and initialize dynamic models of the background.
• Dynamic models: We will develop models that capture the ways images change over time, including such things as motion of vegetation, diurnal variations in temperature and illumination, et cetera.

Event Detection and Recognition

Our approach to event recognition is based on the TI Automatic Video Indexing (AVI) system (see Relevant Images), which analyzes security videotape recordings. In this system, object motions and interactions are described by a directed acyclic graph called a motion graph. Each node of the graph is an observation of an object, and is linked to its predecessor and successor in time. Forks and joins in the graph represent complex interactions. For example, if a person enters a scene, puts down an objects and leaves, the graph will contain a chain of nodes representing the person, with a fork node whose successors are the continuation of the person track and a chain of observations of a stationary object.

Military/Battlefield Relevance

The generic tracking and event recognition capabilities to be developed in this project are applicable to a wide variety of military and defense intelligence needs, and to related problems such as counter-terrorism, drug interdiction, airport security, and urban policing and crime prevention. We will demonstrate the technologies in the context of a system that allows users to interactively specify monitoring and threat detection tasks, and issues alarms or displays results when instructed to do so. The technologies will enable automation of generic tasks such as:

• Counting vehicle traffic that satisfies particular description, e.g. turns into a particular street or parks in a designated area. For example, soldiers in a rural battlefield could use a covert camera to count vehicles moving north along a given stretch of road, or to detect possible mine-laying activity.
• Reporting activities such as depositing or removing an object in a specified region. For example, a security system for a government building can issue an alarm if a person deposits an object against the wall of the building late at night.
• Issuing alarms when specified threatening conditions are detected. For example, soldiers maintaining base security can use the system to detect and photograph persons loitering near the base perimeter, or to issue an alarm if a vehicle parks in a sensitive location.

Planned Demonstrations

TI will present live demonstrations of the technologies at selected VSAM and Image Understanding workshops, using a combination of live and taped video data. In the demonstrations, the technologies for moving object detection and event recognition will be demonstrated both in isolation and in the context of an end-to-end monitoring application. In the end-to-end application, the algorithms will be used to detect events or behaviors that have been defined as threats, and to respond to threats by issuing alarms.

Recent Publications

Courtney, J. "Automatic video indexing via object motion analysis", Pattern Recognition (in press).

Flinchbaugh, B., and T. Bannon, "Autonomous Scene Monitoring System", Proc. 10th Annual Joint Government-Industry Security Technology Symposium, American Defense Preparedness Association, June 1994.

Flinchbaugh, B., and T. Olson, "Autonomous Video Surveillance", presented at 25th AIPR Workshop: Emerging Applications of Computer Vision Washington, DC, October 1996.

Relevant Images

Here are pictures and brief descriptions of TI event detection and recognition capabilities.

Related Sites

DARPA Image Understanding Program