We have developed a "visual odometer" which can visually lock-on to ground objects and sense relative helicopter position in real time. As the tracked objects leave the field of view, the odometer selects and tracks new objects to continue sensing helicopter motion.

The odometer tracks pairs of image templates (32x32) using a custom-made TI C40-based vision system. Pairs of templates are tracked to measure image rotation and range to properly rotate and scale templates before matching. The system supports six DSPs each tracking one pair of templates at 60 Hz with 24 ms latency. Images are filtered by an 8x8 convolution ASIC (GEC Plessey) before matching. The odometer was tested indoors (mpeg) before outdoor flight tests. The visual odometer using a pair of b/w video cameras aided by a set of inexpensive angular sensors (Gyration gyro-engines and KVH digital compass) were the only sensors employed to stabilize and maneuver (less than 15 mph) small to mid-sized model RC helicopters (see mpegs on the right).

We have developed a state estimator which fuses data from an Inertial Measurement Unit (IMU), a GPS receiver, and a compass for accurate helicopter position estimation. The IMU, Litton LN-200, is composed of 3-axis Silicon accelerometers and angular rate sensors. The GPS receiver, NovAtel MillenRt2, is a dual-frequency carrier-phase unit capable of 2 cm accuracy positioning using nearby (less than 20 miles) ground differential correction stations. The compass, KVH Industries, is a flux-gate sensor with a toroidal element. The data from these sensors is fused by a 12th order Kalman Filter which keeps track of latitude, longitude, height, 3-axis velocities, roll, pitch, yaw, and accelerometer biases (Gauss-Markov model).

An on-board classical (feed-forward PD) controller controls the helicopter on smooth (quintic spline) trajectories planned by a flight path editor. As shown in the above figure, the flight path editor's console shows the planned trajectory between goal points supplied by a human operator. The helicopter can be programmed to maintain its heading tangent to the path or always orient itself towards a point in space. The console also displays 3D range data collected by the mapping system (see below) to help in selecting goal points. The auto trajectory mpeg (right) shows the helicopter executing a circular path, planned using the flight path editor, while pointing to a parked car.

We have developed a mapping system which integrates a laser rangefinder (Reigl), custom synchronization hardware, and our inertial state estimator on-board an autonomous helicopter. The systems scans the environment line by line using a spinning mirror during helicopter flight. Each line, consisting of an array of range measurements, is tagged with helicopter state and is registered in memory to incrementally build a 3D map. You can see the system in action in the aerial mapping video on the right.

We have developed an color discriminator which can detect objects based on their color. Built by high-speed digitally controllable analog hardware, the discriminator can be configured at NTSC field rate (60 Hz) to look for as many RGB color combination as necessary in sequential image fields. The discriminator normalizes RGB intensity levels, to eliminate the effects of lighting, determines the difference between each pixel color from the target color, and penalizes each pixel based on this distance. Most recently the discriminator was used to pick up an orange disk from a barrel on the ground for the 1997 Unmanned Aerial Robotics Competition. Although, the system did not work properly at the contest, you can see successful pickups in the videos (see mpegs to the right) shot prior to the contest. The pickup system tracked a blue magnet and aligned it with the orange disk as the helicopter descended to the estimated range to the disk. Helicopter range to the disk was measured by triangulation.

We have also developed a template-based detector to locate objects based on appearance. The detector locates image regions which resemble a picture of the object to find a potential match. Because searching every image for a the object's picture or template in any orientation requires enormous computational power, the detector exploits principal component methods to reduce its work load. Rotated templates look very similar and are typically highly correlated. The detector analyzes each template in every orientation and determines a small set of principal templates which would yield accurate matches without matching every possible templates angle. The above figure demonstrates this by comparing the 25 radioactive hazard symbol templates, rotated in 15 degree intervals, to the 8 principal templates used for matching.