Vision and Mobile Robotics Laboratory | Software
Home | Members | Projects | Publications | Software | Videos Internal

Library Building Tutorial

The purpose of this tutorial is to explain how to build model libraries for multi-object recognition using spin-images. Multiple programs from Mesh Toolbox are applied during library building. The theoretical details of library building are given in Chapter 6 of Spin-Images: A Representation for 3-D Surface Matching.

To build a library, first the spin-images for each model in the library must be made. Using the same same spin-image generation parameters for all of the models makes library building easier. Eigen-spin-image stacks are then computed from the model stacks. Each spin-image is then projected onto its significant eigen-spin-images to create model tuples.

To create library tuples, the eigen-spin-images from the stack made by concatenating the stacks from all models in the library are computed. All of the model spin-images are then projected ont the library eigen-spin-images to get the library tuples. The model meshes, stacks, eigen-stacks, model tuples and library tuples constitute the model library. The location of all of the files in a model library are indicated by a library file. Since library building is an intricate process, please read this document carefully.

Step 1: Generate Models

Before the model library can be built, surface mesh representations of all of the models in the library must be obtained. Models can be built by merging mutiple range images or by creating CAD models of objects which can be tesselated using finite element software. Once the meshes are obtained, it may be neccessary to downsample them to save storage space and execution time during recognition. Suppose the model library has three models robot, duckie and car, which all have very high resolution. To reduce the resolution of the meshes so that each model has 10000 faces, call the following commands. They are explained imore detail in the Mesh Preprocessing tutorial .

Step 2: Compute Spin-Images

The next step is to compute the spin-image stacks for all of the models in the library using the same spin-image generation parameters. For object recognition in cluttered scenes, the following parameters work well

Step 3: Compute Model Eigen-Spin-Image Stacks

The eigen-spin-images for each model need to be computed for model spin-image compression.

Step 4: Create Model Tuples

Multi-object recognition with compression can conceivably work when each model has a different number of significant eigen-spin-images. However, in the current implementation of RecognizeMesh, each model is assumed to have the same number of significant eigen-spin-images. Therefore, some averaging of the number of significant eigen-spin-images needs to occur. To decide on a good number, use CalcStackDim with the same reconstruction level on all of the models, and pick some average of the resulting numbers of significant eigen-spin-images.

Suppose that an average number of significant spin-images turns out to be 10. The next step is to project each model spin-image onto the 10 most significant eigen-spin-images for the model to get a vector array of model tuples.

Steps 1 to 4 are the same for all models, so a model independent script that performs the creation of model stacks, eigen-stacks and vector arrays could be made. The next step describe the process for creating the library tuples.

Step 5: Concatenate Stacks

The model stacks and model resampled stacks need to be appended to each other to make the library stacks. Lets assume the name of the library is toys then the following sequence of commands will make the resampled library stack for computation of library eigen-spin-images.

And the following commands will make the library stack for makeing library tuples.

Step 6: Compute Library Eigen-Spin-Image Stack

The library eigen-spin-images are computed from the resampled stack to insure that each model has equal weight.

Step 7: Create Library Tuples

To create the library tuples, first determine the number of significant library eigen-spin-images.

Suppose the number of significant library eigen-spin-images is 20. Project the model spin-images stored in the library stack onto the library eigen-spin-images to get the library tuples.

Step 8: Create the Library File

All of the computation is carried out in Steps 1 to 9. Once the model is built, place all of the meshes in one directory, all of the model stacks in one directory, all of the model eigen-stacks in one directory and all of the model vector arrays in one directory. RecognizeMesh determines the models and location of files in the library by parsing a library file. A possible library file toys.lf for the library created in this tutorial could have the form:

The directories in the library file are the directories where the various files of the model library are stored; the key words should be self explanatory. NOTE: The order of entries in the library file is fixed. Also the models in the library must be listed in the library file in the order in which they were concatenated to make the model library. The final constraint is that the file suffixes .wrl, .stack .estack and .va be used to indicate meshes, stack, eigen-stacks and vector arrays, respectively.

The model library is now ready for object recognition. A typical object recognition command using RecognizeMesh could be

up

The VMR Lab is part of the Vision and Autonomous Systems Center within the Robotics Institute in the School of Computer Science, Carnegie Mellon University.