For generating realistic images of a three dimensional object, two aspects of information are fundamental: the object's shape (geometric information) and reflectance properties (photometric information) such as color and specularity. Significant improvements have been achieved in computer graphics hardware and image rendering algorithms. However, it is still often the case that three dimensional models are created manually by users. That input process is normally time-consuming and can be a bottle neck for realistic image synthesis. To overcome this limitation, we are developing a new approach to obtain photometric information as well as geometric information of an object model automatically by observing a real object. We believe our approach to be useful for many practical applications.
Previously, we developed a method to analyse a sequence of color images taken under a moving light source [1]. The method is used in our current work to create real object models. A series of color images are examined in a four dimensional space whose axes are the RGB color axes and one geometrical axis. The method is called goniochromatic space analysis (GSA). The significance of the GSA lies in its ability to analyse the change of image color for different geometric relationships between the viewing direction, illumination direction, and surface normal. This allows us to estimate the photometric information of the object model. Conceptually, the two reflection components from the dichromatic reflection model, the diffuse and specular reflection components, form two subspaces in the goniochromatic color space. By using GSA, these two components can be extracted at each pixel in the image locally. Based on the separation result, shape and reflectance parameters of the object can be estimated in a robust manner.
In our current work [2], GSA has been further extended to handle a color image sequence of a moving object. Unlike the previously used GSA, a range image sequence is used for modelling real objects as well as a color image sequence. First, a sequence of range and color images of a real object is measured by rotating the object on a PUMA industrial robot arm. Therefore, the object motion is known while viewing and illumination directions are fixed. Then, the object shape is obtained as a collection of triangular patches by merging multiple range images. Secondly, a sequence of color images are mapped onto the recovered shape. As a result, we can determine observed color changes through the image sequence for all triangular patches of the object surface. The observed color sequence is separated into the diffuse reflection component and the specular reflection component by the modified GSA. Subsequently, the parameters of a reflection function used in our analysis are estimated reliably for the diffuse and specular reflection components. The reflection model used here is described as a linear combination of the Lambertian model and the Torrance-Sparrow model.
Using the recovered object shape and estimated reflectance parameters associated with each triangular patch, realistic images of the real object can synthesized under arbitrary illumination conditions. The proposed approach has been applied to real range and color images of a plastic object. The effectiveness of our approach has been successfully demonstrated by synthesizing images of the object. A desktop virtual reality system was developed as a part of the demonstration. The system allows users to view and manipulate virtual objects interactively in 3D using a CyberGlove device.