Faculty: Katsushi Ikeuchi

The Task-Oriented Vision Laboratory investigates the theory of how the requirements for accomplishing a visual task determine the optimal architecture of a vision system. The issues that are considered are object representation, selection of vision modules and control schemata. Traditionally, computer vision research has emphasized investigating each vision module in a general and isolated condition. Such research efforts often generate unrealistic solutions from ill-defined assumptions. In contrast, we consider a vision system as a whole and emphasize the research on interactions among modules as well as the research on each vision module.

Vision Algorithm Compiler (VAC)

  • Mark Wheeler
  • Takeshi Shakunaga
  • Taku Yamazaki
    • Currently, the majority of vision programs are generated by hand-coding methods. Programmers first carefully consider vision tasks, and then convert their knowledge into vision programs that will accomplish these tasks. Although the completed programs are efficient and effective, such methods are extremely time consuming, expensive, and labor-intensive. To solve this problem, we are developing techniques for the VACs which convert object and sensor models into vision programs automatically. In place of human programmers, a VAC examines both available object features (using object and sensor models) and available vision modules for low-level processing, then chooses the optimal set of features and vision modules to generate the optimal vision program for a given task. Current research topics include VACs for Automatic Target Recognition (ATR) and a Parallel Vision Algorithm Compiler (PVAC) for iWarp parallel computers.

    Assembly Plan from Observation

  • Sing Bing Kang
  • George Paul
  • Jun Miura

    • Currently, the robot programming is done either by manual programming or by the "teach-by-showing" method using a teach pendant. Both methods have been found to have several drawbacks. We have been developing a novel method to program a robot, the assembly-plan-from-observation (APO) method. The APO system observes a human performing an assembly task, understands it, and generates a robot program to achieve that same assembly task. Current research topics include how to: choose meaningful snapshots from an image sequence; infer what kind of operation is performed from given snapshots; determine where and from which direction a gripper should grasp an object to achieve a task; and determine a path to transfer an object while avoiding collision with other objects from observation.

    Representative publications

    Physics-Based Inspection

  • Fred Solomon
  • Yoichi Sato

    • Surface inspection is a vision application area that has great industrial significance. Inspection of machined parts, solder joint surfaces, and plastic sheets are just a few examples of industrial tasks too laborious and time consuming for humans. We are developing basic theories for surface inspection as well as complete end-to-end inspection systems. Our theories are ready to be applied to real inspection systems based on rigorous physics and mathematical theories. Current research topic includes: basic surface reflectance mechanisms, temporal-color space representations that relate how illumination variances effect apparent surface colors, and sensing theories for determining the optimal illumination and camera configurations under a given inspection specification. We are also developing complete inspection systems based on the developed theories: photometric sampler for silicone wafer inspection; range-brightness fusion system for solder join inspection; and 4-photometric system for plastic sheet inspection.

    Modeling from Reality

  • Harry Shum
  • Prem Janardhan

    • Solid modeling is a useful tool for: representing virtual environments for virtual reality systems; representing real environments for robot programming; and modeling real objects for object recognition. Currently, these solid models are constructed by human operators. Often applications provide real objects to be modeled. If we can generate solid models from given real objects, we can reduce the effort and cost of constructing solid models. We are developing techniques to construct solid models from observation of real objects. Research projects include: building a virtual Carnegie Mellon Wean Hall corridor using a movable range finder cart; building human busts from a sequence of images; and building appearance models for vision algorithm compiler.