Assembly Plan from Observation

Currently, the robot programming is done either by manual programming or by the "teach-by-showing" method using a teach pendant or using automatic programming. All these methods have been found to have several drawbacks. We have been developing a novel method to program a robot, the assembly-plan-from-observation (APO) method. The APO system observes a human performing an assembly task, understands it, and generates a robot program to achieve that same assembly task.

The people working on the APO project are

  1. Katsushi Ikeuchi
  2. Sing Bing Kang
  3. George Vikram Paul
  4. Jun Miura
  5. Santiago Conant
  6. Yunde Jiar
  7. Hiroshi Kimura
Contact Bibliography

The APO project is headed by Katsushi Ikeuchi

Sing Bing Kang 's work dealt with the following topics.

  • Choosing meaningful snapshots from an image sequence;
  • Infering what kind of operation is performed from given snapshots
  • Determining where and from which direction a gripper should grasp an object to achieve a task;
  • Determining a path to transfer an object while avoiding collision with other objects from observation.

    • Representative publications

    George V Paul 's work deals with the following topics.

  • Partitioning of Contact State Space using the theory of Polyhedral Convex Cones.
  • Modelling Planar Assembly Paths from Observation.
  • Modelling Assembly Tasks: Representation and Recognition.

    • The following illustrates the working of the APO system for a planar peg in hole assembly task.

    Task Observation (mpeg video)

    NOTE: The apparent movement of the hole is due to the fact that the coordinates about which I zoomed the pictures was not consistent. Only 9 out of 64 scenes are shown here.

    Task Modelling (iros-96.ps 0.5 Meg)

    Task Execution (mpeg video)

    Representative publications

    JunMiura worked on developing Task-Oriented Generation of Visual Sensing Strategies in Assembly Tasks.

    Santiago Conant is working on the execution module of the APO.

    Yunde Jiar is working on the grasping module of the APO.

    Hiroshi Kimura is also working on the grasping module of the APO.

    Contact

    This page is still under construction (last update Apr 14 1997 by gvp@ri.cmu.edu)

    Please send comments to gvp@ri.cmu.edu

    Bibliography

    gvp@ri.cmu.edu is working on it.

    Keywords: Assembly planning, Robot Task level Programming, Task Modeling, Grasp Recognition, Dual Quaternions, Planar Quaternions, Contact States, Robot Programming by Demonstration, Assembly Task Modeling, Assembly Task Segmentation, Modeling Human Assembly Tasks from Observation, Polyhedral Convex Cones, C-Surface, C-Obstacle Features, C-features, Application of Real-time Stereo.