Grasp recognition

Grasp recognition is accomplished using the grasp taxonomy, which is based on the contact web.

While the non-volar grasp can be recognized directly from its branch of taxonomy, volar grasp recognition is not as easy, due to the high degree of contact between the hand and the object. It turns out that using higher-level descriptions of the grasp facilitates the recognition of a volar grasp.

The grasp taxonomy is augmented with higher-level grasp abstraction concepts such as the virtual finger and opposition space. This is done for the following reasons:

They enable the operator to easily check the correctness of the system recognition of the human grasp.
They enable the volar grasp to be further identified.
They provide an intermediate abstract representation which facilitate the mapping from the human hand to that of the manipulator.

In order to recognize the volar grasp, we have to first map the real fingers that are touching the object (i.e., "active") to virtual fingers.

Mapping real fingers to virtual fingers

Associated with each contact in the contact web is its location and the object normal at the contact. The object normal is used as an approximation of the force acting at that contact. The mapping from real fingers to virtual fingers is done by comparing the normals between different fingers. For each virtual finger, the comparison yields an index called the cohesive index (between 0 and 1) which indicate the degree to which all the real fingers in that virtual finger act in a similar manner. If all the fingers act exactly in the same direction, this index would be 1. The grasp cohesive index is a global measure of how all the real fingers act in the similar manner in each of their respective virtual fingers. It is in essence the geometric mean of all the cohesive indices of the virtual fingers. The mapping is automatically done by exhausive consider all possible configurations and choosing one that results in the maximum grasp cohesive index.

The conceptual picture of mapping of real fingers to virtual fingers is shown below.

By performing the mapping of real fingers to virtual fingers, we can then construct a grasp abstraction hierarchy that involves, at the lowest level, positional information of finger segments, to the highest level, namely the identity of the grasp itself.

Volar grasp recognition

It turns out, based on empirical results, that the grasp cohesive index (obtained from the real finger to virtual finger mapping) can be used to discriminate between different types of volar grasps. While the cylindrical and type 2 "coal-hammer" grasps cannot be differentiated based on this index, they can be identified based on the degree of thumb abduction.

Note: The cylindrical grasp is one in which the thumb is adducted (i.e., the thumb is lying in the plane of the palm). A "coal-hammer" grasp is identified by the high degree of abduction (i.e., the thumb is positioned almost perpendicularly to the plane of the palm). The labels of types 1 and 2 associated with the "coal-hammer" grasp are used to indicate contact of the thumb with the object (with type 1 associated with the non-contact case).

References

S.B. Kang and K. Ikeuchi, "Grasp recognition using the contact web," Proc. IEEE/RSJ Int'l Conf. on Intelligent Robots and Systems, Raleigh, NC, July 1992.
S.B. Kang and K. Ikeuchi, A framework for recognizing grasps, Tech. Rep. CMU-RI-TR-91-24, Carnegie Mellon University, Nov. 1991.
S.B. Kang and K. Ikeuchi, "Toward automatic robot instruction from perception: Recognizing a grasp from observation," IEEE Int'l Journal of Robotics and Automation, vol. 9, no. 4, Aug. 1993.

Return to Research interests