We created and implemented a classification model for Cozmo to differentiate between floor and non-floor areas, such as obstacles and edges of the floor, into an occupancy grid.
We first made a simple program to collect patches of the surface Cozmo will be exploring to be referenced as sample floor patches. This program allows Cozmo to adapt to any new surface it tries to explore.
Using local binary patterns and hue information from Cozmo's camera image of the surface, we developed a classification method to differentiate between floor and non-floor image patches based on color and texture. Each patch used for classification is 40 by 40 pixels, though this parameter can be easily changed. These classified patches are then projected onto Cozmo's world coordinates and illustrated in an occupancy grid, showing unexplored areas, known floor and non-floor regions, and Cozmo's path.
The classifier and occupancy grid code is written in separate library files, allowing the whole system to be easily plugged into any FSM file. The patch collection program is a standalone FSM that can be easily run on new surfaces.
Slides for our presentation, with more information on the methods, are here.
Our code is on this GitHub page.