15-494/694 Cognitive Robotics: Occupancy Grid Project

Megan Roche and JD Parker

Goal of Project

We wanted to create an occupancy grid for Cozmo, where it can map out its surroundings. We want to do this by having Cozmo be able to distinguish what surface it's on, and whether or not there is an obstacle in its way or around him.

We broke this up into two broad steps: detecting if something is an obstacle and creating a grid and adding a patch of land once we know its status.

Obstacle Detection

We decided to try for a simpler K-Nearest Neighbors approach before attempting something more complex such as a convolutional neural network. We knew from our professor's experimentation with this topic that using RGB values to test similarity was not reliable- the lighting affected those values too much. We chose to use HSB values- hue, saturation, and brightness- so that we could account for hue and saturation in our comparison, but leave out brightness- which is easily subject to changes of lighting.

We created a histogram of each of those values, and plugged it into K-Nearest Neighbors, finding that we had decent accuracy when it sought to compare two types of surfaces- a floor versus a table.

We also decided to use the filter responses from Sobel Filters so we could get more information about the texture of what we were looking at- if we were looking at wood that had grain we'd see that, but if we saw some large item take up half the screen, it'd disrupt the texture heavily.

After putting these steps together, we had an accuracy rate of over 95% when comparing two different types of surfaces, and needed to decide how to differentiate between tabletops and obstacles.

We used a concept called adaptive thresholding, where we got the average distance value and average standard deviation value of all of the test images in KNN, and when something was far outside of that range for either surfaces, we called it an obstacle.

Now we have our user take many images of their own table area, and have our own image set saved, that of a blue yoga mat- which is used because we need two settings to make K-Nearest Neighbors work, and the yoga mat is a very distinctive pattern and color.

Image Patches:

This is an image patch of a typical floor. This is an image patch of the blue yoga mat. This is an image patch containing an obstacle.

Occupancy Grid

Now that we were confident in the KNN being able to correctly identify obstacles from natural terrain, we started making the grid to present this.

We assume that Cozmo always begins at the center of said grid when it is starting up. From there, we take his position, convert it to world coordinates, and adjust it so that (0,0) in world coordinates is in the center of the grid.

Our obstacles are the color gray on our image, the spaces with free area are green, and the areas labeled as the yoga mat are blue.

We have two options for the robot- either the user can control where he goes, and he takes photos as he goes on his own, or the robot can roam on its own and naturally turn whenever it sees an obstacle in its way.

While roaming, he turns a slight amount when encountering an obstacle before retaking an image. If Cozmo does not see anything blocking his path, he will move forward instead. He will also perform a 150 degree turn when coming to the edge of the table, and perform a 180 degree turn when coming to the edge of our occupancy grid bounds.

Grid in Action

An obstacle is in the way of the Cozmo. The grid shows the spot as being gray- meaning that there is an object there (working as expected). The Cozmo is at the very edge of the table here. The Cozmo, on its own, knew it was an edge and to do a 150 degree turn to avoid falling. This is the occupancy grid sometime after the edge, to show its path before and after.

Resources:

Our Code: Presentation of Our Results: