15-494/694 Cognitive Robotics:
2019 Final Project Presentations

A Neural Net Line Follower That Traverses Intersections

• Tomasz Swierzewski and Haine Cho. Slides.
• Train a convolutional neural network to allow Cozmo to follow lines on the floor made from tape.
• Basic network follows a straight or gently curving line.
• Another network detects T-intersections, and the user can tell the robot to take either the left or right branch.
• Uses the GPU board to process 320x240 gray scale images through the two neural nets in real time.
• Watch the YouTube video.

A Self-Driving Car That Obeys Traffic Signals

• Xinze "Cindy" Deng and Bolaji Bankole. Slides.
• Train a neural network to allow the robot to follow lines on the floor made from tape.
• Basic network follows a straight or gently curving line.
• Another network detects a traffic light that can be red or green. The robot stops at red lights and waits for a green.
• Uses the GPU board to process 160x240 color images through the two neural nets in real time.
• Watch the YouTube videos: track following and traffic light detection.

• Thomas Matson and Rui Yuan Shum. Slides.
• Use a pre-trained deep neural network (VGG16) as the hidden layer for a new, rapid feature learner.
• Using the TensorFlow-GPU and Keras packages.
• Teach Cozmo to recognize hand gestures using the GPU.
• Two types of gestures: pointing in four directions, and four sign language letters.
• Watch the YouTube videos: training on pointing, testing, sign language letters.

Transfer Learning 2: Gesture Recognition

• Amanda Steiner and Ananya Rao. Slides.
• Use a pre-trained deep neural network (VGG) as the hidden layer for a new, rapid feature learner.
• Using the PyTorch package.
• Use the GPU to teach Cozmo to recognize when the hand is making a pointing gesture.
• Recover pointing direction using a marker on the fingertip.
• Watch the video.

Reading Printed Signs

• Dylan Vanmali. Slides.
• Locate printed text on signs in Cozmo's camera image.
• Use a pre-trained deep neural network to read the text.
• Perform the action indicated by the text, or speak the result.

Cozmo Forklifts Things

• Zachary Snow and Ian Lo. Slides.
• Write code to visually detect and dock with a pallet using the Hexnub lifting kit attachment.
• Watch the video and see this final report.

Vector Robot Support

• Aravind Vadali and Fengzhi "Steven" Li. Slides.
• Vector is a more advanced robot than Cozmo, with a similar SDK.
• Vector includes a higher resolution camera with a wider field of view, a laser rangefinger, a capacitive touch sensor on the head, and a microphone array.
• The Vector SDK differs from the Cozmo SDK in a variety of ways, including the event handling mechanism.
• This project will adapt the cozmo-tools package to support Vector.
• See this results writeup for documentation.

• Nikhil Swaminathan Writeup.
• Navigate between rooms using the inter-room path planner.
• Press the elevator call button.
• DreamHouse videos: 1, 2, 3. Path planning videos: 1, 2.