Fall 98
 

Thu 12/3/98 at 5pm in 7220	John Langford	``Probabilistic Inequalities ''  Fun ways to play with them.
Thu 11/19/98 at 5pm in 7220	Daniel Nikovski	``Learning Bayes Nets with hidden state ''  How to learn a bayes net with hidden state.
Thu 11/12/98 at 5pm in 7220	Daniel Nikovski	``Bayes Nets with hidden state ''  Today's talk will be the second part of the tutorial on reasoning and learning with Bayesian nets. Last week Dimitris talked about prior distributions and learning parameters and structure when all variables are completely observable. I will start out with what was left of last week's agenda - the message-passing and join-tree algorithms, and then go on in the following directions: learning with continuous variables; learning parameters when some variables are unobservable; learning structure when some variables are unobservable; learning temporal Bayesian nets. I will also show some results that demonstrate the behavior of some of these algorithms on learning temporal Bayesian nets with mixed discrete/continuous variables, some of which are not observable.
Thu 11/5/98 at 5pm in 7220	Dimitris Margaritas	``Bayes Nets ''  Learning Bayes nets with fully observable variables as well as present an algorithm for exact Bayesian inference on a BN.
Thu 10/29/98 at 5pm in 7220	Nicholas Roy	``Coastal Navigation ''  I'll be talking about motion planner developed for Minerva, the all-singing, all-dancing, tour-guide robot that operated in the Smithsonian this summer. Many of the challenges that arose from the Minerva project were a result of the environment; many large, open spaces and way too many little kids resulted in a uncomfortably high probability that Minerva might periodically get lost. Thus, coastal navigation was born, to reduce the likelihood that Minerva would lose her positional knowledge and emit an irritating whine, "I'm confused". For those distressed by the non-technical nature of this abstract, I say "Partially Observable Markov Decision Processes". http://gs135.sp.cs.cmu.edu/papers/icra99.ps.gz
Thu 10/22/98 at 5pm in 7220	John Langford	``Microchoice Learning Bounds ''  I will present bounds which take advantage of extra degrees of freedom to provide bounds which are better for some hypothesis than for others. These bounds can be applied to the structure of many existing learning algorithms in order (sometimes) justify tigheter bounds than the standard PAC bounds.
Thu 10/15/98 at 5pm in 7220	Daniel Nikovski	``System Identification for Reinforcement Learning ''  I will try to give a short tutorial on system identification (SI) algorithms, with an emphasis on their possible use in reinforcement learning. SI is a mature discipline that in fact subsumes connectionist modeling, without being limited to it only. I will review the classical SI techniques such as ARIMA and ARMAX, their recursive on-line modifications, the instrumental variable method, and then discuss in more detail subspace-based methods. The example application of subspace-based system identification will be on learning local visual models for robotic soccer.
Thu 10/8/98 at noon in 4603	Steffen Gutmann	``The CS Freiburg Robotic Soccer Team ''  Robotic soccer is a challenging research domain because problems in robotics, artificial intelligence, multi-agent systems and real-time reasoning have to be solved in order to create a successful team of robotic soccer players. This talk presents the architecture and key components of the CS Freiburg team. The focus is set on the self-localization and object recognition method based on using laser range finders and the integration of all this information into a common team world model. Using the explicit model of the environment built by these components, path planning, simple ball handling skills and basic multi-agent cooperation have been implemented. The talk concludes by pointing out the key components of the CS Freiburg team that led to the success at RoboCup'98, where CS Freiburg won the competition in the middle size league. Further information at: http://www.informatik.uni-freiburg.de/~gutmann/ and http://www.informatik.uni-freiburg.de/~ki/forschung/robocup.html

Spring 98
 
 

Thu 5/28/98	Tom Mitchell	``Co-training: Learning from Labelled and Unlabelled Data''  Brain teaser: Suppose you have the task of learning to classify web page (e.g., as academic course pages, student home pages, etc.). I give you a small set of labeled training example pages (positive and negative). I also give you a large number of unlabelled pages (e.g., 300,000,000 of them). Question: what learning algorithm can you think of that will use the unlabeled pages to boost accuracy of your learned classifier? That's what I'll talk about. If you'd like to check it out in advance, then see the paper on Co-Training at http://www.cs.cmu.edu/afs/cs.cmu.edu/project/theo-11/www/wwkb/colt98_final.ps
Thu 5/21/98	Cancelled	Cancelled
Thu 5/14/98	Peter Stone	``Team-Partitioned, Opaque-Transition Reinforcement Learning (TPOT-RL)''  In this talk, I present a novel multi-agent learning paradigm called team-partitioned, opaque-transition reinforcement learning (TPOT-RL). TPOT-RL introduces the concept of using action-dependent features to generalize the state space. In our work, we use a learned action-dependent feature space. TPOT-RL is an effective technique to allow a team of agents to learn to cooperate towards the achievement of a specific goal. It is an adaptation of traditional RL methods that is applicable in complex, non-Markovian, multi-agent domains with large state spaces and limited training opportunities. Multi-agent scenarios are opaque-transition, as team members are not always in full communication with one another and adversaries may affect the environment. Hence, each learner cannot rely on having knowledge of future state transitions after acting in the world. TPOT-RL enables teams of agents to learn effective policies with very few training examples even in the face of a large state space with large amounts of hidden state. The main responsible features are: dividing the learning task among team members, using a very coarse, action-dependent feature space, and allowing agents to gather reinforcement directly from observation of the environment. TPOT-RL is fully implemented and has been tested in the robotic soccer domain, a complex, multi-agent framework. This talk presents the algorithmic details of TPOT-RL as well as empirical results demonstrating the effectiveness of the developed multi-agent learning approach with learned features. There is a link to a paper of the same title at http://www.cs.cmu.edu/~pstone/pstone-papers.html.
Thu 5/7/98	Reid Simmons	``Task-Level Robot Control: Languages and Tools''  Years of experience with mobile robots has taught the research community a few things about what architectures, data structures and algorithms are useful for robot systems. Unfortunately, in many cases these insights remain beyond the reach of most roboticists, mainly due to the difficulty of understanding and re-implementing the solutions. To help promulgate such results, we advocate the development of general robot-programming languages and support tools that embed concepts that robotics researchers have found useful. In particular, we are working on a "Task Definition Language" that contains explicit support for robot executive functions (concurrency, task decomposition, resource management, monitoring and exception handling). This talk will introduce the language and the design and analysis tools we are building to support the language.
Thu 4/30/98	Chuck Rosenberg	``Behavior Based Robot Control''  I will discuss two behavior controlled (à la Brooks) robot systems I worked on at IS Robotics, Inc. The first is named Kaa. Kaa is a small autonomous snake-like robot. It has twelve redundant degrees of freedom, all a single plane. A behavior control system implemented basic mobility and grasping algorithms for this system. The second robot I will discuss is named IT. The IT robot consists of a vaguely human shaped head with an expressive face and is meant to interact with people via simulated emotional responses. When specific changes to the environment are sensed, the behavior control system changes the system's internal "emotional state" which in turn changes IT's behavior. For example, when it got dark, IT would become "sad" and eventually "cry". For more information check out: http://www.cs.cmu.edu/~chuck/robotpg/kaapg/ http://www.cs.cmu.edu/~chuck/robotpg/itpg/ The corresponding paper is: Rajiv Desai, Charles Rosenberg and Joseph Jones, "Kaa: An Autonomous Serpentine Robot Utilizes Behavior Control", IROS '95, vol. 3, pp. 250-255, August 1995.
Thu 4/23/98	Daniel Nikovski	``Visual Servo Control for Robot Navigation''  I will present Greg Hager's work with Chris Rasmussen on robot navigation using visual servoing on image sequences. Their approach to robot localization is based on the idea that a location is defined by the visual features seen by the robot. Scenes with the same features are grouped in the same equivalence class and this division into classes is used to build a map of the environment in the form of a graph. Navigating between two locations is performed by planning a sequence of node traversals in the graph and executing the traversals by means of visual servoing and handoffs between segments of the trajectory. I will discuss the features used for servo control, and time permitting, other types and applications of visual servoing. The corresponding paper is: Rasmussen, C. and Hager, G. D. (1996). Robot navigation using image sequences, Proceedings of the AAAI Conference on Artificial Intelligence, pp. 938-943, 1996. ftp://ftp.cs.yale.edu/pub/hager/papers/AAAI96.ps.Z A general introduction to visual servoing is available online too: Hutchinson, S., Hager, G.D., and Corke, P. (1996). A tutorial on visual servo control. IEEE Transactions on Robotics and Automation, 12(5) pp. 651-670, 1996. ftp://ftp.cs.yale.edu/pub/hager/tutorial.ps.gz
Thu 4/16/98	Jamieson Schulte	``Teleo-Reactive Agent Control''  I will describe the Teleo-Reactive (T-R) agent control formalism developed by Nils Nilsson at the Stanford University Robotics Laboratory. A T-R tree is a mapping from states to actions that resembles a control algorithm implemented in an electronic circuit in that the state is continually evaluated based on sensor inputs. With a careful ordering of states within the tree, useful sequences of actions are possible despite the low-level state-action mapping and subsumptive architecture. To ensure that there is some robot learning content, I will summarize some progress that has been made in supervised learning of T-R sequences. After presenting T-R trees and some experimental results, I will encourage group discussion about the applicability of this approach to different types of robot tasks. Much of my talk will be based on the paper: Nilsson, N. (1994) "Teleo-Reactive Programs for Agent Control", Journal of Artificial Intelligence Research, Volume 1, pages 139-158.
Thu 4/9/98	Joseph O'Sullivan	``Specifing transferable knowledge in lifelong learning agents''  I'll shall begin with a short summary of results from my research into transfer mechanisms for lifelong learning agents. This should provide a basis for discussing my current research direction -- improving lifelong learning agents by explicitly specifying knowledge to be transfered. Consider that when you are asked to learn a task, you bring to bear specific knowledge related to the type of task being learned. As a simple example, if I asked you to learn doors in Wean Hall, you would already know to discount changes in light. The questions I'd like to raise are, what are such types of knowledge, and how can they be used by a lifelong learning agent.
Thu 4/2/98	Wolfram Burgard	``Integrating Global Position Estimation and Position Tracking for Mobile Robots: The Dynamic Markov Localization Approach''  Localization is one of the fundamental problems of mobile robots. In order to efficiently perform useful tasks such as office delivery, mobile robots must know their position in their environment. Existing approaches can be distinguished according to the type of localization problem they are designed to solve. Tracking techniques aim at monitoring the robot's position. They assume that the position is initially known and that the robot does not loose track of its position. Global localization techniques, on the other hand, are able to estimate the robot's position starting with complete uncertainty. In this paper we present the dynamic Markov localization technique as a uniform approach to position estimation, which is able to (1) globally estimate the position of the robot, (2) to efficiently track its position whenever the robot's certainty is high, and (3) to detect and recover from localization failures. The approach has been implemented and intensively tested in real-world environments. We present several experiments illustrating the strength of our method.
Thu 3/19/98	John Langford	``Probabilistic Graphplan''  I'll talk about the (graph) representation which graphplan uses. Then I'll talk about some probabilistic extensions Avrim and I have worked out for graphplan which allow planning in MDP domains. I'll come up with some examples to illustrate the usage of graphplan.
Thu 3/5/98	Frank Dellaert	``The Kalman Filter, Part II: Kalman Returns''  This week I will continue, as requested, the introduction to the Kalman filter. I hope to have time to discuss at least one real world example in some detail.
Thu 2/26/98	Frank Dellaert	``That Magical Tool: The Kalman Filter''  Taking the hint from last time, I'll give an overview of the magical tool named the Kalman filter. I will illustrate its usefulness with three scenarios where I have applied it: Dead Reckoning for Xavier Never published: this was my 'warm-up' exercise in using a Kalman filter. Car Tracking How the Kalman filter interacts with simple vision algorithms v Super-Resolved Texture Tracking My latest and exciting research on using KF and texture mapping to track and super-resolve patterns on planar surfaces.
Thu 2/19/98	Nick Roy	``Online Self-Calibration For Mobile Robots''  I will be talking about work that Sebastian and I have been doing on developing a probabilistic model of the kinematics of a mobile robot. The goal is to parameterise the motion model in order to correct position estimates given by odometry, or dead-reckoning. In contrast to previous approaches, which require explicit measurements of actual motion when calibrating a robot, the algorithm proposed here uses the robot's sensors to automatically calibrate the robot as it operates. I'll discuss the formalism, show some experimental results, and be fairly laid back about the whole thing.

Fall 97
 

Thu 11/20/97	Chuck Rosenberg	``Special Purpose GP's for Evolving Artificial Neural Networks''  I will focus on describing the work of Frederic Gruau. His research has centered on using genetic search to generate neural network solutions to specific problems instead of using more traditional techniques like backpropagation. This, in and of itself, is not new. What is unique about his work is that the genetic search evolves a special purpose genetic program which when run, outputs a neural network. The evolved genetic program encodes a developmental process, very loosely analogous to a biological developmental process, which can construct a neural network including its weights and general architecture. This can be contrasted with a genetic algorithm approach which might encode the connection weights in a network as a matrix of values.  His claim is that by the choice of genetic program operators which allow for the construction of neural networks with highly regular architectures, solutions to problems can be found faster than other systems which evolve neural networks. Indeed, in the case of the solutions for the problems Gruau presents in his papers, the neural networks are quite regular in their architecture.  He has successfully used this system to evolve solutions which can solve n-bit parity and n-bit symmetry in the general case as well as controllers for a simulated six legged walking robot and a real eight legged walking robot. He has also experimented with the Baldwin effect and Lamarkian learning.  The three papers I will discuss are:  F. Gruau. Automatic Definition of Modular Neural Networks. Adaptive Behavior V3N2, pages 151-183, 1995. MIT press. Available online at: http://www.cwi.nl/~gruau/gruau/AB.ps.Z  F.Gruau. and Quatramaran. Cellular Encoding for Interactive Evolutionary Robotics. ECAL97, pages ???, MIT Press, 1997. Available online at: http://www.cwi.nl/~gruau/gruau/e.ps.gz  F. Gruau and D. Whitley. The cellular developmental of neural networks: the interaction of learning and evolution. Research report RR93-04, Laboratoire de l'Informatique du Parallelisme, Ecole Normale Superieure de Lyon, 1993. Available online at: ftp://ftp.ens-lyon.fr/pub/LIP/Rapports/RR/RR93/RR93-04.ps.Z  If you are only going to read one paper, I would suggest reading the first paper, which gives a good general overview of his technique.  For more information, refer to Gruau's home page.
Thu 11/5/97	Daniel Nikovski	``Learning Discrete Dynamical Systems''  Learning a model of the world is a fundamental task for intelligent autonomous agents. In the field of automatic control, system identification methods have been used with much success. These methods, though, seem to have had little impact on AI systems, which traditionally model the world with discrete, propositional variables without uncertainty. The relatively recent realization that planning under uncertainty and optimal control are essentially the same thing, has generated a lot of interest in decision-theoretic planning, POMDPs, and system identification of symbolic dynamics. I will discuss some representative work on learning discrete dynamical systems, coming out of Brown University. The paper is: Basye, K., Dean, T., and Kaelbling, L. (1995). "Learning Dynamics: System Identification for Perceptually Challenged Agents", Artificial Intelligence, Volume 72, Pages 139-171, available from http://www.cs.brown.edu/people/tld/postscript/BasyeDeanandKaelblingAIJ-95.ps. The author's approach can be summarized as follows. Discrete dynamical systems are modeled with deterministic finite-state automata. Uncertainty is not inherent in the system, but comes from inability to observe states and state transitions correctly. The number of states, actions, and percepts is known to the learner. Under these assumptions, the authors present algorithms for the cases of fully deterministic dynamics, deterministic actions and uncertain perception, stochastic action and fully observable states, and finally, for the case when both action and perception are stochastic. The algorithms are applied to the task of learning large maps of the environment of mobile robots. I will discuss the positive and negative features of those algorithms as regards their applicability to decision-theoretic planning and control.
Thu 10/30/97	Peter Stone	I will present some robot learning work out of Minoru Asada's lab at Osaka University. I will cover a few innovations in autonomous robot control using reinforcment learning, including learning from easy missions, state/action space construction, and combining independent learned behaviors. Time permitting, I will also briefly touch on their work in adaptive visual servoing. The Asada lab homepage is at http://mike.ccm.eng.osaka-u.ac.jp/Welcome-eg.html
Thu 10/23/97	Karen Zita Haigh	``Cost-sensitive Robot Learning'' Ming Tan Carnegie Mellon University, School of Computer Science, CMU-CS-91-134 Robot learning aims to improve the connection of a robot's perception to action. However, the majority of work in robot learning has not dealt realistically with perception and action. A common assumption is that sensors provide a complete picture of the external world at no cost and actions can be executed instantly. In reality, robots only have limited resources. There are always costs (e.g., sensing or moving speed) associated with perception and action. This dissertation is a study of cost-sensitive robot learning that considers such costs explicitly. The main thesis is that a learning robot should focus on both learning accuracy and learning efficiency and that it is possible to gradually improve the former without sacrificing the latter. Specifically, this dissertation describes a cost-sensitive learning system called CSL that learns to perform an approach-recognize task for a real robot. Given a set of unknown objects and the speeds of sensors and actions, CSL learns where to sense, which sensor to use, and which action to apply. As a result, it learns to approach, recognize, and grasp objects efficiently. CSL integrates robot learning with robot control to handle sensing noise and grasping failures and to balance efficiency and effectiveness. Compared with traditional learning approaches, in experiments involving 15 training objects and 11 test objects, CSL is an order of magnitude faster during training and at least 30% faster during testing. Its overall success rate on novel objects is over 80% and many failures can be eliminated through incremental learning. This dissertation also presents a unified framework for cost-sensitive learning of classification knowledge from examples and constructing examples from scratch. Within this framework, various cost-sensitive learning methods can be instantiated. This dissertation develops two such cost-sensitive learning methods: a decision-tree-based method and an instance-based method, and it evaluates the tradeoff between their learning efficiency and learning accuracy. This dissertation demonstrates the generality of cost-sensitive learning in three ways. First, it uses cost-sensitive learning to accomplish the approach-recognize task. Second, it applies cost-sensitive learning to construct a task-dependent internal representation for use by a reinforcement learning method as it learns a decision policy. Experimental results confirm the efficiency of the overall approach. Talk summary (postscript gzipped)
Thu 10/16/97	Joseph O'Sullivan	``Learning from Demonstration'' Paper (postscript gzipped)
Thu 10/9/97	David Andre	``What GP has done for us lately?'' Genetic programming is a machine learning technique for automatically generating computer programs to solve a given task by means of simulated evolution. Although GP has only been around in force in the 90s, it has been successfully applied to a wide variety of problems. Starting with simple programmatic ingredients and a means for testing the fitness of programs, GP breeds a population of computer programs using artificial analogs of the natural processes of crossover and mutation. GP has been used to solve a wide variety of 'benchmark' problems from the fields of AI, machine learning, neural networks, control theory, game theory, molecular biology, classification, and system identification. In recent years (1995-1997), GP has been used to create solutions to several problems that rival solutions created by humans. These success stories include the areas of evolving electrical circuits, solving hard molecular biology problems, theorem provers, automatic parallel programming, evolving neural networks, and control theory. The goal of this talk is to introduce the concepts and ideas of GP and evolutionary computation and place GP in the field of ML and show its relevance to both AI and robot learning. The ``fitness'' of GP as a tool will be discussed, as will the areas where we all should still be skeptical about GP. Slides (postscript)