SIS Research Seminar by Prashant DOSHI

Inverse reinforcement learning (IRL) investigates ways by which a learner may approximate the preferences of an expert by observing the experts’ actions over time. Usually, the expert is modeled as optimizing its actions using a Markov decision process (MDP), whose parameters except for the reward function are known to the learner. IRL is enjoying much success and attention in applications such as robots learning tasks from demonstrations by human experts and in imitation learning. Robots gather large amounts of multi-modal data as they observe others and must sift it to infer the trajectories.

In this talk, I will first discuss the data processing pipeline that enables IRL in robotic applications. Then, I will present methods for generalizing IRL to contexts that involve multiple observed agents, which may interact and whose observed trajectories are partially occluded from the learner. I will also explore generalizing IRL to settings where additional parameters of others' MDPs are unknown. These are all challenges encountered in real-world contexts. We evaluate these methods in the context of an application setting involving two mobile robots executing simple cyclic trajectories for perimeter patrolling. Both robots’ patrolling motions are disturbed when they approach each other in narrow corridors leading to an interaction. A subject robot observes them from a hidden vantage point that affords partial observability of their trajectories only. It’s task is to penetrate the patrols and reach a goal location without being spotted. Thus, its eventual actions do not impact the other robots. Videos of demonstration runs will be shown.

 

About the Speaker