PhD Dissertation Proposal by LING Jiajing

Many applications of autonomous vehicles (e.g., drone delivery) have been observed in our urban system recently. In these applications, hundreds of autonomous vehicles are required to accomplish tasks in a collaborative way. To manage these vehicles, we have to address several key challenges such as maintaining safety among vehicles, avoiding congestion, minimizing travel time, visiting landmarks, etc. Motivated by the emerging applications, we discuss the large-scale multi-agent decision making for the urban system optimization with different constraints in this thesis proposal. Our main contributions include: (1) We generalize the standard multi-agent path finding (MAPF) model and map it to a Decentralized Partially Observable Markov Decision Process (Dec-POMDP) for urban system optimization; (2) We develop a new multi-agent reinforcement learning (MARL) approach with credit assignment scheme to solve the Dec-POMDP for urban system optimization; (2) We compile domain constraints (e.g., visiting landmarks) using Sentential Decision Diagrams (SDDs) and integrate the compiled domain constraints into MARL approaches; (3) We propose to leverage constrained reinforcement learning to solve general constraints (e.g., congestion, battery of a drone) in urban system optimization. We also propose an approach to address the case in which there are no feasible solutions to the constrained problem due to the misspecification in the constraint threshold.

Jiajing LING is a Ph.D. candidate in Computer Science and is supervised by Prof. Akshat KUMAR. His research fields include reinforcement learning, large-scale multi-agent decision making, urban system optimization, and propositional logic.