Pre-Conference Talk by GENG Minghong | L2M2: A Hierarchical Framework Integrating Large Language Model and Multi-agent Reinforcement Learning

Multi-agent reinforcement learning (MARL) has demonstrated remarkable success in collaborative tasks, yet faces significant challenges in scaling to complex scenarios requiring sustained planning and coordination across long horizons. While hierarchical approaches help decompose these tasks, they typically rely on hand-crafted subtasks and domain-specific knowledge, limiting their generalizability. We present L2M2, a novel hierarchical framework that leverages large language models (LLMs) for high-level strategic planning and MARL for low-level execution, enabling zero-shot planning that supports both end-to-end training and direct integration with pre-trained MARL models. Through extensive experiments on diverse multi-agent navigation scenarios, we demonstrate that L2M2’s end-to-end MARL achieves superior performance while requiring less than 20% of the training samples compared to baseline methods on the VMAS tasks. On complex scenarios in the MOSMAC environment, L2M2 achieves a 98.75% win rate with pre-defined subgoals and maintains a 68.13% win rate without subgoals — where baseline methods fail to perform. Analysis through kernel density estimation reveals L2M2’s ability to automatically generate appropriate navigation plans, demonstrating its potential for addressing complex multi-agent coordination tasks. 

This is a Pre-Conference talk for The 34th International Joint Conference on Artificial Intelligence (IJCAI 2025).