Pre-Conference Talk by WANG Haoyu | ProbGuard: Proactive Runtime Monitoring for LLM Agent Safety via Probabilistic Prediction

LLM agents are increasingly deployed in domains such as autonomous driving, robotics, and web automation, where a single unsafe decision can have serious consequences. Yet most runtime safety monitors are reactive: they detect violations only when unsafe behavior is imminent or has already occurred. We present ProbGuard, a proactive runtime monitoring framework that predicts safety risks before they materialize. ProbGuard abstracts agent executions into symbolic states and learns a Discrete-Time Markov Chain from execution traces to model how agent behavior evolves. At runtime, it estimates the probability that the current execution will remain safe and intervenes when this probability falls below a user-defined threshold. ProbGuard further incorporates semantic validity constraints and provides a PAC-style analysis of the learned model.Across autonomous driving and embodied-agent tasks, ProbGuard consistently anticipates safety violations, providing warnings up to 15.84 seconds before violations with no false alarms. For embodied agents, re-prompting reduces unsafe behavior by 65.37% while retaining 80.4% of baseline task completion, while a stricter halting policy achieves a 93.60% reduction at a higher cost. ProbGuard is implemented as an extensible open-source runtime monitor for LangChain with minimal runtime overhead.

This is a Pre-Conference talk for 41st IEEE/ACM International Conference on Automated Software Engineering (ASE 2026).

Haoyu WANG is currently a PhD candidate at the Singapore Management University co-advised by Prof. C.M. Poskitt and Prof. Jun Sun. His current research interest mainly focus on ensuring the safety and security for AI software. Besides, he also interested in fuzzing fundamental software systems including compilers, smart contracts, etc.