PhD Dissertation Proposal by KONG Jiaolong | Automated Program Repair in the Era of Large Language Models

ABOUT THE TALK

Software bugs are inevitable in modern software systems, with billions of dollars spent annually on finding and fixing them. Automated Program Repair (APR) promises to reduce this burden by generating patches automatically, and Large Language Models (LLMs) have become the most effective foundation for APR, consistently surpassing traditional and learning-based techniques. However, two fundamental questions remain open: how far LLM repair capability can be advanced, and to what extent it reflects genuine program understanding rather than memorization of training corpora. This dissertation investigates both.

For the first question, we study how to strengthen repair capability. We propose ContrastRepair, which utilizes external feedback to build more informative prompts from contrastive test pairs (a failing test and a minimally different passing test), helping the LLM identify the bug's root cause. While effective, external feedback alone remains coarse: it reveals that a patch fails but not which tokens are responsible. We therefore incorporate the model's internal signals and propose TokenRepair, which analyzes token-level uncertainty to localize the most suspicious tokens within a generated patch and rectifies them through Chain-of-Thought (CoT) decoding. Both achieve state-of-the-art results on Defects4J and HumanEval-Java.

For the second question, we examine whether such performance stems from genuine program understanding or memorization. We formalize memorization detection in APR as a general hypothesis testing problem that unifies existing approaches by defining a low-probability event under the null hypothesis of no memorization. Our study reveals that part of correct repairs on Defects4J may stem from memorization, while existing detectors are inadequate for reliably identifying it.

These studies advance and delimit the capabilities of LLMs in APR, motivating future work on memorization-aware evaluation and generalizable repair techniques.

ABOUT THE SPEAKER

KONG Jiaolong is a Ph.D. candidate in Computer Science at Singapore Management University, supervised by Assistant Professor Xie Xiaofei. His research focuses on AI for Automated Program Repair. He has published papers at leading software engineering Journal and conference, including TOSEM, FSE, and ASE.