PhD Dissertation Proposal by WANG Chenyu | Quality Assurance for AI in Software Engineering: Practitioner Perceptions, Training-Time Threats, and Inference-Time Control

ABOUT THE TALK

AI models have become core components of modern software systems, and software engineering itself is increasingly performed by AI: large language models summarize, generate, and repair code, and autonomous agents resolve real-world software issues end to end. The quality of AI models is thus now a software quality problem—yet AI systems are data-driven, stochastic, and largely opaque, so established quality assurance principles do not transfer directly. This dissertation studies how to assure AI quality across the lifecycle through five studies. First, an interview study with 15 industry practitioners, validated by a 50-practitioner survey, establishes which quality attributes practice actually prioritizes—revealing, among other findings, that efficiency is treated as a hard constraint under which correctness is optimized. Second, an empirical study of data-poisoning backdoor attacks on Code LLMs shows that poisoning just 20 of 454,451 training samples (0.004%) successfully implants backdoors, overturning the consensus that attacks are ineffective at extremely low poisoning rates. Third, a fail-fast-and-restart framework for software engineering agents predicts failures online from partial trajectories and reinvests the reclaimed compute, raising issue resolution from 66.6% to 71.8% under bounded overhead. Finally, two planned studies generalize this inference-time control to security-operations agents—repository-level vulnerability detection pipelines—through the first stage-attributed failure taxonomy for that setting and an online monitoring-and-correction mechanism. Together, these studies make AI quality something that can be understood, protected, and controlled, from development through operations.

ABOUT THE SPEAKER

WANG Chenyu is a PhD candidate in Computer Science at the School of Computing and Information Systems, Singapore Management University, supervised by Prof. David Lo. His research focuses on quality assurance for AI in software engineering, spanning studies of practitioners' quality needs, training-time threats such as backdoor attacks on code models, and run-time control of LLM agents for software engineering tasks.