PhD Dissertation Proposal by WONG Wai Tuck | Attacking Numerical Stability in Machine Learning

Attacking Numerical Stability in Machine Learning

ABOUT THE TALK

Numerical instability in machine learning arises when tiny changes in input or computational precision cause large, unpredictable shifts in model output, leading to unreliable predictions in domains like healthcare, finance, and autonomous driving. This instability undermines the robustness and trustworthiness of AI systems in real-world settings.

Our research explores how numerical instability itself can be exploited to cause model failures. The first work shows how adding noise to optimization layers (e.g., OptNet) can trigger NaNs and cause complete failure in model inference. The second investigates attacks on Large Vision-Language Models (LVLMs) that induce instability throughout the network, revealing a novel, distinct threat beyond traditional adversarial perturbations. We conclude by highlighting directions for deeper study into this emerging vulnerability.

SPEAKER BIOGRAPHY

WONG Wai Tuck is Head of Labs Engineering at watchTowr, where he leads engineering offensive security capabilities. He is also a part-time PhD candidate in the School of Computing and Information Systems in Singapore Management University, co-advised by Arunesh Sinha and Sun Jun. His interest lies in the intersection of machine learning and cybersecurity, primarily looking at novel attack vectors in machine learning systems.