PhD Dissertation Proposal by TIAN Zichen| Robust and Scalable Parameter-Efficient Fine-Tuning of Foundation Models

ABOUT THE TALK

Parameter-Efficient Fine-Tuning (PEFT), especially Low-Rank Adaptation (LoRA), has become the de facto approach for adapting foundation models to downstream tasks. Existing PEFT research focuses on reducing the number of trainable parameters, but when the parameter budget is constrained, a more fundamental question is how this limited capacity should be allocated. This dissertation formulates LoRA adaptation as a constrained capacity allocation problem and systematically addresses four dimensions: data, configuration, composition, and architecture. For data: class imbalance causes LoRA's representations to bias toward majority classes; debLoRA (NeurIPS 2024) corrects this through unsupervised feature clustering and calibration. For configuration: PEFT hyperparameters interact non-monotonically, making manual tuning unreliable; MetaPEFT (CVPR 2025, Highlight) automates this via a meta-learned differentiable modulator. For composition: scaling to multiple tasks causes catastrophic performance collapse due to inter-task conflicts; mtLoRA (ICLR 2026) resolves this through spectral-aware regularization, fine-grained routing, and block-level adaptation, outperforming the state-of-the-art by 2.3% with 47% fewer parameters. For architecture: the optimal per-layer expert allocation varies dramatically yet relies on expensive manual search; we propose signal-guided allocation that predicts optimal configurations from frozen-model signals. Together, these contributions provide a systematic solution for reliable LoRA adaptation from single-task deployment to multi-task scaling.

ABOUT THE SPEAKER

Zichen TIAN is a PhD candidate in Artificial Intelligence at the School of Computing and Information Systems, Singapore Management University, advised by Prof. Qianru Sun. He holds an M.Sc. from Nanyang Technological University and a B.Eng. from Beijing University of Posts and Telecommunications. His research focuses on parameter-efficient adaptation of foundation models, systematically addressing robustness and scalability challenges. He has published three first-author papers at NeurIPS 2024, CVPR 2025 (Highlight), and ICLR 2026, and co-authored papers at CVPR 2022, CVPR 2023, and IEEE TMM. He is recipient of the Presidential Doctoral Fellowship and the Doctoral Dean's List, and has been serving as a reviewer for 10+ top-tier AI conferences including NeurIPS, CVPR, ICML, and IJCV.