SCIS Research Cluster Seminars (March 2024)

Training deep networks on increasingly large-scale datasets is computationally challenging. In this talk, we explore the problem of how to train networks faster. Previous popular and effective solutions are Adam-type optimizers, such as Adam and AdamW. However, they often suffer from overshoot issues: slightly large learning rate yields a big update step, leading to oscillation and slow convergence of the network parameter. To solve this issue, we introduce a faster optimizer, i.e. ADAptive Nesterov momentum algorithm (Adan). We first observe that the current optimization step in Adam-type optimizers can be divided into two parts, the previous accumulated momentum and current gradient. Then Adan computes the gradient at a position slightly ahead in the direction of the previous accumulated momentum. This look-ahead step allows the optimizer to look ahead and thus correct its insuitable update, namely, avoiding too large or too small update steps. Accordingly, Adan is about 2x faster than the SoTA optimizers, e.g. Adam, while achieving higher or comparable performance. This has been verified on many networks, e.g., CNNs, ViTs and MAE in the CV field, UNet and ViTs in AIGC field, GPT2 and billion-scale LLaMA in the NLP field, MLPs and CNNs in reinforcement learning. In theory, Adan is also optimally efficient. Moreover, Adan has been used in multiple popular deep-learning codebases, e.g., Timm of HuggingFace (GitHub Star: 25k) and Jittor of Tsinghua University (GitHub Star: 3k).

Built-in pervasive cameras have become an integral part of mobile/wearable devices and enabled a wide range of ubiquitous applications with their ability to be “always-on”. In particular, life-logging has been identified as a means to enhance the quality of life of older adults by allowing them to reminisce about their own life experiences. However, the sensitive images captured by the cameras threaten individuals' right to have private social lives and raise concerns about privacy and security in the physical world. This threat gets worse when image recognition technologies can link images to people, scenes, and objects, hence, implicitly and unexpectedly reveal more sensitive information such as social connections.

In this talk, I’ll address the issue of preserving bystander privacy without compromising the reminiscability of the lifelog images. Subsequently, I’ll present a set of obfuscation strategies that naturally balances the trade-off between reminiscability and privacy (revealing social ties) while selectively obfuscating parts of the images.

Industrial control systems (ICSs) are types of cyber-physical systems in which programs, written in languages such as ladder logic or structured text, control industrial processes through sensing and actuating. Given the use of ICSs in critical infrastructure, it is important to test their resilience against manipulations of sensor/actuator inputs. Unfortunately, existing methods fail to test them comprehensively, as they typically focus on finding the simplest-to-craft manipulations for a testing goal, and are also unable to determine when a test is simply a minor permutation of another, i.e. based on the same causal events. In this work, we propose a guided fuzzing approach for finding 'meaningfully different' tests for an ICS via a general formalisation of sensor/actuator-manipulation strategies. Our approach identifies the causal events in a test, generalises them to an equivalence class, and then updates the fuzzing strategy so as to find new tests that are causally different from those already identified. An evaluation of our approach on a real-world water treatment system shows that it is able to find 106% more causally different tests than the most comparable fuzzer. While we focus on diversifying the test suite of an ICS, our formalisation may be useful for other fuzzers based on intercepting communication channels.

ZHOU Pan is an Assistant Professor at the SCIS of SMU. Before, he worked as a research scientist at Sea AI Lab and Salesforce. His research interests include deep learning optimization, self-supervised learning, and network architecture design, and has published more than 50 top-tire conference and journal papers, such as NeurIPS, ICML, CVPR, and TPAMI. Moreover, he also serves actively as an Area Chair for the conferences of NeurIPS and ICLR.

Thivya is currently an Assistant Professor in the School of Computing and Information Systems at SMU. The specific themes of my research interests are mobile and wearable computing, and human cognition and memorability. Her works have been well recognized in top tier venues such as CSCW, UbiComp, MobiSys, and CHIIR. She serves as an organising committee member for workshops hosted in venues such as Percom and MobiSys.   
 

Chris Poskitt (https://cposkitt.github.io/) is an Associate Professor of Computer Science (Education) at Singapore Management University, where he is part of the Centre for Research on Intelligent Software Engineering. Prior to Singapore, he undertook his doctoral studies at the University of York and held a postdoctoral research position at ETH Zürich. His research broadly addresses the problem of engineering correct and secure software, especially in the context of cyber-physical systems (e.g. industrial control systems, autonomous vehicles). In addition to software engineering, his interests span formal methods, cybersecurity, and computer science education.

David LO     
OUB Chair Professor of Computer Science 
Director, Information Systems & Technology Cluster 
Director, Centre for Research for Intelligent Software Engineering