Pre-Conference Talk by YU Jiongchi | CAShift: Benchmarking Log-Based Cloud Attack Detection under Normality Shift

With the rise of cloud-native computing, securing cloud environments has become an important task. Log-based Anomaly Detection (LAD) is widely adopted for attack detection; however, existing LAD methods and datasets fail to comprehensively consider for the complex behaviors of cloud systems and the occurrence of real-world normality shifts, which significantly limits their effectiveness. To address these limitations, we introduce CAShift, a novel dataset that captures diverse cloud system activities and models multiple types of normality shift in real world practice, including changes of application type, application version, and cloud architectures, alongside 20 realistic cloud attack scenarios. Using CAShift, we systematically evaluate eight LAD methods, including our self-implemented VAE model, and analyze the impact of normality shifts. Our findings reveal that all methods experience substantial performance degradation under these conditions. We further investigate existing continuous learning techniques for normality shift adaptation and observe notable improvements in performance, contingent on both data and algorithmic choices. Our work highlights key challenges and offers practical insights for enhancing the robustness of LAD systems in dynamic cloud environments.

This is a Pre-Conference talk for ACM International Conference on the Foundations of Software Engineering (FSE 2025).