Research Seminar by Santonu Sarkar | Mining Operational Logs for Anomaly Detection

An important research area for the Software Science group at BITS Pilani Goa campus is software analytics for dependable systems, which essentially is a data driven approach to enable people and systems to obtain insightful and actionable information. In this talk we present a class of techniques, known as invariant mining, that have been applied on operational data of cloud based systems to obtain latent anomalies in the execution behaviour of the system. The idea of invariant based detection and monitoring can be used to support a variety of activities; on line anomaly detection through system monitoring to avoid an imminent failure, or post mortem for troubleshooting. The usage scenarios can be capacity planning and detection of failures, execution anomalies and violations of Service Level Agreements.

Likely system invariants are properties that are expected to hold in normal operating conditions of computing systems. Their violation is indicative on runtime anomalies. Invariants are mined offline from training datasets, or they are dynamically inferred during execution. In this talk we present a flow-invariant and value based invariant detection approaches.

From a SaaS application log, we created a time series based modelling to capture various flow-invariants. Detection capabilities of identified invariants are then tested on a portion of available logs, showing that the tool is able to filter information useful for administrators to pinpoint error conditions that usually go unnoticed by the operations personnel. Typically, these invariants can be used to monitor service level agreement violations.

On two widely different datasets from real world systems- one a Google cluster, whose traces are publicly available, and a SaaS platform, we have performed an empirical analysis of three techniques for mining value based invariants: clustering, association rules, and decision list. The assessment is based on common metrics of coverage, recall and precision. Results show that relatively few invariants characterize the majority of operating conditions, that precision and recall may drop significantly when trying to achieve a large coverage, and that techniques exhibit similar precision, though the supervised one a higher recall. Using these invariants we have been able to characterise job failures in google cluster and several extreme cases of silent data corruption in the SaaS application.
 

About the Speaker