PhD Dissertation Defense by HONG Jiaqi | A Virtualization based System Infrastructure for Dynamic Program Analysis

Dynamic malware analysis schemes either run the target program as is in an isolated environment assisted by additional hardware or modify it with instrumentation code statically or dynamically. The hardware-assisted schemes usually trap the target during its execution to a more privileged environment which is not accessible by the untrusted kernel. Thus this approach is often applied for transparent and secure kernel analysis. Nevertheless, the isolated environment induces a virtual address gap between the analyzer and the target, which hinders effective and efficient memory introspection and undermines the correctness of semantics extraction. Code instrumentation mixes the analyzer code with the target. The instrumentation code has native access capabilities to the target's virtual memory, which seamlessly introspects and controls the target. However, code instrumentation based schemes are inadequate to tackle malicious execution since the analysis can be detected, evaded, or even tampered with as noted in many recent works.

In this dissertation, we propose a new system infrastructure, named Onsite Analysis Infrastructure (OASIS), for dynamic program analysis. OASIS combines the virtues of hardware-trapping (strong security and isolation) and code instrumentation (native-access) without their drawbacks. We also propose two new dynamic analysis models based on OASIS: onsite memory analysis and execution flow instrumentation. We build several tools for the two dynamic analysis models respectively to demonstrate their capabilities.

Jiaqi HONG is a Ph.D. candidate in Cybersecurity at School of Information Systems, Singapore Management University. She is advised by Associate Professor Xuhua DING. In her Ph.D. study, she focuses on system security, especially on relying hardware virtualization technology to build secure systems.