Hardware-Assisted Protection and Isolation
Date
2011-08-22
Authors
Wang, Jiang
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Software is prone to contain bugs and vulnerabilities. To protect it, researchers normally go to a lower layer, such as protecting the applications from the kernel or protecting the operating systems from the hypervisor, because the upper layer is controlled and depends on the lower layer. However, even a small hypervisor, which partitions the system hardware resources into different domains to support and isolate multiple virtual machines, may contain some vulnerabilities and is hard to protect within itself. In this dissertation, we use a hardware-assisted method to monitor the integrity of the software running on top it. We present HyperCheck, a hardware-assisted tampering detection framework designed to protect the integrity of hypervisors or operating systems (OS). HyperCheck leverages the CPU System Management Mode (SMM), present in x86 systems and a dedicated commercial network card, to securely generate and transmit the full state of the protected machine to an external server. Using HyperCheck, we were able to ferret-out rootkits that targeted the integrity of both the Xen hypervisor and traditional OSes. Moreover, HyperCheck is robust against attacks that aim to disable or block its operation. Our experimental results show that HyperCheck can produce and communicate a scan of the state of the protected software in less than 40ms. In addition to detecting the intrusion, another promising approach to protect the end user's computer is to separate sensitive tasks, such as financial-related activities, from un-sensitive tasks. For this purpose, we designed a system which has two operating systems installed: one trusted and the other untrusted. The trusted OS runs only the trusted applications and is guaranteed to be separated from the untrusted OS. Without using a hypervisor, we leverage the commercial hardware and the BIOS to enforce the isolation between the two OSes. By utilizing the standard ACPI S3 sleep, we also achieve a short delay when switching between the two OSes.
Description
Keywords
Hardware - Assisted, Digital Forensic, System Management Mode, Hypervisor Security, BIOS, Integrity Monitor