Bluebox: A policy-driven, host-based intrusion detection system

Abstract

Detecting attacks against systems has, in practice, largely been delegated to sensors, such as network intrustion detection systems. However, due to the inherent limitations of these systems and the increasing use of encryption in communication, intrusion detection and prevention have once again moved back to the host systems themselves. In this paper, we describe our experiences with building BlueBox, a host-based intrusion detection system. Our approach, based on the technique of system call introspection, can be viewed as creating an infrastructure for defining and enforcing very fine-grained process capabilities in the kernel. These capabilities are specified as a set of rules (policies) for regulating access to system resources on a per executable basis. The language for expressing the rules is intuitive and sufficiently expressive to effectively capture security boundaries. We have prototyped our approach on Linux operating system kernel and have built rule templates for popular daemons such as Apache and wu-ftpd. Our design has been validated by testing against a comprehensive database of known attacks. Our system has been designed to minimize the kernel changes and performance impact and thus can be ported easily to new kernels. We describe the motivation and rationale behind BlueBox, its design, implementation on Linux, and how it relates to prior work on detecting and preventing intrusions on host systems.