Anomaly detection in large databases using behavioral patterning

Abstract

We present a novel approach for detecting malicious user activity in databases. Specifically, we propose a new machine learning algorithm for detecting attacks such as a stolen user account or illegal use by a user. Our algorithm relies on two main components that examine the consistency of a user's activity and compare it with activity patterns learned from past access. The first component tests for self-consistency, to determine whether the actions performed by a user are consistent with previous patterns. This engine is based on a probabilistic model that we developed to capture a user's normal behavior. The second component checks for global-consistency, to determine whether a user's actions are consistent with the past actions of similar users. We test our algorithm on access data from SQL databases. Experimental results show that we can keep false positive rates while retaining the overall accuracy level. An outlier detection engine based on the presented methods is now included in the standard offering of IBM InfoSphere Guardium1 with positive user feedback.