Date of Award

1-1-2009

Document Type

Dissertation

Degree Name

Ph.D.

Organizational Unit

Daniel Felix Ritchie School of Engineering and Computer Science

First Advisor

Ramakrishna Thurimella, Ph.D.

Second Advisor

Chris Gauthier-Dickey

Third Advisor

Christian Grothoff

Keywords

Anomaly detection, Hidden Markov Model, Intrusion detection, Markov chain, Misuse detection

Abstract

One of the greatest challenges facing network operators today is the identification of malicious activity on their networks. The current approach is to deploy a set of intrusion detection sensors (IDSs) in various locations throughout the network and on strategic hosts. Unfortunately, the available intrusion detection technologies generate an overwhelming volume of false alarms, making the task of identifying genuine attacks nearly impossible. This problem is very difficult to solve even in networks of nominal size. The task of uncovering attacks in enterprise class networks quickly becomes unmanageable.

Research on improving intrusion detection sensors is ongoing, but given the nature of the problem to be solved, progress is slow. Research simultaneously continues in the field of mining the set of alarms produced by IDS sensors. Varying techniques have been proposed to aggregate, correlate, and classify the alarms in ways that make the end result more concise and digestible for human analysis. To date, the majority of these techniques have been successful only in networks of modest size. As a means of extending this research to real world, enterprise scale networks, we propose 5 heuristics supporting a three-pronged approach to the systematic evaluation of large intrusion detection logs. Primarily, we provide a set of algorithms to assist operations personnel in the daunting task of ensuring that no true attack goes unnoticed. Secondly, we provide information that can be used to tune the sensors which are deployed on the network, reducing the overall alarm volume, thus mitigating the monitoring costs both in terms of hardware and labor, and improving overall accuracy. Third, we provide a means of discovering stages of attacks that were overlooked by the analyst, based on logs of known security incidents.

Our techniques work by applying a combination of graph algorithms and Markovian stochastic processes to perform probabilistic analysis as to whether an alarm is a true or false positive. Using these techniques it is possible to significantly reduce the total number of alarms and hosts which must be examined manually, while simultaneously discovering attacks that had previously gone unnoticed. The proposed algorithms are also successful at the discovery of new profiles for multi-stage attacks, and can be used in the automatic generation of meta-alarms, or rules to assist the monitoring infrastructure in performing automated analysis. We demonstrate that it is possible to successfully rank hosts which comprise the vertices of an Alarm Graph in a manner such that those hosts which are of highest risk for being involved in attack are immediately highlighted for examination or inclusion on hot lists. We close with an evaluation of 3 sensor profiling algorithms, and show that the order in which alarms are generated is tightly coupled with whether or not they are false positives. We show that by using time based Markovian analysis of the alarms, we are able to identify alarms which have a high probability of being attacks, and suppress more than 90% of false positives.

Publication Statement

Copyright is held by the author. User is responsible for all copyright compliance.

Rights Holder

James J. Treinen

Provenance

Received from ProQuest

File Format

application/pdf

Language

en

File Size

117 p.

Discipline

Computer science

Download



Share

COinS