Combating threat-alert fatigue with online anomaly detection using isolation forest

Muhamad Erza Aminanto, Lei Zhu, Tao Ban, Ryoichi Isawa, Takeshi Takahashi, Daisuke Inoue

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

13 Citations (Scopus)

Abstract

The threat-alert fatigue problem, which is the inability of security operators to genuinely investigate each alert coming from network-based intrusion detection systems, causes many unexplored alerts and hence a deterioration of the quality of service. Motivated by this pressing need to reduce the number of threat-alerts presented to security operators for manual investigation, we propose a scheme that can triage alerts of significance from massive threat-alert logs. Thanks to the fully unsupervised nature of the adopted isolation forest method, the proposed scheme does not require any prior labeling information and thus is readily adaptable for most enterprise environments. Moreover, by taking advantage of the temporal information in the alerts, it can be used in an online mode that takes in the most recent information from past alerts and predicts the incoming ones. We evaluated the performance of our scheme using a 10-month dataset consisting of more than half a million alerts collected in a real-world enterprise environment and found that it could screen out 87.41% of the alerts without missing any single significant ones. This study demonstrates the efficacy of unsupervised learning in screening minor threat-alerts and is expected to shed light on the threat-alert fatigue problem.

Original languageEnglish
Title of host publicationNeural Information Processing - 26th International Conference, ICONIP 2019, Proceedings
EditorsTom Gedeon, Kok Wai Wong, Minho Lee
PublisherSpringer
Pages756-765
Number of pages10
ISBN (Print)9783030367077
DOIs
Publication statusPublished - 2019
Event26th International Conference on Neural Information Processing, ICONIP 2019 - Sydney, Australia
Duration: 12 Dec 201915 Dec 2019

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11953 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference26th International Conference on Neural Information Processing, ICONIP 2019
Country/TerritoryAustralia
CitySydney
Period12/12/1915/12/19

Keywords

  • Intrusion detection system
  • Isolation forest
  • Stacked autoencoder
  • Threat-alert fatigue

Fingerprint

Dive into the research topics of 'Combating threat-alert fatigue with online anomaly detection using isolation forest'. Together they form a unique fingerprint.

Cite this