Log Message Anomaly Detection using Positive and Unlabeled Learning
Date
2024-01-29
Authors
Seifishahpar, Fatemeh
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Log messages are widely used in cloud servers and software systems. Anomaly
detection of log messages is important as millions of logs are generated each day.
However, besides having a complex and unstructured form, log messages are large
unlabeled datasets which makes classification very difficult. In this thesis, a log
message anomaly detection technique is proposed which employs Positive and Unlabeled Learning (PU Learning) to detect anomalies. Aggregated reliable negative
logs are selected using the Isolation Forest, PU Learning, and Random Forest algorithms. Then, anomaly detection is conducted using deep learning Long Short-Term
Memory (LSTM) network. The proposed model is evaluated using the commonly employed Openstack, BGL, and Thunderbird datasets and the results obtained indicate
that the proposed model performs better than several well-known approaches in the
literature.
Description
Keywords
Anomaly detection, Classification, Deep learning, Log messages, PU learning