Deep Learning for Anomaly Detection
anomaly-detectiondeep-learningautoencodersgenerative-models
Abstraction: Applied research report on deep learning methods for anomaly detection
Key points:
- Semi-supervised approach is standard: train on normal data only, then flag test samples whose reconstruction error exceeds a threshold; labeled anomalies are rare by definition so supervised methods are impractical
- Four deep architectures covered: Autoencoder (AE), Variational Autoencoder (VAE), BiGAN (encoder+generator+discriminator), and Sequence-to-Sequence (LSTM-based) — all use encoder-decoder structure
- Benchmark on KDD network intrusion dataset (18 features): BiGAN best overall ROC AUC 0.972, AE 0.963, OCSVM 0.957; Seq2Seq worst at 0.919 due to non-temporal data
- VAEs and BiGANs support principled uncertainty estimates; Seq2Seq is best for data with temporal dependencies; linear models (OCSVM, PCA) preferred for univariate or low-latency scenarios
- Two prototypes built: Blip (KDD dataset, four algorithm comparison) and Anomagram (ECG5000 dataset, interactive autoencoder training in the browser via TensorFlow.js)
Connections: Cloudera · Anomaly Detection · Autoencoders · Generative Adversarial Networks · Variational Autoencoders