Extracting Hidden Anomalies using Sketch and Non Gaussian Multiresolution Statistical Detection Procedures

Guillaume Dewaele (1) , Kensuke Fukuda (2) , Pierre Borgnat (1) , Patrice Abry (1) , Kenjiro Cho (3)

1 Phys-ENS - Laboratoire de Physique de l'ENS Lyon
2 NII - National Institute of Informatics
3 IIJ - Internet Initiative Japan Research Laboratory

A new profile-based anomaly detection and characterization procedure is proposed. It aims at performing prompt and accurate detection of both short-lived and long-lasting low-intensity anomalies, without the recourse of any prior knowledge of the targetted traffic. Key features of the algorithm lie in the joint use of random projection techniques (sketches) and of a multiresolution non Gaussian marginal distribution modeling. The former enables both a reduction in the dimensionality of the data and the measurement of the reference (i.e., normal) traffic behavior, while the latter extracts anomalies at different aggregation levels. This procedure is used to blindly analyze a large-scale packet trace database collected on a trans-Pacific transit link from 2001 to 2006. It can detect and identify a large number of known and unknown anomalies and attacks, whose intensities are low (down to below one percent). Using sketches also makes possible a real-time identification of the source or destination IP addresses associated to the detected anomaly and hence their mitigation.

Mots clés: Anomaly Detection; Traffic measurement; MAWI database; Sketch; Gamma multiresolution modeling

Domaines: Traitement du signal et de l'image [eess.SP]; Réseaux et télécommunications [cs.NI]; Traitement du signal et de l'image [eess.SP]

Guillaume Dewaele, Kensuke Fukuda, Pierre Borgnat, Patrice Abry, Kenjiro Cho. Extracting Hidden Anomalies using Sketch and Non Gaussian Multiresolution Statistical Detection Procedures. ACM SIGCOMM 2007 Workshop on Large-Scale Attack Defense (LSAD), Aug 2007, Kyoto, Japan. ⟨ensl-00177654⟩