Budapest BI 2015

Data mining is usually about trends, repetitive patterns, generally true relationships. Thus, filtering outliers is a necessary step prior to any modelling operation in order to eliminate anything obviously untypical. However, there are several domains where detection and characterization of outliers indicating unexpected events are in the focus of research. For instance in finance (fraud detection) or security (network intrusion detection), outliers are not considered as measurement errors but important markers of interesting or dangerous phenomena. Therefore, machine learning frameworks — RapidMiner, Scikit-learn, H2O, Weka, ELKI — all have their own subset of detection algorithm implementations.The talk summarizes the basic concepts aiming at detection and characterization of outliers, focusing their implementation in R.