Composite Indicators for Data Mining: A New Framework for Assessment of Prediction Classifiers

Effectiveness and superiority of predictive accuracy of different Data mining (DM) models over the others have traditionally come from results of the empirical studies of DM. Study [4] compared logistic regression, classification tree, neural network, random forest and AdaBoost based on evaluation composite indicators (ECI) built from four parameters like accuracy, interpretability, robustness and speed using four input alternatives (original, aggregated, principal component analysis and stacking based variables), three random indicator weighting criteria and two indicator normalization methods (z-score and min-max). In this study, ECI has been calculated using results from [4] from same four input variable types but using “four plus one” (five) parameters. The fifth parameter of interest (POI) named as Residual Efficiency (RE), has been quantified for this study based on characteristics of interest (COI) described in [10]. Besides, analytical hierarchy process (AHP) of [13] has been used as weighting criteria and step wise utility functions of [12] as normalization technique. Finally we have compared our results with that of [4]. As opposed to study [4], this study has calculated ECIs for all the classifiers used and results have narrower ranges thus are more realistic for comparing the considered classifiers objectively based on type of inputs and POIs.