Modified Entropy Measure for Detection of Association Rules Under Simpson's Paradox Context

The rapid explosion in retail data calls for more effective and efficient discovery of association rules to develop relevant business strategies and rules.Unlike online shopping sites, most brick and mortar retail shops are located in geographically and demographically diverse areas. This diversity presents a new challenge to the classical association rule model which assumes a homogenous group of customers behaving differently. The focus of this paper is centered on the discovery of association rules that were hidden as a result of a geographical and demographically diverse data. We will introduce a novel measure which incorporates the entropy measure with modified weighting for the detection of association rules not detected by the standard measures due to Simpson's paradox. The proposed measure is evaluated using a real-word case study involving a major retailer of fashion good in the context of traditional brick and mortar setting.