Outlier Detection for Data Reliability
RetailZoom is a data science consultancy specializing in analytics, business intelligence, and custom CRM solutions for medium and large businesses in Cyprus and beyond. One of their projects involved calculating key statistics for supermarket product prices. During the data audit, they identified outliers caused by unknown or poorly understood processes, making it challenging to assess the outliers’ frequency and impact.
As part of the EuroCC2 project, NCC Cyprus provided a technical consultation service to help RetailZoom detect these outliers and develop a predictive framework for imputing their values. We proposed leveraging a Bayesian linear model that simultaneously accounts for inliers and outliers, assigning probabilistic estimates to each. This approach utilizes a double-likelihood function within a Bayesian framework, typically requiring high-performance computing and multi-processor resources for efficient execution.
An add-on of our suggested analysis pipeline is a principled way to impute outlier values by leveraging the inlier portion of the likelihood for inference only. By integrating this technique, RetailZoom can enhance the reliability of their data, leading to more accurate insights and informed decision-making.