A Configurable Synthetic Data Quality Error Injection Framework for Testing Data Quality Tools and AI Systems (DQInject-Bench)

Description

Data quality is essential for developing reliable software and Artificial Intelligence (AI) systems. However, real-world datasets often contain errors such as missing values, duplicate records, invalid categories, incorrect formats, outliers, and broken references. These problems can affect data processing, system reliability, and AI performance. This project aims to design and develop a configurable framework that automatically injects realistic data quality errors into clean datasets. The framework will allow users to select different error types, control error rates, and generate corrupted datasets with known errors. These datasets will serve as benchmarks for systematically evaluating how effectively different data quality validation tools and AI systems can detect such errors. The framework will be implemented using Python and data processing libraries such as Pandas and NumPy. It will support six types of data quality errors and multiple injection rates (1%, 5%, 10%, and 20%). Experiments will be conducted using datasets from healthcare, finance, and retail domains. The generated benchmarks will be used to evaluate traditional data validation tools, including Great Expectations and Pandera, alongside a Large Language Model (LLM). Their detection performance will be compared using metrics such as precision, recall, F1-score, false positives, false negatives, execution time, and reproducibility. The expected outcome is an open, reusable, and configurable data quality benchmarking framework that enables researchers and developers to generate controlled data quality problems, evaluate detection capabilities, and identify which types of errors are most challenging for traditional tools and AI-based systems.