Data Quality Assessment Framework (DQAF) as a useful tool in composite indicator construction
Conference
65th ISI World Statistics Congress 2025
Format: CPS Poster - WSC 2025
Keywords: "statistical_quality_control, #officialstatistics, composite indicator, quality frameworks;, quality-assessment
Abstract
A Data Quality Assessment Framework (DQAF) is crucial in the foundational stages of Composite Indicator (CI) construction by ensuring the selection of fit-for-purpose elementary indicators. Whereas data quality dimensions are rooted in the UN Fundamental Principles of Official Statistics, different CI constructors often employ varying criteria, with little information on how it was cultivated. This does not only lead to diverse outputs but also raises justification questions.
This study, which focuses on a CI for service delivery, utilized literature to identify twenty-three potential data quality dimensions. Through the assessment of eighteen experts, an adjusted pedigree matrix of statistical information was developed, prioritizing five dimensions: Relevance, Interpretability, Methodological Soundness, Accuracy, and Statistical Adequacy. An elaboration of these dimensions, the quality criteria, and associated scores ranging from 1 to 5 formed the DQAF. An aggregated quality score below 2.0 served as a cut-off point for accepting elementary indicators into the initial list for CI construction.
The study's findings emphasize the importance of a well-documented DQAF, not only for CI construction but also for its adaptability to other use cases, thereby enhancing its applicability. This developed DQAF features a dual-orientation encompassing two user-oriented and three producer-oriented dimensions. This orientation bridges the gap between data producers and users, providing a rationale for indicator selection and areas for data quality improvement. Applying the DQAF to service delivery indicators resulted in the selection of 51 out of 103 potential elementary indicators.
The main contribution of this study is the development of the DQAF which provides a transparent bridge between the theoretical framework and exploratory data analysis stages of the CI construction stages. This study highlights the importance of expert involvement during the development and application of the DQAF to a respective case study to minimize subjectivity in scoring and ensure comprehensive assessments. By addressing potential overlaps and redundancies in the selected dimensions through correlation analysis of the scores, the DQAF aims to further refine the selection process, thereby enhancing the overall quality and reliability of CIs.