Abstract

This research paper presents a comprehensive case study of the collaboration between the National Institute of Statistics and Geography (INEGI) in Mexico and the National Institute of Statistics (INE) in Chile. This collaboration is at the forefront of integrating advanced data science methodologies into public statistical systems. It focuses on the synergistic relationship developed between these institutions to augment their statistical capacities using big data, machine learning, and artificial intelligence.

A central element of this collaboration is INEGI's development of a sophisticated technological platform, featuring a versatile data lake capable of accommodating a wide range of statistical and geographic digital data formats. This platform is pivotal in facilitating data collection, secure storage, and controlled access. It allows for the application of unconventional data analysis techniques and the development of innovative data product prototypes.

Key features of INEGI's technological platform include:

Workflows for systematic gathering, storage, processing, analyzing, and presenting information from both internal and external data repositories.
Models for statistical and geographic information, maintaining data lineage and integrity for seamless analysis and exploitation based on specific needs.
Tools that support visual representations of analysis results, enhancing interpretation and utilization of the information.
An environment fostering ongoing development and collaborative efforts, enabling continuous evolution and improvement of the platform.
This system seamlessly integrates statistical and geographic information from various open-data sources. Data from these diverse sources are harmonized within a unified storage environment, adaptable for both temporary and permanent storage needs.

The paper details the joint initiatives undertaken by INEGI and INE, including shared data science laboratories and cross-border data projects. These initiatives aim to improve data collection, analysis, and dissemination, leveraging the capabilities of the technological platform.

Challenges such as harmonizing methodologies across different statistical frameworks and ensuring data security are explored in depth. The study also assesses the impact of this collaboration on public statistics quality and policy-making implications.

The value of international cooperation in the data science realm for evolving national statistical systems is underscored. The case study provides insights for global statistical offices looking to modernize through collaborative frameworks.