Experimental Design in One-sided Matching Platforms
Conference
65th ISI World Statistics Congress 2025
Format: IPS Abstract - WSC 2025
Keywords: experiment, gamification;, platform
Session: IPS 812 - Experimental and Observational Causal Inference in the Tech Industry
Monday 6 October 10:50 a.m. - 12:30 p.m. (Europe/Amsterdam)
Abstract
We investigate experimental design problems in one-sided matching platforms, which are commonly found in the online gaming industry and anonymous social networks. These platforms involve users being matched with others for activities such as games or social interactions, resulting in an interdependence of users' metrics on the treatment assignments of their counterparts. We construct a stochastic market model and develop its mean field limit to analyze the experimental dynamics in such platforms. Our focus is on two randomization strategies: user randomization and match randomization. We demonstrate that under Markovian user behavior and homogeneous treatment effects, match randomization can provide unbiased estimations. However, significant biases may arise when these conditions are not met. On the other hand, user randomization is generally biased but exhibits greater resilience to model inaccuracies. We then propose a simple linear regression estimator under user randomization and demonstrate that this estimator consistently outperforms alternatives in various situations.