65th ISI World Statistics Congress 2025

65th ISI World Statistics Congress 2025

Addressing Recruitment Bias in RCTs for Impact Evaluation

Conference

65th ISI World Statistics Congress 2025

Format: IPS Abstract - WSC 2025

Keywords: bias, bias reduction, evaluation, impact, power, sample

Session: IPS 674 - The Role of Statistics and Data Science in Impact Evaluation

Tuesday 7 October 10:50 a.m. - 12:30 p.m. (Europe/Amsterdam)

Abstract

Randomized controlled trials (RCTs) reduce bias by using random assignment to make treatment and control groups comparable. However, biases introduced at other stages, such as participant recruitment, can persist. This talk will focus on recruitment bias introduced before randomization. For example, if a program expands recruitment to meet sample size targets, it may engage participants from new locations or through different methods. While a larger sample size increases statistical power, outcomes may be skewed if these new participants respond differently to the program than the typical participants. We present an analysis of recruitment bias in an RCT impact study of grant-funded employment and training programs and discuss strategies to mitigate this issue.