Decision making in the tech industry is often data-driven, and reliant on causal insights in particular. Online controlled experiments (i.e., A/B tests) have seen a meteoric rise in prominence in the last decade. Such experiments seek to test and improve internet-based products and services using user-generated data to determine what works and what doesn’t. A/B tests have become an indispensable tool for major tech companies when it comes to maximizing revenue and optimizing the user experience, with some companies running hundreds of experiments engaging millions of users each day. Although controlled experiments are the gold standard of causal inference, they may be infeasible or unethical for some questions of interest. In this case, tech companies regularly turn to observational causal inference methods.

In this online setting, where there is a desire test as many ideas as possible, as quickly as possible, novel practical issues and modern challenges abound. This session brings together researchers and practitioners from both industry and academia to discuss such modern challenges and their work on them. The session is intended to provide a glimpse into the experimentation and causal inference issues practitioners face in this context, and the associated research being done.