Conference on Digital Experimentation at MIT (CODE)
In the setting of online experiments, we propose a two-step procedure to improve efficiency for estimating average treatment effect (ATE) by combining synthetic control methods with the popular regression adjustment framework. In particular, we form a synthetic control for each and every subject in the experiment using a donor pool that consists of k nearest-neighbors (kNN) from outside of the experiment. The predicted counterfactuals are then used in the following regression adjustment step. The asymptotic theory of the method can be shown following and is validated in a realistically calibrated Monte Carlo analysis. For both user-level and cluster experiments at Facebook, we show that the proposed method yields significantly narrower CIs compared with the standard difference-in-mean estimator and a widely used OLS adjusted estimator.
Donglai Xiang, Timur Bagautdinov, Tuur Stuyck, Fabian Prada, Javier Romero, Weipeng Xu, Shunsuke Saito, Jingfan Guo, Breannan Smith, Takaaki Shiratori, Yaser Sheikh, Jessica Hodgins, Chenglei Wu
Ludwig Sidenmark, Mark Parent, Chi-Hao Wu, Joannes Chan, Michael Glueck, Daniel Wigdor, Tovi Grossman, Marcello Giordano
Xingyi Zhou, Rohit Girdhar, Armand Joulin, Philipp Krähenbühl, Ishan Misra
