Popularity Prediction for Social Media over Arbitrary Time Horizons
Daniel Haimovich, Dima Karamshuk, Thomas Leeper, Evgeniy Riabenko, Milan Vojnovic
Conference on Digital Experimentation at MIT (CODE)
In the setting of online experiments, we propose a two-step procedure to improve efficiency for estimating average treatment effect (ATE) by combining synthetic control methods with the popular regression adjustment framework. In particular, we form a synthetic control for each and every subject in the experiment using a donor pool that consists of k nearest-neighbors (kNN) from outside of the experiment. The predicted counterfactuals are then used in the following regression adjustment step. The asymptotic theory of the method can be shown following and is validated in a realistically calibrated Monte Carlo analysis. For both user-level and cluster experiments at Facebook, we show that the proposed method yields significantly narrower CIs compared with the standard difference-in-mean estimator and a widely used OLS adjusted estimator.
Daniel Haimovich, Dima Karamshuk, Thomas Leeper, Evgeniy Riabenko, Milan Vojnovic
Liqi Yan, Qifan Wang, Yiming Cu, Fuli Feng, Xiaojun Quan, Xiangyu Zhang, Dongfang Liu
Patrick Lewis, Barlas Oğuz, Wenhan Xiong, Fabio Petroni, Wen-tau Yih, Sebastian Riedel