A Theory of Experimenters - Abhijit Banerjee; Sylvain Chassang; Sergio Montero; Erik Snowberg (Center for Economic Studies and ifo Institute (CESifo), 2017)
This paper proposes a decision-theoretic framework for experiment design. We model experimenters as ambiguity-averse decision-makers, who make trade-offs between subjective expected performance and robustness. This framework accounts for experimenters' preference for randomization, and clarifies the circumstances in which randomization is optimal: when the available sample size is large enough or robustness is an important concern. We illustrate the practical value of such a framework by studying the issue of rerandomization. Rerandomization creates a trade-off between subjective performance and robustness. However, robustness loss grows very slowly with the number of times one randomizes. This argues for rerandomizing in most environments.
Keywords: C90; D78; D81; 330; experiment design; robustness; ambiguity aversion; randomization; rerandomization