Adjusting for differences between treatment and control groups: “statistical significance” and “multiple testing” have nothing to do with it

Jonathan Falk points us to this post by Scott Alexander entitled “Two Unexpected Multiple Hypothesis Testing Problems.” The important questions, though, have nothing to do with multiple hypothesis testing or with hypothesis testing at all. As is often the case, certain free-floating scientific ideas get in the way of thinking about the real problem. Alexander …