If variation in effects is so damn important and so damn obvious, why do we hear so little about it?

Earlier today we posted, “To Change the World, Behavioral Intervention Research Will Need to Get Serious About Heterogeneity,” and commenters correctly noted that this point applies not just in behavioral research but also in economics, public health, and other areas.

I wanted to follow this up with a question:

If variation in effects is so damn important and so damn obvious, why do we hear so little about it?

Here’s my quick response:

It’s difficult to estimate variability in treatment effects (recall the magic number 16), and in statistics we’re often trained to think that if something can’t be measured or estimated precisely, that it can be safely ignored.

When I talk about embracing variation and accepting uncertainty, this is one reason why.
P.S. Thanks to Diana for the above photo of Sisi, who is really good at curve fitting and choice of priors.