Here’s a presentation, Exaggerated Claims Undermine Science by Ignoring the Scientific Method, by Rob Kass, a statistician who over the years has done a lot of interesting work on statistical theory and applications, especially in neuroscience. A few years ago, we discussed Kass’s thoughts on statistical pragmatism. And here’s a discussion of a couple of papers by Kass and Brad Efron, which may be the first time I quoted Hal Stern that “the big divide in statistics is not between Bayesians and non-Bayesians but rather between modelers and non-modelers.” (That’s similar to the other Hal Stern quote that “what’s important in a statistical method is not what it does with the data but what data it uses.” I guess Hal has a certain style with his aphorisms.)
Rob argues that the way that a big problem in science communication is that the way that science is presented—as a set of known facts—does not match the give-and-take of the scientific process.
True science is collaborative or kaleidoscopic in many ways: many models of the world, many sources of data, many techniques of measurement, many researchers (including yourself at different times). The exciting process of scientific discovery is not well captured either by dry textbooks (at one extreme) or the scientist-as-hero framework of Freakonomics and Ted talks.
In his talk, Kass goes into details on a social neuroscience example, discussing among other things the often misunderstood distinction between correlation and causation. As someone who’s done a lot of work in survey research, I’d also like to emphasize that correlation does not even imply correlation. Also, a minor thing: Kass discusses how a regression “controls” for variables. I prefer to say “adjust for.” But that’s all minor. Overall I recommend Rob’s talk in that it connects general issues of science to more specific questions about what is statistics.