“How to be Curious Instead of Contrarian About COVID-19: Eight Data Science Lessons From Coronavirus Perspective”

Rex Douglass writes:

I direct the Machine Learning for Social Science Lab at the Center for Peace and Security Studies, UCSD. I’ve been struggling with how non-epidemiologists should contribute to COVID-19 questions right now, and I wrote a short piece that summarizes my thoughts.

8 data science suggestions

For people who want to use theories or models to make inferences or predictions in social science, Douglass offers the following eight suggestions:

1: Actually Care About the Answer to a Question

2: Pose a Question and Propose a Research Design that Can Answer It

3: Use Failures of Your Predictions to Revise your Model

4: Form Meaningful Prior Beliefs with a Thorough Literature Review

5: Don’t Form Strong Prior Beliefs Based on Cherry Picked Data

6: Be Specific and Concrete About Your Theory

7: Choose Enough Cases to Actually Test Your Theory

8: Convey Uncertainty with Specificity not Doublespeak

2 more suggestions from me

I’d like to augment Douglass’s list with two more items:

9: Recognize that social science models depend on context. Be clear on the assumptions of your models, and consider where and when they will fail.

10: Acknowledge internal anomalies (aspects of your theories that are internally incoherent) and external anomalies (examples when your data makes incorrect real-world predictions).

Both these new points are about recognizing and working with the limitations of your model. Some of this is captured in Douglass’s point 3 above (“Use Failures of Your Predictions to Revise your Model”). I’m going further, in point 9 urging people to consider the limitations of their models right away, without waiting for the failures; and in point 10 urging people to publicly report problems when they are found. Don’t just revise your model; also explore publicly what went wrong.

Background

Douglass frames his general advice as a series of critiques of a couple of op-eds by a loud and ignorant contrarian, a law professor named Richard Epstein.

Law professors get lots of attention in this country, which I attribute to some combination of their good media connections, their ability to write clearly and persuasively and on deadline, and their habit and training of advocacy, of presenting one side of a case very strongly and with minimal qualifications.

Epstein’s op-eds are pretty silly and they hardly seem worth taking seriously, except as indicating flaws in our elite discourse. He publishes at the Hoover Institution, and I’m guessing the people in charge of the Hoover Institution feel that enough crappy left-wing stuff is being published by the news media every day, that they can’t see much harm in countering that with crappy right-wing stuff of their own. Or maybe it’s just no big deal. Stanford University publishing a poorly-sourced opinion piece is, from a scholarly perspective, a much more mild offense than what their Berkeley neighbor is doing with a professor who engages in omitting data or results such that the research is not accurately represented in the research record. If you’re well connected, elite institutions will let you get away with a lot.

When responding to criticism, Epstein seems like a more rude version of the cargo-cult scientists who we deal with all the time on this blog, people who lash out at you when you point out their mistakes. In this case, Epstein’s venue is not email oor twitter or even Perspectives on Psychological Science; it’s an interview in the New Yorker, where he issues the immortal words:

But, you want to come at me hard, I am going to come back harder at you. And then if I can’t jam my fingers down your throat, then I am not worth it. . . . But a little bit of respect.

Dude’s a street fighter. Those profs and journalists who prattle on about methodological terrorists, second-string replication police, Stasi, Carmelo, etc., they got nothing on this Richard Epstein guy.

In this case, though, we can thank Epstein for motivating Douglass’s thoughtful article.

P.S. I’d been saving the above image for the next time I wrote about Cass “Stasi” Sunstein. But a friend told me that people take umbrage at “sustained, constant criticism,” so maybe best not to post more about Sunstein for awhile. My friend was telling me to stop posting about Nate Silver, actually. It’s ok, there are 8 billion other people we can write about for awhile.