Coronavirus: the cathedral or the bazaar, or the cathedral and the bazaar?

Raghu Parthasarathy writes:

I’ve been frustrated by Covid-19 pandemic models, for the opposite reason that I’m usually frustrated by models in science—they seem too simple, when the usual problem with models is over-complexity. Instead of doing more useful things, I wrote this up here.

In his post, Parthasarathy writes:

Perhaps the models we’re seeing are not the models we need. . . . What models do I mean? It pains me as a physicist to write this, but: detailed ones. For disease spreads, we can imagine various modeling approaches. There are: (1) Simple differential equation based models, for example SIR models. These are elegant and interesting, describing in a generic sense the rise and fall of diseased populations as a function of just a few parameters. (2) Models that include network connectivity. (3) Models that include the realistic connectivity networks of the world we live in. Why? Because the network matters! The general approach of (2) above already tells us that geometry matters, not for the existence of infectious spreading but for its characteristics. It strikes me as analogous to percolation theory in physics — asking what fraction of sites on a lattice I’ll have to randomly occupy with a stepping stone before I can cross from one side to the other. That fraction varies a lot on different lattices, though it exists for all of them.

More importantly, our goal in analyzing an ongoing pandemic is to understant that pandemic, not the general case. . . . It’s only minimally helpful to apply the same naive model to all, even as a curve-fitting exercise, because while we can always fit effective parameters, the regions’ curves themselves may be profoundly different.

One possibility is that we do, but these models are not widely publicized. I would like to believe this, but I am doubtful. . . . Another possibility is that it’s much easier to make poorer models. Certainly it seems like everyone these days is doing it. . . .

I [Parthasarathy] would like to imagine that like the Manhattan Project, there is a sequestered band of scientists somewhere doing the difficult work of slogging through demographic, transport, and city planning data to construct realistic pandemic models . . .

My reply:

I think these more elaborate models make sense, and the way they happen is that people build up from simpler models. I was just talking with someone today about the flaws of individualistic social science, and we see it in part in the horrible b.s. about “case fatality rate” etc. as if it’s a Platonic number just sitting there to be estimated and argued about.

Regarding the “Manhattan project” idea: no, I’m pretty sure that’s not happening. Instead we’re following the model of the “war on cancer,” which is to have a large number of research groups, all competing for attention and funds. I suspect that, during the time I spend blogging, various medical researchers are on the phone getting million-dollar donations…

Raghu’s response:

Well before Covid-19, I’ve been thinking that our present structure of academic science, mainly made up of lots of small research groups, is highly flawed—it’s what you’d create if you didn’t really care about the answers to questions, but rather cared about keeping people busy. That’s an exaggeration, of course—one needs some number of small, random groups (like mine!) hopefully finding new and unusual directions—but I think a smaller number of groups that are larger, more stable, and publishing less would be an improvement. It worked for the Manhattan Project. It also works for High Energy Physics, painful though it is to write that. I would be miserable as one cog in a CERN project, and I find their questions scientifically very boring, but they are undeniably successful at actually answering the questions they ask.

My further thoughts:

I’m not sure what to think here. On one hand, I like my freedom. It’s a lot more fun, and lucrative, to be a college professor and statistical consultant than I imagine it would be to be a full-time government employee. On the other hand, if the government really got a few thousand of us together to work on this stuff full time, we could probably make more progress than under the current ultra-decentralized system. On the third hand, if the government really did convene a panel of experts, who knows who they’d get? Maybe they’d get a bunch of politically-connected incompetents. Yes, the Manhattan Project in World War 2 was successful, but (a) that was just one project, and (b) the world of physics was pretty small back then, so maybe it was easier for them to find top people. Also, there were not such easy-to-find lucrative and comfortable non-governmental options the way there are now. But that in turn is partly the government’s choice. They choose to fund NSF, NIH, etc., rather than just hiring a bunch of us directly.

Maybe a Manhattan Project or the equivalent “cathedral” could not be constructed today. But we do see problems with the “bazaar,” especially in the way that the news media and social media create frenzies based on whatever bits of news come up on any given day. The best that can be said about these frenzies, I think, is that it’s not clear what the alternative would be. The news media have major problems, but I wouldn’t want the New England Journal of Medicine or Lancet or PNAS being our gatekeeper either. (Sorry, Lancet and PNAS, but we’re not going to forget Andrew Wakefield, himmicanes, and all the rest. You’re not evil; you’re just imperfect, like any other human institution.)

All that said, given that the bazaar is what we have now, we should think about how to make best use of it. Siloing data seems like a problem, releasing results with no report seems like a problem—it’s easy to list problems, not so easy to figure out what to do next, beyond each of us reacting to information as it comes in, and attacking problems from many directions (as here).

So far I’ve been talking about the actions of the scientific community, but the same issues arise for everyday decisions. I wrote about this a few weeks ago: the problem is that the necessary actions are at the societal level, but for years we’ve been socialized to think of everything in terms of individual decisions. Prepping might be a good idea for some people, but in any case it’s not scalable.

To get back to this cathedral/bazaar thing: that’s just a shorthand. In the famous article with that slogan, the “bazaar” was open source, but the current scientific “bazaar” is some mix of data sharing and data hiding.

Ideally we could have a best of both worlds, with a large government-organized effort that also includes some data and communication experts that would publicly release data, code, and preliminary findings as they came in. The cathedral would facilitate the bazaar, as it were.

In World War 2, we were fighting human enemies, and so the Manhattan Project was kept secret (not that it stopped the Soviets from infiltrating). Now we’re fighting a virus, so no need for secrets.