How much of Trump’s rising approval numbers can be attributed to differential nonresponse?

Josh Marshall writes:

If you follow polling you know that over the last couple weeks President Trump’s approval numbers have been trending up. . . . 538’s composite average has notched up a couple points over the second half of January and this morning Gallup released the highest approval rating of Trump’s presidency: 49%. Why is this happening?

Marshall goes through several natural explanations (economic prosperity, trade agreements, acceptance of Trump’s behavior), and then continues:

But there is another plausible explanation. Pollsters call it differential response. When one side gets enthused or energized their numbers go up but in an ephemeral fashion. The pumped-up side is a bit more eager to answer the phone or fill out the survey. The demoralized side is a bit less eager. This is a real and demonstrated phenomenon, not just a concept or speculation. It’s not necessarily an error per se in the polling. It’s picking up something real. It’s just ephemeral.

That’s right. It’s what Sharad Goel, Douglas Rivers, David Rothschild, and I found in our analysis of polling data from 2012, and the same pattern appeared in 2016, as illustrated in this graph from Alan Abramowitz.

image001

Returning to today, Marshall continues:

There are good reasons to think that at least some of that is happening today — the President’s impending acquittal and Republican unity have been the driving news of the last two or three weeks. Republicans are energized and enthused by the certainty of President Trump’s acquittal. Many Democrats are demoralized by seeing an overwhelming and exacting case made for the President’s guilt and seeing it simply not matter.

This is something we should be able to see in the data. It’s testable.

Most simply, if there’s differential nonresponse, then when the president’s approval is going up, we should also see an increasing percentage of self-declared Republicans in the sample.

Let’s take a look at that Gallup poll where Trump has an unprecedentedly high 49% approval:

Additionally, the poll finds 48% of Americans identifying as Republicans or leaning toward that party, compared with 44% Democratic identification or leaning. Recent Gallup polls had shown a fairly even partisan distribution, after the Democratic Party held advantages for much of 2019.

So, yeah, some of that shift can be explained by differential nonresponse: more Republicans and fewer Democrats answering the poll. This explanation for the change is not mentioned in the Gallup report, but we can read between the lines and see it.

For another example, let’s look at the Harris poll, which puts Trump at 46% approval:

Results were weighted for age within gender, region, race/ethnicity, marital status, household size, income, employment, education, political party and political ideology where necessary to align them with their actual proportions in the population.

They adjust for party ID, so differential nonresponse should not be an issue.

And from the CBS News poll, where Trump has 43% approval, we see that the weighted number of respondents of each party are 298 Republicans, 362 Democrats, and 542 Independents: that’s 25% R, 30% D, 45% I.

Marshall concludes:

I’m pretty confident that at least some of this is ephemeral differential response. But I am not confident that’s all of it. We just don’t know and won’t know for a while.

I agree. Not just about the “we just don’t know” etc. but also that I think that some, but not all, of the recent trend is attributable to differential nonresponse. Again, that’s consistent with what we saw in 2012.

To answer the question more quantitatively, you’d want to look at a large number of polls and track both presidential approval and party identification. It shouldn’t be hard to do this.