Inaccurate Polling
Opinion polling has no lazier cliché than “snapshot in time.”
. . . [T]he phrase is a refuge or metaphoric shield for pollsters when their pre-election surveys misfire. In such cases, “snapshot in time” is cited in attempting to defend or rationalize polls that careen well off-target, as many of them did in the 2020 presidential election.
Joe Biden was elected to the presidency four years ago by margins well short of the double-digit blowout suggested by the polls of CNN, Quinnipiac University, Economist/YouGuv and NBC/Wall Street Journal. Those polls estimated Biden’s end-of-campaign lead at 10 to 12 percentage points over then-President Donald Trump.
Biden won the popular vote by 4.5 points.
The discrepancy in 2020 between election results and polls overall was the most pronounced in 40 years, and prompted characterizations that the outcome was a “train wreck” and “a disaster for the polling industry,” as David A. Graham wrote in the Atlantic.
This starts with the assumption that polls are intended to be predictive. If that were the case, it seems to me that there would be a concerted effort among the pollsters' customers -- legacy media -- to increase their predictive accuracy, especially if their performance in the last two presidential election cycles was as disastrous as Prof Campbell says it was.I think a better starting point is to recognize what Rush Limbaugh understood throughout his career, that polls are intended to manipulate the news, not report it. As he said on September 10, 2012,
The polls are just being used as another tool of voter suppression. The polls are an attempt to not reflect public opinion, but to shape it. Yours. They want to depress the heck out of you, and they want to suppress your vote.
On January 19, 2018, he said,
I don’t trust the polls that are published anymore. As wrong as they were about the general election, as wrong as they were about the special elections that happened last year, the polls are nothing more than a tool for making public opinion. The polls are not taken to reflect public opinion at all. The primary purpose of polls is to depress and dispirit Republican voters by making them feel constantly like they’re in the minority.
The purpose of polls is to reinforce among Republican voters that there’s no hope, that no matter what it might look like, Republican voters are a very tiny minority of the thinking in this country.
He generally explained that the pollsters do this by consistently oversampling self-identified Democrats and undersampling Republicans with various justifications, and their results then reinforce the opinions they prefer -- which are often what their customers, the legacy media outlets, prefer.In the wake of the 2020 election, the Pew Research Institute did a detailed study on the possible effects of oversampling Democrats in polling, almost certainly in implicit response to Limbaugh's allegations. It looked at potential polling outcomes with an oversampling of Democrats versus a more realistic sampling and found little difference:
The adjustment from the tilted version (a 12-point Biden advantage with a 10-point Democratic advantage in party affiliation among nonvoters) to the balanced version (a 4.4-point Biden advantage with equal numbers of Democrats and Republicans among nonvoters), makes very little difference in the balance of opinion on issue questions. Across a set of 48 opinion questions and 198 answer categories, most answer categories changed less than 0.5%. The average change associated with the adjustment was less than 1 percentage point, and approximately twice that for the margin between alternative answers (e.g., favor minus oppose). The maximum change observed across the 48 questions was 3 points for a particular answer and 5 points for the margin between alternative answers.
One 3-point difference was on presidential job approval, a measure very strongly associated with the vote. In the balanced version, 39 percent approved of Trump’s job performance, while 58 percent disapproved. In the tilted version, 36 percent approved of Trump’s performance and 60 percent disapproved. Two other items also showed a 3-point difference on one of the response options.
But 3 points nationally is still a 3% error, not negligible at all, and this omits the bigger problem in recent presidential elections, where they've hinged not on the national popular vote, but on the outcomes in swing states and their effect on the Electoral College vote. What if we were to revise Biden's numbers in current polls down by 3 points? One thing that now seems to be happening is that polls that have proven highly inaccurate in the recent past, like Quinnipiac, are feeding outlier results into the Real Clear Politics average. This must certainly be deliberate, and RCP is enabling this effect.Thus we're seeing what I think are troubling indicators of the national mood, like the emrgence of conspiracy theories surrounding relatively minor events like the collapse of the Francis Scott Key bridge, or a real uptick in self-immolation protests, contrasting with happy-face statistics like an RCP national polling average showing Trump ahead by only 0.3 points. Real things are happening that aren't being reported.
Frankly, this can't be right. Part of the mainstream obliviousness is Prof Campbell's assumption that the pollsters' customers want accuracy. He quotes David A. Graham's Atlantic piece:
Graham’s essay anticipated the “snapshot-in-time” defense favored by pollsters, writing: “If their snapshots are so far off, where were they aiming the lens? Why bother?” He noted that “the public uses opinion polls to try to understand what happens [in elections]. If the polls and their analysts don’t offer the service that customers are seeking, they’re doomed.”
But I think Limbaugh was much more insightful, the customers want to shape the news, not report it. Prof Campbell completely misses that point.