Complex Super Tuesday Vote Calls for Careful Analysis of Poll Performance
With 14 states holding presidential primaries on March 3, Super Tuesday poses challenges for journalists, pundits and polls. The nation’s largest organization of survey researchers, the American Association for Public Opinion Research, is calling on analysts to be deliberate in their judgments of the performances of the Super Tuesday state polls.
“With the most complex election day of 2020 so far, we must all be careful to wait for final vote totals before rushing to judgement on how the polls performed,” said Nora Cate Schaeffer, AAPOR President.
By the time the polls close on March 3, dozens of polls will have been conducted in those states. In past years, pundits and journalists have been tempted to quickly render a judgement on the polls’ performance after such major events, haste that can lead to incorrect conclusions.
Democratic presidential primaries will be held in 14 states on March 3. Democrats in American Samoa will also hold caucuses that day. Republican primaries will be held in 13 states.
Democratic primaries will be held in Alabama, Arkansas, California, Colorado, Maine, Massachusetts, Minnesota, North Carolina, Oklahoma, Tennessee, Texas, Utah, Vermont and Virginia. Republican primaries are scheduled in all those states except for Virginia.
“We must wait until all the votes are counted before analyzing the polls’ performance,” said Schaeffer.
- AAPOR strongly urges journalists and analysts to wait for final primary results before analyzing polls’ performance. Without final official vote results, poll performance analyses can be far off the mark. The vote returns from California and other states on March 3 may pose special analytical problems.
- California’s system of early and absentee voting means millions of votes will not be counted until well after March 3. And the late-counted votes can be very different from the early returns. In 2016, Hillary Clinton defeated Bernie Sanders by a final margin of 53%-46% among the more than 6 million votes cast. But Clinton’s lead was in double digits, 13 percentage points, two days after the primary with only 3.5 million votes counted.
- Other states voting on March 3, such as Colorado and Utah, have also typically left a large portion of the vote uncounted until after Election Day.
- Events often have an impact on public opinion. With the Nevada Democratic caucuses and the South Carolina Democratic primary being held in the 10 days before Super Tuesday, voters’ views and preferences may well shift as a result of those events. The timing of a poll in a Super Tuesday state must be considered when judging how well or poorly that poll did in anticipating the results of the Super Tuesday voting.
- Analyses of poll performance must consider the fundamental uncertainty in all sample surveys, usually stated as the margin of sampling error. Failure to do so could result in a flawed analysis. This year, margins of error are critical. Results from both the Iowa caucuses and the New Hampshire primary showed the top Democratic finishers were within 2 percentage points of each other. That margin is substantially smaller than the margin of sampling error in every state’s pre-primary polling. In addition, many polls in Super Tuesday states have shown the candidates to be closely bunched, well within those polls’ margins of sampling error.
In January, AAPOR announced the formation of a special task force to examine the performance of public opinion polls during the 2020 presidential election. This effort builds on the work of the task force that showed the national polls performed well in the 2016 election. The task force — comprising 19 academic experts, pollsters, and statisticians — will evaluate the accuracy of 2020 pre-election polling for both the primaries and the general election on the presidential race and other races. The task force will collect information on the polls continuously through 2020, but will not be issuing interim analyses during the primary season.
Performance of Polls in Recent Elections
Pundits and journalists have paid increasing attention to how closely poll results match election results in the last few decades, even as some in the polling industry have debated the value of giving substantial attention to how closely poll results have matched election returns.
Major studies in the last four years show polling is as accurate as it has ever been. The national presidential polls in the 2016 general election were quite accurate by historical standards, despite frequent mistaken claims to the contrary. In fact, they were among the most accurate in estimating the national popular vote since 1936. (See https://www.aapor.org/education-resources/reports/an-evaluation-of-2016-election-polls-in-the-u-s.aspx.) State-level polling was not as accurate in the 2016 general election (notably in Pennsylvania, Michigan and Wisconsin), but it was reasonably accurate in the 2016 primary elections.
The 2016 presidential primary polls generally performed on par relative to past elections. The vast majority of primary polls predicted the right winner, with the prediction widely off the mark in only a few states. In short, the primary polls held their own in 2016.
–An Evaluation of 2016 Election Polls in the U.S.,
AAPOR Ad Hoc Committee On 2016 Election Polling, page 16.