The upcoming election is about uncertainty. So it’s no surprise that many Americans rely on the polls to provide comfort, or warning. Not all polls are created equal, however – and polling experts warn that some are more biased than others.
Not only that, pollsters vary widely in their level of access, and how closely they adhere to industry standards regarding data accuracy.
Take this poll released Wednesday by Quinnipiac University, which says Vice President Kamala Harris was ahead of former President Donald Trump by 3 percentage points in the battleground state of Michigan. On Twitter/X, Harris’ improved result encouraged his supporters, while Trump supporters challenged the validity of the poll.
The Tweet may have been deleted
Samara Klar, Ph.D., a political science professor at the University of Arizona’s School of Government and Public Policy, emphasizes transparency when it comes to deciding which polls to approve.
“The poll buyer should be able to clearly see how the data was collected, where it was handled, how many people are in the sample, and who the demographics are,” Klar tells Mashable.
A weighty story
Pay attention to whether the survey results are weighted, Klar adds, referring to the statistical analysis performed on the data after collection. Weighting aims to correct for sampling error by weighting certain responses differently to account for underrepresented groups in the survey.
For example, if a minority of survey respondents are Gen Z, or women, that surveyor may give more weight to the responses of younger women than older, male participants.
“If the data has weight, it helps to know how it’s measured,” Klar said.
Ideally, he adds, polls should have sample sizes closer to 1,000 respondents, “as this allows for smaller margins of error and closer estimates.”
Why Elon Musk jumped on Donald Trump’s stage
The margin of error – usually around 3% of 1,000 respondents – is a warning, acknowledging that the sample will not be able to provide the full picture. The American Association for Public Opinion Research (AAPOR) defines margins of error as “that wide [a respondent’s] the answer would probably fall in the middle if we had talked to everyone instead of just a sample.
Mashable Top Stories
“For example, if a national poll of adults with a margin of error of plus or minus 3 points finds that 58% of the public approve of the job their governor is doing, we can be sure that the true number would be somewhere in between. 55% and 61% if we did a survey of all adults in the province.”
‘There is no way to ensure that the survey is reliable’
Even accounting for appropriate sample sizes, weighted data, and margins of error, David Wasserman, senior editor and election analyst at the nonpartisan Cook Political Report, paints a less than rosy picture of polling accuracy.
“There’s no way to be sure the poll is reliable because response rates are so low these days,” Wasserman said. “Every pollster makes a different assumption about who’s going to turn out and vote that may or may not happen. You can give the same structured data to 10 different pollsters and you might get seven or eight top-line results. In a survey based on what pollsters think each group of voters will bring together voters. “
If it seems like random polls are popping up everywhere lately — not just those from established pollsters like YouGov or the New York Times/Siena College — that’s because they are. “There are a lot of new voters with no records or limited records this cycle, just like there was in 2022,” Wasserman said. “Democrats like to target Republicans who are all over the place with Republican polls.”
“There is an effort by the general public and other pollsters to fix the sub-samples of Trump’s base of support in 2016 and 2020. Voters do that in different ways but one of the most common ways is to measure their sample by the way they vote. Remember to vote in the 2020 election.”
The weight of the so-called recall vote is intended to correct the reluctance of some voters to agree to vote for the loser of the president. So the voters who have weighted recall votes this cycle will emphasize those who agree to vote for Trump in 2020.
One thing that all good therapists have in common, according to both Klar and Wasserman, is adherence to the standards set by AAPOR. Members of the association, which include highly respected voters, agree to abide by the association’s Code of Ethics and Conduct. That includes training standards, transparency, sampling methods, and weighting.
Reaching out to voters in the present
The lack of responses in many polls requires careful consideration in relation to weighting, Wasserman said. While the idea of ​​voters calling landlines is outdated, Wasserman says, even reaching out to people via cell phones, texts, or online panels is a challenge.
Many voters have started using email to reach respondents, according to the report New York Times — often with the offer of a financial incentive to take an online survey, called an opportunity panel. The new approach is a way to combat the low response rates of random polls, which only one notable pollster, Quinnipiac, still does.
“It’s common for telephone polls, even cell phone samples, to use less than 1% completed responses,” Wasserman said. “For every 100 calls you make, you might get one completed survey, sometimes less than that.
“The text of the web modality reaches younger voters. But it is difficult to reach voters aged 18-34 years old no matter what method you use, so what ends up happening is that the pollsters raise the weight of the respondents who receive in that age. to show their expected share of their voters.
The Cook Political Report includes national polling on its website, which pulls the latest data from reputable and diverse pollsters, such as Fox News and ABC News/Washington Post. Three times this year, Cook has conducted his battleground polls with a large online panel.
“We cannot hope that our numbers reflect the true state of play, but we have made a great effort to come up with a way that our partners vote, the Democratic Alliance company and the Republican company, that both feel comfortable with,” said Wasserman. .
While imperfect, polls still serve an important purpose, Klar insists.
“Polls are good at showing us a snapshot in time: what people are thinking now,” he says. “Forecasting requires polls to predict the future: Who will actually turn out to vote weeks, or months, or sometimes years, from now? Will people change their minds between now and then? Then polls are very important.”
On the other hand, “if you want a crystal ball to predict the future, you have to take the poll results with a grain of salt.”