Inside America’s political polling problem

The Associated Press reporting that Clinton is in a commanding position

Donald Trump will be the 45th president. Nationally, it shows Joe Biden leading President Donald Trump

The race between President Donald Trump and Joe Biden still too close to call

Our new poll with Ipsos has Kamala Harris right now back in front

The next president of the United States is Donald Trump. Three presidential elections in a row

The polls pointed one way and reality went the other. Many people boil it down to a simple theory

People lied. But is that really what's wrong with modern polling? To find out, Straight Arrow turned to a swing state pollster

one of the nation's leading survey researchers, and a veteran TV news producer

Polling is strategic intelligence. Are you getting good intelligence or bad intelligence

And that matters, because if not, you might be surprised. That's why I think a lot of people thought the election was stolen

Mike Noble of Noble Predictive Insights has been polling in Arizona for more than a decade

Yes, that Arizona. Arizona, are you 100% sure of that call and when you made it and why did you make it

Absolutely. He says the problem isn't that people are dishonest. For him, it comes down to who is still willing to talk

People that have, let's say, a college education or white-collar workers, they're taking surveys like it's a hobby

They have more flexibility in their schedule. They can sit there and do a 5-10 minute interview. However, what you're missing is that people like more blue-collar jobs

or high school or less education, do you think they got time to stop in the middle of the thing and do a survey

So basically response rate-wise, you get a big imbalance there. 30 years ago, about one in three Americans would answer a pollster's phone call

Now, that number is fewer than one in 50. The polling industry says it's adjusted for that

but Noble says that's harder than it sounds. Making a poll is like a chef A chef you only see the final meal that comes out But again the ingredients used how he prepares it blah blah blah blah blah That how basically polling does So if you follow this recipe you should get this result

John Krosnick, a professor at Stanford University, has spent his career studying exactly how people answer survey questions

So when he went searching for evidence that voters were hiding their true opinions from pollsters

he kept coming up empty. Researchers who want to claim that social desirability is a big problem can point to those studies as long as they don't point to how big the effects are

The minute they start paying attention to the fact that the effects are pretty small, then it becomes less of a problem

When Krasnick and his team reviewed every study comparing what people say in surveys to what they do

What we discovered is that about 85 percent of people's self-reports in surveys are accurate

And that's often, that's quite a bit higher than many professionals guess before they know what that literature says

But what about more sensitive questions, where telling the truth might come back with a social cost

Krosnick set out to explore the merit behind an oft-touted theory on why people conceal their true views, racial attitudes

We found no more evidence of racial prejudice being expressed, for example, by white people against black people in the anonymous condition than in the oral condition

And so that's yet another piece of evidence, in my opinion, that really raises serious questions about the claim that you can't trust self-reports because people are lying

Despite it being a popular theory, there's weak evidence that people are lying to pollsters

That raises the harder question. If voters are mostly telling the truth

why do polls keep getting it wrong? It's the abandoning of random sampling that's the problem

The night before the 2016 election aggregated polls gave Hillary Clinton a 75 to 80 percent chance of winning Most of those polls were opt surveys made up of people who volunteer to take surveys online sometimes for money In the battleground states that decided the election that year

roughly 70 polls were conducted in the final week. The scene here is so different than it was a few hours ago

The random probability samples had an average error in predicting the Trump and Clinton share

of the vote of less than one percentage point. Really accurate. Whereas the average error of those opt-in sample surveys was much higher and sometimes a great deal higher

We just didn't have good quality telescopes to be able to see the outcome of the selection clearly in those places where it mattered

A few good telescopes were no match for dozens of blurry ones. And those blurry ones drove most of the headlines, which turned out to be wrong

Ben Bogardus spent years producing newscasts in Washington, Houston, and Jacksonville before becoming the chair of the journalism department at Quinnipiac University, a school that happens to host a major polling center

Bogardus says responsible newsrooms will evaluate the source. You want to go down and almost have a checklist. Who is this poll done by? Who paid for it? Who commissioned it? Is it an independent poll? Is it by a network? Or is it by one of the campaigns or an advocacy group

what is the quality of the poll in terms of the internals

How did they try to reach people? How much did they weigh it

What's the sample size? What's the margin of error? And then you have to say, what is the wording of the poll

Is it phrased in a non-biased, subjective way? Bogartas can rattle this off because of his many years of journalism experience

something increasingly missing in today's newsrooms. A lot of the people who do have experience, the lifers, are retiring or leaving the profession

And the younger journalists replacing them are facing even more pressures to be fast be everywhere and produce more stories with fewer people So if a press release comes in with a great headline it looks like a shocking news story based on the poll results you going to be sort of interested in putting that in your newscast

as something new, something different, something exciting. And on social media, where people are increasingly getting their news

the polling picture is even less in focus. If someone doesn't click through the article to read the actual specifics of the poll

they could get a sort of a lopsided view of what the poll actually says because you only have one or two sentences on your social media post

The problem isn't just polling. It's the system around it. And despite high stakes, flubs and diminishing public confidence in them, polls are still attention grabbers

Since the beginning of time, there's been this sort of problem in journalism where when you're covering political campaigns, it becomes more about the horse race

It becomes about who's ahead, who's going down. It's not really about the issues that are out there

The audience wants a simple answer. The algorithm rewards a simple answer

And polls, stripped of context, provide the simplest answers out there. Now, polling isn't broken

John Krosnick's research shows that when the right methodology is used, polls are still remarkably accurate

Mike Noble has built a career on getting those methods right. And Ben Bogart has spent years in newsrooms that strive to report polling responsibly

When done correctly, polls are still the best tools we have to understand public opinion

That is, when you know how to interpret them. I don't care what you say about prediction markets, all this other stuff

It is still the best tool out there. Is it infallible, perfect? No, because errors can happen in that process of a poll

However, it's still the best thing out there. The polls are coming

They always do. But the question isn't whether to pay attention, it's whether you're looking through the right telescope