5 Questions: Polling Is Not an Exact Science Says Statistician Andrew Gelman
With the presidential election just two weeks away, the latest polls suggest that Hillary Clinton will win, some saying her chances are better than 90 percent. But individual poll results vary widely and some still give Trump a chance of turning things around.
Why the discrepancies? “Polling is not an exact science,” said Andrew Gelman, professor of statistics and political science and founding director of the Applied Statistics Center at Columbia. “You have to be careful.”
With colleagues, he recently studied 4,221 polls to compare poll results to the actual outcome in 608 federal, state and local elections since 1998. They found that the actual margin of error once the votes were counted was between 6 and 7 percent, about twice the estimated margin reported with poll results.
Here he answers questions about the polling in the current election campaign.
Q. Why is polling not an exact science?
A. Different polls use different methods and survey people on different days. They reach different people. The non-response rate is estimated at more than 90 percent. Poll takers try their best to get a random sample, but if they don’t, they adjust for differences between the sample and the population at large. Those adjustments are based on assumptions. When a candidate is losing, his or her supporters are less likely to respond to a poll. That doesn’t mean they are less likely to vote. I think that if the election were held today, Donald Trump would do better than what the polls are showing.
Q. Is there a chance Trump can win?
A. A one-in-ten chance seems like a reasonable number. For presidential elections, one in ten takes us back 40 years. In that period we have had just one election when there were big changes just before the election: In 1980 Ronald Reagan won the presidency after losing in the polls for months. Based on what has happened before, things don’t change much in the last three weeks. By now most people have decided who they’re going to vote for. But you can’t be sure.
Q. Will the third-party candidates make a difference?
A. Although the third-party candidates this year are doing better than many in the past, they are still far behind the Republicans and the Democrats. Once it’s a two-candidate race, you don’t want to waste your vote. It’s hard to break the threshold of getting people to vote for you if you don’t have a chance. The federal system, with the electoral college, makes it hard for third-party candidates to succeed.
Q. Should the polls still be getting so much attention?
A. You shouldn’t bother paying attention to individual polls. The aggregators such as pollster.com are more accurate. Many polls are done for commercial purposes in partnership with a survey organization, with questions about the election added to questions about such things as what kind of toilet paper you use. Horse-race polls are a loss leader, a way to get your poll in the news. This is not to say that these polls are no good, it just gives some insight as to why there are so many polls out there.
Q. What’s your outlook for this election?
A. This is not a bad year for the Democrats; the economy is doing okay and the president is fairly popular. There has been a narrative that after two terms of a Democratic president, it is time for the Republicans to take over, that the party in the White House usually changes after two consecutive terms. I don’t see that strongly supported in the data. In 1988, Reagan was succeeded by George H.W. Bush. In 2000 [after Bill Clinton’s two terms], as we know, Al Gore won the popular vote. At the start of this campaign I would have given the Democratic candidate 52 percent of the vote, others said about 49 percent. That’s not too far from where we are now. All that is without reference to the particularities of this year’s candidates and their campaigns.
—By Georgette Jasen