秘密研究所

Polling 'Vital,' So Are Fixes

---

Posted on November 6, 2020

Jeb Schrenk

---

---

It probably doesn鈥檛 take a poll to gauge public opinion on political polling right now. Pollsters have been roundly criticized for a steady stream of data that missed the mark in reflecting voter attitudes in the 2020 election cycle. Dr. Thomas Shaw, associate professor of political science and director of the USA Polling Group, talks about how polling needs to evolve, why it鈥檚 still useful and what he鈥檚 telling his students this week.

How are political pollsters feeling right about now?

Probably not that good. I鈥檝e primarily been keeping an eye on places like Wisconsin, Michigan, Pennsylvania and Florida. Particularly in Michigan, Pennsylvania and Wisconsin, they are certainly not going to be anywhere near the numbers they were projecting Joe Biden to be up by. So, if you鈥檙e a state-level pollster, not so good.

We鈥檝e now had two presidential election cycles where polls under-counted the position of GOP voters. We鈥檙e not even in the margin of error on some of these, are we? What鈥檚 going on?

I think we have got pretty sound questions in terms of identifying the likelihood of voting. I think it is probably more just overall methodology. There are a lot of issues with telephone polling these days. It is just too easy for people to screen out calls. I鈥檓 a pollster and I鈥檝e been screening out my phone. And so you get a lot of systematic biases with telephone polling that are hard to account for, and pollsters this time were trying to make allowances for this 鈥� they were trying to revise some of their methodologies. So I am surprised, and in addition to the presidential polling there was polling suggesting the Senate would flip.

And the Democrats are losing House seats, which was not predicted.

Exactly. They were supposed to gain in the House. I think they will keep control in the House, but they were supposed to make sizable gains, according to most of the pollsters.

Well, polling is a science, right? Has the science not evolved to keep up?

It is a science, but it is a dynamic environment. Between the mid-70s and probably the early 2000s, you had a period in which telephone polling did a great job. And then as we transitioned into the mid- to late-2000s, that landscape really starts to change. We get the transition to cellphones. And then of course all the technology that goes along with your communication devices now that allows you to do screening. So what is going to have to happen with polling is a transition to the new landscape. Some of that is going to be moving to more electronic means, but right now there is such a diversity in terms of your provider and your email address, that it鈥檚 difficult to get a proper sample from a population. The other place I think we will see some changes is in the development of data analytics and other ways to get a public opinion. Dr. Philip Habel, who is the chair of our department, does a lot of work with social media, and I know he does a lot of what鈥檚 called scraping on Twitter, where you go in and see the things that are trending and not trending and how public opinion is fixing itself those trends. So I think what we will probably see is a continuation of traditional survey research that needs to adapt itself to the new landscape, and then augmented with other ways of trying to look at public opinion. Then, particularly to the extent that one can, we need to triangulate between the traditional methods and the other methods and see that there is definitely an underlying phenomenon going on.

For political polling, it seems like there鈥檚 a cottage industry on election modeling, and people pay close attention to those forecasts. Has that been a mistake? Should news organizations abandon some of this, or will people lose their trust in that type of forecasting?

It鈥檚 fascinating because exit polls are traditionally pretty unreliable. But they still do a lot of exit polls. So I don鈥檛 think they鈥檒l give up on them completely. For one thing, it lets them put numbers on stuff and one of the things we know that people love is a number. They love the horse race. I think you will see trust continue to erode. I see a lot of it in the social media that I look at. There was an immense amount of distrust particularly amongst the people that I know are President Trump supporters and were apparently right on their part. Clearly Trump over-performed what the polls were suggesting, so it鈥檚 just going to reinforce the mistrust they had to begin with. But I don鈥檛 think we will see it completely go away. It still gives the news people a tool to utilize. I think it is going to force the pollsters to have to continue to try to work to improve what they are doing. The science was there between that period of the 1970s and early 2000s. I think that science needs to be tweaked and adjusted to better accommodate where we are.

There鈥檚 a story in The Atlantic, and it argues that the biggest problem posed by the missed polling is not with elections. 鈥淭he real catastrophe,鈥� it states, 鈥渋s that the failure of the polls leaves Americans with no reliable way to understand what we as a people think outside elections 鈥� which in turn threatens our ability to make choices, or to cohere as a nation.鈥� Is that too doomsday?

I think it might be a little excessive. Here is the downside to the election polling as far as it relates to nonelection polling: It鈥檚 usually been the success of election polling that has allowed us to suggest that other polling is good. If we are way off on the elections, how do we not know we are way off on these other things? I would say we have to have mechanisms to get a public opinion. And polling is still one of the best ways that we have to get at that even if it is a little further off than what we might think.

I will give you an example. When I was in Cincinnati, Hamilton County was trying to start a developmental project on the eastern side of the city, and one of the things they did, the engineering firm that they hired came in and the first thing they did is they started scheduling a number of town hall meetings to get community members to come in and actually talk about this developmental project. But then they also commissioned the company where I worked to come in and do a survey of the area in order to be able to find out what people thought about it. And in a lot of ways it was really good for them to have both of those perspectives, because not everybody goes to town meetings. You know a lot of people have kids, they have soccer practice, they just can鈥檛 make it. But the people who go to town hall meetings are usually your very vocal opponents or supporters. So it is absolutely vital to hear what those messages are because those messages are going be there. The survey is not going give you your big critics or your big supporters, but it鈥檚 going tell you what the person who doesn鈥檛 have time for the town hall meeting 鈥� the 鈥榮ilent majority鈥� 鈥� thinks about things. It鈥檚 important to again triangulate the information and not to rely just on one source.

What are you telling your students this week?

In my methods class we talked about survey research on Monday. And so I referenced some of the polling that was going on then. Now with the vote in, we will probably talk about how it鈥檚 definitely going be a pretty serious setback for the polling industry. And that it鈥檚 going to continue to level a lot of criticism at polls, and as students they need to be aware of that; they need to realize the significance of polls. It鈥檚 vital that we have polling, but we have got to know if it鈥檚 not working so we can fix it.

---

The USA Polling Group has conducted more than 600 surveys for various clients on topics including anthropology, community relations, criminal justice, educational administration, healthcare, marketing, public administration and sociology. It did not conduct political polling in the 2020 election cycle.

---