MaT’s Journal #17: Understanding polls

Around 8 or 9 p.m. Thursday night I headed over to Drudge and was greeted by this “shocking” headline. When I woke up the next morning to head to class, it was still the blaring headline only in black font instead of the eye-catching red. He wants you to think this is a big deal. Somehow, John McCain is within striking distance of winning! There’s also an underlying implication that there is some liberal media conspiracy that isn’t reporting the closeness of the race because they simply want Obama to win. Predictably, conservatives took notice and got all excited while liberals started freaking out like they always do. This mini-controversy all stems from this data. Let me try to provide some context.

I think everyone knows that the vast majority of polls are showing Obama up a relatively substantial margin. Usually anywhere from 5-10 points. So how does this jive with Gallup, considered the “Gold Standard” of polling, only showing a 2 point Obama lead? First things first, let’s look at the numbers. Drudge is cherry-picking the most positive number he can from the three seperate data pieces available to him. He’s going with the “Likely Voters (Traditional)” method to present his case. Gallup has recently begun showing three seperate models. One is the “Registered Voters” model. What this means is that it only reflects people who actually tell them that they are registered to vote. The “Likely Voters (Expanded)” means that Gallup has adjusted the weighting of the poll to reflect a more accurate (in their estimation) representation of the makeup of the country. For example, the Expanded model assumes that there have been shifts in such things as Republican and Democratic party affiliation and youth vote, etc. Essentially, they take all the data they have received over the past couple of years, crunch the numbers, then put out their best guess as to what the public looks like right now; what the makeup of the elecotorate is.

Now, what’s getting all the attention in the blogosphere is the “Likely Voters (Traditional)” model, which shows McCain only down 2 points. There are a number of reasons why anyone should be skeptical of this last one and wary of those who push it into the media narrative.

First off, the obvious. This model assumes that there has been absolutely no shift at all in the makeup of the electorate. As the word says, this is the “traditional” model. There are a certain percentage of Democrats, Republicans, and Independents and historically this is how they vote. To make it easier to understand, the Traditional model essentially says that this year is pretty much exactly how it was in 2004. In this scenario, it is easy to see how McCain could be only 2 points down, right? The problem is that there is no evidence to suggest that we are simply re-living 2004 all over again. Massive voter registration is one key aspect. There are now more registered Democrats then there were four years ago. The public is more engaged in the election, largely due to the extended Democratic primaries and historic nature, then they were. And probably the most telling reason is that, in the last four years, pollsters have now begun picking up hard numbers on the “Cell Phone Effect”. In other words, you’d be hard pressed to find anybody, liberal or conservative, that would tell you the electorate is the same now as it was four years ago. The evidence just doesn’t back that up.

So why is Gallup even showing a Traditional model? One of the biggest reasons is to publicly find out if there is a difference. Precisely because there is a unanimous agreement amongst pollsters that this is a weird time we’re living through, no one is quite sure what to expect. This is another example that solidifies the argument that this year is not 2004. Pollsters have been frighteningly off many times this year. The famous example from the primaries is New Hampshire in which every poll showed Obama winning by a huge margin. He lost, of course. The result of which caused Gallup to stop doing state polls until they could figure out just what the hell went wrong. I’ll give you three guesses as to which model was used back then?

Frankly, the Likely Voter method has been horrendous at predicting anything this year. One case in point came a couple months back when Gallup’s tracking poll showed Obama ahead while their USA Today poll, done in the same timeframe, showed McCain leading. It was an embarrasing moment and one that left the company red faced, especially after the New Hampshire debacle. Looking deeper into the numbers, it became apparent that what was actually a better gauge of what was happening on the ground was the Registered Voter model. When pollsters call people to do a survey they ask screener questions in order to decide how likely it is that the person they are talking to will actually vote. Every company has a different methodology and their own science as to how, and in what order, they ask the screener questions. Gallup is actually quite famous for their methodology, as it is one of the most intensive, and question heavy, in the business. So how does this relate to what I’m discussing today?

Well let me use an example that happened yesterday. I made my mom go to the election place here in Lincoln to vote early. The place was packed and as I waited for her to fill out a ballot, I saw a guy come in and say “I’d like to register to vote”. Now let’s pretend that I called this guy last week to do a survey with him. I ask him if he’s registered to vote and he would have said “No”. I’d ask him if he plans to vote and he’d have said “Yes”. This voter would then be placed into the “Likely Voters” model. It’s important to understand that if someone tells a pollster that they plan on voting, we sort of have to account for them in some fashion, even if they never show up to the polls. Hence the “Registered Voters” and “Likely Voters” models. Now when the guy asked the lady to register him, she told him that the deadline had passed. For the sake of this example, what you have here is a voter who will not be able to vote, yet one that is represented in a poll of “Likely Voters”. As you can imagine, results will inevitably be skewed a bit. The conventional wisdom amongst pollsters is that if you haven’t registered to vote by now, then odds are you won’t. In many states, the voter registration deadline is passed, so this late in the game, Likely Voter models of polling are suspect. One interesting note on Gallup in particular is that when they were embarrassed by showing two separate polls in the field at the same time showing two separate results, they themselves said to put more stock in the Registered Voter method (of which Obama was leading). So if the headline is being based on the Gallup name in order to induce “shock”, then it should also be pointed out that they actually stand by the Registered Voter model…something that Drudge obviously would not point out.

That being said, it’s important for everyone to understand that nobody, even Gallup, really knows what is going to happen on November 4th. Nobody expects the Traditional model to hold up this year. Gallup is posting it so readers can have a side by side comparison and to give the public a more complete picture of how things stand in comparison to 2004 . Though unlikely, it is still possible that the expected turns out to be the unexpected. If the Traditional model holds up then it is quite possible that Obama could lose. The problem is that pimping that model is disingenuous at best or flat out deceptive at worst. The reason Drudge ran this headline is to push the media narrative of a “McCain Comeback” (which was actually his headline on Monday). And there is nothing wrong with that. All sorts of sites cherry pick polls and push narratives for Obama. The difference is that Drudge has a lot more sway than other sites. Because his audience is mostly conservative (love those post-debate polls he does where 75% of visitors picked Palin-McCain ;) ) he is giving them what they want. Like the link to some AP-yahoo .pdf file that showed a poll where McCain was also down 2 points. How many people are actually going to open a .pdf file and read through the minutia to discover that the poll isn’t even designed for horse race numbers? Not very many. Most people are just going to see the headline, believe it reinforces the narrative based on the Gallup number, and start getting excited again. Again, I don’t begrudge Drudge (droppin mad rhymes) for doing it, I just laugh at the people who take it so seriously, and that goes for both liberals and conservatives.

If you’ve read this and said to yourself “I’m confused. How am I supposed to interpret the polls in the next two weeks?”, I’ll help you out. First, expect the race to tighten up. All elections do. As long as Obama stays in the upper 40’s in most polls, he’ll be in good shape. The fact that he has consistently been over 50% in multiple polls for two weeks straight is sort of eye-popping and definitely not normal. But if you really want to know what is happening, don’t pay much attention to the horse race numbers (if you must, watch the Registered Voters model). Instead, watch the state polls. In particular, Colorado and Virginia. Obama isn’t going to lose any Kerry states and he’s already got two Bush states in the bag with Iowa and New Mexico. All he needs is either Colorado or Virginia to be president. Virginia is where all the attention is because since it’s on the east coast, it’ll be called early on November 4th. If Obama wins it, supporters can start drinking early.

Obama has hit his ceiling in terms of support. Supporters expecting him to be at 55% or more are just delusional. His numbers will go down. Expect a tough fight.

P.S. If you want an example of how campaigns are constantly trying to one-up each other look no further than Obama’s recent comments that he could “lose”. It is no coincidence that this coincides perfectly with Drudge’s pro-McCain polling push. It’s one of the oldest tricks in the book: take advantage of your opponent’s perceived advantage. If Drudge tries to give hope to the millions of conservatives who read his site, Obama is using that same meme as an opportunity to keep his supporters from complaceny. Political one-upsmanship :)

Leave a Reply