What’s in a Poll?

by Sal on September 25, 2008

in Election 2008,Media Bias,Politics

The latest round of polls are out, and they are somewhat all over the map.  Polls currently range from McCain up by 1 (Battleground) to Obama ahead by 9 points (Washington Post), with most giving Obama a slight edge.  Looking at the poll averages over at RealClearPolitics, it is worth noting that mos of the polls listed are of registered voters, not likely voters.  I tend to discount any poll of registered voters, because it always skews Democrat, and it has not been an historically accurate indication of voter preference on election day.  Taking out polls of registered voters, we are left with the following: 

  • Battleground:  McCain +1
  • Rasmussen:  Obama +3
  • ABC News/Washington Post:  Obama, +9
  • LA Times / Bloomberg:  Obama +4
  • CNN / Opinion Research:  Obama +4

So even with Likely voter polls, the spread is all over the map.  What accounts for this?  Polls are usually weighed by Party Affiliation, meaning, if a pollster has data that shows that the nation is 38% Democrat, 34% Republican, and 18% independent, then that is how the poll sample is composed.  There are arguments on both sides of the equation of whether or not this is an accurate indicator, but for the sake of argument, let’s say that it is.  Of the five polls above, there was party affiliation data available for three of them.  Looking at the demographics for the three polls indicated shows the following: 

  • Battleground:  Democrats +3
  • Rasmussen:  Democrats +5.1%
  • ABC News/Washington Post:  Democrats +16 (780 LV + 183 AM)

Now, normally I consider Rasmussen to be the most reliable of pollsters, although Battleground does have a decent track record.  However, this year, the party identification weighing seems way off.  Jim Geraghty over at the National Review Online’s Campaign Spot points to a review of party identification over the last five Presidential elections, beginning with George H.W. Bush’s victory over Michael Dukakis in 1988.  The Party identification statistics for each of the presidential election years is as follows:

  • 1988:  Democrats +3, Bush (R) wins
  • 1992:  Democrats +3, Clinton (D) wins
  • 1996:  Democrats +4, Clinton (D) wins
  • 2000:  Democrats +4, Bush (R) wins
  • 2004:  Even, Bush(R) wins

Even in 2006, when the Republican brand was in trouble and the Democrats swept congress, the spread was only Democrats +3.  So the wild swings in Party Identification seem way off.  In the ABC News/Washington Post poll, it is obscene.  To weigh your poll by Democrats +16 is laughable on its face.  In addition to that, they sampled 780 Likely voters and then added an over sample of 183 African Americans on top of that.  Now, we all know that 90% of African Americans will be going for Obama, so this further skewes their 9-point lead. 

As far as Rasmussen, I am skeptical this year of his party weighing system.  About 50% of his current party identification numbers were gathered before the pick of Sarah Palin and the Republican convention, since he goes on a 6-week rolling average.  The 5.2% spread seems high historically, and so may be skewing things slightly in Obama’s favor in a way that is not representative of the actual electorate.  Battleground seems to have the most accurate sample, with a +3% advantage for the Democrats, which is in line or within a point of 4 of the last 5 elections. 

In summary, What’s in a poll?  The data presented can be skewed by any number of factors, and although useful for measuring trends, they can be used to skew the news.  Obama may have gained ground in the past two weeks, but his lead is likely not as large as many of the polls currently predict.

{ 1 trackback }

Dear Demoralized Republicans, | Axis of Right
September 25, 2008 at 5:08 pm

{ 2 comments… read them below or add one }

Mike September 25, 2008 at 11:01 am

Those party ID numbers caught my attention because Ryan and I had this discussion very recently. The wild swings in Party ID this year should be a major red flag because people RARELY change their party identification. Even people who label themselves as independent tend to support one party over another. The academic literature on this has been very clear. Party ID is fairly static.

Sure, turnout might increase or decrease slightly in a given year and some people to split their tickets. However, Ds who vote R or Rs who D in a given year still keep their own party ID even when supporting the other side. This dynamic between long-term forces (party) and short-term forces (a particular vote) is at play in every election. That’s why each side wins occasionally. But to have fluctuating party ID is a major red flag.

For the record, Rasmussen (“Independent”) and Battleground (One Democrat, one Republican) are excellent national polls with sound methodology. Gallup is a good one in terms of looking at trends. Wash Post, Fox (National polls), AP, CBS, Newsweek, Time, NBC, and Hotline are garbage.

At the state level, Mason-Dixon has been the gold standard for over a decade now.


Ryan September 25, 2008 at 4:37 pm

There has been some panic amongst some Republicans due to the polling data that has shown a swing (sometimes dramatic) towards Obama over the last ten days. This post is very helpful to those who aren’t quite sure what to make of “RV”, “LV”, or the garbage “Adults” sampling. Also, Mike’s comment about the short-term and long-term forces in each poll indicate an important point about the nature of the sampling.

To delve into anecdote: I know plenty of people who are registered Dems, likely to vote this year, but hate Obama. They’d be Registered, Likely Voters, identified as Democrats… who are voting McCain. The same is likely true for the other side as well.

I’m still concerned that McCain’s rarely ever up, even in the close polls, and that the trends have been pro-Obama lately — even despite the bias, some wind has fallen out of McCain’s sails. He needs to get his Mojo back by taking it to Obama at the deba— oh yeah, forgot about that.


Leave a Comment

You can use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Previous post:

Next post: