Inaccurate State Polls

by Mike on October 20, 2008

in Election 2008,Media Bias,Politics

Many websites have done a great job of dissecting national polls to determine how reliable they are. Because of these efforts, polls conducted by the Washington Post, Newsweek, CBS, etc. have been exposed for the garbage they are. Unfortunately, state polls have not been as heavily scrutinized.

Motivated by my own curiosity, I recently looked at the internals of some state polls cited by Real Clear Politics in their state-by-state poll averages. What I found is that many of the state polls out there aren’t worth the paper they’re printed on. This is especially true in Virginia and Pennsylvania.

The polls coming out of Virginia show Obama leading anywhere between 6 and 10 points. It is not unbelievable to see a Democrat leading in the Republican state. After all, the Democrats have won two consecutive Gubernatorial races and the Commonwealth’s most recent Senate race. What is unbelievable is the size of Obama’s lead, if he has one at all.


One poll published by an outfit called CNU (PDF) shows Obama leading McCain by six points. 35 percent of the polls respondents were Democrats and 30 percent were Republicans. Another poll by Survey USA recently showed Obama with a ten point lead. 39 percent of that polls respondents were Democrats and 30 percent were Republicans. Finally, a Suffolk poll showed Obama leading McCain by twelve points. The sample size: 45 percent Democrats and 31 percent Republicans.

Democrat party ID advantages in these three polls were 5, 9, and 14 points respectively. How do these numbers compare to recent elections in Virginia? They don’t. Party identification rarely shifts from election to election, even when the results do. For example, when President Bush won Virginia in 2004, 39 percent of voters were Republican and 35 percent of voters were Democrats, a four-point edge for Republicans. But that was a Republican year. Fair enough. In 2006, when moderate Democrat Jim Webb won statewide in Virginia, 39 percent of voters were Republican and 36 percent were Democrats, a three point edge for Republicans. A different party won, but the voters were the same. That’s because party ID rarely changes.

The polls coming out Virginia would have us believe that a state in which more Republicans show up at the polls regardless of whether a Republican or Democrat wins the election, all of a sudden not only has more Democrats, but as many as fourteen percent more. Sorry. And another thing on the CNU poll, 58 percent of the respondents were women.


Pennsylvania polls are no better. A recent poll from Morning Call shows Obama leading McCain by 12 points. The internals showed that 54 percent of the respondents were Democrats and 40 percent were Republicans, a 14 point edge for the Democrats. Another poll from Survey USA showed Obama leading McCain by 15 points. The sample size showed that 54 percent of the respondents were Democrats and 35 percent were Republicans, a whopping 19 point edge for Democrats. How do these spreads compare with recent election in Pennsylvania? They don’t.

When John Kerry won Pennsylvania against President Bush in 2004, 41 percent of voters were Democrats and 39 percent were Republicans. But that was a good year for the GOP. How were the numbers when the Democrats swamped the Republicans across the country and when Bob Casey slaughtered Rick Santorum in 2006? Answer: 43 percent of the voters were Democrats and 38 percent were Republicans, a five point edge for the Democrats.

Pennsylvania Democrats have outnumbered Republicans in recent years and the difference between the number of voters in each party is basically the same regardless of whether it was a Republican or Democrat year. That’s because party ID rarely changes. The current polls in Pennsylvania would have us believe that the Democrats have a double-digit party ID advantage even though they only managed a five point edge the Democrat tidal wave of 2006.

Final Thought

Obama may be leading in these states, but we have no way of knowing for sure. When analyzing a poll, there are many other factors to continue, such as demographics, income, gender, hometown, etc. In many of the above-mentioned polls, party ID was not the only internal measure that is skewed. Skewed party ID is a red flag however.

I didn’t want to write a post to rival the length of War and Peace, so I’ll stop here. Polls out of Ohio, Minnesota, and Missouri are also questionable. Head over to Real Clear Politics for more info, but be forewarned, many polls don’t even publish their internals. I wonder why.

UPDATE: Welcome Campaign Spot readers.

{ 2 trackbacks }

Reasons for Optimism | Axis of Right
October 30, 2008 at 12:56 pm
Mike’s 2008 Election Predictions | Axis of Right
November 3, 2008 at 8:12 pm

{ 14 comments… read them below or add one }

Ken October 21, 2008 at 7:35 am

You’re comparing apples and oranges; these state polls aren’t necessarily asking voters how they are registered to vote, but who they more identify with, or what they consider themselves.

And in Pennsylvania, at least, Democrats DO have a double digit party registration advantage, 51% to 37% at the moment.

See these voter statistics (XLS).


Mike October 23, 2008 at 9:52 pm

Thanks for reading the post and commenting.

With all due respect, I think you’re comparing apples and oranges in that you’re comparing party registration at the voting booth with registration. Democrats have had a registration advantage since the days of FDR, but never turn out at the rate their registration percentages suggest. As my post showed (and many other elections), major fluctuations in party ID from election to election are rare. The difference in PA from 04 to 06 was 2 points, even though the results were the polar opposite.

Every year, we hear about Democrat registration advantages that don’t materialize. This year may turn out to be different, but nothing in previous data says it will. Party registration doesn’t either.


RW October 24, 2008 at 10:15 am

One thing to consider when simply looking at party registration in some states: Operation Chaos.

Google it. ’twas no small matter.


Fisher October 24, 2008 at 11:26 am

I enjoyed the post. One thing for sure, if McCain is able to pull this out there are some polling organizations and corporate media folks that will have a hell of lot of explaining to do.


KG October 24, 2008 at 12:01 pm

Time and time again we have heard that the voter turnout will be huge, that Democrats are being registered in record numbers, etc. But unless these people actually VOTE all the registrations and dire predictions do not matter.

I thought LAST time we were supposed to have this huge surge of voters appear at the polls, and we still had under 60% turnout, I believe.

I have read in my local paper (I’m in VA) that they are expecting an 80% turnout. Luckily, I live in a rural area, we have short lines, and I don’t have to be anywhere on Election Day, so I can wait in line as long as necessary.

For those Republican voters out there, do whatever it takes to be at your polling place EARLY. Please consider being late for work and staying at your polling place no matter how long it takes.

A woman this morning told me when she went to vote in the last presidential election, the woman checking her in to vote asked her party affiliation. DON’T ANSWER those questions. They are NOT allowed to ask those things, as far as I know. It also makes me very suspicious of this woman’s motives for asking.

I don’t believe the polls for the very reasons listed here. I think many voters remember 2004 and the screwy exit polling, so I hope they are just as wary of polls as I am.


Dr Irish October 24, 2008 at 1:28 pm


unfortunately they are never required to answer for anything. they say woops and thats it and they get right back to purposely skewing things toward liberals.


rightwingprof October 24, 2008 at 4:30 pm

There’s no way to directly measure the effect of Operation Chaos, but note that about a week after Rush started it came the first stories in the press here in Pennsylvania about people changing their registration. And if McCain’s internal pollsters were telling him Obama had a double digit lead here, he wouldn’t be working Pennsylvania as hard as he is.


Joe C. October 24, 2008 at 8:56 pm

Since 1988, the largest party ID differential for any election – that includes 2 Clinton wins, the Rep takeover in ‘94, the Dem takeover in ‘06 – has been Dem +4 in ‘96.

Every election Dems think THIS will be the year that all those winos, homeless, and college students are going to vote, and it hasn’t happened yet. Some day it may (possibly ‘08), but I have no reason to believe that this will be the year. Remember, all those newly registered voters were unregistered for a reason – THEY DON’T VOTE! I would assume any other national poll with a Dem advantage >4 is likely suspect, and needs to be adjusted at the point of consumption.


The Ohio Poll – the most accurate polling service in Ohio – which has projected Election Day winners in 34 of 34 statewide contests polled since ‘94 has McCain +2 in Ohio. All other polling services give Obama a lead anywhere from +2 to +5 (some more). I’ll believe The Ohio Poll – based on track record – which suggests that the other polls are 4-7 off to Obama’s favor. The question is, “Does this reflect polling in other swing states?”


Rick C October 27, 2008 at 1:53 pm

I understand you analysis but there is a problem. The idea that perty ID does not change much has met up with the lowest rating for the GOP in the history of tracking.

A look at party ID changes in key states is telling

This tells the tale unfortunately.


2000 35 34
2001 35 32
2002 35 32
2003 35 35
2004 33 33
2005 33 33
2006 36 30
2007/8 37 25

While the year by year changes might be small (except for this year) the declining GOP trend over the 4 years in Ohio is startling.

Ohio GOP had its special problems there but it’s baked in.

From 2004 to today though the GOP ID is down 8 points.
The DEM ID is up 4 points.

That is a huge 12 point swing. They may not all vote DEM but the race was only 1 point in 2004.

Pennsylvania is similar

2000 41 36
2001 40 37
2002 39 38
2003 40 39
2004 41 37
2005 42 36
2006 40 36
2007/8 43 31

The huge shift this year is why Obama is so comfortable saying McCain=Bush=GOP Policy.

But taken again as a 4 year swing
DEM +2
GOP -6
Another 8 point swing! I can’t see it despite McCain’s best effort in PA.

North Carolina is similar. Though a DEM in NC is not exactly a DEM in PA or Ohio typically. The “new” DEMS though are which is part of the problem.

2004 39 35
2007/8 39 26

GOP is down 9!


2000 29 32
2001 26 39
2002 31 31
2003 30 31
2004 32 32
2005 34 31
2006 33 30
2007/8 32 28

Again the GOP has fallen.

But from 2004 to 2008 its DEM +0, GOP -4 for a four point gap.

The bottom line is this in Blue states the DEM gap has grown, in swing states it has reversed or grown for the DEMS, and in red states it has narrowed.

Red states

2004 25 44
2007/8 26 38

From a 19 point gap in 2004 to a mere 6 now.

What this all means is that state polls in places like OH are probably right to give a party ID gap. Thought the VA gap might be too big its a big change since 2004.

When you look at the changes in governors and senators in these states it becomes apparent as well.

2008 is going to be a rough year. I may not like it but denying reality is not going to help.


Rick C October 27, 2008 at 1:54 pm

sorry about the typos


Mike October 27, 2008 at 2:04 pm

With all due respect, you just proved my point that the polls, in your case PEW, overstates Democrats. The point spread in Ohio was not +6 Dem in 2006 as your survey suggests. It was +3. In Virginia, the spread on election day 2006 was GOP +3, not Dem +3 as your survey suggests. Polls overstate Dem margins, and the survey you cite is no exception.

It was one thing to have new registrations, as there were in 2004 and 2006. It is quite another to have a drastic change in party ID at the ballot box. There is a reason why realignments are so rare and that is because party ID rarely changes drastically from election to election. Your survey did not cite the party ID of voters on election day. At the end of that day, that is what matters. My point stands.


Rick C October 27, 2008 at 2:49 pm

I understand your point and I agree with the oversampling ID issue depending on state. My point is about the differntials between 2004 and 2008. Obviously not all DEMS voted DEM. This is highly variable state by state and especially skewed in southern states. (I.e. DEMS vote GOP in national elections and the difference between DEM and GOP candidates is quite differnet ideologically. i.e. More like 2 conservative leaning candidates with different brand names. Thank God for that at least.)

However, whether DEM ID is overstated or not in a given year, assuming the same methodology yearly, the differentials from 04 to 08 are the ominous tell.

OH -12
PA -8
VA -4
NC -9
red states down
swing states down or reversed
blue states even wider

If it’s the same error, the delta is still a major problem.

these are serious headwinds and suggest certainly that whatever the true ID delta was in ’04 its shifted unfavorably for ’08. I agree the pollsters shoud probably let the voting confirm the weights (and delta) before assuming such a shift in polls. But on the other hand they are trying to predict the ID picture for the election as well so using ’04 is probably wrong too with “rare deltas” in the mix.

I would say that PEW has identified a shift regardless of there basic overweighting of DEM error.

“Party identification rarely shifts from election to election, even when the results do.”

I agree 100%. But the key word is “rarely”. The PEW shift is indeed rare according to their historical data. It might be dead wrong but if it’s not this election could get ugly.

The pollsters may have overall ID skewed too high, but on delta alone OH would fall given the closeness of the race in ’04 and the huge delta in ID. Given this I’d suggest NC could be more a problem than VA but time will tell.

the basic problem though is that CO , NM, and IA are probably lost. And everything, regardless of poll skew, is being played on “our end of the field.” A simple look at where all candidates are tells that tale.

FL, OH, CO, VA, NC, IN, MO and oh yeah PA. Go PA we need it bad.

I think Obama does NOT have a 8-10 point lead in VA. But I do think he has a small one. Same for Ohio. TURNOUT is what we need because it certainly trumps “ID”. So let’s hope the Alaska Blasta revs up the troops.


Mike October 27, 2008 at 3:18 pm

We actually agree more than I thought. My point is that although our guy is down, the skewed state polling tells us Obama’s lead isn’t what they say it is.

“I agree 100%. But the key word is “rarely”. The PEW shift is indeed rare according to their historical data. It might be dead wrong but if it’s not this election could get ugly.”

Ultimately, that is the question. Seeing as party registration has trended D in recent years without a matching jump in actual voters, its the other side banking on realignment. It’s possible of course, but the side doing that is betting against history.

“I think Obama does NOT have a 8-10 point lead in VA. But I do think he has a small one. Same for Ohio. TURNOUT is what we need because it certainly trumps “ID”. So let’s hope the Alaska Blasta revs up the troops.”

I basically agree with you here and that was my point. Though I think McCain has the small lead in OH (though we’re both within the MOE).


Rick C October 27, 2008 at 3:59 pm

We’ll see how the turnout war works out. Obama has changed the strategy to get people with a “D” to actually vote in many states that actually had larger apparent deficits four years ago. The last two dems really only tried to get the 49s to change to 50s. (i.e. FL ,OH, NH, NM, WI battleground strategies).

If we wasn’t able to rasie so mush $$$ he would have lost with his broad and ambitious strategy. As it is it will be closer than polls say in my opinion because FL and OH are still close and HIS strategy means he has to win CO/VA if FL/OH stay Red.

I’m pretty disappointed in the overall campaign by McCain though. Seems to me they should have worked to destroy Obama in CO, VA, and maybe even NH on taxes which would have let the polling meme play into a deflating electoral one. As it is Obama has gained a momentum of pop vote plausability AND electoral plausability that has his support energized all over the freakin map instead of staying at home like a defeated party ID lazy dem often does in places like OH and VA. lol

In the few polls that have tracked “Already voted” the Obama ground game is perhaps too much in a close contest. He has enough $$ to have a big rally with onsite taxi (Volunteer) service to the polls in these early voting battlegrounds. And that’s not really a joke even if it is an exaggeration.

We’ll see but it’s uphill from my vantage point that’s for sure.


Leave a Comment

You can use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Previous post:

Next post: