* How did the pollsters do on the Illinois governor’s race? I don’t think it’s ever been this bad. Nobody’s average even came close. From RCP via Zorn. Click the pic for a larger image…

* October polling. Again, click the pic for a better view…

PPP should’ve stopped in mid October when it was ahead.

* Rasmussen was also way off nationwide. From FiveThirtyEight, also via Zorn…

I did a quick check on the accuracy of polls from the firm Rasmussen Reports, which came under heavy criticism this year — including from FiveThirtyEight — because its polls showed a strong lean toward Republican candidates.

Indeed, Rasmussen polls quite consistently turned out to overstate the standing of Republicans tonight. Of the roughly 100 polls released by Rasmussen or its subsidiary Pulse Opinion Research in the final 21 days of the campaign, roughly 70 to 75 percent overestimated the performance of Republican candidates, and on average they were biased against Democrats by 3 to 4 points.

Every pollster is entitled to a bad cycle now and again — and Rasmussen has had some good cycles in the past. But their polling took a major downturn this year.

* But FiveThirtyEight’s famed computer model didn’t exactly work like a charm, either…

Thoughts?

       

29 Comments - returning dog - Friday, Nov 5, 10 @ 12:28 am: I was surprised to see the exit polls going Quinn’s way, and even more surprised to see him winning over the night, because I listened to the polls. This will be the election I remember from now on when I start trying to read the tea leaves from the polls.. - Louis G. Atsaves - Friday, Nov 5, 10 @ 12:29 am: 538 blew Dold and Bean races big time as well. Not even close. Too many pollsters, too many polls, too many variations this election cycle. - roger - Friday, Nov 5, 10 @ 12:33 am: Facsinating - G. Willickers - Friday, Nov 5, 10 @ 12:48 am: > But FiveThirtyEight’s famed computer model didn’t exactly work like a charm, either… Nate’s model is based on a lot of factors, but polling is at the heart of it. If the polling is way off then so to will be the entire formula (witness IL-10 and IL-8 also with fivethirtyeight giving Seals and Bean significant shots at winning). Fiverthirtyeight gave Quinn an 18% chance of winning, nearly 1 in 5. It’s a small chance, but it’s still a chance. - Vote Quimby! - Friday, Nov 5, 10 @ 12:53 am: IMHO I think pollsters are not keeping up with technology, skewing their numbers with reality. Heck, it wasn’t 10 years ago everyone had a land line and not everyone had a cell phone. In the same way advertisers are using a shotgun approach these days, pollsters are starting with bad info… - Louis Howe - Friday, Nov 5, 10 @ 1:00 am: Now we see how applying “modeling techniques” to Wall Street sometimes backfires. The problem with forecasting the future human behavior is that some things are not just risky bets, but unknowable. - shore - Friday, Nov 5, 10 @ 1:05 am: I said yesterday on the congresscritter blogpost that predicting the electorate right now is very very hard. an instution of government has flipped in every cycle for 3 straight cycles, that’s the first time that’s happened in modern america (house-white house-house) political history. My theory would be that in the suburbs they underestimated the gop vote and OVERestimated the downstate vote in the brady race. - siriusly - Friday, Nov 5, 10 @ 1:26 am: I actually agreed with 538. Quinn probably had about a 1/5 chance to win. Somehow he did it. - Been There - Friday, Nov 5, 10 @ 6:33 am: This why I’ve never really put much stock in RCP. They don’t add anything new, they merely average polls that are already out there. One could argue that the polls were accurate and the public shifted in the last 72 hours. I don’t believe it. A better explanation is that Quinn voters were there all the time, and Quinn’s GOTV was better, just enough to edge out Brady’s. I think this election is a perfect example of the Yogi Berra saying, ‘It ain’t over ’til it’s over.’ - Wensicia - Friday, Nov 5, 10 @ 7:33 am: I never follow polls, they’re really not representative of the changing moods and trends. My own little poll, consisting of just me, came closer than anyone else. I said Kirk would win by 3% and the governor’s race was too close to call. - bored now - Friday, Nov 5, 10 @ 8:09 am: i don’t know what the republican’s internals looked like, but private polling done for democrats showed a distinct surge for quinn in the last two weeks. a *lot* of it was democrats coming home, but it felt to me that you could also trace this surge to quinn actually getting out there and campaigning… - Expat - Friday, Nov 5, 10 @ 8:23 am: So the cell phone bias against democrats theory doesn’t hold up. The pollsters got the bean race wrong, and understated the margins of victory of other republicans. The pollsters just got it wrong. Exit polls predicted right result, but they were off as well. - Observing - Friday, Nov 5, 10 @ 8:32 am: Brady’s organization started to believe the polls and relaxed. Brady himself either decided to or was advised to start to appear to be the winner…to act more gubernatorial…to start governing….before the chicks were hatched. - Ray del Camino - Friday, Nov 5, 10 @ 8:58 am: Couple things happening–first, polls are not rifles, they’re shotguns. Some of those polls are within or close to the edge of their margins for error. Second, “likely voter” modeling is imprecise and kicks out the kinds of respondents who tend to be Dems. If the D ground game is good, it can skew the whole thing. - Segatari - Friday, Nov 5, 10 @ 9:04 am: This is the question - how did Quinn swing it so far on election day? What was the catalyst? There wasn’t some major gaffe that suddenly turned people off at the last second. So what happened? Anyone? “Well he wasn’t a good candidate” doesn’t cut for me - I want an actual cause and effect here. - Berkeley Bear - Friday, Nov 5, 10 @ 9:07 am: In defense of Nate, when Ras is off even a little is causes a cascade since they do so many of the public polls. This poor performance will cause them to be weighted lower in the future which should improve the regression analysis. The 1/5 chance given Quinn might have been a little low (I had it at 1/3, but still less than 50-50) but that’s a direct reflection of the polling. Plus, I don’t think many people thought Scott Lee Cohen would draw more GOP protest votes than Dem ones, so I’m not sure how 538 could have modeled it correctly. What I hope people take from this is any poll within the margin of error or right around it should not be reported as evidence, on its own, of anything other than the fact an election is going to be close. - Carl Nyberg - Friday, Nov 5, 10 @ 9:08 am: My observation based on going door-to-door in Park Ridge and visiting Dem leaners that were not consistent voters was that Alexi was losing lots of voters who were otherwise voting for the entire Dem ticket. Also, Quinn wasn’t losing anybody to Brady. Why was the polling off? Best guess: the people who voted for Brady were sure they wanted Brady. The people who voted for Quinn were pretty iffy. They were not solidly enough with Quinn that they were telling pollsters. (And pollsters pretty consistently over estimate GOP strength. My sense is that almost every case of where the bulk of pollsters pick one candidate and the other candidate wins, it’s almost always a Dem who pulls off the upset.) So, the GOTV and last minute persuasion, including the ground game, was key to getting the iffy voters to swing to Quinn. But as my mother observed, people from far outside the Chicago area have often have significant gaps in their understanding of Chicago. They say things and do things that are off-putting without realizing it. Brady didn’t connect with Chicago voters. And the GOP doesn’t have much infrastructure in the region. Having a “Right Flight” party at the Cubby Bear doesn’t replace have an indigenous party apparatus. - Carl Nyberg - Friday, Nov 5, 10 @ 9:21 am: G. Willickers makes a good point. If Nate Silver assigns an 80% chance of winning in five races, one should expect that in one of the five races the other candidate is going to win. If every candidate NS gives better than 70% chance of winning actually wins, it means the model is too conservative in assigning percentages. So, rather than being evidence the model is flawed, Quinn’s “upset” shows the percentages make some sense. - Louis G. Atsaves - Friday, Nov 5, 10 @ 9:24 am: Berkeley: In many ways 538 had Dold losing the same way as Quinn losing, and showing Walsh even more poorly. There has to be some explanation for this failure which cuts both ways on a party basis. This election, “conventional wisdom” repeatedly failed along with polling. Those who polled these races in particular need to do some serious analysis and explaining about their performances. Did their polling cause Seals supporters to relax and not turn out? Or Brady supporters to relax and not turn out? Explain the Bean-Walsh race as being anything more than a David vs. Goliath thing, no money vs. $1 million dollars, energized volunteers just banging on doors vs. a Rose Garden strategy. How do you poll that or do a 538 analysis on that? - Ghost - Friday, Nov 5, 10 @ 9:37 am: This was a fairly to porperly wait the sample. If you talk to 600 poeple statewide, your poll number match the election returns. But you need to weight the result to account for population density. If the sample 100 people in chicago have the same impact as the 500 people sampled downstate, the model is flawed. - dave - Friday, Nov 5, 10 @ 9:53 am: **If the sample 100 people in chicago have the same impact as the 500 people sampled downstate, the model is flawed.** Every pollster weights their results, and not just be geography. They weight by geography, gender, party affiliation, etc. They also make assumptions about turnout, and weight accordingly. The assumptions are where the problems lie. - Ghost - Friday, Nov 5, 10 @ 9:57 am: Thats generally where I was going, just more aptly stated. - MrJM - Friday, Nov 5, 10 @ 10:08 am: The polls all have to make two presumptions: 1) who the “likely voters” are, and 2) how “cell phone-only voters” should be weighted. When they get one of those presumptions wrong, they’re results are off. If they get both wrong, the errors are compounded and the poll results are wildly off. – MrJM - Jake from Elwood - Friday, Nov 5, 10 @ 10:31 am: Maybe the Europeans have it right when they limit by law the reporting of polling data for a block of time prior to the election. One wonders if some potential Brady supporters saw the slew of favorable polls and decided not to vote. We will probably never know. - Muskrat - Friday, Nov 5, 10 @ 10:37 am: 538 arguably got it just right. There were, what, 39 Governor’s races, and how many upsets? 33 Senate races and three upsets (NV, CO and AK… maybe). 18% doesn’t look big, bit it’s one in five. One in five chances happen every day. Overall the purely statistical method is *designed* to fail on some occasions, to avoid erring too much one way or the other. If they’d tweaked it to lean Dem enough to show Quin with a 50% chance, they’d have shown Alex Sink winning gin Florida. Intrade had both Harry Reid and Quinn as underdogs, at about .30 and .40 respectively. Is there anyone who does a roundup of how the various political prognosticators did (the ones who handicap all the races, like Rothenberg, Cook, etc)? - Loop Lady - Friday, Nov 5, 10 @ 10:48 am: I said this during the primary and and I’ll say again that polls are good for putting money in a pollsters pocket…I’m have serious doubts as to their merit unless the race is a runaway, and then why would you need to poll? Save your money candidates for lit printing or have someone else pay for the poll, that’s my advice future candidates… - Living in Oklahoma - Friday, Nov 5, 10 @ 11:07 am: All the polls were accurate. They simply were not taken the day of, or day before the election. Four or five days before the election I think the polls were right on the money. the democratic ground game the last four days before the election changed the game. - dave - Friday, Nov 5, 10 @ 11:17 am: **the democratic ground game the last four days before the election changed the game. ** Or, in other words, the polls made wrong assumptions about turnout. So, in turn, they were inaccurate. - Ghost of John Brown - Friday, Nov 5, 10 @ 2:26 pm: Living in Oklahoma got it correct. Dave - the pollsters MAY have actually gotte the turnout correct a week ahead of time, but Madigan et. al, changed that paradigm over the weekend. Sorry, comments for this post are now closed.