Nate Silver Ranks the Best and Worst Pollsters of 2012

Does Nate Silver ever sleep, or even rest? Apparently not. Even after a grueling election season in which he (rightfully) became a superstar, and also somehow found time to write and promote a new book (The Signal and the Noise, which I'm currently reading), he's still hard at work. The latest example: this past Saturday night (as I said, he never rests), Silver posted his rankings of "the 90 polling firms that conducted at least one likely voter poll in the final three weeks of the campaign." You can see those on the "flip."But first, here are Nate Silver's rankings of the "roughly two dozen polling firms that issued at least five surveys in the final three weeks of the campaign" (click on the image to "embiggen!"). As you can see, Gallup wasn't just bad, it was abysmally, disastrously off course - both inaccurate AND wildly biased (towards Republicans). Sadly for Gallup, this election cycle wasn't a fluke; as Nate Silver points out, this debacle makes "three poor elections in a row" for Gallup. So much for being a respected pollster anymore, at least by anyone who knows anything (e.g., not many pundits or much of the corporate media). Other than Gallup, other godawful pollsters were - shocker - Rasmussen (both inaccurate and heavily Republican biased), Mason-Dixon (inaccurate and Republican biased), and American Research Group (ditto). The best pollsters, interestingly, were ones that much of the "mainstream" media (e.g., the Washington Post, whose own polling unit was mediocre at best) disdain, such as internet pollsters Google Consumer Surveys, RAND Corporation, Ipsos/Reuters, and Angus Reid. Also doing very well were live phone (including cell) pollsters IBD/TIPP, Mellman, CNN/Opinion Research, Quinnipiac, and Marist. Robodialer PPP, which for whatever reason the "mainstream media" insists on calling a "Democratic" firm (do they identify Rasmussen as wildly biased towards Republicans?), turned out to actually have a 1.6-point Republican bias. Pretty funny. Anyway, check out the list of pollsters with a minimum of 5 polls in the last 21 days of the election, and when you're done, head to the "flip" for the longer list, and a few more comments by yours truly (including on the utterly abysmal, laughable, Roanoke College polling unit).

lowkell :: Nate Silver Ranks the Best and Worst Pollsters of 2012

A few more thoughts on these pollster rankings. First, note that almost all the polls had a bias - not towards Democrats, as the crazy "unskewed" folks would have us believe, but towards Republicans! I mean, it's not even close; there is a significant pro-Republican bias, even among supposedly pro-Democratic polling firms like PPP. So much for THAT theory!Second, to quote Nate Silver, "I'm not as certain about the future for automated telephone polls." Instead, surprisingly, it's the internet pollsters of all people who seem to be doing better in this day and age. Fascinating. Third, one Virginia pollster, Roanoke College, ranks close to the bottom in terms of both average error (they only did one poll in the final 21 days of the election, but man was it a bad one - off by 8.1 points!) and also bias (8.1 points in the direction of Republicans). Ouch. Sadly, this finding isn't an aberration, as Roanoke College's polling efforts have been wildly erratic, with all kinds of "internal" problems, and just a horrible track record in general. In the future, until proven otherwise, these polls should be ignored (along with Christopher Newport University's, not to mention any poll that's conducted over several weeks, as a lot of these college polls are...). Anyway, enjoy the full list, and thanks to Nate Silver for his continued hard (and impressive) work!