After Elections, Pollsters Take Stock
Driven by an unprecedented volume of polling that often proved unreliable and contradictory, even victorious Democrats lost sleep over the reliability of their data in the closing weeks of the 2012 campaign.
Guy Cecil, executive director of a Democratic Senatorial Campaign Committee that helped the party gain two Senate seats on a treacherous playing field, ultimately trusted his party’s numbers. But down the stretch of close races in Republican-leaning states, facing nearly complete opposite findings touted by competing GOP campaigns, he would hold private calls with his pollsters.
“I spent a lot of time with pollsters,” Cecil told Roll Call this week. “We would do the campaign call where everyone is on the call, or we would do it on our side, and then usually afterward I would call them up myself. I would set up a separate call, where I would just say I need you to explain — I obviously didn’t know what the specifics of the Republican numbers were, but I knew just from talking to everybody that they were much more optimistic — and I think genuinely optimistic — about places like North Dakota.”
In multiple Senate contests Republicans lost, including open seats in North Dakota and Indiana and their challenge of Montana Sen. Jon Tester, the survey data gathered by GOP pollsters for the National Republican Senatorial Committee and their candidate clients turned out to be flat wrong. A similar problem afflicted some Republican House candidates, although for them and the National Republican Congressional Committee — which easily held its majority — the issue was less pronounced.
The Republican campaign committees and GOP political strategists have been hesitant to criticize the party’s pollsters directly, either on background or on the record. But privately, they concede feelings from disappointment to outright hostility, depending on the race in question. But at least some Republican pollsters have acknowledged that they missed the mark, at least as a community if not as individual consultants. Public Opinion Strategies, among the premiere Republican polling firms, said as much this week.
“There is no question that the 2012 election was challenging for pollsters,” according to a memorandum it released Monday. “The number of assumptions that must be made has increased significantly: including enough cell phone interviews, ensuring that younger voters are represented in a sample like they are in the electorate, projecting the white and minority percents of the electorate, and having the right partisan mix are all tough calls.”
A Republican operative who worked on congressional races this past cycle was more blunt, saying his party’s data was often wrong in contests where the Hispanic or youth voters, or both, were factors. Where a race was largely decided by the white vote, GOP data tended to be reliable, said this operative, who declined to speak on the record in order to provide a candid assessment of the surveys produced by GOP pollsters.
However, in states with small populations of ethnic minority voters, such as Montana and North Dakota, Republican pollsters surveying Senate races still misfired, which Democratic and GOP strategists have suggested was a result of failing to include the right mix of other key demographics in their turnout models, such as gender and age.
Even reliable public pollsters fared poorly in their reading of the electorate. Gallup, considered the gold standard among public pollsters, projected a Nov. 6 electorate substantially more favorable to Republicans than actually showed up.
Mark Mellman, who polled for Senate Democrats in Montana and North Dakota, said the public polls that consistently showed Republicans in the lead in those races might have been too dependent on “likely voter” screens, and thus gave a false impression to political observers that Tester and Sen.-elect Heidi Heitkamp, D-N.D., were behind when in fact they were ahead heading into Election Day. “You’re never going to have an electorate exclusively made up of likely voters,” Mellman said.
Republican pollster David Winston, who advises House and Senate Republicans, also is suspicious of relying on “likely voter” screens in surveys, and said the misjudgments of the public pollsters could stem from their decision to exclude too many minorities and young voters from their turnout models. Winston said he prefers to rely on “registered voter” screens in his polling.
In losing to President Barack Obama, Mitt Romney’s campaign badly miscalculated the partisan, ethnic and demographic composition of the electorate. To the extent Romney’s pollsters — considered among the best — and others misunderstood voter turnout, Winston suggested that not enough was done to question turnout assumptions, such as that GOP enthusiasm and winning independents would equal victory on Election Day.
“Without situation awareness how do you make good decisions?” Winston asked, adding that the Obama campaign did a much better job of analyzing its data in order to refine it and find flaws.
Democratic pollster Pete Brodnitz, whose partner Joel Benenson polled for the Obama campaign, said he believes Democratic pollsters were more successful than their Republican counterparts because they had a better grasp of the voting population, suggesting that Republican pollsters appear to have failed to detect a shift in percentages among the various demographics that benefitted the president and other Democratic candidates.
Brodnitz said that he and other Democratic pollsters rigorously challenged their data, particularly when it was favorable to their candidates, and indicated this gave his party an advantage in forecasting turnout. Brodnitz polled for Virginia Sen.-elect Tim Kaine, and often found higher levels of African Americans who were coming up as likely to vote — about 18 percent — than he thought was reasonable, which led him to weight their turnout at 16 percent, despite the fact that they comprised about 20 percent of the commonwealth’s electorate in 2008.
Brodnitz said Republicans appear to him to have put too much stock in voter enthusiasm and national polls, which suggested that Romney and GOP candidates down ticket would perform better. He also said that they might have read too much into the massive crowds Romney was attracting to his campaign rallies in October.
“If you’re going to tell the story of this election, name of the story would be the case of the wrong metrics. Republicans were using wrong metrics to try and determine what was going on this year,” he said. “If Democrats did a better job than Republicans this time, it’s because Democrats were being more objective about what was going on.”
Kyle Trygstad contributed to this report.