Subscribe via RSS Feed

Search Results for 'pollin'

Polling Failure in the UK

[ 50 ] May 12, 2015 |


As the occasional (charitably) LGM Senior British Correspondent, I’m going to weigh in with a series of thoughts on the British General Election last week.  Cards on table: one of several reasons for my extended sabbatical(*) from LGM has been my active involvement, time permitting, in the Labour Party campaign here in Plymouth.

When I walked into the Plymouth Guildhall Thursday night around 10:15 to participate in the counting of the ballots as a count agent for my Constituency Labour Party, following 14 hours on the campaign trail, I was immediately confronted with the exit poll conducted by several pollsters for several media organisations.  The poll was a shock as it was inconsistent with the narrative created by the last year of public polling released by the major houses.  All the major polls, and the five or six seat projection models, suggested a hung parliament. The largest single party in this new parliament would likely be the Conservatives, but they would only have anywhere from five to 15 more seats than Labour, and the maths suggested there was no way the Tories could form a stable minority government, let alone a coalition:

The exit poll will be out very shortly, and then we’ll have a good idea (or a false one). But first, here’s the game. No one is going to win an overall majority, so it’s all about who can cobble together 323 seats – the number needed for a majority – by banding together with other parties.

Second, Labour seem the most likely to win that game. May2015’s Poll of Polls, which has averaged all the latest polls since September, has finally finished adding numbers up. It’s conclusion? The Tories are going to win 33.8 per cent of the vote, and Labour are going to win 33.7.

This was the narrative the pollsters stood by, and the narrative that those of us academics called upon by the media used as the foundation for discussion (with our own various caveats).  Quite obviously this was wrong, and I’m plastered all over the media both in Devon and the Southwest of England as getting it very wrong.

For the 2015 General Election, we had access to considerably more, and richer, data than in elections past. It felt like an embarrassment of riches, and a certain hubris resulted.  In addition to the national level polling, Lord Ashcroft released around 130 constituency level polls of marginal seats.  These, with the expected N of around 1000, for the first time allowed us to understand how the national numbers and trends were being reflected at the constituency level not only occasionally, but systematically. We were in a position where we could finally bury the swingometer based on the mythical uniform national swing.

The interesting academic question from this election is why polling in the UK failed as bad (if not worse) than it did in 1992.  At this early point, we don’t know, and anybody offering a definitive explanation is taking a significant risk.  There are working theories and interesting questions; four can be found here from YouGov, ICM, Populus, and ComRes. Labour’s internal pollster has an observation here, which is intriguing given the methodological insight revealed for the internal polling. Of course, as these data are not in the public domain, any conclusions drawn are not definitive.  Finally, Eric Kaufmann (one n removed from a relation with our own SEK) has this intriguing take here at the LSE blog. I have one potential minor critique of the Kaufmann piece — his methodology is based on 130 of the Lord Ashcroft constituency level polls, and some of these were ancient in political terms.  Speaking for the two constituencies that represent 15 of Plymouth’s 20 electoral wards, Plymouth Sutton & Devonport was polled in August, while Plymouth Moor View was polled in December.  (The third, Southwest Devon, wasn’t touched, as it’s a very safe Conservative seat.)  These polls estimated a 13 and 11 point Labour victory respectively; on May 7th, the Conservatives won both constituencies by 1.1% and 2.4%.

It is likely that no single cause will explain the polling failure of 2015.  I have an additional theory that I’ll discuss soon assuming it passes prima facie.

(*) The other reasons include being elevated to an administrative role in my department (cue up Bunk to McNulty here), and that this election has increased my media calls significantly. I’ve done somewhere north of 60 appearances in the past twelve months. Rob suggested last week that I link these to LGM.  I will in the future when I can, if only that you, too, can laugh along with the audience at home.


Social Desirability and Response Validity in Current Polling

[ 22 ] September 28, 2012 |

Several days ago in these very pages, discussion ensued regarding the latest conservative attempt to rewrite reality through re-weighting polls to one guy’s liking.  Of course, polling is not an exact science, but it is a science, and the latest wingnut delusion has no grounding in theory or empirical evidence.  Like any science, survey research continuously attempts to improve upon the validity and reliability of its measures and findings.  While I’m not at all concerned about some nefarious (and successful) attempt by the MSM and that paragon of power, the Democratic Party, to turn otherwise professional and reputable polling houses into duplicitous shills.

However, I have been somewhat interested (note, not concerned) if there might be something else going on that causes the polls to over estimate support for Obama.  Social desirability bias is something I’ve published on in the past (direct link to the paper here).  While that article suggests a contextual effect that causes variance in social desirability across countries (regarding accurately reported turnout in survey research), relevant here is what is colloquially known as the Bradley Effect.  It’s possible (though I consider it unlikely in the specific context of the 2012 Presidential election) that this helps explain Obama’s consistent polling advantage in an election where many if not most structural conditions suggest an incumbent defeat.

It’s difficult with the data I have available to examine this hypothesis to any satisfaction, but that’s not going to stop me from trying.

To begin with, we have the current state of the polls.

Obama Romney Advantage
RCP 28 Sep. 48.6 44.6 4.0
538 28 Sep. 52.2 46.5 5.7
538 6 Nov. 51.5 47.4 4.1


RCP’s running average has Obama up 4 points, Nate Silver’s “nowcast” model up 5.7, and his current prediction for election day 4.1 points.

If social desirability is at work here, a poll respondent will state that she or he supports the President because internally, our not entirely sincere respondent is seeking the socially desirable response, and not supporting the black guy might be racist.  However, this is done knowing that they will ultimately support the white guy.  Practically, this would mean that Obama’s support in these polls is inflated.

I’m approaching this from several directions.  First, I’ve averaged the final month of polls for Presidential elections going back to 1976 (an arbitrary cut off) to examine how accurate the polls were in predicting the final outcome between two white men, with 2008 to serve as a benchmark for 2012.  Shift represents how wide of the mark the final polls were, to the benefit or detriment of the incumbent party.

Poll Result Shift
2008 D 7.6 D 7.3 0.3
2004 R 1.5 R 2.4 0.9
2000 R 3.0 D 0.5 3.5
1996 D 11.0 D 8.0 -3.0
1992 D 12.0 D 6.0 -6.0
1988 R 12.0 R 7.0 -5.0
1984 R 18.5 R 18.0 -0.5
1980 R 4.0 R +10.0 -6.0
1976 D 2.0 D 2.0 0.0


Social desirability response bias in an election can take on many forms, not just race.  To wit, the 1992 general election in the United Kingdom is a good case study (one I lectured on here at Plymouth about six or seven years ago, shame I have no clue where those lecture notes now reside) as the polls largely predicted a narrow Labour outright victory or a hung parliament with Labour having the plurality of seats, yet the Conservatives under John Major easily won by 7.5%.  This is called the Shy Tory Factor on this island, which is simply another manifestation of social desirability.  But in 2012, I’m primarily considering race, and comparing 2008 to past elections does not support the hypothesis that this might be a problem for Obama’s numbers in 2012.

I also considered several of the primary elections in January, 2008.  This was the beginning of a primary where Obama was a somewhat unknown junior senator only four years into his Congressional career, going up against the assumed nominee.  For these data, I average all polls from the last week of the campaign in the given state (there were 20 in New Hampshire alone).

Poll Result Shift
NH Obama 35 37 2
NH Clinton 30 39 9
NV Obama 33.25 45 11.75
NV Clinton 37 51 14
SC Obama 41 55 14
SC Clinton 26 27 1


This evidence is more ambiguous than the examination of previous general elections.  Both Clinton and Obama received shifts in their favor, which isn’t surprising considering the undecideds presumably made a decision of some sort once voting.  However, in both New Hampshire and Nevada, the shift was stronger towards Clinton than Obama: a 7 and 2.25 point advantage respectively.  Both are dwarfed by Obama’s advantage in South Carolina.

These are the wrong data to be analysing this with, of course; ideally we’d have individual level data.  While not individual level data, the following figure, by Greenwald and Albertson, offers a more holistic view of the 2008 primaries.

The above shows that among 32 states where data were available, the “Bradley effect” was only evident in three states, yet 12 states demonstrated what has been termed (erroneously, in my opinion) the “reverse Bradley effect”: states where Obama’s support in the primaries was under, not over, estimated (see South Carolina above).  I consider this an erroneous classification because where the theoretical explanation for the Bradley effect hinges on social desirability, the reverse has been hypothesized as a function of systematic sample bias, through either the under-representation of African Americans in polling samples, or the cell-phone effect.  However, some have hypothesized that “black voters might have been reluctant to declare to pollsters their support for Obama”, and the link above does discuss that

After the Super Tuesday elections of February 5, 2008, political science researchers from the University of Washington found trends suggesting the possibility that with regard to Obama, the effect’s presence or absence may be dependent on the percentage of the electorate that is black. The researchers noted that to that point in the election season, opinion polls taken just prior to an election tended to overestimate Obama in states with a black population below eight percent, to track him within the polls’ margins of error in states with a black population between ten and twenty percent, and to underestimate him in states with a black population exceeding twenty-five percent. The first finding suggested the possibility of the Bradley effect, while the last finding suggested the possibility of a “reverse” Bradley effect in which black voters might have been reluctant to declare to pollsters their support for Obama or are under polled.

There are numerous possible explanations for the “reverse” effect, including faulty likely voter models, under sampling of blacks, sampling bias due to cell phones, to name a few.  There might be some sort of contextual effect at work here, but to ascribe it to behavioural motivations (rather than factors exogenous to the individual, such as sampling bias) such as blacks being shy to state their support for Obama fails the face validity test to me.

Ultimately, given the wide array of mediocre data presented here, I am not concerned about social desirability biasing the estimates of support for Obama in any significant, substantive manner.  However, much as I’d like to, I wouldn’t say that the Republican conspiracy is more likely, if only because that is so creatively ludicrous I initially thought it was an Onion piece.

Push Polling!

[ 0 ] March 20, 2010 |

Ten minutes ago, the wife received a call Family Research Council. The first question: “Do you have a favorable opinion of President Obama’s government run health care proposal?”

After she answered “Yes,” there were no further questions. I suspect that there would have been more than a few additional queries if the answer had been “No.”

UPDATE: IPE@UNC marshaled the mighty power of empirical science to determine the truth of my last statement:

They called the wife’s phone about 10 minutes later and she let me answer. This time when asked if I supported the Marxist take over of everything great about America I said “No”. Were there follow-ups? Why….. yes! Of course there were. The next question was some variant of “Do you support abortion?” I said “No” and then the robot lit into a 30 second rant about how Obama pledged to Planned Parenthood during the campaign that he’d do everything in his power to eliminate all restrictions on abortion (which I think is untrue, but maybe not) and strongly implied, without directly saying so, that passage of the health care bill would lead to just that outcome. Then the robot asked if I’d be willing to contact my congressman about this issue and I said “Yes”. The robot then helpfully gave me my representatives’ information.

More on British Polling and Margins of Error

[ 0 ] March 2, 2010 |

I promise that this will not be a daily habit of mine, but the Tory +2% lead poll did generate the predictable breathless excitement on these islands.

YouGov’s daily tracker indeed moved back towards +6, as anticipated, at Conservative +7. Additionally, a ComRes poll was released yesterday for The Independent which places the Tory lead at +5 (37/32/19). I want to emphasize that the +2 poll released on Sunday, while at the fringes assuming a true value of +6, is still within the margin of error. It was not an outlier in a pedantic understanding of the word, which is a case three standard deviations removed from the mean.
In other words, the findings reported in the Sunday poll were not “wrong”. 95% of samples will yield the true value within the error band of +/- 3%, as this was. This is why we have margins of error in the first place.

What we should focus on is not the point estimate itself, but the trend — and the consistent trend is clearly away from the Tories and towards Labour.

Speaking of which, the +5% Tory lead reflected in The Independent’s poll still yields a distribution of seats as Labour 287, Conservative 272, Liberal Democrat 59 — assuming a uniform national swing.

UPDATE ( 3/3/10): Again as expected, Wednesday’s YouGov tracker is rather consistent, at Conservative +5 (C 38 L 33 LD 16). Given the last seven to ten days of polling, this suggests a true value of support around + 6%, and if not precisely 6 (and it isn’t) it’s slightly below 6%.

Preliminary Colorado exit polling data

[ 0 ] November 4, 2008 |

This is weighted by affiliation but doesn’t attempt to incorporate early voting:

Obama 52

McCain 47

Senate Polling

[ 0 ] October 31, 2006 |

Some new CNN data. It still looks like 50-49-1 to me, with Tennessee looking like a write-off. The one ray of hope is Virginia–the polls are still even, and given the D.C. suburbs I can see Virginia going Democratic in a toss up race the way I can’t see Tennessee. On the other hand, MO is also still a toss-up, although I’m inclined to think the Dems will take it.

Polling Data

[ 0 ] September 17, 2004 |

Via Kos, John Zogby makes a compelling argument that telephone polling, as we know it, is obsolete:

Zogby points out that you don’t know in which area code the cell phone user lives. Nor do you know what they do. Beyond that, you miss younger people who live on cell phones. If you do a political poll on land-line phones, you miss those from 18 to 25, and there are figures all over the place that show there are 40 million between the ages of 18 and 29, one in five eligible voters.

This is pretty interesting, and does not admit an obvious solution. Zogby had moved to internet polling, which may or may not give a more accurate account. We’ll have to wait for November to clear some of this up. In any case, the huge spread of current polls is probably due to something more than the difference in likely voter methodology.

I’m not current with the legalities of polling via cell phone, although I have noted that I tend to receive very few unsolicited calls on my cell (very few solicited ones, either; I’m one hell of a loser). Myself and my roommate have gone the cellular/cable modem route, and no longer have a landline at all, meaning that we are apparently politically invisible. How will the invisibile people sort themselves out in November? Well, we’re young, which may help the Dems, and we’re probably a touch more affluent than normal (correcting for the fact that we’re young) which favors the Republicans.

rebRand Paul Bawls as Campaign Stalls

[ 71 ] January 13, 2016 |

Rand Paul got bumped from the big kids’ GOP debate. No fair! says he.

They have been saying for months they’re going to narrow the field, but I don’t think it’s the job of the establishment in the Republican Party to decide who is and who isn’t [in].

Like a legitimate board of ophthalmology? Maybe he should create his own Republican Party, and then abandon it when he realizes it’s a big hassle!

And it isn’t about his polling numbers, it’s about the fact that he’s so super and they’re so stupid.

Paul … said he’s being pushed out because he has a “unique voice.”

Nah. Whiny bluster is a hallmark of the GOP.

Naturally he’s threatening to take his slit lamp and go home.

Do they really not want liberty voters in their party?

[Lip wobble]

Someone who could be reached for comment says the liberty voters will probably hang around with Cruz.

Those liberty-oriented voters are still vacillating between Rand Paul — which is where their heart is and where they’ve been with the Paul family for the past two cycles — or Ted Cruz, where their head may be on a candidate that may not align with them perfectly on anything but will be closer than anyone else to have the potential to go the distance.

So there.

As an aside – Where did “liberty voter” come from, and why? I assume this means libertarians, but why the rebranding?

A Man in My Position Can’t Afford to Be Made to Look Ridiculous

[ 117 ] January 10, 2016 |

nbc-fires-donald-trump-after-he-calls-mexicans-rapists-and-drug-runnersAt the risk of eventually looking deeply ridiculous, I’m going to have to depart from Paul (and to some extent from Scott) regarding Trump’s candidacy. At this point, I’d rate the chances as Cruz 50%, Rubio 35%, Bush 10-15%, and Trump 0-5%.

Cruz’s path to the nomination is fairly clear; he wins Iowa, and Trump and the vote in the next few primaries is sufficiently divided between Trump and the remaining mainstream candidates that he wins or does very well in the first few contests.  The Establishment dislikes Cruz, but it will rally around him.

Rubio and Bush have very similar paths.  Each has to do decently enough in Iowa to have an impact on New Hampshire, forcing the Establishment to choose one or the other in order to avoid Cruz or Trump.  I appreciate that it’s trendy to be completely dismissive of Bush’s chances at this point, but expectations have gotten so low that if he stages any kind of rally in Iowa or New Hampshire, he’ll earn a strong comeback narrative and he’ll get a lot of his Establishment support back.  Given that he’s a known quantity and that the situation is fluid, I would not at all be surprised to see voters go with him as the safe choice.  But then it’s also possible he’ll be out when polls close in New Hampshire.

My case against Trump is pretty much identical to that of Nate Silver.  The party hates him; he’s polling worse in the states that have the earliest primaries than he is nationally; the people who’ve expressed a preference for him are the lowest information voters and the least likely to actually vote; there are deep questions about his organization (a problem tied to the party hating him); state polls and national polls are extremely volatile in the early going.   I think there’s a very strong chance that he’ll underperform in Iowa, which will make things very difficult moving forward. It’s also important to remember that while we’re treating state polls in isolation at this point, that’s not at all justifiable. Especially for the first few states, these polls have historically seen wild shifts based on the outcome of the previous contests.

While Trump may see some isolated success, I think the most likely outcome is that he doesn’t see the early success he’s expecting, and goes on to regularly run second or third for as long as he remains in the race, behind Cruz and whichever of Bush and Rubio survive. In the unlikely event that Trump does win some early primaries, I suspect that the party will fairly quickly unite around Cruz or Rubio, and that Trump will struggle to put together majorities. Either way, he’s drawing dead.

The impact of Trump, such that it is, will most likely manifest in giving Ted Cruz the advantage he needs to take the race, although it’s possible that Cruz might have won anyway.  The more interesting wild card is, of course, if Trump runs third party; even a limited impact (and I’d say he’ll manage something less than Perot ’92) will give Clinton a huge advantage in the general.

Taking Trump Seriously

[ 173 ] January 7, 2016 |


Sam Wang:

For comparison I include Hillary Clinton, this year’s overwhelming favorite for the Democratic nomination. This emphasizes the fact that based on polling data, Donald Trump is in as strong a position to get his party’s nomination as Hillary Clinton in 2016, George W. Bush in 2000, or Al Gore in 2000. The one case in which a lead of this size was reversed was the 2008 Democratic nomination, which very was closely fought.

Obviously, polls are not the entire story of the campaign. Unlike past nominees, Trump does not have the national party behind him. In that respect, he is emblematic of the overall weirdness of this year’s GOP primaries.

Other factors are said to influence the nomination process: candidate experience, campaign finance, and party endorsements. These are described in the New York Times feature Who’s Winning the Presidential Campaign? (Here is one entertaining recent discussion over at FiveThirtyEight.) In my view, these factors are likely to matter under normal conditions – until a political party undergoes a major upheaval. That happens about every 40-50 years (see this excellent XKCD explainer graphic). Trump-as-nominee could fairly be seen as such an upheaval. This is one reason to pay attention not just to data pundits, but also to grizzled old historians.

My guess is that Trump’s complete lack of elite support, combined with opposition from a viable candidate with strong base support in Cruz and a potentially viablish candidate with elite support in Rubio, means that he’s still not the most likely Republican nominee. But I can see the case that the fact that Trump has been able to maintain such strong numbers for so long is an indication that we’re in a context where general tendencies just don’t apply. You can make a decent case that Trump should be considered the frontrunner, and a pretty solid case that he should be considered a stronger candidate than Rubio.

Prop 13

[ 42 ] January 6, 2016 |


Sasha Abramsky makes the argument that maybe California voters have finally had enough of the impact of Proposition 13, the 1978 ballot measures that decimated the state budget and set in motion the anti-property tax movement nationwide that has contributed to a variety of problems, including spiraling tuition rates for college as states reduce their contributions to higher education in response to lower tax revenues. Not surprisingly, Prop 13 has served the interests of the extremely wealthy, as they have developed fictions to get around its limitations while allowing it to retain its electoral magic. Tax reformers are pushing for what’s call a split-roll approach to property taxes. An explanation:

 Coupal and his allies have recently come out in favor of a legislative fix to tackle the sorts of “abuses” embodied in the Dell case. They support a law that defines “ownership change” as having occurred whenever at least 90 percent of a property shifts hands, regardless of whether any one owner ends up with more than 50 percent. But they have drawn a line in the sand against the idea of a “split-roll tax,” which would impose a higher burden on corporations. Coupal accepts that such a tax would easily boost state revenues in the short term by several billion dollars annually. But his organization, the California Chamber of Commerce, and other opponents of change argue that the cost in lost jobs and leakage from businesses relocating out of state would more than cancel out the benefits in the long run. “Our position has always been that if you’re going to have a tax increase, it should be broad-based and universally applicable,” says California Chamber of Commerce policy advocate Jennifer Barrera. “A split-roll tax treats residential property differently from commercial property, so it’s discriminatory.”

 Reform advocates, however, believe that a split-roll tax is exactly the way to go, and their polling research suggests that, for the first time in a generation, they have a decent chance of persuading a majority of the electorate to support them. Over the last few years, they have been calling for a reform that would protect homeowners and renters while taxing corporations at closer to the market value of their properties. Far from being discriminatory, they argue, it is simply a matter of equity: In an era of growing inequality and wealth concentration, this reform would generate desperately needed funds to maintain and expand vital public services.

Economists at the University of Southern California’s Program for Environmental and Regional Equity were recently hired by Make It Fair, a statewide coalition of reform groups seeking to put a split-roll-tax initiative on the ballot. When the PERE team crunched the numbers on more than 1 million properties across the state, they reached an astonishing conclusion: If a reform were enacted that maintained lower tax rates for residential homes but raised them to market rates for commercial and industrial properties, the state would generate $8.2 billion to $10.2 billion in additional annual revenues. It’s a figure large enough to restore the state’s education system, improve its mental-health infrastructure, and reform many of the other areas that have been left to lag in the decades following the 1978 tax revolt.

You can color me skeptical that this passes, but I’d love to be wrong. Abramsky cites California’s changing demographics, and that’s certainly true. But mobilizing young voters of color for criminal justice reform is a different beast than mobilizing them for property tax reform because the former is more obviously a justice issue, even as the latter is in reality as well. And the money pouring into California from corporations to defeat this is going to be amazing. But I am at least glad to see this issue on the table and maybe something positive will happen.

Donald Trump will shoot water flow restrictors with his guns

[ 48 ] December 16, 2015 |

to save us from Obomanable restrictions on our bath time freedoomz.

Asked about the Environmental Protection Agency’s (EPA) “Waters of the United States” rule at a campaign event in Aiken, S.C., on Saturday, Trump said it significantly impairs his ability to rinse, lather and repeat.
“I’ll give you one regulation,” Trump said. “So I build, and I build a lot of stuff. And I go into areas where they have tremendous water. … And you have sinks where the water doesn’t come out.
“You have showers where I can’t wash my hair properly,” he added. “It’s a disaster.

What a liar. Everyone knows Trump can’t even build a decent argument. And cotton candy dissolves in water. Sheesh.

But the more I see of this loud, grasping, terminally bitchy bigot, the less I’m surprised he is polling so well.

Page 1 of 1912345...10...Last »