Subscribe via RSS Feed

Search Results for 'pollin'

Polling Failure in the UK

[ 50 ] May 12, 2015 |

UK_opinion_polling_2010-2015

As the occasional (charitably) LGM Senior British Correspondent, I’m going to weigh in with a series of thoughts on the British General Election last week.  Cards on table: one of several reasons for my extended sabbatical(*) from LGM has been my active involvement, time permitting, in the Labour Party campaign here in Plymouth.

When I walked into the Plymouth Guildhall Thursday night around 10:15 to participate in the counting of the ballots as a count agent for my Constituency Labour Party, following 14 hours on the campaign trail, I was immediately confronted with the exit poll conducted by several pollsters for several media organisations.  The poll was a shock as it was inconsistent with the narrative created by the last year of public polling released by the major houses.  All the major polls, and the five or six seat projection models, suggested a hung parliament. The largest single party in this new parliament would likely be the Conservatives, but they would only have anywhere from five to 15 more seats than Labour, and the maths suggested there was no way the Tories could form a stable minority government, let alone a coalition:

The exit poll will be out very shortly, and then we’ll have a good idea (or a false one). But first, here’s the game. No one is going to win an overall majority, so it’s all about who can cobble together 323 seats – the number needed for a majority – by banding together with other parties.

Second, Labour seem the most likely to win that game. May2015’s Poll of Polls, which has averaged all the latest polls since September, has finally finished adding numbers up. It’s conclusion? The Tories are going to win 33.8 per cent of the vote, and Labour are going to win 33.7.

This was the narrative the pollsters stood by, and the narrative that those of us academics called upon by the media used as the foundation for discussion (with our own various caveats).  Quite obviously this was wrong, and I’m plastered all over the media both in Devon and the Southwest of England as getting it very wrong.

For the 2015 General Election, we had access to considerably more, and richer, data than in elections past. It felt like an embarrassment of riches, and a certain hubris resulted.  In addition to the national level polling, Lord Ashcroft released around 130 constituency level polls of marginal seats.  These, with the expected N of around 1000, for the first time allowed us to understand how the national numbers and trends were being reflected at the constituency level not only occasionally, but systematically. We were in a position where we could finally bury the swingometer based on the mythical uniform national swing.

The interesting academic question from this election is why polling in the UK failed as bad (if not worse) than it did in 1992.  At this early point, we don’t know, and anybody offering a definitive explanation is taking a significant risk.  There are working theories and interesting questions; four can be found here from YouGov, ICM, Populus, and ComRes. Labour’s internal pollster has an observation here, which is intriguing given the methodological insight revealed for the internal polling. Of course, as these data are not in the public domain, any conclusions drawn are not definitive.  Finally, Eric Kaufmann (one n removed from a relation with our own SEK) has this intriguing take here at the LSE blog. I have one potential minor critique of the Kaufmann piece — his methodology is based on 130 of the Lord Ashcroft constituency level polls, and some of these were ancient in political terms.  Speaking for the two constituencies that represent 15 of Plymouth’s 20 electoral wards, Plymouth Sutton & Devonport was polled in August, while Plymouth Moor View was polled in December.  (The third, Southwest Devon, wasn’t touched, as it’s a very safe Conservative seat.)  These polls estimated a 13 and 11 point Labour victory respectively; on May 7th, the Conservatives won both constituencies by 1.1% and 2.4%.

It is likely that no single cause will explain the polling failure of 2015.  I have an additional theory that I’ll discuss soon assuming it passes prima facie.

(*) The other reasons include being elevated to an administrative role in my department (cue up Bunk to McNulty here), and that this election has increased my media calls significantly. I’ve done somewhere north of 60 appearances in the past twelve months. Rob suggested last week that I link these to LGM.  I will in the future when I can, if only that you, too, can laugh along with the audience at home.

Social Desirability and Response Validity in Current Polling

[ 22 ] September 28, 2012 |

Several days ago in these very pages, discussion ensued regarding the latest conservative attempt to rewrite reality through re-weighting polls to one guy’s liking.  Of course, polling is not an exact science, but it is a science, and the latest wingnut delusion has no grounding in theory or empirical evidence.  Like any science, survey research continuously attempts to improve upon the validity and reliability of its measures and findings.  While I’m not at all concerned about some nefarious (and successful) attempt by the MSM and that paragon of power, the Democratic Party, to turn otherwise professional and reputable polling houses into duplicitous shills.

However, I have been somewhat interested (note, not concerned) if there might be something else going on that causes the polls to over estimate support for Obama.  Social desirability bias is something I’ve published on in the past (direct link to the paper here).  While that article suggests a contextual effect that causes variance in social desirability across countries (regarding accurately reported turnout in survey research), relevant here is what is colloquially known as the Bradley Effect.  It’s possible (though I consider it unlikely in the specific context of the 2012 Presidential election) that this helps explain Obama’s consistent polling advantage in an election where many if not most structural conditions suggest an incumbent defeat.

It’s difficult with the data I have available to examine this hypothesis to any satisfaction, but that’s not going to stop me from trying.

To begin with, we have the current state of the polls.

Obama Romney Advantage
RCP 28 Sep. 48.6 44.6 4.0
538 28 Sep. 52.2 46.5 5.7
538 6 Nov. 51.5 47.4 4.1

 

RCP’s running average has Obama up 4 points, Nate Silver’s “nowcast” model up 5.7, and his current prediction for election day 4.1 points.

If social desirability is at work here, a poll respondent will state that she or he supports the President because internally, our not entirely sincere respondent is seeking the socially desirable response, and not supporting the black guy might be racist.  However, this is done knowing that they will ultimately support the white guy.  Practically, this would mean that Obama’s support in these polls is inflated.

I’m approaching this from several directions.  First, I’ve averaged the final month of polls for Presidential elections going back to 1976 (an arbitrary cut off) to examine how accurate the polls were in predicting the final outcome between two white men, with 2008 to serve as a benchmark for 2012.  Shift represents how wide of the mark the final polls were, to the benefit or detriment of the incumbent party.

Poll Result Shift
2008 D 7.6 D 7.3 0.3
2004 R 1.5 R 2.4 0.9
2000 R 3.0 D 0.5 3.5
1996 D 11.0 D 8.0 -3.0
1992 D 12.0 D 6.0 -6.0
1988 R 12.0 R 7.0 -5.0
1984 R 18.5 R 18.0 -0.5
1980 R 4.0 R +10.0 -6.0
1976 D 2.0 D 2.0 0.0

 

Social desirability response bias in an election can take on many forms, not just race.  To wit, the 1992 general election in the United Kingdom is a good case study (one I lectured on here at Plymouth about six or seven years ago, shame I have no clue where those lecture notes now reside) as the polls largely predicted a narrow Labour outright victory or a hung parliament with Labour having the plurality of seats, yet the Conservatives under John Major easily won by 7.5%.  This is called the Shy Tory Factor on this island, which is simply another manifestation of social desirability.  But in 2012, I’m primarily considering race, and comparing 2008 to past elections does not support the hypothesis that this might be a problem for Obama’s numbers in 2012.

I also considered several of the primary elections in January, 2008.  This was the beginning of a primary where Obama was a somewhat unknown junior senator only four years into his Congressional career, going up against the assumed nominee.  For these data, I average all polls from the last week of the campaign in the given state (there were 20 in New Hampshire alone).

Poll Result Shift
NH Obama 35 37 2
NH Clinton 30 39 9
NV Obama 33.25 45 11.75
NV Clinton 37 51 14
SC Obama 41 55 14
SC Clinton 26 27 1

 

This evidence is more ambiguous than the examination of previous general elections.  Both Clinton and Obama received shifts in their favor, which isn’t surprising considering the undecideds presumably made a decision of some sort once voting.  However, in both New Hampshire and Nevada, the shift was stronger towards Clinton than Obama: a 7 and 2.25 point advantage respectively.  Both are dwarfed by Obama’s advantage in South Carolina.

These are the wrong data to be analysing this with, of course; ideally we’d have individual level data.  While not individual level data, the following figure, by Greenwald and Albertson, offers a more holistic view of the 2008 primaries.

The above shows that among 32 states where data were available, the “Bradley effect” was only evident in three states, yet 12 states demonstrated what has been termed (erroneously, in my opinion) the “reverse Bradley effect”: states where Obama’s support in the primaries was under, not over, estimated (see South Carolina above).  I consider this an erroneous classification because where the theoretical explanation for the Bradley effect hinges on social desirability, the reverse has been hypothesized as a function of systematic sample bias, through either the under-representation of African Americans in polling samples, or the cell-phone effect.  However, some have hypothesized that “black voters might have been reluctant to declare to pollsters their support for Obama”, and the link above does discuss that

After the Super Tuesday elections of February 5, 2008, political science researchers from the University of Washington found trends suggesting the possibility that with regard to Obama, the effect’s presence or absence may be dependent on the percentage of the electorate that is black. The researchers noted that to that point in the election season, opinion polls taken just prior to an election tended to overestimate Obama in states with a black population below eight percent, to track him within the polls’ margins of error in states with a black population between ten and twenty percent, and to underestimate him in states with a black population exceeding twenty-five percent. The first finding suggested the possibility of the Bradley effect, while the last finding suggested the possibility of a “reverse” Bradley effect in which black voters might have been reluctant to declare to pollsters their support for Obama or are under polled.

There are numerous possible explanations for the “reverse” effect, including faulty likely voter models, under sampling of blacks, sampling bias due to cell phones, to name a few.  There might be some sort of contextual effect at work here, but to ascribe it to behavioural motivations (rather than factors exogenous to the individual, such as sampling bias) such as blacks being shy to state their support for Obama fails the face validity test to me.

Ultimately, given the wide array of mediocre data presented here, I am not concerned about social desirability biasing the estimates of support for Obama in any significant, substantive manner.  However, much as I’d like to, I wouldn’t say that the Republican conspiracy is more likely, if only because that is so creatively ludicrous I initially thought it was an Onion piece.

Push Polling!

[ 0 ] March 20, 2010 |

Ten minutes ago, the wife received a call Family Research Council. The first question: “Do you have a favorable opinion of President Obama’s government run health care proposal?”

After she answered “Yes,” there were no further questions. I suspect that there would have been more than a few additional queries if the answer had been “No.”

UPDATE: IPE@UNC marshaled the mighty power of empirical science to determine the truth of my last statement:

They called the wife’s phone about 10 minutes later and she let me answer. This time when asked if I supported the Marxist take over of everything great about America I said “No”. Were there follow-ups? Why….. yes! Of course there were. The next question was some variant of “Do you support abortion?” I said “No” and then the robot lit into a 30 second rant about how Obama pledged to Planned Parenthood during the campaign that he’d do everything in his power to eliminate all restrictions on abortion (which I think is untrue, but maybe not) and strongly implied, without directly saying so, that passage of the health care bill would lead to just that outcome. Then the robot asked if I’d be willing to contact my congressman about this issue and I said “Yes”. The robot then helpfully gave me my representatives’ information.

More on British Polling and Margins of Error

[ 0 ] March 2, 2010 |

I promise that this will not be a daily habit of mine, but the Tory +2% lead poll did generate the predictable breathless excitement on these islands.

YouGov’s daily tracker indeed moved back towards +6, as anticipated, at Conservative +7. Additionally, a ComRes poll was released yesterday for The Independent which places the Tory lead at +5 (37/32/19). I want to emphasize that the +2 poll released on Sunday, while at the fringes assuming a true value of +6, is still within the margin of error. It was not an outlier in a pedantic understanding of the word, which is a case three standard deviations removed from the mean.
In other words, the findings reported in the Sunday poll were not “wrong”. 95% of samples will yield the true value within the error band of +/- 3%, as this was. This is why we have margins of error in the first place.

What we should focus on is not the point estimate itself, but the trend – and the consistent trend is clearly away from the Tories and towards Labour.

Speaking of which, the +5% Tory lead reflected in The Independent’s poll still yields a distribution of seats as Labour 287, Conservative 272, Liberal Democrat 59 — assuming a uniform national swing.

UPDATE ( 3/3/10): Again as expected, Wednesday’s YouGov tracker is rather consistent, at Conservative +5 (C 38 L 33 LD 16). Given the last seven to ten days of polling, this suggests a true value of support around + 6%, and if not precisely 6 (and it isn’t) it’s slightly below 6%.

Preliminary Colorado exit polling data

[ 0 ] November 4, 2008 |

This is weighted by affiliation but doesn’t attempt to incorporate early voting:

Obama 52

McCain 47

Senate Polling

[ 0 ] October 31, 2006 |

Some new CNN data. It still looks like 50-49-1 to me, with Tennessee looking like a write-off. The one ray of hope is Virginia–the polls are still even, and given the D.C. suburbs I can see Virginia going Democratic in a toss up race the way I can’t see Tennessee. On the other hand, MO is also still a toss-up, although I’m inclined to think the Dems will take it.

Polling Data

[ 0 ] September 17, 2004 |

Via Kos, John Zogby makes a compelling argument that telephone polling, as we know it, is obsolete:

Zogby points out that you don’t know in which area code the cell phone user lives. Nor do you know what they do. Beyond that, you miss younger people who live on cell phones. If you do a political poll on land-line phones, you miss those from 18 to 25, and there are figures all over the place that show there are 40 million between the ages of 18 and 29, one in five eligible voters.

This is pretty interesting, and does not admit an obvious solution. Zogby had moved to internet polling, which may or may not give a more accurate account. We’ll have to wait for November to clear some of this up. In any case, the huge spread of current polls is probably due to something more than the difference in likely voter methodology.

I’m not current with the legalities of polling via cell phone, although I have noted that I tend to receive very few unsolicited calls on my cell (very few solicited ones, either; I’m one hell of a loser). Myself and my roommate have gone the cellular/cable modem route, and no longer have a landline at all, meaning that we are apparently politically invisible. How will the invisibile people sort themselves out in November? Well, we’re young, which may help the Dems, and we’re probably a touch more affluent than normal (correcting for the fact that we’re young) which favors the Republicans.

Searching for Meaningful Meaning with David Brooks

[ 56 ] May 8, 2015 |

This is not the first time Brooks’ increasingly-more-maudlin columns have been documented, but I’m not sure they’ve ever been documented in such exhaustive and hilarious detail.

I’ve been a pretty regular reader of the New York Times columnist since before he even came to the Times, going all the way back to his seminal 2000 book Bobos in Paradise: The New Upper Class and How They Got There, which revealed, to the astonishment of various residents of the East Coast media world’s upper crust, that the rich white people of the Clinton years were different from previous generations of rich white people, because they used their money to buy elite refrigerators instead of jewel-encrusted top hats.

Speak for yourself, dude. Some of us super-classy elites do both. Me, I bought a jeweled refrigerator in the shape of a top hat because I am a tastemaker.

You’d see a headline like “The Slow Virtues” or “The Hollow Century” or “Why the Teens Are Despicable,” and you’d know ol’ Dave’s coffee shop was out of plain croissants a week ago and the barista had a nose-ring and he’d decided he’d witnessed the death of the Western moral tradition.

In fairness, the barista was wearing the nose ring on his toe and was serving the coffee with his feet. They call it “coffee-surfing.” It’s the new planking. Millennials.

 

David Brooks is telling us something dark and sad—about loneliness and the search for connection; about social desolation and sexual frustration and sadness. Something deeply personal, about discovering, too late in life, that accomplishment and position and thinkfluence are no ameliorative for the rejection of your gross old-man wiener by cute millennials. Something not about what priorities he guesses Whole Foods Uncles will take into the voting booth in 2016, but about himself.

 

This is so silly,  and Brooks shouldn’t take it personally. Cute millennials only eat gluten-free, vegan weiner. Also, gluten-free, vegan weiners are what Whole Foods Uncles take into the voting booth with them, which is why you never, ever want to man polling places in hoity-toity neighborhoods.

The hot millennials do not want a New York Times columnist from whom to receive stimulating discourse about the moral and attitudinal deficiencies of the poor. No, they want a “not-repulsive person” who “does not look like a waxed talpid,” thanks to some cockamamie notion that “sexual attraction” might be a more fruitful basis for a relationship than “being lectured by a fusty boomer pissbaby about how masculine chivalry is the bedrock of civilization and both were destroyed by the sexual revolution.”

I got nothin’. Nothin’ but applause. This guy is good.

David Brooks sent his ex a dong shot. And then used his New York Times column to tell the world about it. Our man perches upon the edge of The Void, and hears its howl. Does it call his name? Or does he only want it to?

 

This is bad, sure, but I’m not checking on David ’til he sends a tweet saying simply “Poopin’.” Then again, I guess this guy is arguing that David’s been sending that long-form tweet for awhile now.

Selective Health Care Populism

[ 62 ] April 6, 2015 |

Obama-sells_Obamacare-620x320

Anyone who participated in blog comments sections back in the previous decade knows that there was an inevitable response to criticisms of the Bush administration: “Republicans keep winning elections neener-neener.”  This was true until it wasn’t, as well as being irrelevant to the criticisms.  As Jon Chait argues, with virtually all of their empirical predictions about the ACA having proven to be spectacularly wrong, all that critics of the ACA have are public opinion surveys:

This is the reality that the entire Republican Party has failed to come to grips with. The American health-care system before Obamacare was an utter disaster — the most expensive in the world and also the only one that denied access to millions of its own citizens. Obamacare set out to change those things, and it has worked.

There is one remaining indictment of the law that Tanner makes, and it’s true. “The law remains extraordinarily unpopular, with opponents topping supporters by nearly 11 percentage points, according to the latest Real Clear Politics average,” he argues. It is notable that opponents of Obamacare have fixated on the law’s poor polling. In a recent column, Reason’s Peter Suderman quibbles halfheartedly with the law’s demonstrable success in carrying out its goals — suggesting that the astonishing drop in medical inflation may be owed to outside forces — before reveling for six paragraphs in his major point, which is continued lack of public approval. “Obamacare is simply not well liked,” he concludes, “This is the political reality — and President Obama still refuses to embrace it.”

It is telling that, having lost every substantive argument about the law’s operation, their sole remaining refuge is an argument about its perception. It’s true: Their lies got halfway around the world before the truth could get its pants on. Indeed, if you google most of the factual disputes I discuss above, you’ll get a lot more hits from conservatives making hysterical and false predictions than you will find from reports showing those predictions failed to come true. Those myths still hold enormous sway over public opinion. Far more Americans believe Obamacare has death panels, which is false, than believe its costs have come in under projections, which is true. Conservatives have won the propaganda war over Obamacare. The trouble is that they think this is an indictment of Obamacare, when in fact it’s an indictment of them.

As Chait says, it’s not an accident that much conservative criticism has focused on assertions that the ACA would fail on its own terms. The position of most American conservatives on health care — i.e. that in 2009 too many people had insurance and the insurance that many people did have was too good — is not only morally barbarous but would make the ACA look more popular than free beer in comparison. And in this sense, while the argument that the ACA is unpopular is unusually true for an anti-ACA talking point, it’s still very misleading. Preserving or expanding the ACA remains more popular than cutting it back or rescinding it. The ACA is the least popular health care proposal on offer except for all the others. Which is yet another reason for why Obama is not going to embrace Peter Suderman’s “reality.”

Israeli election

[ 36 ] March 12, 2015 |

This is starting to get interesting. I’ve long had this vague notion that Likud would pull away just before the election, although I admit I can’t give a particularly compelling narrative about why, other than generalized pessimism about anything good ever happening again in that part of the world. But the polling seems pretty robust at this point, and time is very nearly up. At this point it seems pretty clear the speech stunt didn’t do him any favors. The man who may be kingmaker, Moshe Kahlon, seems to be playing it pretty close to the vest.

This is mostly just an Israeli election open thread. Is Bibi really in trouble? And what’s Kahlon’s deal? If he ends up in a position to choose the next prime minister, what sort of concessions might he seek to extract from Bibi, or Herzog?

The GOP Will Not Have An Alternative to the ACA. Do They Need To?

[ 73 ] January 31, 2015 |

With the Supreme Court quite likely to willfully misread the ACA and destroy the insurance markets in a majority of states, the GOP is again pretending that it will at some point have some alternative policy.  Ed Kilgore (correctly, in my view) thinks there’s virtually no chance of that happening, especially as the fight for the 2016 nomination heats up:

No one at this point in the GOP is addressing how they deal with the ecstatic reaction of their party’s conservative activist base if and when the news blares out on Fox that SCOTUS has landed a lethal spear in the hide of the Great White Whale. Just yesterday polling data came out showing Republican rank-and-file opposed the idea of Congress doing anything to “repair” Obamacare. Ya think maybe the already difficult process of agreeing on a “fix” might be complicated a bit more by the shrieks of “NO! NO! NO!” from every Republican who has been told again and again that the Affordable Care Act is the worst thing to happen to America in living memory? Is it possible a Republican presidential candidate or three would exploit the situation by starting a crusade to destroy any GOP member of Congress who even thinks about “fixing” Obamacare?

The fact that congressional Republicans are highly unlikely to even be able to pretend to have an alternative to the ACA may make the Roberts and/or Kennedy marginally less likely to embrace pure lawlessness in support of Republican policy goals. But should the Court reverse King, will the lack of an alternative hurt the GOP? I don’t really think so. Obama, not congressional Republicans, is likely to take the brunt of the political hit if the Court wrecks the markets. Voters who don’t follow policy details are going to tend to blame the president for bad things that happen, irrespective of who’s actually responsible.

For the same reason, I don’t agree with the arguments I’ve seen in some quarters that the Supreme Court upholding King while using Chevron deference would be barely better than the Court wrecking the exchanges immediately, because a future Republican president would remain free to wreck the markets. I don’t agree. I’m not at all sure that a Republican president would do that unilaterally — the GOP can largely escape political retribution for raising taxes taking insurance away from 10 million people if the Court does it, but not if a Republican president does it. President Walker might do it anyway — but if it’s going to happen, it’s still better that a Republican White House takes appropriate responsibility for it.

Warren, public opinion, and inequality

[ 38 ] January 18, 2015 |

Paul Rosenberg has a good response to Amitai Etzioni’s rather lame attempt at a hatchet-job on Elizabeth Warren at The Atlantic:

Taking Norton and Ariely’s results seriously, we can say that the American people want a much fairer society than they live in, but that the means for articulating this desire—the stories, concepts, policy proposals, etc.—are in scandalously short supply, a de facto example of hypocognition thwarting what people want. Elizabeth Warren is particularly popular precisely because she provides some of the missing means that people are so hungry for—an antidote to the hypocognition that thwarts their desire for a fairer, more just vision of America, which respects both their hard work and their compassionate values. There may be relatively little polling to support this view (though there’s considerably more than you’d expect) but that’s partly just another example of how elites dominate the landscape of acceptable thought to protect their interests, as underscored by recent research by Martin Gilens and Benjamin Page. Warren represents a clear alternative to this narrow-minded view. Her popularity derives in large part from her ability to shape narratives that reflect the hidden majority’s shared values and articulate them in policy terms, reversing a decades-long trend by which elites of both parties have turned their backs on the welfare of ordinary Americans.

While Rosenberg offers a much more accurate portrait of American public opinion than does Etzioni, there are reasons to think this offers an overly optimistic account. He’s right, of course, that Americans want a more egalitarian distribution of wealth and income than they’ve got. But it’s almost certainly the case that partisan identity is likely to significantly diminish the ‘hidden majority’ support for redistribution when it turns into an actual plan promoted by and associated with Democratic politicians (as the continued unpopularity of something called “Obamacare” demonstrates). Raising the minimum wage manages to remain broadly popular despite the partisan divide, so it’s important not to be too fatalistic about this. (I suspect one reason for this is the simplicity of the policy; it’s harder to spin or dissemble the basic fairness of it away.) But the lack of specific policy proposals cuts both ways–lots of inequality-reducing proposals could be quite popular in the abstract, but once they become “Democratic” proposals support is likely to conform to a more familiar partisan pattern.

Page 1 of 1712345...10...Last »