Subscribe via RSS Feed

Search Results for 'pollin'

Polling Failure in the UK

[ 50 ] May 12, 2015 |

UK_opinion_polling_2010-2015

As the occasional (charitably) LGM Senior British Correspondent, I’m going to weigh in with a series of thoughts on the British General Election last week.  Cards on table: one of several reasons for my extended sabbatical(*) from LGM has been my active involvement, time permitting, in the Labour Party campaign here in Plymouth.

When I walked into the Plymouth Guildhall Thursday night around 10:15 to participate in the counting of the ballots as a count agent for my Constituency Labour Party, following 14 hours on the campaign trail, I was immediately confronted with the exit poll conducted by several pollsters for several media organisations.  The poll was a shock as it was inconsistent with the narrative created by the last year of public polling released by the major houses.  All the major polls, and the five or six seat projection models, suggested a hung parliament. The largest single party in this new parliament would likely be the Conservatives, but they would only have anywhere from five to 15 more seats than Labour, and the maths suggested there was no way the Tories could form a stable minority government, let alone a coalition:

The exit poll will be out very shortly, and then we’ll have a good idea (or a false one). But first, here’s the game. No one is going to win an overall majority, so it’s all about who can cobble together 323 seats – the number needed for a majority – by banding together with other parties.

Second, Labour seem the most likely to win that game. May2015’s Poll of Polls, which has averaged all the latest polls since September, has finally finished adding numbers up. It’s conclusion? The Tories are going to win 33.8 per cent of the vote, and Labour are going to win 33.7.

This was the narrative the pollsters stood by, and the narrative that those of us academics called upon by the media used as the foundation for discussion (with our own various caveats).  Quite obviously this was wrong, and I’m plastered all over the media both in Devon and the Southwest of England as getting it very wrong.

For the 2015 General Election, we had access to considerably more, and richer, data than in elections past. It felt like an embarrassment of riches, and a certain hubris resulted.  In addition to the national level polling, Lord Ashcroft released around 130 constituency level polls of marginal seats.  These, with the expected N of around 1000, for the first time allowed us to understand how the national numbers and trends were being reflected at the constituency level not only occasionally, but systematically. We were in a position where we could finally bury the swingometer based on the mythical uniform national swing.

The interesting academic question from this election is why polling in the UK failed as bad (if not worse) than it did in 1992.  At this early point, we don’t know, and anybody offering a definitive explanation is taking a significant risk.  There are working theories and interesting questions; four can be found here from YouGov, ICM, Populus, and ComRes. Labour’s internal pollster has an observation here, which is intriguing given the methodological insight revealed for the internal polling. Of course, as these data are not in the public domain, any conclusions drawn are not definitive.  Finally, Eric Kaufmann (one n removed from a relation with our own SEK) has this intriguing take here at the LSE blog. I have one potential minor critique of the Kaufmann piece — his methodology is based on 130 of the Lord Ashcroft constituency level polls, and some of these were ancient in political terms.  Speaking for the two constituencies that represent 15 of Plymouth’s 20 electoral wards, Plymouth Sutton & Devonport was polled in August, while Plymouth Moor View was polled in December.  (The third, Southwest Devon, wasn’t touched, as it’s a very safe Conservative seat.)  These polls estimated a 13 and 11 point Labour victory respectively; on May 7th, the Conservatives won both constituencies by 1.1% and 2.4%.

It is likely that no single cause will explain the polling failure of 2015.  I have an additional theory that I’ll discuss soon assuming it passes prima facie.

(*) The other reasons include being elevated to an administrative role in my department (cue up Bunk to McNulty here), and that this election has increased my media calls significantly. I’ve done somewhere north of 60 appearances in the past twelve months. Rob suggested last week that I link these to LGM.  I will in the future when I can, if only that you, too, can laugh along with the audience at home.

FacebookTwitterGoogle+Share

Social Desirability and Response Validity in Current Polling

[ 22 ] September 28, 2012 |

Several days ago in these very pages, discussion ensued regarding the latest conservative attempt to rewrite reality through re-weighting polls to one guy’s liking.  Of course, polling is not an exact science, but it is a science, and the latest wingnut delusion has no grounding in theory or empirical evidence.  Like any science, survey research continuously attempts to improve upon the validity and reliability of its measures and findings.  While I’m not at all concerned about some nefarious (and successful) attempt by the MSM and that paragon of power, the Democratic Party, to turn otherwise professional and reputable polling houses into duplicitous shills.

However, I have been somewhat interested (note, not concerned) if there might be something else going on that causes the polls to over estimate support for Obama.  Social desirability bias is something I’ve published on in the past (direct link to the paper here).  While that article suggests a contextual effect that causes variance in social desirability across countries (regarding accurately reported turnout in survey research), relevant here is what is colloquially known as the Bradley Effect.  It’s possible (though I consider it unlikely in the specific context of the 2012 Presidential election) that this helps explain Obama’s consistent polling advantage in an election where many if not most structural conditions suggest an incumbent defeat.

It’s difficult with the data I have available to examine this hypothesis to any satisfaction, but that’s not going to stop me from trying.

To begin with, we have the current state of the polls.

Obama Romney Advantage
RCP 28 Sep. 48.6 44.6 4.0
538 28 Sep. 52.2 46.5 5.7
538 6 Nov. 51.5 47.4 4.1

 

RCP’s running average has Obama up 4 points, Nate Silver’s “nowcast” model up 5.7, and his current prediction for election day 4.1 points.

If social desirability is at work here, a poll respondent will state that she or he supports the President because internally, our not entirely sincere respondent is seeking the socially desirable response, and not supporting the black guy might be racist.  However, this is done knowing that they will ultimately support the white guy.  Practically, this would mean that Obama’s support in these polls is inflated.

I’m approaching this from several directions.  First, I’ve averaged the final month of polls for Presidential elections going back to 1976 (an arbitrary cut off) to examine how accurate the polls were in predicting the final outcome between two white men, with 2008 to serve as a benchmark for 2012.  Shift represents how wide of the mark the final polls were, to the benefit or detriment of the incumbent party.

Poll Result Shift
2008 D 7.6 D 7.3 0.3
2004 R 1.5 R 2.4 0.9
2000 R 3.0 D 0.5 3.5
1996 D 11.0 D 8.0 -3.0
1992 D 12.0 D 6.0 -6.0
1988 R 12.0 R 7.0 -5.0
1984 R 18.5 R 18.0 -0.5
1980 R 4.0 R +10.0 -6.0
1976 D 2.0 D 2.0 0.0

 

Social desirability response bias in an election can take on many forms, not just race.  To wit, the 1992 general election in the United Kingdom is a good case study (one I lectured on here at Plymouth about six or seven years ago, shame I have no clue where those lecture notes now reside) as the polls largely predicted a narrow Labour outright victory or a hung parliament with Labour having the plurality of seats, yet the Conservatives under John Major easily won by 7.5%.  This is called the Shy Tory Factor on this island, which is simply another manifestation of social desirability.  But in 2012, I’m primarily considering race, and comparing 2008 to past elections does not support the hypothesis that this might be a problem for Obama’s numbers in 2012.

I also considered several of the primary elections in January, 2008.  This was the beginning of a primary where Obama was a somewhat unknown junior senator only four years into his Congressional career, going up against the assumed nominee.  For these data, I average all polls from the last week of the campaign in the given state (there were 20 in New Hampshire alone).

Poll Result Shift
NH Obama 35 37 2
NH Clinton 30 39 9
NV Obama 33.25 45 11.75
NV Clinton 37 51 14
SC Obama 41 55 14
SC Clinton 26 27 1

 

This evidence is more ambiguous than the examination of previous general elections.  Both Clinton and Obama received shifts in their favor, which isn’t surprising considering the undecideds presumably made a decision of some sort once voting.  However, in both New Hampshire and Nevada, the shift was stronger towards Clinton than Obama: a 7 and 2.25 point advantage respectively.  Both are dwarfed by Obama’s advantage in South Carolina.

These are the wrong data to be analysing this with, of course; ideally we’d have individual level data.  While not individual level data, the following figure, by Greenwald and Albertson, offers a more holistic view of the 2008 primaries.

The above shows that among 32 states where data were available, the “Bradley effect” was only evident in three states, yet 12 states demonstrated what has been termed (erroneously, in my opinion) the “reverse Bradley effect”: states where Obama’s support in the primaries was under, not over, estimated (see South Carolina above).  I consider this an erroneous classification because where the theoretical explanation for the Bradley effect hinges on social desirability, the reverse has been hypothesized as a function of systematic sample bias, through either the under-representation of African Americans in polling samples, or the cell-phone effect.  However, some have hypothesized that “black voters might have been reluctant to declare to pollsters their support for Obama”, and the link above does discuss that

After the Super Tuesday elections of February 5, 2008, political science researchers from the University of Washington found trends suggesting the possibility that with regard to Obama, the effect’s presence or absence may be dependent on the percentage of the electorate that is black. The researchers noted that to that point in the election season, opinion polls taken just prior to an election tended to overestimate Obama in states with a black population below eight percent, to track him within the polls’ margins of error in states with a black population between ten and twenty percent, and to underestimate him in states with a black population exceeding twenty-five percent. The first finding suggested the possibility of the Bradley effect, while the last finding suggested the possibility of a “reverse” Bradley effect in which black voters might have been reluctant to declare to pollsters their support for Obama or are under polled.

There are numerous possible explanations for the “reverse” effect, including faulty likely voter models, under sampling of blacks, sampling bias due to cell phones, to name a few.  There might be some sort of contextual effect at work here, but to ascribe it to behavioural motivations (rather than factors exogenous to the individual, such as sampling bias) such as blacks being shy to state their support for Obama fails the face validity test to me.

Ultimately, given the wide array of mediocre data presented here, I am not concerned about social desirability biasing the estimates of support for Obama in any significant, substantive manner.  However, much as I’d like to, I wouldn’t say that the Republican conspiracy is more likely, if only because that is so creatively ludicrous I initially thought it was an Onion piece.

Push Polling!

[ 0 ] March 20, 2010 |

Ten minutes ago, the wife received a call Family Research Council. The first question: “Do you have a favorable opinion of President Obama’s government run health care proposal?”

After she answered “Yes,” there were no further questions. I suspect that there would have been more than a few additional queries if the answer had been “No.”

UPDATE: IPE@UNC marshaled the mighty power of empirical science to determine the truth of my last statement:

They called the wife’s phone about 10 minutes later and she let me answer. This time when asked if I supported the Marxist take over of everything great about America I said “No”. Were there follow-ups? Why….. yes! Of course there were. The next question was some variant of “Do you support abortion?” I said “No” and then the robot lit into a 30 second rant about how Obama pledged to Planned Parenthood during the campaign that he’d do everything in his power to eliminate all restrictions on abortion (which I think is untrue, but maybe not) and strongly implied, without directly saying so, that passage of the health care bill would lead to just that outcome. Then the robot asked if I’d be willing to contact my congressman about this issue and I said “Yes”. The robot then helpfully gave me my representatives’ information.

More on British Polling and Margins of Error

[ 0 ] March 2, 2010 |

I promise that this will not be a daily habit of mine, but the Tory +2% lead poll did generate the predictable breathless excitement on these islands.

YouGov’s daily tracker indeed moved back towards +6, as anticipated, at Conservative +7. Additionally, a ComRes poll was released yesterday for The Independent which places the Tory lead at +5 (37/32/19). I want to emphasize that the +2 poll released on Sunday, while at the fringes assuming a true value of +6, is still within the margin of error. It was not an outlier in a pedantic understanding of the word, which is a case three standard deviations removed from the mean.
In other words, the findings reported in the Sunday poll were not “wrong”. 95% of samples will yield the true value within the error band of +/- 3%, as this was. This is why we have margins of error in the first place.

What we should focus on is not the point estimate itself, but the trend — and the consistent trend is clearly away from the Tories and towards Labour.

Speaking of which, the +5% Tory lead reflected in The Independent’s poll still yields a distribution of seats as Labour 287, Conservative 272, Liberal Democrat 59 — assuming a uniform national swing.

UPDATE ( 3/3/10): Again as expected, Wednesday’s YouGov tracker is rather consistent, at Conservative +5 (C 38 L 33 LD 16). Given the last seven to ten days of polling, this suggests a true value of support around + 6%, and if not precisely 6 (and it isn’t) it’s slightly below 6%.

Preliminary Colorado exit polling data

[ 0 ] November 4, 2008 |

This is weighted by affiliation but doesn’t attempt to incorporate early voting:

Obama 52

McCain 47

Senate Polling

[ 0 ] October 31, 2006 |

Some new CNN data. It still looks like 50-49-1 to me, with Tennessee looking like a write-off. The one ray of hope is Virginia–the polls are still even, and given the D.C. suburbs I can see Virginia going Democratic in a toss up race the way I can’t see Tennessee. On the other hand, MO is also still a toss-up, although I’m inclined to think the Dems will take it.

Polling Data

[ 0 ] September 17, 2004 |

Via Kos, John Zogby makes a compelling argument that telephone polling, as we know it, is obsolete:

Zogby points out that you don’t know in which area code the cell phone user lives. Nor do you know what they do. Beyond that, you miss younger people who live on cell phones. If you do a political poll on land-line phones, you miss those from 18 to 25, and there are figures all over the place that show there are 40 million between the ages of 18 and 29, one in five eligible voters.

This is pretty interesting, and does not admit an obvious solution. Zogby had moved to internet polling, which may or may not give a more accurate account. We’ll have to wait for November to clear some of this up. In any case, the huge spread of current polls is probably due to something more than the difference in likely voter methodology.

I’m not current with the legalities of polling via cell phone, although I have noted that I tend to receive very few unsolicited calls on my cell (very few solicited ones, either; I’m one hell of a loser). Myself and my roommate have gone the cellular/cable modem route, and no longer have a landline at all, meaning that we are apparently politically invisible. How will the invisibile people sort themselves out in November? Well, we’re young, which may help the Dems, and we’re probably a touch more affluent than normal (correcting for the fact that we’re young) which favors the Republicans.

It Is Getting Hard to See Any Evitability

[ 170 ] April 26, 2016 |

151221122025-trump-shrug-super-alt-11

I have generally thought that it would be enormously difficult for Donald Trump to win on a second or later ballot in Cleveland, given his organizational weaknesses. Who knows, this might even be right! However, after tonight’s massive blowout it seems pretty clear that it’s a moot point:

Donald J. Trump is essentially two key states from the nomination.

By sweeping five states on Tuesday, he pulled only a few hundred Republican delegates short of the 1,237 he needs to win without a contested convention.

He has long been favored in the polls in two of the remaining primary states, New Jersey and West Virginia. That leaves Indiana and California as the crucial prizes that would put Mr. Trump over the top — and while he was once thought to be vulnerable in both states, polls have shown him with a modest lead.

And not only that:

One thing that has to greatly worry the anti-Trump forces is that Trump is now exceeding his poll averages. Since New York, Trump has performed at least 6.5 percentage points better in every state than the average of polls taken within 21 days of the election. Before that, Trump tended to hit his polling average and win no undecideds. Now, he’s winning his fair share of undecideds and then some. That’s very bad news for his opponents, given that Trump is already ahead in Indiana, a must-win state for Cruz.

He has a very good shot at 1,237 pledged delegates, and if not he’ll probably get close enough to lock it up on the first ballot. Living in a satirical novel is weird, but I guess eventually you get used to it.

No, Hillary Isn’t Stealing the Primary

[ 252 ] April 19, 2016 |

beb4bf31ba44e38e58cbdfaa03747ef7

In this Democratic primary that is both inspiring and infuriating in equal measure, one of the least appealing aspects of it how many Bernie supporters are accusing Hillary Clinton or the “Democratic Machine” of stealing the election for her. Now, there’s no question that the primary system is a hot mess–caucuses are undemocratic, many voter registration laws are ridiculous (including New York’s), states won’t pony up for primaries, states have different standards on who can vote in these primaries (I do not believe that non-Democrats should have the right to choose the Democratic Party nominee), etc. But these allegations that the problem is a conspiracy are totally ridiculous. First, the “party machine,” whatever that even means today, is the weakest it has ever been. Remember that for the vast majority of Democratic Party history, the primaries didn’t even mean very much and actual Democratic Party insiders picked the president. This has only changed in the last 50 years. Second, as Holland points out, why would Hillary steal a primary when she has no reason to do so?

It’s a conspiracy theory, and conspiracy theories tend to fall apart under the weight of their own internal illogic. Consider the 9/11 attacks: If they were a “false flag” operation pulled off by a cabal of extremists within the Bush administration in order to create a casus belli for the invasion of Iraq, why wouldn’t they make the attackers evil Iraqi intelligence officers rather than citizens of Saudi Arabia? And if they really felt the need to bring down the World Trade Center, why bother shooting a missile at the Pentagon? Why crash an airplane into a field in the middle of nowhere in Pennsylvania? Even judged on its own terms, the theory makes no sense.

Obviously, if campaign operatives were caught rigging a primary or caucus, their candidate would face a media shit-storm, their political careers would be over, and they might well end up in prison. So let’s set aside the specifics for a moment and consider whether there’s a coherent motive for these crimes that would be worth the risk.

Put yourself in the shoes of a vicious Clinton operative with loose ethics and a desire to win at all costs. Why would such a person bother rigging the vote in Wyoming, the state with the fewest delegates up for grabs? That’s a significant risk and a lot of trouble to go through to turn what might have been an 8-6 or perhaps 9-5 delegate split into a 7-7 outcome.

The same problem holds more generally. While Sanders has run an excellent campaign and exceeded all expectations, at no point during the Democratic primaries has he been on track to win. Sanders has held a lead in a handful of national polls, but at no time in the past year has his support broken 42 percent in FiveThirtyEight’s weighted polling average. And at no point in the race has Clinton held a lead narrower than 9.7 percentage points in that average. Why would any campaign, no matter how unprincipled, fix a race that it’s been winning from the start?

The reasons Sanders isn’t winning are not a conspiracy theory. They are quite explainable. It’s that a) he had a late start and couldn’t really compete in many of the states between Nevada and when his early success led to more fundraising and b) he has done very poorly with people of color and a Democrat can’t win the nomination without sizable black and Latino support.

But for a lot of people, conspiracy theories are more palatable than actual political analysis. It’s easier to blame some nefarious power than own up to a candidate’s or a position’s shortcoming, not to mention the work of structural analysis.

Life After Shelby County

[ 72 ] April 13, 2016 |

Chief-Justice-John-Roberts

“Instead, the Court strikes §4(b)’s coverage provision because, in its view, the provision is not based on “current conditions.”  It discounts, however, that one such condition was the preclearance remedy in place in the covered jurisdictions, a remedy Congress designed both to catch discrimination before it causes harm, and to guard against return to old ways. Volumes of evidence supported Congress’ determination that the prospect of retrogression was real. Throwing out preclearance when it has worked and is continuing to work to stop discriminatory changes is like throwing away your umbrella in a rainstorm because you are not getting wet.”

Shelby County v. Holder (Ginsburg, J., dissenting.)

This year marks the first presidential election in 50 years without a functioning Voting Rights Act — and it’s not going well.

Republican lawmakers have been devising efforts to make it harder for Americans to vote for many years, since the GOP took over statehouses across the country in 2010. Those efforts culminated in 2013 with the Supreme Court’s decision to strike down key portions of the Voting Rights Act (VRA), a formerly untouchable cornerstone of civil rights. For the past three years, states and jurisdictions have no longer had to clear potentially discriminatory laws with the Department of Justice — at least not as long as a Congress controlled by radical, right-wing factions continues to sit on the rewrite of the VRA.

Meanwhile, emboldened by that Supreme Court victory, other Republican-controlled states have accelerated their efforts to disenfranchise low-income, minority voters, whether it’s through photo identification requirements or limiting the number of polling locations. In 2016, 17 states will have new voting restrictions in place for the first time in a presidential election.

While Republican lawmakers like to claim that the laws are not suppressing votes, stories from across the country speak for themselves. In Arizona, voters waited up to five hours in line in order to cast ballots. In South Carolina, people were given misleading information about the state’s voter ID law. And in Wisconsin, students were disenfranchised because polling places refused to accept their out-of-state IDs.

Ginsburg’s dissent is and remains unanswerable, but the problem is that the majority wanted a hard rain after they threw away the umbrella.

Voter Suppression Works Well

[ 137 ] March 24, 2016 |

az_lines_usatoday

Now that the relevant parts of the Voting Rights Act have been declared unconstitutional by our lovely Supreme Court, the states that long engaged in racist voting practices are now free to do so again. Such as Arizona, where the statewide effort to stop brown people from voting worked great in this week’s primary.

Phoenix Mayor Greg Stanton requested a federal investigation Wednesday into long voter wait times during Arizona’s presidential preference election.

“Throughout the county, but especially in Phoenix, thousands of citizens waited in line for three, four, and even five hours to vote,” he wrote in a letter to U.S. Attorney General Loretta Lynch.

Stanton alleged that, by cutting down the number of polling places, the Maricopa County Recorder’s Office unfairly made minorities wait longer to vote.

“In Phoenix, a majority-minority city, county officials allocated one polling location for every 108,000 residents,” the letter read.

“The ratios were far more favorably in predominantly Anglo communities: In Cave Creek/Carefree, there was one polling location for 8,500 residents; in Paradise Valley, one for every 13,00 residents; in Fountain Hills, one for 22,500 residents; and in Peoria, one for every 54,000 residents.”

One polling location for 108,000 residents. That’s good news for John McCain!

Also, the idea that this is some sort of Hillary scheme is really stupid and irresponsible. That’s Bernie supporter conspiracy theories at their very worst. Yeah, I’m sure the Republican controlled Arizona government is totally rigging the game so their candidate doesn’t have to run against an avowed socialist. That makes a ton of sense….

Where We Are

[ 221 ] March 16, 2016 |

rubioself

  • Bernie Sanders has run a valiant, historically consequential campaign. But barring force majeure, Hillary Clinton will be the Democratic nominee. The already nearly prohibitive delegate math just got considerably worse for Sanders, with New York, California and Maryland still on the table, and this time Clinton pretty much held her polling leads in every state with the possible exception of Illinois. Her lead in national polls is double digits and trending upward, in a context in Sanders would need win by substantial margins going forward. It’s over.
  • This is a useful explanation for why Clinton did so much better in Ohio than Michigan.
  • The Republican race is obviously more complicated. Trump is the only candidate who can win a majority of delegates, which means that the Republican candidate will either be Trump or someone who had to depose trump at a contested convention. Either way, this is Excellent. News. For. The. Democratic. Party, albeit with a downside risk horrible enough that one can’t be unambiguously happy.
  • Obviously, nobody is going to top Chris Mathews for the evening’s most molten take, but these are certainly solid entries.
  • Vote suppression in North Carolina.
  • More on the National Review‘s new war on the white working class.
  • An entertaining requiem for the hapless Rubio.
Page 1 of 2012345...1020...Last »