Subscribe via RSS Feed

Search Results for 'pollin'

Social Desirability and Response Validity in Current Polling

[ 22 ] September 28, 2012 |

Several days ago in these very pages, discussion ensued regarding the latest conservative attempt to rewrite reality through re-weighting polls to one guy’s liking.  Of course, polling is not an exact science, but it is a science, and the latest wingnut delusion has no grounding in theory or empirical evidence.  Like any science, survey research continuously attempts to improve upon the validity and reliability of its measures and findings.  While I’m not at all concerned about some nefarious (and successful) attempt by the MSM and that paragon of power, the Democratic Party, to turn otherwise professional and reputable polling houses into duplicitous shills.

However, I have been somewhat interested (note, not concerned) if there might be something else going on that causes the polls to over estimate support for Obama.  Social desirability bias is something I’ve published on in the past (direct link to the paper here).  While that article suggests a contextual effect that causes variance in social desirability across countries (regarding accurately reported turnout in survey research), relevant here is what is colloquially known as the Bradley Effect.  It’s possible (though I consider it unlikely in the specific context of the 2012 Presidential election) that this helps explain Obama’s consistent polling advantage in an election where many if not most structural conditions suggest an incumbent defeat.

It’s difficult with the data I have available to examine this hypothesis to any satisfaction, but that’s not going to stop me from trying.

To begin with, we have the current state of the polls.

Obama Romney Advantage
RCP 28 Sep. 48.6 44.6 4.0
538 28 Sep. 52.2 46.5 5.7
538 6 Nov. 51.5 47.4 4.1


RCP’s running average has Obama up 4 points, Nate Silver’s “nowcast” model up 5.7, and his current prediction for election day 4.1 points.

If social desirability is at work here, a poll respondent will state that she or he supports the President because internally, our not entirely sincere respondent is seeking the socially desirable response, and not supporting the black guy might be racist.  However, this is done knowing that they will ultimately support the white guy.  Practically, this would mean that Obama’s support in these polls is inflated.

I’m approaching this from several directions.  First, I’ve averaged the final month of polls for Presidential elections going back to 1976 (an arbitrary cut off) to examine how accurate the polls were in predicting the final outcome between two white men, with 2008 to serve as a benchmark for 2012.  Shift represents how wide of the mark the final polls were, to the benefit or detriment of the incumbent party.

Poll Result Shift
2008 D 7.6 D 7.3 0.3
2004 R 1.5 R 2.4 0.9
2000 R 3.0 D 0.5 3.5
1996 D 11.0 D 8.0 -3.0
1992 D 12.0 D 6.0 -6.0
1988 R 12.0 R 7.0 -5.0
1984 R 18.5 R 18.0 -0.5
1980 R 4.0 R +10.0 -6.0
1976 D 2.0 D 2.0 0.0


Social desirability response bias in an election can take on many forms, not just race.  To wit, the 1992 general election in the United Kingdom is a good case study (one I lectured on here at Plymouth about six or seven years ago, shame I have no clue where those lecture notes now reside) as the polls largely predicted a narrow Labour outright victory or a hung parliament with Labour having the plurality of seats, yet the Conservatives under John Major easily won by 7.5%.  This is called the Shy Tory Factor on this island, which is simply another manifestation of social desirability.  But in 2012, I’m primarily considering race, and comparing 2008 to past elections does not support the hypothesis that this might be a problem for Obama’s numbers in 2012.

I also considered several of the primary elections in January, 2008.  This was the beginning of a primary where Obama was a somewhat unknown junior senator only four years into his Congressional career, going up against the assumed nominee.  For these data, I average all polls from the last week of the campaign in the given state (there were 20 in New Hampshire alone).

Poll Result Shift
NH Obama 35 37 2
NH Clinton 30 39 9
NV Obama 33.25 45 11.75
NV Clinton 37 51 14
SC Obama 41 55 14
SC Clinton 26 27 1


This evidence is more ambiguous than the examination of previous general elections.  Both Clinton and Obama received shifts in their favor, which isn’t surprising considering the undecideds presumably made a decision of some sort once voting.  However, in both New Hampshire and Nevada, the shift was stronger towards Clinton than Obama: a 7 and 2.25 point advantage respectively.  Both are dwarfed by Obama’s advantage in South Carolina.

These are the wrong data to be analysing this with, of course; ideally we’d have individual level data.  While not individual level data, the following figure, by Greenwald and Albertson, offers a more holistic view of the 2008 primaries.

The above shows that among 32 states where data were available, the “Bradley effect” was only evident in three states, yet 12 states demonstrated what has been termed (erroneously, in my opinion) the “reverse Bradley effect”: states where Obama’s support in the primaries was under, not over, estimated (see South Carolina above).  I consider this an erroneous classification because where the theoretical explanation for the Bradley effect hinges on social desirability, the reverse has been hypothesized as a function of systematic sample bias, through either the under-representation of African Americans in polling samples, or the cell-phone effect.  However, some have hypothesized that “black voters might have been reluctant to declare to pollsters their support for Obama”, and the link above does discuss that

After the Super Tuesday elections of February 5, 2008, political science researchers from the University of Washington found trends suggesting the possibility that with regard to Obama, the effect’s presence or absence may be dependent on the percentage of the electorate that is black. The researchers noted that to that point in the election season, opinion polls taken just prior to an election tended to overestimate Obama in states with a black population below eight percent, to track him within the polls’ margins of error in states with a black population between ten and twenty percent, and to underestimate him in states with a black population exceeding twenty-five percent. The first finding suggested the possibility of the Bradley effect, while the last finding suggested the possibility of a “reverse” Bradley effect in which black voters might have been reluctant to declare to pollsters their support for Obama or are under polled.

There are numerous possible explanations for the “reverse” effect, including faulty likely voter models, under sampling of blacks, sampling bias due to cell phones, to name a few.  There might be some sort of contextual effect at work here, but to ascribe it to behavioural motivations (rather than factors exogenous to the individual, such as sampling bias) such as blacks being shy to state their support for Obama fails the face validity test to me.

Ultimately, given the wide array of mediocre data presented here, I am not concerned about social desirability biasing the estimates of support for Obama in any significant, substantive manner.  However, much as I’d like to, I wouldn’t say that the Republican conspiracy is more likely, if only because that is so creatively ludicrous I initially thought it was an Onion piece.

Push Polling!

[ 0 ] March 20, 2010 |

Ten minutes ago, the wife received a call Family Research Council. The first question: “Do you have a favorable opinion of President Obama’s government run health care proposal?”

After she answered “Yes,” there were no further questions. I suspect that there would have been more than a few additional queries if the answer had been “No.”

UPDATE: IPE@UNC marshaled the mighty power of empirical science to determine the truth of my last statement:

They called the wife’s phone about 10 minutes later and she let me answer. This time when asked if I supported the Marxist take over of everything great about America I said “No”. Were there follow-ups? Why….. yes! Of course there were. The next question was some variant of “Do you support abortion?” I said “No” and then the robot lit into a 30 second rant about how Obama pledged to Planned Parenthood during the campaign that he’d do everything in his power to eliminate all restrictions on abortion (which I think is untrue, but maybe not) and strongly implied, without directly saying so, that passage of the health care bill would lead to just that outcome. Then the robot asked if I’d be willing to contact my congressman about this issue and I said “Yes”. The robot then helpfully gave me my representatives’ information.

More on British Polling and Margins of Error

[ 0 ] March 2, 2010 |

I promise that this will not be a daily habit of mine, but the Tory +2% lead poll did generate the predictable breathless excitement on these islands.

YouGov’s daily tracker indeed moved back towards +6, as anticipated, at Conservative +7. Additionally, a ComRes poll was released yesterday for The Independent which places the Tory lead at +5 (37/32/19). I want to emphasize that the +2 poll released on Sunday, while at the fringes assuming a true value of +6, is still within the margin of error. It was not an outlier in a pedantic understanding of the word, which is a case three standard deviations removed from the mean.
In other words, the findings reported in the Sunday poll were not “wrong”. 95% of samples will yield the true value within the error band of +/- 3%, as this was. This is why we have margins of error in the first place.

What we should focus on is not the point estimate itself, but the trend – and the consistent trend is clearly away from the Tories and towards Labour.

Speaking of which, the +5% Tory lead reflected in The Independent’s poll still yields a distribution of seats as Labour 287, Conservative 272, Liberal Democrat 59 — assuming a uniform national swing.

UPDATE ( 3/3/10): Again as expected, Wednesday’s YouGov tracker is rather consistent, at Conservative +5 (C 38 L 33 LD 16). Given the last seven to ten days of polling, this suggests a true value of support around + 6%, and if not precisely 6 (and it isn’t) it’s slightly below 6%.

Preliminary Colorado exit polling data

[ 0 ] November 4, 2008 |

This is weighted by affiliation but doesn’t attempt to incorporate early voting:

Obama 52

McCain 47

Senate Polling

[ 0 ] October 31, 2006 |

Some new CNN data. It still looks like 50-49-1 to me, with Tennessee looking like a write-off. The one ray of hope is Virginia–the polls are still even, and given the D.C. suburbs I can see Virginia going Democratic in a toss up race the way I can’t see Tennessee. On the other hand, MO is also still a toss-up, although I’m inclined to think the Dems will take it.

Polling Data

[ 0 ] September 17, 2004 |

Via Kos, John Zogby makes a compelling argument that telephone polling, as we know it, is obsolete:

Zogby points out that you don’t know in which area code the cell phone user lives. Nor do you know what they do. Beyond that, you miss younger people who live on cell phones. If you do a political poll on land-line phones, you miss those from 18 to 25, and there are figures all over the place that show there are 40 million between the ages of 18 and 29, one in five eligible voters.

This is pretty interesting, and does not admit an obvious solution. Zogby had moved to internet polling, which may or may not give a more accurate account. We’ll have to wait for November to clear some of this up. In any case, the huge spread of current polls is probably due to something more than the difference in likely voter methodology.

I’m not current with the legalities of polling via cell phone, although I have noted that I tend to receive very few unsolicited calls on my cell (very few solicited ones, either; I’m one hell of a loser). Myself and my roommate have gone the cellular/cable modem route, and no longer have a landline at all, meaning that we are apparently politically invisible. How will the invisibile people sort themselves out in November? Well, we’re young, which may help the Dems, and we’re probably a touch more affluent than normal (correcting for the fact that we’re young) which favors the Republicans.

Warren, public opinion, and inequality

[ 38 ] January 18, 2015 |

Paul Rosenberg has a good response to Amitai Etzioni’s rather lame attempt at a hatchet-job on Elizabeth Warren at The Atlantic:

Taking Norton and Ariely’s results seriously, we can say that the American people want a much fairer society than they live in, but that the means for articulating this desire—the stories, concepts, policy proposals, etc.—are in scandalously short supply, a de facto example of hypocognition thwarting what people want. Elizabeth Warren is particularly popular precisely because she provides some of the missing means that people are so hungry for—an antidote to the hypocognition that thwarts their desire for a fairer, more just vision of America, which respects both their hard work and their compassionate values. There may be relatively little polling to support this view (though there’s considerably more than you’d expect) but that’s partly just another example of how elites dominate the landscape of acceptable thought to protect their interests, as underscored by recent research by Martin Gilens and Benjamin Page. Warren represents a clear alternative to this narrow-minded view. Her popularity derives in large part from her ability to shape narratives that reflect the hidden majority’s shared values and articulate them in policy terms, reversing a decades-long trend by which elites of both parties have turned their backs on the welfare of ordinary Americans.

While Rosenberg offers a much more accurate portrait of American public opinion than does Etzioni, there are reasons to think this offers an overly optimistic account. He’s right, of course, that Americans want a more egalitarian distribution of wealth and income than they’ve got. But it’s almost certainly the case that partisan identity is likely to significantly diminish the ‘hidden majority’ support for redistribution when it turns into an actual plan promoted by and associated with Democratic politicians (as the continued unpopularity of something called “Obamacare” demonstrates). Raising the minimum wage manages to remain broadly popular despite the partisan divide, so it’s important not to be too fatalistic about this. (I suspect one reason for this is the simplicity of the policy; it’s harder to spin or dissemble the basic fairness of it away.) But the lack of specific policy proposals cuts both ways–lots of inequality-reducing proposals could be quite popular in the abstract, but once they become “Democratic” proposals support is likely to conform to a more familiar partisan pattern.

Is Climate Change the Next Gay Marriage?

[ 39 ] November 19, 2014 |

This piece argues that climate change could be the next gay marriage in terms of young Republicans supporting meaningful action on it while old Republicans hate the idea of even considered it. Well, maybe. But I think there are some problems here. Primarily, I’m not sure how this manifests itself in policy. Gay marriage has a simple solution: making gay marriage legal. But climate change is far more complicated with no clear situation. Polling showing young people would pay $20 extra a month in home heating have some value but let’s be clear, that ain’t solving climate change. So what happens then? And the mechanism for change is much murkier. Ballot measures mean people can vote to legalize gay marriage. Lawsuits can force states to do the same. There isn’t really a similar mechanism for climate change. Weaning us off coal is great, but it doesn’t solve the problem either. So I’m glad to see young Republicans reasonable on this issue, but it’s not gay marriage.

Friday Links

[ 63 ] October 24, 2014 |

Those Who Fail To Learn From the Results of Nominating Shitty Candidates Are Doomed To Keep Losing

[ 76 ] October 22, 2014 |

The Titantic, the Hindenberg, the 2007 New York Mets, Martha Coakley:

National Democrats are haunted by memories of Martha Coakley’s unforced stumbles and missteps in 2010, which cost them a U.S. Senate seat in one of the country’s bluest states.

Four laters later, the Massachusetts attorney general might be about to blow another major contest: The race to succeed Deval Patrick as governor.

With two weeks left to go, a new poll by WBUR, which tracks the race weekly, found Coakley trailing for the first time against Republican Charlie Baker, a former health care CEO who served as secretary of finance and health under Gov. William Weld in the 1990s.

It’s still a close contest: Baker has 43 percent while Coakley has 42 percent, well inside the poll’s 4.4 percent margin of error.

But the troubling sign for Coakley is that Baker appears to be gaining steam down the stretch after consistently trailing throughout the campaign.

“It’s one of several polls which over the last week or so have shown a movement toward Baker,” Steve Koczela, the president of MassINC Polling Group, which conducts the polls, said. “Coakley has essentially been treading water while Baker’s been climbing.”

Coakley’s late drop-off seems eerily reminiscent of the 2010 special election against upstart Republican candidate Scott Brown, when the Democrat blew a huge lead, fell behind in the final stretch, and went on to lose.

Hopefully she’ll pull it out anyway, but it’s ridiculous that the Massachusetts bench is so shallow that someone who ran one of the worst campaigns in known human history — a bad campaign with very substantial consequences for the country, yet — could get nominated for a competitive race again.

Another Winner From Late-Period Posner

[ 46 ] October 16, 2014 |

Judge Posner’s Voter ID opinion — which, alas, came in the form of a dissent from the denial of an en banc hearing — is indeed a beauty. My favorite graf:

Voter-impersonation fraud may be a subset of “Misinformation.” If so, it is by all accounts a tiny subset, a tiny problem, and a mere fig leaf for efforts to disenfranchise voters likely to vote for the political party that does not control the state government. Those of us who live in Illinois are familiar with a variety of voting frauds, and no one would deny the propriety of the law’s trying to stamp out such frauds. The one form of voter fraud known to be too rare to justify limiting voters’ ability to vote by requiring them to present a photo ID at the polling place is in-person voter impersonation.

Or maybe it’s this one:

As there is no evidence that voter impersonation fraud is a problem, how can the fact that a legislature says it’s a problem turn it into one? If the Wisconsin legislature says witches are a problem, shall Wisconsin courts be permitted to conduct witch trials? If the Supreme Court once thought that requiring photo identification increases public confidence in elections, and experience and academic study since shows that the Court was mistaken, do we do a favor to the Court-do we increase public confidence in elections-by making the mistake a premise of our decision? Pressed to its logical extreme the panel’s interpretation of and deference to legislative facts would require upholding a photo ID voter law even if it were uncontested that the law eliminated no fraud but did depress turnout significantly.

You also have to love the appendix titled “Scrounging For Your Birth Certificate in Wisconsin” (a direct shot at Easterbrook’s embarrassing opinion.) The dissent also does a good job distinguishing the case from Crawford, although Posner and the Supreme Court were wrong then too.

I really hope that Ponsner ends up hearing a case brought by the ACA troofers; that could possibly result in the most entertaining opinion in judicial history.

Thoughts on the Scottish Independence Referendum III: Ramifications

[ 133 ] September 18, 2014 |

Unlike my previous two posts on today’s referendum in Scotland, on electoral ramifications for the remainder of the United Kingdom, and on interpreting polling data, this piece is more of a speculative nature. Here, I consider constitutional, political, and international ramifications of a yes vote, as well as the constitutional ramifications of a no vote.

A No result, which I consider likely, will have both constitutional and political ramifications throughout the United Kingdom.  Constitutionally, the devolved powers to the Scottish Parliament will be greatly enhanced.  This, of course, presents as many challenges to the constitutional order of the UK as problems it solves by retaining the union. The legendary unwritten British “constitution” encourages muddling through, and the implementation of devolution in 1997-99 makes for clearly delineated (and fair) distribution of powers as Heathrow makes for efficient air travel.  What the UK has at present is a vague form of ersatz federalism, and one that is asymmetrically distributed at that. Among its powers, Scotland’s parliament has control over health, education, and policing primary legislation, and it can vary its income tax by 3% (up or down) from the national baseline. Among things it can not touch are corporation tax. This is considerably greater power than the Welsh Assembly, and of course infinitely more than any English region.

Any increase of these powers (an excellent overview of the nature of these enhanced powers on offer and some of the pitfalls surrounding implantation was posted yesterday at the UK Constitutional Law Association) will cause resentment not only in Wales and Northern Ireland, witnessing yet more constitutional preferential treatment given to Scotland, but perhaps most of all, in England. Made famous by then MP Tam Dalyell, arguing in opposition to the devolution legislation under consideration in 1977 (which eventually went on to be referenda in both Scotland and Wales in 1979), perhaps the most stinging critique of the current implantation of devolution in the UK is known as the “West Lothian Question”, which identifies the bizarre situation: MPs representing Scottish constituencies get to vote on legislation that impacts England, while MPs representing England (as well as those MPs representing Scotland for that matter) can not vote on a range of devolved areas of policy.  This perhaps was most stark in 2004 when tuition for English and Welsh universities was raised to £3000 per year (from something around £1500 if I recall correctly, and I likely do not). Education, including higher education, is a devolved matter in Scotland and thus under the remit of the Scottish Parliament. Even to this day, Scottish universities are free for residents of Scotland. Yet, in the 2004 debate,  while Labour had a strong majority, the legislation passed by only five votes. Remove the Scottish Labour MPs from voting, the Act would have failed. Hence, Scottish MPs voted on legislation affecting only English universities (hence, students) while those same MPs can not act in that policy area for Scotland (nor can English MPs, for that matter).

With the promise of expanded devolution should No prevail, this asymmetry will only become more apparent, and English resentment is emerging:

Support in England for Scottish devolution has fallen from 57% in 1999 to 43% now; on the one hand a quarter now think Scotland should leave the Union, while on the other almost as many feel that Scotland should not have any kind of Scottish Parliament at all.  Meanwhile, although it remains the case that only a minority feel that Scotland gets more than its fair share of public spending, the proportion that do feel that way has more than doubled from 21% in 2000 to 44% now.

That was written a year ago. The current edition of The Economist has several articles on this subject (unsurprisingly), and brings more current public opinion data to bear. In one case, a poll in April suggests that by a four to one margin, the English believe that Scotland should receive a smaller share of public expenditure. This isn’t surprising, where north of the border universities are free (as mentioned above) and so too are prescriptions. As it stands, under the Barnett formula, Scotland receives a larger share of public expenditure per capita than England. Additionally, as reported in The Economist, the Future of England Survey identifies a growing desire for Scottish MPs to not be eligible to vote on England-only issues, from 18% in 2000 to 55% in 2012.

None of the proposed methods to circumnavigate the West Lothian Question are perfect, so long as the existing unitary parliamentary structure is retained. For example, if Labour were to win in 2015, but with a majority dependent on Scottish MPs, and England-only rules were in force such that English (or English and Welsh) MPs could vote on matters not impacting Scotland, the Government’s majority suddenly becomes a minority, and the Government can’t pass legislation meant to affect the largest nation in the UK. Assuming a No victory tonight, and the implantation of Devolution Max with the beginning of the next government following the May 2015 election, calls for some sort of representational fairness will grow louder in England, and to me it seems logistically only a true federal response will ensure equitable representation combined with a workable parliamentary system.

A Yes result brings up many issues of its own, of course. The Yes campaign assumes that the admittance of an independent Scotland into the European Union will be a mere formality.  However, it might not be that easy:

First it was claimed that Scotland would automatically remain in the EU, inheriting its UK membership. Highly unlikely. Then it was asserted that Scotland would be put on a fast-track to membership under a different article in the Lisbon treaty from the one dealing with accession and the only process that has been used so far to admit new members. This is also extremely improbable.

While admitting an independent Scotland to the EU would be a smoother process than, say, Turkey, it’s not going to be automatic. Furthermore, there are several countries with regional separatist problems (smile for the camera, Spain) which would not want to set this particular precedent. It’s not difficult to imagine Spain blocking Scotland’s admittance, or at least make it extremely cumbersome. Cyrpus as well. To a lesser extent, Italy, France, Belgium, and even Germany would not want to see a precent of automatic entry to the EU for break-away nations. Remember, every one of the 28 member states has to agree on membership. While the right to self-determination should result in virtually unanimous recognition of Scotland as an independent state, admitting it to the European Union is a different issue altogether.

This neatly segues into what currency Scotland would use upon independence. The Yes campaign insists it will be the Pound Sterling in a formal currency agreement with the remaining United Kingdom. There are only two problems with this. First, all three major party leaders in Westminster reject this idea, and the governor of the Bank of England (which would remain the central bank of a “Sterling zone”) recently stated that this would be “incompatible with sovereignty”. The rUK has no interest in a formal currency union with an independent Scotland while having no control over fiscal policy; the Eurozone crisis has taught them that much. The Scottish government has since threatened to not pay any share of the accrued public debt of the United Kingdom if it is not allowed a currency union, which is, well, bonkers.

Scotland could continue to use the Pound regardless, as several minor countries use the Euro or the US Dollar as their de facto currency, but this would leave Scotland at the mercy of the Bank of England’s monetary policy, which would not be responsible for Scotland. Furthermore, financial services in Scotland, which is a significant share of the Scottish economy (12.5% of Scottish GDP according to the Economist, 7.1% of Gross Value Added according to the BBC) would flee to London.

The second problem takes us back to the European Union. As it’s likely accession negotiations with an independent Scotland would be treated like any other new member application, Scotland would be required to adopt the Euro eventually as a condition of membership.

Finally, Scottish independence would also have far-reaching ramifications in the rUK. On Tuesday I suggested that an independent Scotland would make a Labour government in the rUK less likely, and if it does happen, more fragile. This would seem to, eventually at least, give the Conservatives an opportunity to form an outright majority. The Conservatives are on record as promising a referendum on continued EU membership in 2017 for the United Kingdom. Without Scotland, the rUK becomes even more Eurosceptic, thus increasing the probability of a “British” exit from the EU.  Again, The Guardian:

If Salmond wins his vote and Cameron wins his for a second term next year, the bizarre situation may arise where a new country called Scotland is clamouring to be let in to the EU after having forfeited 41 years of membership at the same time as a shrunken UK is heading for the EU door marked Brexit.




Page 1 of 1612345...10...Last »