Subscribe via RSS Feed

Search Results for 'pollin'

Social Desirability and Response Validity in Current Polling

[ 22 ] September 28, 2012 |

Several days ago in these very pages, discussion ensued regarding the latest conservative attempt to rewrite reality through re-weighting polls to one guy’s liking.  Of course, polling is not an exact science, but it is a science, and the latest wingnut delusion has no grounding in theory or empirical evidence.  Like any science, survey research continuously attempts to improve upon the validity and reliability of its measures and findings.  While I’m not at all concerned about some nefarious (and successful) attempt by the MSM and that paragon of power, the Democratic Party, to turn otherwise professional and reputable polling houses into duplicitous shills.

However, I have been somewhat interested (note, not concerned) if there might be something else going on that causes the polls to over estimate support for Obama.  Social desirability bias is something I’ve published on in the past (direct link to the paper here).  While that article suggests a contextual effect that causes variance in social desirability across countries (regarding accurately reported turnout in survey research), relevant here is what is colloquially known as the Bradley Effect.  It’s possible (though I consider it unlikely in the specific context of the 2012 Presidential election) that this helps explain Obama’s consistent polling advantage in an election where many if not most structural conditions suggest an incumbent defeat.

It’s difficult with the data I have available to examine this hypothesis to any satisfaction, but that’s not going to stop me from trying.

To begin with, we have the current state of the polls.

Obama Romney Advantage
RCP 28 Sep. 48.6 44.6 4.0
538 28 Sep. 52.2 46.5 5.7
538 6 Nov. 51.5 47.4 4.1

 

RCP’s running average has Obama up 4 points, Nate Silver’s “nowcast” model up 5.7, and his current prediction for election day 4.1 points.

If social desirability is at work here, a poll respondent will state that she or he supports the President because internally, our not entirely sincere respondent is seeking the socially desirable response, and not supporting the black guy might be racist.  However, this is done knowing that they will ultimately support the white guy.  Practically, this would mean that Obama’s support in these polls is inflated.

I’m approaching this from several directions.  First, I’ve averaged the final month of polls for Presidential elections going back to 1976 (an arbitrary cut off) to examine how accurate the polls were in predicting the final outcome between two white men, with 2008 to serve as a benchmark for 2012.  Shift represents how wide of the mark the final polls were, to the benefit or detriment of the incumbent party.

Poll Result Shift
2008 D 7.6 D 7.3 0.3
2004 R 1.5 R 2.4 0.9
2000 R 3.0 D 0.5 3.5
1996 D 11.0 D 8.0 -3.0
1992 D 12.0 D 6.0 -6.0
1988 R 12.0 R 7.0 -5.0
1984 R 18.5 R 18.0 -0.5
1980 R 4.0 R +10.0 -6.0
1976 D 2.0 D 2.0 0.0

 

Social desirability response bias in an election can take on many forms, not just race.  To wit, the 1992 general election in the United Kingdom is a good case study (one I lectured on here at Plymouth about six or seven years ago, shame I have no clue where those lecture notes now reside) as the polls largely predicted a narrow Labour outright victory or a hung parliament with Labour having the plurality of seats, yet the Conservatives under John Major easily won by 7.5%.  This is called the Shy Tory Factor on this island, which is simply another manifestation of social desirability.  But in 2012, I’m primarily considering race, and comparing 2008 to past elections does not support the hypothesis that this might be a problem for Obama’s numbers in 2012.

I also considered several of the primary elections in January, 2008.  This was the beginning of a primary where Obama was a somewhat unknown junior senator only four years into his Congressional career, going up against the assumed nominee.  For these data, I average all polls from the last week of the campaign in the given state (there were 20 in New Hampshire alone).

Poll Result Shift
NH Obama 35 37 2
NH Clinton 30 39 9
NV Obama 33.25 45 11.75
NV Clinton 37 51 14
SC Obama 41 55 14
SC Clinton 26 27 1

 

This evidence is more ambiguous than the examination of previous general elections.  Both Clinton and Obama received shifts in their favor, which isn’t surprising considering the undecideds presumably made a decision of some sort once voting.  However, in both New Hampshire and Nevada, the shift was stronger towards Clinton than Obama: a 7 and 2.25 point advantage respectively.  Both are dwarfed by Obama’s advantage in South Carolina.

These are the wrong data to be analysing this with, of course; ideally we’d have individual level data.  While not individual level data, the following figure, by Greenwald and Albertson, offers a more holistic view of the 2008 primaries.

The above shows that among 32 states where data were available, the “Bradley effect” was only evident in three states, yet 12 states demonstrated what has been termed (erroneously, in my opinion) the “reverse Bradley effect”: states where Obama’s support in the primaries was under, not over, estimated (see South Carolina above).  I consider this an erroneous classification because where the theoretical explanation for the Bradley effect hinges on social desirability, the reverse has been hypothesized as a function of systematic sample bias, through either the under-representation of African Americans in polling samples, or the cell-phone effect.  However, some have hypothesized that “black voters might have been reluctant to declare to pollsters their support for Obama”, and the link above does discuss that

After the Super Tuesday elections of February 5, 2008, political science researchers from the University of Washington found trends suggesting the possibility that with regard to Obama, the effect’s presence or absence may be dependent on the percentage of the electorate that is black. The researchers noted that to that point in the election season, opinion polls taken just prior to an election tended to overestimate Obama in states with a black population below eight percent, to track him within the polls’ margins of error in states with a black population between ten and twenty percent, and to underestimate him in states with a black population exceeding twenty-five percent. The first finding suggested the possibility of the Bradley effect, while the last finding suggested the possibility of a “reverse” Bradley effect in which black voters might have been reluctant to declare to pollsters their support for Obama or are under polled.

There are numerous possible explanations for the “reverse” effect, including faulty likely voter models, under sampling of blacks, sampling bias due to cell phones, to name a few.  There might be some sort of contextual effect at work here, but to ascribe it to behavioural motivations (rather than factors exogenous to the individual, such as sampling bias) such as blacks being shy to state their support for Obama fails the face validity test to me.

Ultimately, given the wide array of mediocre data presented here, I am not concerned about social desirability biasing the estimates of support for Obama in any significant, substantive manner.  However, much as I’d like to, I wouldn’t say that the Republican conspiracy is more likely, if only because that is so creatively ludicrous I initially thought it was an Onion piece.

Push Polling!

[ 0 ] March 20, 2010 |

Ten minutes ago, the wife received a call Family Research Council. The first question: “Do you have a favorable opinion of President Obama’s government run health care proposal?”

After she answered “Yes,” there were no further questions. I suspect that there would have been more than a few additional queries if the answer had been “No.”

UPDATE: IPE@UNC marshaled the mighty power of empirical science to determine the truth of my last statement:

They called the wife’s phone about 10 minutes later and she let me answer. This time when asked if I supported the Marxist take over of everything great about America I said “No”. Were there follow-ups? Why….. yes! Of course there were. The next question was some variant of “Do you support abortion?” I said “No” and then the robot lit into a 30 second rant about how Obama pledged to Planned Parenthood during the campaign that he’d do everything in his power to eliminate all restrictions on abortion (which I think is untrue, but maybe not) and strongly implied, without directly saying so, that passage of the health care bill would lead to just that outcome. Then the robot asked if I’d be willing to contact my congressman about this issue and I said “Yes”. The robot then helpfully gave me my representatives’ information.

More on British Polling and Margins of Error

[ 0 ] March 2, 2010 |

I promise that this will not be a daily habit of mine, but the Tory +2% lead poll did generate the predictable breathless excitement on these islands.

YouGov’s daily tracker indeed moved back towards +6, as anticipated, at Conservative +7. Additionally, a ComRes poll was released yesterday for The Independent which places the Tory lead at +5 (37/32/19). I want to emphasize that the +2 poll released on Sunday, while at the fringes assuming a true value of +6, is still within the margin of error. It was not an outlier in a pedantic understanding of the word, which is a case three standard deviations removed from the mean.
In other words, the findings reported in the Sunday poll were not “wrong”. 95% of samples will yield the true value within the error band of +/- 3%, as this was. This is why we have margins of error in the first place.

What we should focus on is not the point estimate itself, but the trend – and the consistent trend is clearly away from the Tories and towards Labour.

Speaking of which, the +5% Tory lead reflected in The Independent’s poll still yields a distribution of seats as Labour 287, Conservative 272, Liberal Democrat 59 — assuming a uniform national swing.

UPDATE ( 3/3/10): Again as expected, Wednesday’s YouGov tracker is rather consistent, at Conservative +5 (C 38 L 33 LD 16). Given the last seven to ten days of polling, this suggests a true value of support around + 6%, and if not precisely 6 (and it isn’t) it’s slightly below 6%.

Preliminary Colorado exit polling data

[ 0 ] November 4, 2008 |

This is weighted by affiliation but doesn’t attempt to incorporate early voting:

Obama 52

McCain 47

Senate Polling

[ 0 ] October 31, 2006 |

Some new CNN data. It still looks like 50-49-1 to me, with Tennessee looking like a write-off. The one ray of hope is Virginia–the polls are still even, and given the D.C. suburbs I can see Virginia going Democratic in a toss up race the way I can’t see Tennessee. On the other hand, MO is also still a toss-up, although I’m inclined to think the Dems will take it.

Polling Data

[ 0 ] September 17, 2004 |

Via Kos, John Zogby makes a compelling argument that telephone polling, as we know it, is obsolete:

Zogby points out that you don’t know in which area code the cell phone user lives. Nor do you know what they do. Beyond that, you miss younger people who live on cell phones. If you do a political poll on land-line phones, you miss those from 18 to 25, and there are figures all over the place that show there are 40 million between the ages of 18 and 29, one in five eligible voters.

This is pretty interesting, and does not admit an obvious solution. Zogby had moved to internet polling, which may or may not give a more accurate account. We’ll have to wait for November to clear some of this up. In any case, the huge spread of current polls is probably due to something more than the difference in likely voter methodology.

I’m not current with the legalities of polling via cell phone, although I have noted that I tend to receive very few unsolicited calls on my cell (very few solicited ones, either; I’m one hell of a loser). Myself and my roommate have gone the cellular/cable modem route, and no longer have a landline at all, meaning that we are apparently politically invisible. How will the invisibile people sort themselves out in November? Well, we’re young, which may help the Dems, and we’re probably a touch more affluent than normal (correcting for the fact that we’re young) which favors the Republicans.

The GOP Will Not Have An Alternative to the ACA. Do They Need To?

[ 73 ] January 31, 2015 |

With the Supreme Court quite likely to willfully misread the ACA and destroy the insurance markets in a majority of states, the GOP is again pretending that it will at some point have some alternative policy.  Ed Kilgore (correctly, in my view) thinks there’s virtually no chance of that happening, especially as the fight for the 2016 nomination heats up:

No one at this point in the GOP is addressing how they deal with the ecstatic reaction of their party’s conservative activist base if and when the news blares out on Fox that SCOTUS has landed a lethal spear in the hide of the Great White Whale. Just yesterday polling data came out showing Republican rank-and-file opposed the idea of Congress doing anything to “repair” Obamacare. Ya think maybe the already difficult process of agreeing on a “fix” might be complicated a bit more by the shrieks of “NO! NO! NO!” from every Republican who has been told again and again that the Affordable Care Act is the worst thing to happen to America in living memory? Is it possible a Republican presidential candidate or three would exploit the situation by starting a crusade to destroy any GOP member of Congress who even thinks about “fixing” Obamacare?

The fact that congressional Republicans are highly unlikely to even be able to pretend to have an alternative to the ACA may make the Roberts and/or Kennedy marginally less likely to embrace pure lawlessness in support of Republican policy goals. But should the Court reverse King, will the lack of an alternative hurt the GOP? I don’t really think so. Obama, not congressional Republicans, is likely to take the brunt of the political hit if the Court wrecks the markets. Voters who don’t follow policy details are going to tend to blame the president for bad things that happen, irrespective of who’s actually responsible.

For the same reason, I don’t agree with the arguments I’ve seen in some quarters that the Supreme Court upholding King while using Chevron deference would be barely better than the Court wrecking the exchanges immediately, because a future Republican president would remain free to wreck the markets. I don’t agree. I’m not at all sure that a Republican president would do that unilaterally — the GOP can largely escape political retribution for raising taxes taking insurance away from 10 million people if the Court does it, but not if a Republican president does it. President Walker might do it anyway — but if it’s going to happen, it’s still better that a Republican White House takes appropriate responsibility for it.

Warren, public opinion, and inequality

[ 38 ] January 18, 2015 |

Paul Rosenberg has a good response to Amitai Etzioni’s rather lame attempt at a hatchet-job on Elizabeth Warren at The Atlantic:

Taking Norton and Ariely’s results seriously, we can say that the American people want a much fairer society than they live in, but that the means for articulating this desire—the stories, concepts, policy proposals, etc.—are in scandalously short supply, a de facto example of hypocognition thwarting what people want. Elizabeth Warren is particularly popular precisely because she provides some of the missing means that people are so hungry for—an antidote to the hypocognition that thwarts their desire for a fairer, more just vision of America, which respects both their hard work and their compassionate values. There may be relatively little polling to support this view (though there’s considerably more than you’d expect) but that’s partly just another example of how elites dominate the landscape of acceptable thought to protect their interests, as underscored by recent research by Martin Gilens and Benjamin Page. Warren represents a clear alternative to this narrow-minded view. Her popularity derives in large part from her ability to shape narratives that reflect the hidden majority’s shared values and articulate them in policy terms, reversing a decades-long trend by which elites of both parties have turned their backs on the welfare of ordinary Americans.

While Rosenberg offers a much more accurate portrait of American public opinion than does Etzioni, there are reasons to think this offers an overly optimistic account. He’s right, of course, that Americans want a more egalitarian distribution of wealth and income than they’ve got. But it’s almost certainly the case that partisan identity is likely to significantly diminish the ‘hidden majority’ support for redistribution when it turns into an actual plan promoted by and associated with Democratic politicians (as the continued unpopularity of something called “Obamacare” demonstrates). Raising the minimum wage manages to remain broadly popular despite the partisan divide, so it’s important not to be too fatalistic about this. (I suspect one reason for this is the simplicity of the policy; it’s harder to spin or dissemble the basic fairness of it away.) But the lack of specific policy proposals cuts both ways–lots of inequality-reducing proposals could be quite popular in the abstract, but once they become “Democratic” proposals support is likely to conform to a more familiar partisan pattern.

Is Climate Change the Next Gay Marriage?

[ 39 ] November 19, 2014 |

This piece argues that climate change could be the next gay marriage in terms of young Republicans supporting meaningful action on it while old Republicans hate the idea of even considered it. Well, maybe. But I think there are some problems here. Primarily, I’m not sure how this manifests itself in policy. Gay marriage has a simple solution: making gay marriage legal. But climate change is far more complicated with no clear situation. Polling showing young people would pay $20 extra a month in home heating have some value but let’s be clear, that ain’t solving climate change. So what happens then? And the mechanism for change is much murkier. Ballot measures mean people can vote to legalize gay marriage. Lawsuits can force states to do the same. There isn’t really a similar mechanism for climate change. Weaning us off coal is great, but it doesn’t solve the problem either. So I’m glad to see young Republicans reasonable on this issue, but it’s not gay marriage.

Friday Links

[ 63 ] October 24, 2014 |

Those Who Fail To Learn From the Results of Nominating Shitty Candidates Are Doomed To Keep Losing

[ 76 ] October 22, 2014 |

The Titantic, the Hindenberg, the 2007 New York Mets, Martha Coakley:

National Democrats are haunted by memories of Martha Coakley’s unforced stumbles and missteps in 2010, which cost them a U.S. Senate seat in one of the country’s bluest states.

Four laters later, the Massachusetts attorney general might be about to blow another major contest: The race to succeed Deval Patrick as governor.

With two weeks left to go, a new poll by WBUR, which tracks the race weekly, found Coakley trailing for the first time against Republican Charlie Baker, a former health care CEO who served as secretary of finance and health under Gov. William Weld in the 1990s.

It’s still a close contest: Baker has 43 percent while Coakley has 42 percent, well inside the poll’s 4.4 percent margin of error.

But the troubling sign for Coakley is that Baker appears to be gaining steam down the stretch after consistently trailing throughout the campaign.

“It’s one of several polls which over the last week or so have shown a movement toward Baker,” Steve Koczela, the president of MassINC Polling Group, which conducts the polls, said. “Coakley has essentially been treading water while Baker’s been climbing.”

Coakley’s late drop-off seems eerily reminiscent of the 2010 special election against upstart Republican candidate Scott Brown, when the Democrat blew a huge lead, fell behind in the final stretch, and went on to lose.

Hopefully she’ll pull it out anyway, but it’s ridiculous that the Massachusetts bench is so shallow that someone who ran one of the worst campaigns in known human history — a bad campaign with very substantial consequences for the country, yet — could get nominated for a competitive race again.

Another Winner From Late-Period Posner

[ 46 ] October 16, 2014 |

Judge Posner’s Voter ID opinion — which, alas, came in the form of a dissent from the denial of an en banc hearing — is indeed a beauty. My favorite graf:

Voter-impersonation fraud may be a subset of “Misinformation.” If so, it is by all accounts a tiny subset, a tiny problem, and a mere fig leaf for efforts to disenfranchise voters likely to vote for the political party that does not control the state government. Those of us who live in Illinois are familiar with a variety of voting frauds, and no one would deny the propriety of the law’s trying to stamp out such frauds. The one form of voter fraud known to be too rare to justify limiting voters’ ability to vote by requiring them to present a photo ID at the polling place is in-person voter impersonation.

Or maybe it’s this one:

As there is no evidence that voter impersonation fraud is a problem, how can the fact that a legislature says it’s a problem turn it into one? If the Wisconsin legislature says witches are a problem, shall Wisconsin courts be permitted to conduct witch trials? If the Supreme Court once thought that requiring photo identification increases public confidence in elections, and experience and academic study since shows that the Court was mistaken, do we do a favor to the Court-do we increase public confidence in elections-by making the mistake a premise of our decision? Pressed to its logical extreme the panel’s interpretation of and deference to legislative facts would require upholding a photo ID voter law even if it were uncontested that the law eliminated no fraud but did depress turnout significantly.

You also have to love the appendix titled “Scrounging For Your Birth Certificate in Wisconsin” (a direct shot at Easterbrook’s embarrassing opinion.) The dissent also does a good job distinguishing the case from Crawford, although Posner and the Supreme Court were wrong then too.

I really hope that Ponsner ends up hearing a case brought by the ACA troofers; that could possibly result in the most entertaining opinion in judicial history.

Page 1 of 1612345...10...Last »