Subscribe via RSS Feed

Silver, Economic Indicators, etc.

[ 27 ] June 4, 2011 |

Silver has a new article up critiquing an over-reliance on economic models in predicting Presidential elections.  While this is in general good work, I’m starting to note a trend in his narrative.  To wit:

“But I also got a few replies wondering how to reconcile these findings against the claims, made with some frequency by political scientists, that presidential elections can be forecast with pinpoint accuracy provided that you know the economic fundamentals.”

This could be passed off as being attributed to some comments, but I’ve noticed this tendency in other articles as well.  It seems to me that his discussion of political scientists and our models etc. is increasingly becoming rather straw-esque (though I might be wrong and am quite open to being told so).  As I’ve said often, I’ve always respected his work since the days of BP, so this new approach seems unworthy of him.  Speaking as a political scientist who does electoral behavior, I would never use the the phrase “pinpoint accuracy” about any of my work.  I’m quite aware of the limitations of my models and methods, and likewise convey this awareness when teaching methods.  I strongly suspect that I’m not rare in this understanding and approach.

One thing I like about Silver is that he doesn’t have an apparent axe to grind beyond evidence-based analysis.  That said, while he’s examining the extant knowledge of political science, lately he’s doing so using rather unsophisticated bivariate analyses.  I share his appreciation for parsimony in modelling, but bivariate analyses don’t really tell us much at all, especially when attempting to confirm or critique existing sophisticated multivariate models.  As he’s well aware.  (That said I like what he did in exploring retro-predictive ability of the the Hibbs “Bread and Peace” model).

If I had the time this weekend I’d do a more thorough read of his latest article, but I typically play singe dad on weekends, and my four and a half year old daughter requires near constant attention, so we’re left with the above somewhat superficial analysis.

 

Comments (27)

Trackback URL | Comments RSS Feed

  1. Alan in SF says:

    Silver has very little understanding of politics, and has a major axe to grind in favor of Obama’s policies. He’s always combined his sound statistical analyses with faith-based conclusions, and appropriate straw men who differ with his faith-based conclusions. Ask him to show his prediction that Obama’s and the Democratic Congress’s actions from 2008-2010 would result in Republicans taking over the House. His actual prediction was that the Democrats would soon revisit the ACA and make it much better than it already was.

    • Ask him to show his prediction that Obama’s and the Democratic Congress’s actions from 2008-2010 would result in Republicans taking over the House.

      I don’t understand what this is supposed to mean.

      1) Obama and the Democrats’ actions didn’t result in Republicans taking over the House; rather, the economy and the usual dropoff in young and minority voting during off-year elections did, and

      2) Silver’s almost never bases his predictions on policy and performance in office.

  2. Patrick Walz says:

    I think that his move to the NY Times has been a disaster. His columns are considerably less interesting (and less frequent, bizarrely, since he is presumably getting paid quite a bit more), and the “Mr. Obama”, “Ms. Palin”, etc references are jarring even after over a year having to write with that style. Further he seems to be extremely obsessed with NYC. I am happy for him that he loves NYC so much, but his repeated dropping of NYC facts, notes, etc, on his twitter and in the column, are getting draining.

    (Yes I am aware he writes for a ‘local’ paper but his column is clearly part of the Times’ national paper of record strategy).

    • Dave Brockington says:

      I had a line to conclude in a draft of this post that openly questioned the effect of the move to the NYT, wondering if it was editorial policy to simplify his analysis or if he was consciously doing so in order to be consumable to a broader audience.

    • Incontinentia Buttocks says:

      This.

    • mpowell says:

      I noticed this trend as well, but I thought there might be other factors. Namely that there is not that much interesting to be said about election forecasting. Silver doesn’t go more than skin deep into the politics so he is skipping the vast majority of the content there. And maybe this is appropriate given what he is doing. But the consequence is that after 2 or 3 years of analyzing the subject, how much more is there to be said? He’s got a great model for using polls to predict elections and provide confidence intervals. There’s not a lot more explaining to do.

      • Namely that there is not that much interesting to be said about election forecasting.

        There are a lot of interesting things to say about election forcasting…when there’s an election going on.

        When there’s not, not so much.

    • J.W. Hamner says:

      His columns might be less interesting, but the quality of his work has vastly improved. The 2008 model that he built his rep on was an ornately overfit piece of garbage, so I’m glad to see him favoring parsimony and interpretable parameters these days. Silver’s greatest asset is his ability to talk in plain language about fairly complicated math… and I think that is clearly in evidence in his NYT columns.

  3. shah8 says:

    I don’t know…

    People have pet theories about this sort of thing, and which they are emotionally invested in (being able to predict stuff can feel good, regardless of whether you were right, and that your thinking was right).

    People, being lazy creatures of habit, latch on to simple cause and effect, with the only difference between smart and dumb people is a continuum between crude and Ptolemaic hedging for obvious contingencies. So I see Silver as thinking he needs to be direct and pop that stuff, so what he said doesn’t get incorporated into some wierdo theory. Statistics are fundamentally about ambiguity, and humans have a fundamentally antagonistic relationship with ambiguity, and the strawmanesqe tactics might be necessary. That being said, I’m not sure why a political scientist takes that seriously what is being told to a layman audience, so long as it’s correct. That’s like professional sports gamblers mumbling about Football Outsiders, since those guys help line up the suckers.

    • elm says:

      But it’s not correct to claim that political scientists think they can make “pinpoint accurate” predictions based on economic fundamentals. It is true that political science models tend to produce more-accurate-than-an-outsider-might-expect predictions that do not include any mention of the candidates themselves or the campaigns they ran using only economic fundamentals, incumbency status, and (sometimes) whether the country is at war or not.

      But Silver seems to be misrepresenting what political scientists are saying. Maybe we’re a touchier profession than others, but I think that’s one of the reasons we care what he’s telling laymen about us. (We also care about him because a lot of his work is interesting even if targeted for a general audience.)

  4. calling all toasters says:

    I would never use the the phrase “pinpoint accuracy” about any of my work.

    I’m sure you wouldn’t, but the emerging meme (that I’ve heard on more than one network) is that no President since Roosevelt has won re-election with an unemployment rate over 7.2%. Therefore, they conclude, Obama is in trouble. I’m sure Silver is responding to this ridiculous precision.

    • elm says:

      See Dave’s previous posts on this exact issue.

      Also, note that the meme is not being offered by political scientists, whose pinpoint accuracy Silver is criticizing here.

    • brandon says:

      I think Silver may be responding to Matthew Yglesias & such, that is, second-hand political science.

      • bh says:

        This strikes me as a more sound critique. I have seen quite a administration-aligned pundits and bloggers use election economic determinism as a sort of Unified Theory of Administration Blamelessness.

        But those guys aren’t political scientists, which makes Silver’s quote bad exactly in the way Dave B said. It’s almost doubly bad, actually, since the academic they most like to cite, Ray Fair, is actually an economist. Though I don’t think Fair himself subscribes to such a simple view either.

  5. elm says:

    I typically play singe dad on weekends, and my four and a half year old daughter requires near constant attention

    I’m not a parent, but I do believe that if you are singeing your daughter regularly on weekends, she would indeed need near constant attention.

  6. This post gets an awful lot of mileage out of pointing out that the obviously-hyperbolic phrase “pinpoint accuracy” isn’t literally true.

  7. Nate Silver says:

    Hi Dave,

    I think this is a fair criticism and I have no doubt that you and many/most political scientists are well aware of these sorts of issues.

    At the same time … I don’t think it’s all that hard to find examples of credentialed political scientists overstating the power of models like these. Or perhaps more to the point, Hibbs himself. In 2000 he writes:

    “I conclude with just two sentences: A simple Bread and Peace model shows that aggregate votes for President in postwar elections were determined entirely by weighted-average growth of real disposable personal income per capita during the incumbent party’s term and the cumulative numbers of American military personnel killed-in-action as a result of U.S. interventions in the Korean and Vietnamese civil wars. No other variable, or set of variables, I have been able to find in the extensive literature on presidential voting adds value to the Bread and Peace model or significantly perturbs its coefficients.”

    Yes, there are some qualifications embedded in there (e.g. “in postwar elections”) and the statement is retrospective rather than prospective in nature.

    At the same time, both the concluding remarks and the entirety of this article are quite self-confident (I would argue overconfident) in dismissing critiques of the model. Even a relatively well-informed reader would probably not come away with an adequate understanding of its limitations.

    In another article, Hibbs seems to take the confidence intervals from his in-sample dataset at face value when applied to an out-of-sample result:

    Given the combination of subpar real income growth and the liability of American fatalities in Iraq, the expected Republican share of the major-party vote is 48%-49%. The Democrats led by Barack Obama are therefore favored to win the presidential election by a margin of 2 to 4 percentage points. But how likely is an Obama victory under these those circumstances? I can calculate Obama’s chances by computing the statistical probability that the actual election result will fall into a certain interval around the forecasted vote share. According to my calculations, Obama’s chances are quite good – The odds he will win are approximately 3 to 1.

    If you reverse-engineer the math, this implies that he believes the standard error for out-of-sample forecasts of the incumbent party’s vote share is only about 1 percentage point.

    In a third article, he replies to another researcher (Nordhaus) who (as I did) found that the model encountered substantial problems when applied to pre-1952 data, noting:

    According to the Bureau of Economic Analysis at the Department of Commerce (which publishes the official US National Income and Product Accounts), quarterly data series on disposable personal income begin in 1947. I conclude that it was not possible for Nordhaus to estimate the Bread and Peace model back to 1916 using the proper income variable.

    This is a questionable argument, in my view, given that _annual_ data on per-capital real disposible income is readily available going back quite a number of years before 1947, and that using annual rather than quarterly data would have a trivial effect on the model’s forecasts. It’s not a good excuse for ignoring this data, particularly given that the small sample sizes instead ought to have us begging and pleading for additional data points.

    Among the other political scientists and economists who have contributed to this literature, I find that Ray Fair tends to do a better job of acknowledging the limitations of his models than Hibbs, whereas someone like Alan Abramowitz is somewhere in between Fair and Hibbs.

    There is also a universe of people (e.g. bloggers and journalists) outside of academia who refer to this literature with some frequency, some of whom have a deeper understanding of it than others.

    In terms of the audience that I’m writing for, I assume it falls into a spectrum of about four different categories:

    1. Practitioners (e.g. academics, journalists, bloggers, and maybe even people directly involved in politics)
    2. Regular readers of 538.
    3. Regular readers of political blogs who read 538 occasionally.
    4. People coming in from the NYT front page who are likely to have only a passing familiarity with 538.

    Maybe this is an impossible task or I’m not doing a good job of it — but I try to keep all of these audiences in mind when writing articles.

    Certainly, I’m more aware of Group 4 than I was before 538 migrated to NYT and that might change the tone of the articles. At the same time, I’m also trying to make sure that Groups 1 and 2 still have plenty to chew on. Sometimes, this results in (overly) long articles that might have a snappy lead and headline but which I hope are reasonably sophisticated once you get down into the article body.

    Thank you for linking to the article and for the good debate here.

    -Nate

Leave a Reply

You must be logged in to post a comment.

  • Switch to our mobile site