Home / General / When markets become too confident in markets

When markets become too confident in markets



Andrew Gelman and David Rothschild have a really interesting piece on some paradoxes that may be arising in the workings of political prediction markets:

Prediction markets took a lot of heat for their prediction on Brexit (holding steady with a predicted probability 25 percent during the week leading up to the vote while polls did not flip from “Leave” to “Remain” until the final day). Probabilistic predictions can and will “fail, ” but it is good to understand why this is happening. Prediction markets have a strong track record and people trust them. And that actually may be the problem right now . . .

Prediction markets have developed an odd sort of problem. There seems to be a feedback mechanism now whereby the betting-market odds reify themselves.

What do we mean by that? In the case of Brexit, it goes like this: Different surveys give different results, and we all know not to trust the polls, which have notoriously failed in various British elections such as the 2014 Scottish Referendum and the 2015 British parliamentary elections. But we do watch the prediction markets, which all sorts of experts have assured us capture the wisdom of crowds.

So, serious people who care about the election watch the prediction markets. The markets say 25 percent for “Leave.” Then there’s other information, the latest poll, and so forth. How to think about this information? Informed people look to the markets. What do the markets say? Twenty-five percent. OK, then that is the probability.

This is not an airtight argument or a closed loop. Of course, real information does intrude upon this picture. But something is amiss when prediction markets stay stable for too long.

In the past, traders followed the polls too closely and sent the prediction markets up and down. But now the opposite is happening. Traders are treating market odds as correct probabilities and not updating enough based on outside information. Belief in the correctness of prediction markets causes them to be too stable. . .

The implication for bettors is that when the odds seem too stable, trade on the news. Don’t assume that the markets have already incorporated all openly available information. Right now the prediction market prices for the presidential election have been incredibly stable for nearly a month, despite waves of news on business scandals, fundraising money, poll swings, and possible indictments. A mature prediction market should show measured, but real, swings in prices in response to news.

Long term, or even medium term, this should right itself; as investors become aware of this bias (in part because of this article!), it should diminish or disappear. But right now, it appears that prediction markets have arrived at a paradoxical place: Their reliability, the very source of their prestige, is causing them to fail.

Read the whole thing.

  • Facebook
  • Twitter
  • Google+
  • Linkedin
  • Pinterest
  • jim, some guy in iowa

    I have always been kind of bemused that people took those things seriously. I just figure it’s another ritual of market-god worship

    • ThrottleJockey


      Just look at the Dow…markets are never wrong my ass….

      • Warren Terra

        RE the Dow, I’m consistently amazed at the breathless reporting of the day’s fluctuation in the Dow, as if (1) the Dow 30 were an adequate representation of the economy; (2) the Dow accurately reflected the state even of those 30 companies to within a percent or two; and (3) it did so on a day-to-day timescale.

        • (((Hogan)))

          I’m even more amazed that they claim to know exactly what caused that day’s fluctuation (news from China, ExxonMobil’s earning report, new jobs numbers, whatever). It’s like evo psych for business majors.

        • muddy

          My mother did not like the stock market (gambling) and she would announce to my dad in thrilling tones when the Dow was down 20 points or whatever, proving she was correct. She never noticed on the up days somehow. And as we see the number has risen tremendously over time, there are obvs more ups than downs!

          I tried to explain to her about the 30, how you might not even own a one of those. I said the Dow is similar to giving the daily temperature average of 30 cities across the globe. It won’t tell you if you need a sweater today. But the next day, “Oooh, look at the Dow, it lost 5 points, dear!” argh

        • CD

          Well, it’s 100% accurate as an index of the share prices of those firms, *precisely* on a day to day scale. That’s the point of it, no? Whether anything else should be inferred from that figure or its fluctuations is a different question.

    • Warren Terra

      MY favorite was the post-9/11 DARPA fantasy that if you collected the impressions of a bunch of American investors who individually knew nothing worthwhile about the future of the middle-east, a collective wisdom would emerge.

      • JustRuss

        Yeah,that never made any sense to me. At the time, I couldn’t decide if I was the idiot or the people pushing for a Middle-Eastern conflict “futures market”. Pretty sure I know the answer now.

        • Warren Terra

          To be generous, it was unfairly portrayed in the media as being a predictions market for terrorist events, which would have been beyond absurd. It is perhaps not theoretically impossible to predict from afar social movements that topple repressive autocracies in the same way you can’t hope to remotely anticipate the actions of a dozen madmen without actually knowing about them as individuals and spying on them.

          Still: the reality was plenty ridiculous enough.

          • xq

            Why? Prediction is important, prediction markets have had at least some success at it, and it was proposed as a research idea, not as the only method analysts would use to make decisions. Seems like something worth researching to me.

      • Just_Dropping_By

        The DARPA prediction market wasn’t going to be limited to “American investors” as far as I can recall, but nice straw man. (If the proposal had a major design flaw, it was that contracts were going to be limited to no more than $100 so as to avoid accusations of people “profiting off of terrorism,” but putting such a low cap on contract values basically guarantees that no one with serious inside information would want to participate. Most people who participate in stock markets do so because they expect to make money, not because they especially enjoy the abstract game aspect of it.)

  • Schadenboner

    Goddamn these last 8 years have been rough on President Obama’s face.

    • ThrottleJockey

      Not as rough as on George W. The only recent president to look better finishing up his term was Reagan, primarily because he was so old to begin with.

      • JonH

        Judging by the photos from the Dallas memorial service, W has taken to dying his hair.

      • so-in-so

        Plus he got LOTS of rest. And really didn’t care about much of anything that went on, even if his handlers told him about it.

        • The Dark God of Time

          He probably wore sunscreen when he was gallivanting clearing brush on his so-called “ranch”.

    • spearmint66

      Almost everyone looks worse at 55 than at 47. Never been a big believer in the theory that the presidency is especially aging.

    • Turkle

      I mean, they have also been rough on my face, and I’m not even the POTUS.

  • NonyNony

    Why should I, as a citizen, care about prediction markets for politics?

    I don’t mean that snarkily. If I’m not a gambler is there some reason I should worry about the fact that prediction markets are not the godsend predictors that they were claimed to be?

    Also I have to say – as someone who thinks that there’s a fine line between “wisdom of crowds” and “mob rule” anything that deflates the “wisdom of crowds” bullshit I’ve been exposed to for the past 20 years is excellent.

    • Vance Maverick

      I believe your third paragraph is the answer to the questions in the first two.

      • NonyNony

        I mean, yes it answers the question of why I should care in that they provide evidence that the “wisdom of crowds” bullshit is bullshit.

        But what I mean is – suppose they didn’t have this problem. So what? Why should I as a non-gambler care what a prediction market thinks about an election? Other than as a poll of the gut feelings of a whole lot of gamblers willing to place bets on elections?

        It’s like – as a non-gambler I don’t care what odds the bookies are giving the Indians when they play the Yankees. So is there a case that says that I should, as a non-gambler, care about the odds that the bookies are giving for a Clinton victory over Trump in the general election? Either as a “good citizen” or in the interest of political science research or something?

        (It really feels like people use betting markets as a substitute for doing their own research. “Oh those guys are putting money on the line so they must be doing the research that I’m too lazy to do so I’ll trust the way the odds are flowing.” Which strikes me as very problematic as most gamblers are doing what this piece suggests – using the odds themselves as a proxy for the true probabilities of success. Which is why most sports gambling is a lucrative operation for bookies.)

        • politicalfootball

          As a non-gambler, I found it vexing that the Yankees took three out of four despite not having been favored in any game.

          You may regard gambling odds as being unrepresentative of reality, but you set up the terms of your comment this way: “suppose they didn’t have this problem.”

          Well, if they didn’t have this problem, people are sensibly very interested in the probabilities of certain events – especially really important stuff, such as Indians games or presidential elections.

  • junker

    I’m having a hard time reading the endpoint of the graphic. Does this mean that the last odds right before Obama was nominated had him at a 77% chance to be nominated? Even after Clinton conceded six weeks earlier?

    • dmsilev

      The screenshot looks like was taken maybe 1/4th of the way through 2008, so around March or so. Obama being a 3:1 favorite at that point is probably reasonable.

  • Jon_H11

    I see some value in prediction markets. Politics resembles the classic Keynesian beauty contest, there is a non-trivial proportion of voters who vote for who they think will win, not necessarily who the want to win (these tend to be “undecideds” and not have strong political views or opinions), I think that prediction markets are better able to predict these voters’ behaviors than traditional polling (where undecided just means undecided and doesn’t imply any distribution of votes). William James’s notion of “cash-value” of a belief is useful here.

    I mean, my father keeps lamenting that he thinks Trump will win, but when I offer to bet him 4:1 odds that Hillary gets more than 300 electoral college votes (I pay him 4 dollars if she doesn’t, he pays 1 if she does), he backs off, even at small stakes. So sometimes willingness to risk real assets can be a better benchmark for beliefs than their mere statement. People are more willing, when money is on the table, to discount information like a new poll that confirms there prejudices and take polls which aren’t favorable under more consideration than they are when its just a matter of stating opinions.

  • politicalfootball

    I’m not terribly impressed with the argument from Gelman and Rothschild.

    The fact that the betting markets didn’t track the polls on Brexit is, in fact, what prediction markets are supposed to do – they are supposed to take everything into account; they aren’t just summaries of polls. Trump’s surprise-of-the-day has become, paradoxically, entirely predictable, and it is reasonable to suppose that tomorrow’s outrage is already factored into today’s odds.

    Betting markets track the conventional wisdom, which is generally pretty stable. (I’d argue that the key value of betting markets is that they provide a quick reference for people who want to know what the conventional wisdom is.)

    You’ll note that Gelman and Rothschild don’t put their money where their mouths are. They don’t identify a single market that they think is unbalanced today. Rather, they identify inefficient markets in the way that they are most often identified: With the benefit of hindsight.

    • Aexia

      You’re right, political markets are nothing more than windows into what the conventional wisdom currently is. Sometimes the conventional wisdom is right and a lot of time the conventional wisdom blows it.

      They’re been around a long time (check out IEM for older stuff). In 2004, they had Dean winning the Dem nomination right up until Iowa caucus night. In 2008, it seesawed between Clinton and Obama. There’s no special insight, just crowds chasing whatever is hot that moment. Knowing what they’re chasing is useful information, but it’s not predictive of the final outcome.

      Brexit is particularly galling because Remain was trading at like 70-80 even a couple hours after polls closed.

      • Phil Perspective

        In 2004, they had Dean winning the Dem nomination right up until Iowa caucus night.

        Don’t forget they had Jeb! or Empty suit Rubio winning the GOP nomination even after Trump won the first primary or two. I don’t think they changed until Ferret-head won South Carolina. Think about that. The bettors chose not to believe all those polls before the primaries even started that had Ferret-head away ahead.

      • xq

        Sometimes the conventional wisdom is right and a lot of time the conventional wisdom blows it.

        OK, but this could be said about any prediction method. I think the only relevant comparison is to other predictors. Is there a predictor that does reliably better than the prediction markets?

        • Aexia

          Survey research but doing it correctly is hard and expensive.

          The data peeps on Obama’s re-elect did a lot of it and while the prediction markets and public polls were bouncing around, their predictions remained steady. That’s why they never panicked. That’s why they played in (and won) Florida while everyone else was worrying about PA and MI.

          Most “movement” that press breathlessly reports is just statistical noise. Presidential races aren’t particularly dynamic at the national level.

          • jim, some guy in iowa

            do you know (or have the sense) that the Clinton campaign is doing the same kind of research the Obama team did?

            • CD

              They are. Lot of the same people.

          • xq

            Yes, you are right. You can definitely do a lot better by doing heavier sampling (and it’s not just a pure sample size issue, that also allows you to build better models). But that’s kind of cheating. The interesting scientific question here, I think, is how well different predictors perform with access to the same public information.

  • The moral of the story is to not use a single data point to form an opinion. These markets are merely a single data point, not the complete picture.

    • The markets aren’t a single data point (nor is any individual market). They are (decentralised, bottom up) aggregations. Now, sometimes, we can regard such as “a” data point (e.g., as we do polls). But in both cases, we really are dealing with data informed *models*.

      The reason I’m being picky on this is…er…I guess just because :) Sorry!

      • Yet, from a decision standpoint, if this is all you are looking at, then it is essentially a single data point. It should be vetted against other sources to get a more rounded understanding :-)

        • :)

          Well, multiple models should be vetted against other sources, always.

          As I said, I was just feeling a bit randomly picky.

      • The Dark God of Time

        The correct term is datum, not data point, IMHO. Treating a conglomeration as a single entity is a capital mistake.

        • Yankee

          Garbage in, garbage out.

      • Yankee

        Since the markets embody feedback loops, they can’t be regarded as simple aggregates.

        “Models” are even less aggregates. So I don’t get what you’re being picky about? The point is you ought to be making a scatter plot of all these different results, represented as “points”, right??

        • kenjob

          an autocorrelated aggregate is still an aggregate.

          i think Bijan’s point is that prediction markets and e.g. polls are models of voting behavior. the process of validating a model is not precisely the same task as the process of sampling from a population.

  • xq

    I don’t know. What changed in the last month that should lead one to revise one’s views on the chances of Clinton vs. Trump? The poll gap has remained relatively stable. They mark “Brexit vote” and “Clinton not indicted” on their graph, but I agree with the markets that the first shouldn’t have any effect and the effect of the latter should be small (because there was little probability Clinton would have been indicted in the first place).

    • Schadenboner

      Also, strictly speaking, “Clinton not indicted” is a (current and in all probability for the foreseeable future) condition, not an event

  • so-in-so

    Also noting that a 25% chance of “Leave” (or Trump) is clearly entirely possible to end up at Leave (or Trump). Unlikely != impossible.

  • JCougar

    That’s very interesting. I wonder if the emergence of sites like FiveThirtyEight have shifted people’s mental framework about stuff like this by suggesting that aberrant polls are aberrations–and not legit deviations based on changed factual circumstances. Obviously, a poll that appears aberrant can be one or the other. Sites like FiveThirtyEight tend to go with the narrative of things are more stable, because statistical aggregation by definition relies overwhelmingly on past information. Therefore, it’s always going to lag behind in predicting change.

    There’s really no logical shortcut to be had in predicting complex stuff like the outcome of elections that relies on social behavior. If you are deadset on guarding yourself from making a Type II error, you’re just making it all the more easy to make a Type I error, and vice versa.

    • njorl

      A system of weighted polls informing a model of 51 elections for electoral votes has been proven to be much more accurate than the “conventional wisdom” of political pundits. That electoral prediction markets should mirror 538 is no accident. 538 had the odds at 77/22 for about 2 weeks, with a 5 point jump for Trump today. I wonder if the markets will track that jump. If they do, I expect it will be a lagged response.

      • JCougar

        That’s not what I’m saying, though. I’m not arguing that one way isn’t more accurate than the other. I’m just arguing that one way isn’t always the most accurate way.

  • Dilan Esper

    It’s weird. Everyone uses the horse race metaphor to describe politics, but FYI, there is actually extensive literature on pari-mutuel pools in horse racing, what they do and do not signify, their reliability (and to what extent they incorporate all relevant information a la the efficient markets hypothesis), and similar issues. And yet to my knowledge nobody ever connects those particular dots to the political betting markets.

  • ConcernTroll

    Brexit had a 25% chance of passing – that’s not a black swan event or even a problem. If political events with a 75% chance happened all the time, then the betting markets would be off. We need a quantitative analysis showing actual events occurring outside of expected probabilities. This is just gibberish.

    • Dilan Esper

      I agree with this too. Brexit was like a 3 to 1 shot at the racetrack. 3 to 1 shots win all the time, and the fact that they do is not proof that the odds were an overlay.

It is main inner container footer text