Perhaps the most central problem with the kind of analysis Simkovic and McIntyre are doing is inherent to any method that uses statistical correlations to try to prove causation. The authors demonstrate that the median law degree holder in their study has a higher lifetime income than the referent group of undergraduate degree holders. As I pointed out in earlier posts, the difference they find is actually fairly small after taking into account taxes and investment costs, and there are various reasons to think these relatively small differences may disappear altogether, or generate a negative present value for law degree holders who received their degrees in the last decade and going forward.
But beyond this, a correlation between higher lifetime earnings, such as they are, and law school attendance does not prove that graduating from law school caused those higher earnings. Perhaps law school graduates have other characteristics that differentiate them from college graduates in general, which accounts for part or all of the earnings gap. For example, law school graduates could, crudely speaking, be smarter, harder working, more ambitious, and more materialistic than college graduates in general. The authors are aware of this possibility, and attempt to control for it, by matching law school graduates with otherwise similar college graduates who don’t go to law school (their method eliminates some of the earnings gap).
Consider a loose analogy: lots of social science research indicates that marriage correlates with all sorts of good outcomes: longer life expectancy, lower rates of chronic disease, higher socio-economic status, lower rates of depression, etc. But it’s far from clear the extent to which, if at all, marriage causes these outcomes, as opposed to correlating with other causal factors that are responsible for both better health, wealth etc., and higher rates of marriage among the healthy and wealthy.
It would be quite reckless, in other words, to get married in order to reap the supposed benefits caused by marriage.
In the case of law school, the fundamental question this sort of analysis faces can be clarified by stating two opposing views in their strong forms:
(1) Graduating from law school increases lifetime earnings relative to otherwise similar college graduates because legal education enhances human capital, by improving people intellectually, emotionally, and ethically, in ways that make them more valuable to employers, or to themselves as self-employers.
(2) Graduating from law school is a social sorting mechanism that enhances earning power, for reasons that have nothing to do with “enhanced human capital” (if anything law school tends to make people dumber than they were before they started), and everything to do with the pernicious effects of empty credentialism and arbitrary barriers to entry.
Now note that both views are perfectly consistent with the conclusion that law school enhances a graduate’s lifetime earnings. The issue, of course — and it is a deeply political and ethical issue — is that law school may “cause” this result in completely different ways. In the first case, law school is adding value in a defensible sense of “value;” in the second case, law school is merely extracting rents.
Finally, perhaps the single biggest weakness in the authors’ analysis is that it does not in any way stratify data in regard to different sorts of law schools. Just as whether getting married is likely to be beneficial to one’s health is far more dependent on whether one is marrying a mature, emotionally stable person who loves his or her work, or an abusive unemployed alcoholic than it is on the generic act of “getting married,” whether law school makes sense as an investment varies just as enormously, depending on the school and its cost.
Under present circumstances, the question of whether or not one should go to “law school” is especially meaningless. There are a handful of law schools at which the answer for most people admitted to them is probably yes. There are far, far more law schools where the answer for most people admitted to them is almost certainly no.
A study that mashes up the extrapolated estimated lifetime earnings of the graduates of a couple of hundred law schools, who graduated from those schools over a 50-year period, then asserts that the statistics generated by this mash up demonstrate that the decision to go to a legal fiction called “law school” will be economically rational for most prospective students going forward, is stretching the predictive power of econometric analysis well past the breaking point.