Who can solve the mystery of why California bar exam pass rates are falling?

The deans of California’s several hundred ABA-approved law schools (actually 21, but there are dozens of California-accredited and unaccredited ones as well, including my personal favorite, Taft law school, a strictly online operation named after America’s healthiest president, and the alma mater of Orly Taitz, the Atticus Finch of the Birther movement) are up in arms about how many of their graduates failed the state’s bar exam this past July.  The first-time pass rate for ABA graduates was 62%.  Until they started slipping three years ago, the typical first-time pass rate for ABA law school grads was generally in the mid-70s, so the latest numbers represent a substantial decline.

Erwin Chemerinsky wonders whether the California bar is secretly making the test harder, in a covert campaign to raise barriers to entry:

The State Bar needs to be much more transparent and explain this statistical drop. Is it a result of a conscious desire to limit the number of attorneys in California, a goal which should be widely condemned? I have heard many speculate that this is what is going and the State Bar would be well served to address this concern directly.

I’m going to try really hard not to be rude here.  Chemerinsky has been dean of a California law school for nine years.  His job is to get the students the school charges $45,000 to $51,000 per year in tuition to pass the California bar.  That’s literally his main job.  Not in the sportscaster sense of literally, but literally literally.  And he apparently doesn’t know anything about the California bar exam, unless he’s feigning ignorance for political purposes, which I’m pretty sure would be unconstitutional.

The California bar exam grading process is already transparent.  Anybody who is interested can look up the technical reports on how it’s done.  The exam has two main parts: the Multi-State Bar Exam (MBE), and the California portion of the exam.  The MBE is a scaled and equated test, which means that the scores of each administration are adjusted to reflect the relative difficulty of the exam.  And the grading of the California portion of the test is always curved to the MBE.  In other words, to the extent possible in this wondrous age in which we live, the variation in pass scores on the exam is a product of the ability of the people who take it to pass it, not by any variation in the exam itself.

Again, we’re not talking about knowing Allen Iverson’s lifetime free throw shooting percentage here.  We’re talking about understanding the grading of the test your students have to pass to qualify to engage in legal practice, which, I repeat, is pretty much a law school dean’s main job.

OK, deep breath.  Serenity now . . . But Erwin the Unready isn’t done yet:

But I think that the more important question is whether the bar exam is measuring the skills that show a person is likely to be a competent attorney. Having taught students preparing for the bar for over 30 years, I am very skeptical and would like to see much more careful consideration of how to devise a bar exam that measures what it is purporting to assess: basic competence to be a lawyer.

That’s a really great point.  We wouldn’t want to have too many wildly expensive and unnecessary barriers to entry now would we?

Except . . . guess what’s by far the best predictor of whether somebody is going to pass the bar?  No you don’t have to guess because I’m going to tell you: law school grades.  These are uncannily good predictors of likely bar success or failure.  At my own school, we recently did a study of every graduate over a five year period and found that law school class rank, when considered by decile, was a perfect predictor of the future risk of bar failure.

So if the bar exam isn’t measuring skills that show a person is likely to be a competent attorney, what exactly is law school — which takes three years not three days to complete, and costs hundreds of times more to attend than the bar exam itself — measuring?  Besides the ability to pass the bar exam of course.  Ironic, isn’t it? (Not sportscaster irony, irony irony).

So why are bar exam pass rates falling? David Faigman, the new dean at UC-Hastings, is particularly interested in this question, given the remarkable drop in the pass rates of the school’s graduates.

Pass rates of Hastings graduate first-time takers of the California bar exam:

2010:  80.3%

2011: 78.6%

2012: 76.4%

2013: 74.2%

2014: 68.4%

2015: 67.5%

2016: 51%

Faigman, to his credit, didn’t sugarcoat this disaster in an email to students and faculty, calling the 2016 numbers “horrific.”  The 2014 and 2015 numbers had been unacceptable, he wrote, but this latest performance “takes unacceptability beyond the pale.”  Nevertheless he lambasted the state bar examiners:

As an aside, let me express my utter incredulity with the conduct of the Committee of Bar Examiners of the State Bar of California.  The pass-rate for first-time takers of ABA accredited California law schools was 62%.  In comparison, New York’s bar-pass rate was 83%.  The California Bar is effectively saying that 38% of graduates from ABA accredited law schools are not qualified to practice law.  This is outrageous and constitutes unconscionable conduct on the part of a trade association that masquerades as a state agency.

Well . . . maybe.  There’s certainly an argument to be made that the California bar is too hard, although as far as I know Hastings wasn’t making it when 80% of their grads were passing it on the first try not long ago.  But this is a side issue.  Faigman’s primary concern, of course, is with the question of why the school’s bar passage rate has collapsed.

This is one of those questions that’s actually very easy to answer, but since the answer is economically unacceptable, more esoteric explanations must be jury-rigged one way or another:

[Faigman] wants Hastings professors to adjust their testing techniques so students can have more practice with the bar exam’s timed, closed-book format, and he suggested that additional midterm exams could provide students with valuable feedback.

“I think our basic obligation as a professional school is to ensure that our students and graduates are prepared to practice law, and that means training them to be great lawyers, which Hastings has been very good at historically, but also ensuring that they get through the entrance exam to be a lawyer,” Faigman said Tuesday. “As I’ve been very honest about and very clear about over the last few years, we as a faculty have not done as much in that regard.”

Again, this ignores the awkward fact that most law school exams are apparently similar enough to the bar exam to function as excellent proxies for it in regard to the risk of failure.  It also puts forth, or at least implies, the implausible theory that widespread and radical pedagogic changes at Hastings between 2007 and 2013 played a major role in the collapse of the bar passage rate of its grads between 2010 and 2016.

In fact the real explanation for that collapse is as plain as day: Hastings slashed its entrance requirements in the face of cratering application totals.  Here’s the median and 25th percentile LSAT score for the school’s entering classes (keep in mind that almost everyone takes the bar for the first time three years after they matriculate), along with the size of the 1L class in each year:

2007:  163/160   (401)

2008: 163/61     (426)

2009: 164/161   (469)

2010: 164/160   (383)

2011: 162/157    (414)

2012: 162/157   (317)

2013: 159/155   (333)

There’s a moderately strong correlation between LSAT scores and law school grades, and an extremely strong correlation between law school grades and bar passage rates, so . . .

BTW Hastings’ last three entering classes have pretty much the same credentials as the entering class of 2013 so I doubt that trying to get professors to change their teaching methods will lead to any changes in bar passage rates, but it will provide many opportunities for selective bureaucratic harassment, which is what could be called a silver lining from an administrative point of view.

Anyway, as bad has Hastings’ bar results were they were still better than those of seven other ABA California schools:

Western State: 42%

Southwestern: 38%

San Francisco: 36%

Golden Gate: 31%

Laverne:  31%

Thomas Jefferson: 31%

Whittier:  22%

Again these are all first-time taker rates.  (Second-time etc. takers have much lower rates naturally).

If the ABA actually adopts its proposed new standard requiring 75% of a school’s graduates who take the bar to pass it within two years of graduation (final approval could take place in February), the carnage is going to be something to behold.