On Evals

I don’t think that these thoughts will be all that shocking to anyone familiar with the industry:

The problem is that students are terrible judges of who’s a good teacher. Because learning is not always pleasant, they end up punishing teachers who teach the most and rewarding the instructors who challenge them the least. An extensive body of research shows no correlation—or even a negative correlation—between how students do on objective learning assessments and how they score their professors. One experiment found that Harvard physics students learned more from “active learning” instruction but thought they learned more by passively listening to a lecture. Another study demonstrated that Air Force Academy students who were taught by highly rated professors tended to do worse in subsequent classes.

Evaluations are also vulnerable to just about every bias imaginable. Course-evaluation scores are correlated with students’ expected grades. Studies have found that, among other things, students score male professors higher than female ones, rate attractive teachers more highly, and reward instructors who bring in cookies. “It’s not clear what the evaluations are measuring, but in some sense they’re a better instrument for measuring gender or grade expectations than they are for measuring the instructor’s actual value added,” Philip Stark, a UC Berkeley statistics professor who has studied the efficacy of teacher evaluations, told me.

Despite their well-documented shortcomings, evaluations matter quite a bit to academics’ careers. “Having been on many promotion and tenure committees, this is one of the main ways, if not the main way, that your teaching is evaluated when you’re being evaluated for a promotion,” Daniel Willingham, a psychology professor at the University of Virginia, told me. Valen Johnson, a statistics professor and former dean at Texas A&M University, told me that evaluations are a “prominent” factor in tenure decisions.

Administrators like numbers, and teaching evals are numbers. I am fortunate to be in a position where I don’t really have to care about evals (no one cares what graduate students think about anything), and… I can’t remember the last time that I paid any serious attention to numbers (comments are often useful and are a different story altogether). But numbers gotta number, even when they have no meaningful relation to teaching effectiveness.