Home / General / Standardized Tests are a Horrible Way to Evaluate Teachers

Standardized Tests are a Horrible Way to Evaluate Teachers


Of all the facets of Rheeism, the one that always made the least sense to me is evaluating teachers based upon the standardized tests of students. Even if you believe that the teacher union busters sincerely want to improve education for children, all that basing employment on standardized test scores means is that any teacher who can get out of teaching poor children will get out of teaching poor children. Why risk your job teaching poor kids when you can teach middle class or wealthy kids who you know will do fine on standardized tests? But try telling that to the Rheeists. Luckily, there is finally some real pushback on this ridiculousness. Rachel Cohen:

The Houston teachers union scored a legal victory in May when a federal judge found that the Houston school district’s system of evaluating teachers violates due process rights. The lawsuit centered on the system’s use of value-added modeling (VAM), a controversial statistical method aimed at isolating a teacher’s effectiveness based on their students’ standardized test scores.

United States Magistrate Judge Stephen Smith concluded that the metric’s impenetrability rendered it unconstitutional. Because, he wrote, teachers have “no meaningful way to ensure” that their value-added ratings are accurate, they are “subject to mistaken deprivation of constitutionally protected property interests in their jobs.” More specifically, he continued, because the school district denies its teachers access to the computer algorithms and data that form the basis of each teacher’s VAM score, it “flunks the minimum procedural due process standard of providing the reason for termination ‘in sufficient detail to enable [the teacher] to show any error that may exist.’”

It’s unclear whether the Houston school district will now negotiate a settlement with the teachers union or end up back in court, but either way, the decision comes at a significant time for the test-based accountability movement, which has faced a number of legal and political challenges over the past several years. The outcomes of the court battles have so far been a mixed bag: Teachers challenging VAM have scored some wins, lost other big cases, and a few major suits are still pending. Outside the courtroom, states have begun implementing the new federal education law—the Every Student Succeeds Act—which imposes far less pressure on the states to use VAM or similar measures than what they faced during the Obama administration.
Donald Trump’s education secretary Betsy DeVos has also signaled she’s less interested in using test scores to define school performance.

Donald Trump’s education secretary Betsy DeVos has also signaled she’s less interested in using test scores to define school performance. (“I’m not a numbers person in the same way you are,” she said in March, in response to a question about measuring school success. “But to me, the policies around empowering parents and moving decision-making to the hands of parents on behalf of children is really the direction we need to go.”) Considering all this, some experts have gone so far as to say that regardless of what ends up happening in the judicial system, the political momentum for using test-based accountability measures is all but over.

Never forget that Barack Obama holds enormous responsibility for the neoliberalization of American education. Demand accountability from all Democratic candidates, from school board to president, to support teacher-friendly initiatives to improve education, such as fighting poverty and providing teachers more pay and resources.

  • Facebook
  • Twitter
  • Google+
  • Linkedin
  • Pinterest
  • DAS

    Actually isn’t one of the problems with VAM that it penalizes teachers who teach high achieving kids. If you are basing teaching effectiveness on improvements in standardized test scores, if your students already are excellent on such tests, they won’t show any improvement. And if your students don’t show improvement then, VAM!, you must be an ineffective teacher!

    • Linnaeus

      I don’t know if this is the same thing that you’re describing, but I’ve heard that VAT/VAM is unreliable in that you can get varied results from year to year from the same teacher depending on the particular cohort that the teacher is instructing – that is, if I understand the concept of statistical unreliability properly.

      • ASV

        Generally speaking it would depend on the model, which apparently no one gets to see. And this is if we stipulate that it would be possible to model this sort of thing.

      • BigHank53

        “Last year you made a good copy of a Stradivarius cello, but this year it’s just a set of bookshelves. And they’re nasty bookshelves, too.”

        “Last year I had bookmatched spruce, properly aged maple, and enough ebony for the fingerboards. This year I had two sheets of oriented strand board and a broken pallet.”

        “….you people always have an excuse, don’t you?”

        • wjts

          “Sometimes you just have to do more with less. Now if you’ll excuse me, I’m going to spend my administrative pay raise on a designer sweater for my dog.”

      • sanjait

        We’d expect teacher scores to vary year over year just due to the stochastic nature of the universe.

        Just like batters in baseball will vary in their game to game performance, even more than could be explained by the quality of pitchers they face.

        • xq

          But that seems to me the biggest problem with a VAM, even if implemented in an ideal way. The teacher quality signal is very small, so one year of data on a class of 20-30 doesn’t tell you very much. Maybe if you had 10 years it would work, but that seems like a long time to wait to make an evaluation.

          • BigHank53

            Between life changes (new siblings, divorce, family illness) and the usual churn of current students transferring out and new ones in, there’s no way to extract useful data on something as small as a single teacher’s influence on a single class. The dataset’s good for identifying trends: bad (or good) fads in pedagogy, lousy textbooks, demographic shifts, environmental degradation or improvement, etc. But there’s way too much noise to make out a single voice within it.

            • sapient

              This seems true. So how do we assess teachers’ performance?

            • sanjait

              A class size of 2-3 dozen is a decent sampling. We’d expect individuals in that pool to have varied problems like you describe, but the question is how much variation *between cohorts* occurs and how the effects of that variation (the noise) compares with the variation in effect between teachers (the signal). There are certainly other confounding factors that could emerge, but population and system level factors wouldn’t, I think, typically happen fast enough to do much *within an evaluation period* for a single teacher’s class, which is 9 months or less.

              The keys to good management-by-metrics, in any case, is to aggressively collect information that is useful but also to avoid interpreting it in stupid ways. Admittedly, many people struggle with that.

              • xq

                A class size of 2-3 dozen is a decent sampling

                I don’t think it is for the teacher effect sizes found in high-quality studies. The Prospect article Erik quotes from links a few studies that seem to agree; e.g. https://ies.ed.gov/ncee/pubs/20104004/pdf/20104004.pdf

                • sanjait

                  “I don’t think it is for the teacher effect sizes found in high-quality studies. The Prospect article Erik quotes from links a few studies that seem to agree”

                  Statistics can be like this. Somehow you saw confirmation of what you believed in that paper, and what I saw is confirmation of exactly what I’ve been saying. The authors show that single-year assessments will be “wrong” about a third of the time in any given year, if we use the thresholds they use. I find that unsurprising, and evidence that, like I said, care must be taken in what one infers from data.

              • weirdnoise

                Students aren’t assigned randomly; administrators can assign more promising pupils to favored teachers. The community students are drawn from can change. One or two problem students can drag down the whole class (either through disruption or in time and effort by the teacher to attempt to bring them along).

                But the MBA-ification of education ignores these variables and makes the same assumptions you did.

                • N__B

                  To an MBA, everything looks like a widget. Even students.

                • xq

                  VAM does not assume students are assigned randomly. I think VAM is basically useless as an evaluation method, but the reasons are not so obvious as that.

                • sanjait

                  Progress metrics are pre- and post-, and complex statistical models like VAM do attempt to account for changes in the composition of classrooms.

                  So no, those variables aren’t getting ignored, speaking of people making assumptions. The open questions are whether they are properly accounted and whether the inferences drawn from those measures are appropriate.

          • sanjait

            It depends on the details. How much expected variation occurs and how significantly does it impact employment and pay?

            If teachers are getting fired because of an anomalously bad year, that’s not right.

            If teachers’ bonuses vary year to year by a few percentage points of their total income, that’s shouldn’t be seen as a big problem.

            • xq

              If teachers’ bonuses vary year to year by a few percentage points of their total income, that’s shouldn’t be seen as a big problem.

              But the benefit of VAM was supposed to come from firing the worst teachers and retaining the best. Varying bonuses by a few percentage points accomplishes neither goal. Sure, you can make the system less capricious by using it less, but that doesn’t address its basic flaws as an evaluation tool.

              • sanjait

                “But the benefit of VAM was supposed to come from firing the worst teachers and retaining the best. Varying bonuses by a few percentage points accomplishes neither goal. Sure, you can make the system less capricious by using it less, but that doesn’t address its basic flaws as an evaluation tool.”

                I’m not following your logic here.

                All such measurements are noisy. We can call that a “flaw” but it doesn’t make measurements useless.

                They are useful *if* we are careful in how we draw inferences. In this case, that means firing teachers if and only if their scores are so low over a sufficient range of time as to leave no reasonable doubt, in the statistically sense, about their poor effectiveness. Basically, the null hypothesis of “not bad” must be soundly rejected by data first.

                This assumes that the problem with the measure is one of noise rather than a flaw that is systemic to the measure. It’s critical to understand these are different things.

                I don’t know the details of VAM. The lawsuit against it seems to be based on the fact that it is not transparently available, and therefore those punished under it are not able to evaluate the possibility that it is systemically flawed. For all I know, it may very well be, but that’s not the same as being merely noisy.

                • xq

                  In this case, that means firing teachers if and only if their scores are so low over a sufficient range of time as to leave no reasonable doubt, in the statistically sense, about their poor effectiveness.

                  My point is that I’m very skeptical that N=30 or even N=90 is enough to figure this out. Maybe you can make this determination by the time the teacher retires but that is not practically useful.

            • nominal

              So your theory is that if VAM is shit, it’s not a big deal if it only costs 5% of your salary? Ok then. I’m going to flip a coin. If I get heads 3 times in a row, that’s proof your a shit whatever-you-do and you lose 5% of your salary. Granted, not a great evaluation system, but it’s only a few percentage points, right?

              • sanjait

                “So your theory is that if VAM is shit, it’s not a big deal if it only costs 5% of your salary?”

                No. I literally said none of that.

          • Snarki, child of Loki

            You want a BIG improvement? Make sure that the kids parents have enough income.

            It might even cost less, overall, but it involves giving stuff to poors and blahs, and not breaking teachers unions, so unpossible.

            • N__B

              That comment is suspiciously snark-free.

    • sanjait

      I don’t know if that’s the case, but in an “equitable” system, it should be the case.

      Teachers who can march into schools with low performing students and rack up big progress gains should be the most richly rewarded.

      • DAS

        The teachers who can reach low performing students should be rewarded. But teachers who teach high performing students shouldn’t be punished for their students’ lack of improvement if those students are acing the standardized tests. But that has actually happened: there was a case in our area where a teacher got a poor performance evaluation because her students didn’t improve enough on standardized tests (because they were already acing them).

        • sanjait

          If the performance metrics don’t account for that, they are amateurishly designed.

          What I was thinking about though was how teachers generally prefer teaching in prosperous schools for intrinsic reasons. So I’d be happy with a system where quality teachers were more richly rewarded extrinsically to choose to provide the same quality instruction to lower income students. That to me sounds a more “fair” world.

      • nominal

        Christ, you’re dense. The problem is that VAM DOESNT TELL YOU IF A TEACHER IS RESPONSIBLE FOR THE GAINS. And the biggest problem is that you can expect a new teacher to randomly do better than average half the time. And vice versa. And to do much better and worse a decent number of times. So you’re just randomly giving out raises and cuts a decent amount of the time.

        And there’s no real way to correct for that.

        • Richard Gadsden

          Wait a couple of decades? Then you’d know who was good and who wasn’t. Of course, you still don’t know whether they will stay good, but it’s a reasonable bet.

        • sanjait

          I never said I knew how reliable the VAM was, but I strongly suspect your assertions that it has literally no signal above random noise is wrong.

          And, like I said already, signal to noise is an important question in evaluating such systems.

          But sure, I’m dense, or something.

  • MarekKulak

    But seriously, harrumph.

  • Linnaeus

    I didn’t know public school teachers in Texas had collective bargaining rights. You’d think the state government there would be rather hostile to that.

    • Joseph Slater

      They don’t.

    • MikeG

      There are 10 US states where teachers are not unionized.
      Eight of these are in the bottom-10 states for educational achievement.
      Kinda puts the lie on the whole “Teachers’ unions are strangling student performance” BS.

  • zoomar2

    I teach in NYC. We are being evaluated based in part on test scores for subjects we don’t even teach. It’s crazy.
    Never forget that Barack Obama holds enormous responsibility for the neoliberalization of American education. Demand accountability from all Democratic candidates, from school board to president, to support teacher-friendly initiatives to improve education, such as fighting poverty and providing teachers more pay and resources.
    Thank you for that. Too many Democrats and Liberals just don’t want to hear it.

    • Van Buren

      I teach in the city also. I will never forget the look on our fresh out of college Music teacher’s face when it was explained to her that her ratings was based partially on how my 5th graders did on tests.

    • Sly

      I suppose we should be grateful that we at least get to see the quantum mechanics equations that are used to spit out our APPR rating.

  • United States Magistrate Judge Stephen Smith concluded that the metric’s impenetrability rendered it unconstitutional.

    Well, obviously you can’t tell the teachers what they are being scored on, because they would then set about improving themselves in those areas and rendering the intellectual property value of the VAM algorithm worthless. Which, after all, is the most important issue here.

    • Sentient AI From The Future

      I share your cynicism, but I’m quite glad to see consistency with judgements like http://www.twincities.com/2009/04/30/breath-test-ruling-jeopardizes-thousands-of-state-dwi-cases/

      Or am I being prematurely optimistic?

      • I think the two issues are merely similar on the surface. At the end of the day, a breathalyzer is merely testing the level of alcohol in a person’rs breath. It it something for which the accuracy can be readily tested. The VAM is a data-driven analysis that, like every data-driven analysis, is only as good as the model and the data. What is the model? What are the assumptions? How good is the data? If the model is valid, then we should be encouraging teachers to improve based on the criteria, instead of keeping the model a mystery.

        • Sentient AI From The Future

          In my mind the same criticisms you have for VAM apply to what I assume are Fourier transforms intended to calculate the AUC in the case of the breathalyzer. Perhaps the breathalyzer samples only twice during a test, but how would we know that without access to the source code?

    • zoomar2

      Bingo!!! This meshugas will go away and all problems magically solved when the UFT is busted, schools mostly privatized, pensions become 401K and teachers go back to making 40K yr. They keep changing criteria, moving the goalposts on us to make it impossible to meet the criteria. Danielson criteria was not supposed to be used to evaluate us, and no one understands it or knows how to train it. Yet we’re forced to design our classes around it.

  • nobody

    From experience, we all know that certain teachers are “better” than others. But it’s not that simple. A teacher could be good in one subject but terrible in another. A teacher could be good in one environment but bad in another.

    But a student’s background and home environment matters the most. And that a teacher can’t easily overcome that.

    If you were to take a school with the best performance and one with the worst and swapped teachers, I bet there would be little difference in performance. This is what people need to understand!

    • Sorry, but all those snowflakes in the white upper middle class schools are just better. Otherwise those inner city schools would be filled with snowflakes, too.

      • Samquilla

        No, it’s that 6 hours per day cannot counteract the disadvantages of poverty and community-wide deprivation. My children, special little now considered white snowflakes that they are, get near round-the-clock educational opportunities from playing board games and hearing the words “vertex” and “vertices” to building with legos and magnatiles and marble shutes. They never spend time and mental energy worrying about whether there will be food for dinner, whether there will be violence in their home, or whether they will have shoes/clothes/winter coats.

        It is an exceptional teacher, expending exceptional personal resources, that can make a dent in the disadvantages of growing up in an impoverished community. And they don’t do it in 6 hours per day. And it’s impossible to fully populate a school system with exceptional people willing to expend exceptional personal resources (at a teacher’s salary).

        Poverty is why poor kids perform worse on standardized tests. And it’s asking the impossible for schools to eliminate the effects of concentrated poverty.

        • sapient

          My guess is that there is violence in some of their homes.

          • BigHank53

            There’s violence, abuse, and neglect in plenty of lily-white middle class homes as well. Probably less of it, and there are fewer of the compounding factors that poverty contributes, but…

        • Therein lies the problem. School is meant to be a place of learning. Poverty is a real and dire issue, of which education is merely one small part of the solution, not the solution.

  • MariedeGournay

    Teacher accountability is Orwellian language to make them scapegoats for society’s failure to protect its children from the ravages of fortune. How dare teachers not make up for poverty, discrimination, illness, accident, tragedy. It’s denial boiled down to pure ressentiment.

    • sapient

      Nonetheless, we need to find a way to evaluate teachers, because some of them aren’t good at teaching.

  • njorl

    It all seems counterproductive to me. Label some of your teachers as “good”, and you’re just painting the target for the recruiters from wealthier suburban districts.

  • Schadenboner

    Until proven otherwise, any seemingly pro-union ruling coming out of Texas should be assumed to be a false-flag designed to get it in front of the currently-existing SCOTUS and allow them to outlaw unions.

    • Unions and trial lawyer fund Dem candidates. Crushing them both has long been a Repub goal. It’s like the goal of management is to have $0 labor costs.

      • postmodulator

        I think it was the Poor Man who came up with the formulation “slaves who forage elsewhere for food and shelter.”

  • Joe Paulson

    “Never forget that Barack Obama holds enormous responsibility for the neoliberalization of American education.”

    I’ll remember he had some responsibility but I’d need more to determine how “enormous” responsibility he personally had for something that went on before he was in office.

    • John F

      He appointed Arne Duncan

      • Joe Paulson

        Yes, he had some responsibility.

        Arne Duncan applied policy set forth by Congress etc. Cabinet heads have some discretion but only so much. And, the “neoliberalization” was there already, including as a matter of local policy nation-wide.

        • nemdam

          I feel I should point out that Hillary wanted to reverse Obama’s education policies. She was an enthusiastic backer of the teacher’s unions and was not a charter school/standardized test champion.

          So this is to say Obama’s policies are not Democratic doctrine.

          • She was an enthusiastic backer of the teacher’s unions and was not a charter school/standardized test champion.

            Un-possible! Everyone knows HRC is the neoliberalest neoliberal to ever neo!

    • FlipYrWhig

      I’d also like to know what “neoliberalization” means here. Because it involves competition? Even reading “neoliberal” to mean “unduly trusting in markets” I’m not getting what’s “neoliberal” in particular, rather than, say, “bad” or “shit.”

      • Alex


      • guthrie

        Driving down wages, privatising education, etc.

  • Gepap

    The proof that education reformers are mostly scammers comes from the fact they rarely discuss pedagogy – instead focusing more on labor contracts or administrative issues, which are relatively marginal issues when it comes to the question of the ability of children to learn, and only because they might affect the ability of a school to implement a superior pedagogic system.

    • Sly

      One of the metrics that Students First, Rhee’s own ed reform mercenary outfit, uses to evaluate the effectiveness of state education practices is the structure of their pension system, with states getting higher ratings if they use a defined contribution system akin to a 401(k), and get lower ratings if they’ve stuck with a traditional defined benefit plan.

      This, of course, has nothing at all to do with the fact that the board of directors of Students First, as is the case with most if not all organizations like it, is entirely peopled with investment bankers.

      Scammers is putting it mildly.

      • applecor

        Not sure I follow this. The finance industry likes big piles of money to play with. DB plans create those. 401(k)s don’t, in addition to having more investor protections under the law.

        • Sly

          1) Public pension systems are typically run by state-chartered non-profits that are subject to much more rigorous oversight and scrutiny than your typical DC fund manager. In New York, even the makeup of the NYSTRS board of trustees is set by state law.

          2) I use 401(k) as a well-known example; the actual type of defined contribution retirement accounts used in lieu of traditional pension funds for public employees are 401(a)s. Unlike 401(k)s, 401(a)s let the employer set contribution requirements and give them exclusive control over the beneficiaries investment portfolio. So you basically get all the drawbacks of a 401(k) without any of its supposed benefits vis-a-vis investment choice (which are usually what DC advocates focus on when pushing them as a better alternative to DB plans).

          • applecor

            Well that is quite a different story, up to a point. 401(k)s are not managed by fund managers, they are managed by the individual beneficiaries, who are not allowed to invest in hedge funds etc. Yes, public pension DB plans are subject to different regulations but they are still vulnerable to the predations of fund managers. State legislators don’t want to fully fund DC plans so they just pretend the plans will earn an 8% compounded annual return for 50 years. Since you can’t do that by buying T-bills, you have to make some risky investments. I have spent a good portion of my professional life trying to protect DC plans from money managers. Some DC plans have quality internal managers that will really rake an outside money manager over the coals before investing, but others don’t.

            Please note that this is an ENTIRELY different issue from turning SS into a DC plan, because in that case the finance industry is getting its hands on money that it did not previously have access to, so of course they are all for privatization.

        • MikeG

          410(k)s and their ilk cost less and shift risk, which is why most corporations have switched to them. Plus it’s easier to sneak in benefit cuts later, and to blame employees when they end up with insufficient funds to retire.

          • applecor

            I agree with this part, but that would explain why any rich GOPer would be for DC plans, not finance types in particular.

            • Sly

              Admittedly on my part, the really big reason a lot of finance types are supporters of Ed Reform is because of the tax provisions surrounding charter schools; because of the way the public/private and non-profit/for-profit parts move together in that system, its easy for a lot of large institutional investors to make big profits off of what is often simultaneously treated as both loans and charitable giving under the law.

              Essentially, the money that investors put in to charter schools is subject to a tax credit on the principle that can reach as high as 40%. Combine that with other tax breaks like “Job Creation” credits as well as the interest payments they’re collecting on the loans to charter operations, and you’re basically looking at a pretty big cash cow for large institutional investors.

              Transforming pension funds into 401(a)s that are managed by the employing charter operation probably only adds a few pennies for every dollar they’re already collecting, but the comparatively low rate of return relative to what they’re making on loan interest and tax benefits isn’t really all that important for people whose answer to the question “How much is enough?” is always “More.” It’s icing on the cake.

        • CPPB

          You’d think. But many of the individuals who make up the finance industry have made it their practice to support right wing organizations that work to dismantle DB pensions (not even just indirectly by opposing unions, but in fact making direct attacks and lobbying for pension freezes, etc). It’s not like after 2008 we need to pretend they are long term rational thinkers.

          It’s why many pension funds, pushed, not coincidentally by the AFT, have started requiring disclosure of their investment managers donations to organizations like Students First that oppose public sector pensions. Actually getting pension fund money out of their hands is a bit more complicated, but knowing who has a conflict of interest is part of the calculation.

    • MikeG

      The whole MBA ideology is about commoditizing teachers and the primacy of metrics and management. Not coincidentally, the ideology that managers are all-important justifies authoritarian practices and huge salaries for managers while driving down the paychecks of everyone else. Cheap Labor Republicans.

      • The end game is to destroy public schools and give them to the private sectors….meaning teaching our kids to be sheep who’ll vote more 1%’s.

  • Sly

    More specifically, he continued, because the school district denies its teachers access to the computer algorithms and data that form the basis of each teacher’s VAM score, it “flunks the minimum procedural due process standard of providing the reason for termination ‘in sufficient detail to enable [the teacher] to show any error that may exist.’”

    Fucking Christ. At least here in New York we get to see the quantum physics equations that are used to spit out our APPR rating. Can’t even get that in Texas.

  • Thirtyish

    And a horrible way to evaluate students, as it happens.

  • Sentient AI From The Future

    I was just ejected from an educational program a few months ago because my standardized test performance while being taught by highly paid professionals was consistently awful, when part of the reason I was let into the program was my quite excellent entrance exam performance, which I prepared for entirely on my own. Ok, so I pirated some textbooks too.

    (I’m only bitter on the outside. Inside, I’m creamy nougat)

    I guess the difference is whose pocket the money is coming out of.

  • David

    “But to me, the policies around empowering parents and moving decision-making to the hands of parents on behalf of children is really the direction we need to go.”

    What a load of…I’ve been on the phone and talked in person to COLLEGE student’s parents. It’s horrifying. They raised an idiot that has spent the last ten days trying to lay the blame anywhere but with their self, where do you think they learned that?

    Sorry, just, and awful flashback or twenty.

  • EvanHarper

    It would make a lot more sense to criticize the specific failings of test-based evaluation in practice, which are serious. Instead we have Loomis dogmatically insisting that test-based evaluation is an impossibility while carefully ignoring how it actually works (everyone knows you can’t use *levels* as a deliverable, you have to look at *changes* in some kind of more sophisticated way, which is what’s actually being discussed here.)

    • sapient

      Thank you for trying to move the discussion to: “What should we do to evaluate teachers?” and “How do we know that students are learning?” Because that’s actually what people are trying to do.

      I attended public schools. It was obvious that many teachers were not good at their jobs. Teacher unions were vibrant when I was growing up, and many bad teachers had tenure. This is a fact. I know some dedicated teachers, and love them, and want to protect and pay them. How to do that?

      • Linnaeus

        We would need to be careful not to excessively rely on test scores and understand their limitations, instead of assuming that because they put numbers to something, they must be accurate and objective. Assuming for argument’s sake that testing can play a role in fair teacher evaluation, you’d need to combine it with other methods of evaluation, e.g., observation of junior teachers by senior teachers, etc.

        What’s more, this evaluation rubric should be developed in conjunction with the faculty being evaluated. Beyond that, there should be avenues for due process, for teacher mentoring and improvement and so forth, if we’re really serious about wanting to develop quality teachers.

        That, however, takes time and takes money, which we seem to be less inclined as a society to give to our public institutions. In the educational context, this is even more serious because of the silver bullietization of education in light of a relatively weak social safety net.

        • Adam Short

          Well and this may seem a bit pedantic, but “accurate and objective” are not actually sufficient qualities for a good measurement. They aren’t even really that important. “Relevant and predictive” are much more important.

          Shoe size and height are objective, accurate, precise measurements. The problem (which is the same problem achievement tests have) is that they aren’t relevant or predictive when it comes to assessing teacher quality.

          Meanwhile there are subjective measures that, when used properly, can be very effective. For example the subjective assessments by fourth-grade teachers of the preparedness of incoming students is probably a better measure of a third-grade teacher’s effectiveness than an achievement test.

          Teachers that are substantially less effective than a replacement teacher would be ought to be replaced. I dunno what it’s like where y’all live but at the moment the median replacement teacher in Virginia is essentially no one, as people are leaving teaching for other professions faster than they are being replaced. So… there probably are teachers that are worse than “no one.” If you can find them, by all means fire them. But that won’t fix the schools; I’m almost positive.

          • Linnaeus

            I was using “accurate and objective” in a loose sense, i.e., a good way to evaluate someone, but “relevant and predictive” is a better way to put it.

          • sapient

            Thanks. This is a really important point. On the other hand, living in VA, I thought it was hard to get a teaching gig here. It would be interesting to see a link to support what you’re saying, because my impression was that it’s hard to become a teacher here in the first place.

            • Adam Short

              Well, the details are important. If you are a math or science teacher or can teach ESL you can get a job anywhere you want. Humanities teachers and elementary school teachers and stuff like that there sort of have to go where the jobs are, but there are lots of jobs.

              The job market was very soft in 2011 or so when all the counties were broke because of the banking crisis. At this point we’re one of the tighter teaching job markets in the country. If you look at Teach Virginia they maintain a “critical need areas” page or some such and it currently lists basically everything.

        • EvanHarper

          > We would need to be careful not to excessively rely on test scores and understand their limitations, instead of assuming that because they put numbers to something, they must be accurate and objective

          The thing is that we have strong evidence from longitudinal studies. Children who study under those teachers who tend to deliver larger increases in test scores go on to do better on a host of life outcomes.

          There are genuine problems with the “test and incentivize” agenda that I don’t want to minimize, and it’s annoying that support for it has become a signalling device for 90s style “neoliberals” to separate themselves from the “progressive” left. But at the same time the progressive left’s understanding of the issue is pathetic. As others have pointed out they’re literally at the Betsy DeVos level of ignorance. They just completely ignore the substance of the actual debate and repeat a handful of talking points.

          • mds

            Children who study under those teachers who tend to deliver larger increases in test scores go on to do better on a host of life outcomes.

            One reform I could get behind is bringing back the old curriculum that included Latin. It might help in translating “Post hoc ergo propter hoc.”

      • Alex

        I had the conversation with my brother about this, and he was all in favor of firing “bad” teachers using standardized testing. He cited one terrible teacher that we both hated in high school, mainly because she spent her entire career chasing test scores on a specific state-wide test.

        Which, you know, is a kind of terrible-ness that standardized testing will not eliminate.

        “Many bad teachers had tenure. This is a fact.” I’m going to go out on a limb here and say that you, in high school, were not an expert on your teachers’ contracts, what constituted fireably-bad teaching, or your school’s hiring/firing procedures.

        I had 2 teachers in high school who were bad. They had tenure. They were both fired when I was in high school. This is also a fact.

  • sanjait

    Loomis unfortunately misses something here.

    DeVos was rightfully excoriated for not knowing the difference between benchmark and progress metrics.

    With the former, the teacher or system is evaluated based on the student’s test score at a given moment, and in the latter they are evaluated based on comparisons of score changes over time.

    So teachers don’t get ahead simply by avoiding low incomes students in a progress based system, like the Houston VAM, contrary to the OP.

    That said, progress metrics still are at best a simplified view of teacher or system quality. Public education should be a holistic endeavor conducted by schools that are run as coordinated teams. Simple measures of subject matter mastery don’t capture the whole picture, especially for single teachers.

    But that said … without some kind of performance metrics, managing a large organization becomes an unaccountable mess. That’s not “neoliberalism”, it’s basic management common sense. For all the critics of test score-based performance measures, we don’t typically see any offer alternative solutions for how to run schools.

    • EvanHarper

      > we don’t typically see any offer alternative solutions for how to run schools

      Sure we do. The solution is to run schools however teachers unions would prefer to be run. Do everything by seniority and never fire anyone even if they’re completely useless. That’s the underlying agenda in all of this.

      • wjts

        …never fire anyone even if they’re completely useless. That’s the underlying agenda in all of this.

        Yeah, that’s exactly what unions want. Sinecures for incompetents, which is all of them.

        • sanjait

          Dueling straw men?

          • wjts

            How else can you interpret, “…never fire anyone even if they’re completely useless. That’s the underlying agenda.”?

            • sapient

              It happens. Or, at least, it used to when I was paying attention. Also, although many [most, perhaps] teachers are saints, there are some that inappropriately gossip about kids, have huge failures of compassion, are clueless about their students’ struggles, etc. Also, some good teachers burn out. Not every teacher is fabulous, so we have to come up with ways to get rid of the bad ones. How do we do that fairly?

              • Steve LaBonne

                By administrators getting off their lazy asses and actually doing the work that is required by any functioning system of due process, instead of lying about the “impossibility” of firing bad teachers.

                • zoomar2


                • Adam Short

                  Right, and actually there is another side to this. I was involved in a system that had to fire an art teacher for being impossible to work with, and a complete nut to boot. It was not an easy process but she was eventually let go. After an extensive search (and, IIRC, one hire that didn’t work out) she was rehired. The school did, after all, need an art teacher.

                  The underlying problem wasn’t the difficulty of firing the teacher but the fact that we couldn’t afford someone better.

              • Paul Thomas

                The level of incompetence a lawyer has to show to actually get disbarred is unbelievable– you have to be more or less a functionally illiterate drunk who sleeps through court dates before the bar will be interested.

                It doesn’t take nearly that level of incompetence to be fired as a GOVERNMENT lawyer, of course. Miss a few filing deadlines and you’ll start getting folks real interested in showing you the door.

                That sort of thing doesn’t happen real often, but from my experience, the people who are on their way to getting fired tend to take a hint and resign under a cloud.

            • sanjait

              I would interpret “…never fire anyone even if they’re completely useless. That’s the underlying agenda.” as indeed implying that unions want sinecures for the incompetent, but I would not interpret it as implying that all teachers are incompetent.

              • wjts

                Then why do they want sinecures for the incompetent? Is it just most of them that are incompetent?

                • sapient

                  Nobody wants to be held to account. I understand the feeling, but with teachers, they really need to be. Just like lawyers and doctors. Unfortunately, we can’t pay them as much.

                • wjts

                  Are lawyers disbarred after they lose an arbitrary percentage of cases? Do doctors lose their license after an arbitrary number of patient deaths?

        • sapient

          I am 60 years old. I suggested that we have a discussion about why people [neo-liberals?] stopped wholesale supporting unions. It was because unions seemed to be supporting lifetime tenure for crappy-ass teachers. That was actually a thing. The teachers in my current public school system are mostly excellent, but we also have many “teach to the test” and other things that they don’t like. The kids, however, learn stuff.

          • wjts

            This sounds slightly pissier than I mean it to, but there is no system, no not one, that will make sure that every single person hired to teach will be excellent or even better than average. Not even stripping tenure protections.

            • zoomar2

              There is/was a good system. Student teaching. A year or longer probationary. All the while being closely supervised by experienced mentors, and administrators. No other profession except medical has as many hurdles and benchmarks to meet. The bad ones wash out. It’s also a myth about getting rid of bad teachers. There’s a clear process. If it’s followed, they’re out. It can be done in a reasonable timeframe with due diligence and documentation. It takes 5 years to make a teacher. Thats from the first day in a classroom. It ain’t MacDonalds. Firing a teacher isn’t easy and shouldn’t be.

              • sapient

                This is probably true, and I respect that firing a teacher should be hard.

                But teachers burn out, and it should also be possible to humanely relieve them of their duties.

          • Erik Loomis

            This is not why “people stopped wholesale supporting unions.” This is just you generalizing from your personal beliefs into a broad experience.

            • sapient

              What’s your plan to assess teachers’ performance?

              • zoomar2

                A good plan certainly wouldn’t include vague and amateur blanket characterizations like “burn out.”

                • Drew

                  I don’t otherwise agree but my impression was that burnout is actually recognized by psychologists as an actual thing, complete with diagnostic criteria.

                • zoomar2

                  Thanks. I was going to add to my comment that if an administration is concerned about a teacher being “burned out”, The answer isn’t to punish that person by firing them or bitching about how the union protects them. A good teacher doesn’t just burn out. Something burns them out. I suspect that the medical diagnosis you describe is relatively uncommon. If suspected burnout were to surface, get it diagnosed and treated as one would for PTSD, addiction or any other condition. Investigate the organizational culture for unnecessary stress. “Burnout” is a term thrown around by a lot of administrators and non-teaching education professionals to pinpoint older well paid teachers. Experienced teachers who may not show enough of a gung ho attitude to some new initiative or in meetings. I once had a young Bloomberg Academy principle who operated like this. Older, tenured teachers at the top of their salary tiers are target #1 with many principals trained there.

          • zoomar2

            Are you still in the classroom? Because if you are you’d know that “teaching to the test is what education “reformers” like Rhee and Joel Klein have been forcing us to do for the last 15 years. The rest of your anti tenure comment is just crap. If you’re retired with a defined benefits pension and had a long career teaching, and took the salary steps you had coming, you should be thanking your union, instead of pissing on it.

            • sapient

              I’m not a teacher. I have no pension, because I’m a “professional” who had no union. So get a grip, and maybe read what I said. And if you’re a teacher, and made those invalid assumptions, ummmm …..

              • zoomar2

                The teachers in my current public school system are mostly excellent, but we also have many “teach to the test…”
                That’s something you said. And it’s a good example of you not understanding what’s really been going on in classrooms and what so-called reformers have been putting teachers through for quite a few years.

    • SWIOTI

      It’s awfully difficult to create meaningful performance metrics for education.
      Not because it is difficult to create the metrics, but because it costs money to do so. It will cost real money to do the tracking, both across time as students progress, and the analysis necessary to figure out what’s important for improving education outcomes, and making sure the results are statistically significant.

      For education, different metrics may be more important at some levels than at others. In schools where graduation rates are lower, is it better to have a teacher who keeps students engaged and going to school, versus one who will help a smaller number achieve higher test scores? There are legitimate questions like these that don’t fit into the simple framework that testing, whether high or low stakes, implicates.

      So when you do create metrics, you want to be sure that you actually are measuring what is important, both for individuals (teachers, students, and administrators), and for the organization as a whole, so that you can understand how, by improving the metric, you help further the goals of the organization. If someone can show how someone scoring so high on a particular test means they’re X% more likely to advance at a regular pace and graduate, there would be much greater buy-in. Without such an observable connection between the measurement and the goal, it’s not a performance metric – it’s just statistical noise.

      I’m not opposed to using test-scores as a performance measure. But there’s little support, AFAIK, for their use as anything more than a data point in teacher evaluation. If we are to talk about performance measures in education, test scores are low-lying fruit. They’re easy to pick up. But it tells us little beyond what scores a student did on any particular day. By themselves, they tell us very little about instructional quality, about whether individual teachers are effective, and whether they will lead to better overall metrics like higher advancement and graduation rates.

      • Adam Short

        Well and there is this article of faith that bad educational outcomes are the fault of bad teachers and if we could just find them and fire them everything would be fine.

        There is actually no point in identifying bad teachers (or failing schools) if you have nothing to replace them with. That’s what happened with NCLB – they created the tests, found that oodles of schools couldn’t pass them… and then that was that because they didn’t have a bunch of well-functioning schools in their back pocket to sprinkle into the education system like fairy dust.

        Pay teachers like top professionals and you will get a higher quality of teacher. There will still be goldbricking douchebags and bullies and all that. The idea that you can get rid of the “bad ones” and that fixes everything seems intuitive to those who have suffered at the hands of these assholes, but there are tumbling choads in every profession.

        • sapient

          Pay teachers like top professionals and you will get a higher quality of teacher.

          I agree. But with this comes malpractice accountability, which is enormous. Lawyers and doctors don’t have unions; they have [anticompetitive] licencing associations and pay huge premiums in insurance.

          • Paul Thomas

            The pedant in me feels obligated to point out that there is nothing stopping (non-supervisory) doctors and lawyers from unionizing–aside from the things stopping anyone from unionizing, i.e. mostly the unbearable crappiness of US labor law.

            The rabble-rouser in me wonders why more junior associates and medical residents have not availed themselves of this option, given the otherwise-horrific conditions they slave away under.

            • SWIOTI

              Good way to prevent yourself from ever joining the profession.
              Junior associates don’t actually have any economic power, and wouldn’t even if they did unionize. Don’t like the job? Ok, we’ll hire two more lawyers at half the rate we’re paying you and ask them to each work 3/4s of the hours we ask of you.
              The situation with medical residents is similarly messy, but complicated uniquely because of how they are funded by Congress. Congrats with your medical degree! Good luck using it without a residency.

              The Bar is a guild, IMO, and I’m a member.

              • Drew

                Also, everybody hates lawyers. Hard to imagine any public sympathy for lawyers unionizing.

          • Adam Short

            This is sort of a weird objection. I don’t have to pay it, and I’m sure a lot of people on this blog do, but legal malpractice insurance is not expensive. The malpractice exposure of doctors is unique.

  • y10nerd

    I’ve been working within the evil empire of national charters in stats and assessment for most of the decade and it’s funny how charter schools are more hesitant about their VAM/VAT than the states and DOE’s were.

    In general though, this is a topic (and anything involving teacher’s unions/charters/k-12 education) where there is so much bad faith on both sides that it becomes sort of no man’s land very often trying to straddle this.

    One big one on this is obvious – a lot of the ‘reformer’ types at the school and district level (principals, curriculum leaders, etcs) aren’t MBA types who don’t like teachers but often are former teachers who have spent some time in the classroom and truly want to figure out what has the most impact so to either coach teachers or suggest they find another profession if coaching doesn’t help.

    On the other hand, it’s totally true that unions, advocates and people critical of testing aren’t trying to keep shitty people in the classroom – they are trying to figure out how to fairly judge someone and also see education as more than a freaking test score.

    More objective evaluation i think is important in the American context, if only to stop some of the absurdities I’ve seen where an adult is damn good at building relationships with students and admin but does a very bad job of actually teaching kids keeping their role over someone who had frostier relationships with both but kids were more successful with and was often at threat of being let go.

    • sapient

      This is a really good comment, with so much nuance that needs to be unwound.

    • Paul Thomas

      It is a somewhat interesting facet of this whole debate that what the reformers are theoretically trying to do– evaluate teachers based on objective metrics rather than subjective management judgments– is typically the sort of thing unions favor.

      That said, obviously the unions aren’t morally obligated to support a system that produces absurd, irrational results, and the idea that they would be denied the right even to LOOK at the metrics is confounding.

      More generally: education is probably the one area where this administration’s cronyist incompetence might end up actually being better (or at least “less bad”) than the Obama administration’s aggressively moronic policies. There’s a truly special level of idiocy involved in actively trying to undercut, diminish and defund your own biggest campaign contributors.

      • y10nerd

        Certainly, the way most VAM systems have been conceived of, implemented and forced onto unions has been dreadful. Like I said, internally, a lot of big charters had some VAM models years ago and they’ve been hesitant to implement them in any way shape or form similar to the DOE.

        I think the framing on issues of union vs democratic politicians is also important. At the end of the day, this all came down to a lot of parents didn’t think the system was working (a situation created largely by income inequality, a bit by inadequate school funding, a lot of bad administrators and yes, often unions who saw their goal as a defense of the status quo). There were a ton of obvious ways this manifested itself, plus global comparisons just made it ripe for people to come in and claim they were going to make drastic change. There is a big constituency for charters within a lot of poor urban communities and it would be important to think why that occurred. I say this as someone who often thinks we make bad choices in the charter world.

  • Harlequin

    The mathematician and data scientist Cathy O’Neil has a great breakdown of VAM in the talk she gave at Google on the use of algorithms to make decisions in society: https://www.youtube.com/watch?v=TQHs8SA1qpk The teachers bit starts at 11:30, but actually the whole thing is worth watching.

  • SpiderDan

    I don’t necessarily agree that standardized testing = “neoliberalism” (where, as always, “neoliberalism” = “a bad thing that we should avoid”). I think it’s more of an issue of using the right tests and correctly setting the parameters for desired results, such as benchmark vs. progress mentioned earlier in the comments.

    But we have to have some sort of way to measure whether teachers are doing their jobs well. If you think standardized testing is a bad idea, what superior option do you propose instead?

    • Erik Loomis

      as always, “neoliberalism” = “a bad thing that we should avoid”)

      No–neoliberalism is an actual thing. Whether you think standardized testing per se means neoliberalism is inherently neoliberal (and I wouldn’t really agree that it is), saying that neoliberalism has no real meaning does as much disservice to a set of ideas that describe both the economy and the modern condition as the leftists who use it as a pejorative.

      • sapient

        Maybe, as I’ve suggested, we should have separate discussions on what neoliberalism is. I, with all my heart, as a public school graduate, and as someone responsible for certain children, had questions about teachers’ unions. I wasn’t “against them”, but I didn’t like a lot of the things they did.

      • SpiderDan

        I wouldn’t say that “neoliberalism” has no real meaning as much as I would say that the meaning has been so thoroughly co-opted that if you don’t mean “a bad thing that we should avoid,” you should really use a different word instead. At this point, “neoliberalism” is like “separate but equal” or “states’ rights”: the term has been irrevocably poisoned and it’s futile to try to reclaim the dictionary meaning when you could just use a different term instead.

        But just to be clear: when you referenced Obama’s legacy of neoliberalization of American education, you were talking about a bad thing we should avoid, right? Or are you saying it was a good thing?

        • SWIOTI

          I disagree. I think people are confused, and are misusing a word. They are using “neoliberalism” as an epithet rather than as a description. It’s meaning hasn’t changed, but its use has, and because its use has changed, people confused the way “neoliberalism” is used with what it actually means.

          This sort of confusion is bound to happen with terms that are both used by proponents and opponents of controversial theories and ideas. But the rigorous thing to do is attempt to use words in a way that allows them to have both meaning and use as descriptors. I think Erik is right to do so, because the threat of neoliberalism still exists, and must be grappled with honestly.

          No term is irrevocably poisoned: all it takes is for people to properly use words for the poison to abate, though ignorance of certain background knowledge certainly helps. But in a case like “neoliberalism”, it is futile to search for a different term: that is the term one uses to refer to certain political and economic theories with an emphasis on laissez-faire free-market principles in combination with some variety of liberalism.

          • SpiderDan

            I mean, if you want to embark on a crusade to reclaim the true meaning of neoliberalism, be my guest, but I see that quest as futile for a very simple reason: there are effectively no proponents of “neoliberalism” named as such, only opponents.

            More to the point: it is used so frequently as an epithet that attempts to reclaim it as a descriptive term seem like a waste of time, as no one admits to using it as an epithet in the first place; ask anyone using it and they will claim that they are merely describing policy positions.

            And I refuse to believe that it is futile to try to find another term to describe the idea you are trying to convey. If we can coin the term “limousine liberal” to describe an idea, I’m pretty sure we can find more than one term to describe “certain political and economic theories with an emphasis on laissez-faire free-market principles in combination with some variety of liberalism.”

            • SWIOTI

              I think most of the proponents of neoliberalism cloak themselves in friendlier sounding-terms like, “deregulationists,” “market-oriented reformers,” or “libertarians.” Well, scratch that last one, and not just because libertarians have historically had issues with central banking and fiat money. Most could easily describe themselves as preferring “liberal” policies, and they wouldn’t be wrong, for some meaning of “liberal,” of which there are several.

              I see no reason to throw out a useful word, one that does describe a particular set of values and ideology, simply because other people misuse it. Because in the end, they’ll misuse any new word as well. So here’s to “neoliberalism”!

              • SpiderDan

                I would argue that the reason why you say “deregulationists” and “market-oriented reformers” are friendlier sounding-terms than “neoliberal” is precisely because “neoliberal” is exclusively used as an attack.

                In other words, the explicit goal in using the word “neoliberal” is to make sure that the reader knows that the writer disapproves of whatever policy or politician that is being discussed… because that’s the only time anyone ever uses the word “neoliberal.” That’s what makes it an epithet.

                • Adam Short

                  Technical terms aren’t automatically rendered useless when they begin to be used incorrectly by people who don’t know what they are talking about. If they were, there would be no technical terms.

                • SpiderDan

                  I don’t see it as much different than trying to salvage “idiot” or “moron” as technical psychological classifications. At a certain point, terms are used often enough as insults that they simply become insults, and “neoliberal” has certainly reached that point among the left on the internet.

                  And just to reemphasize: when Erik used the term above, it’s not like he was using it as a dispassionate description free of value judgment. He was specifically using it as condemnation… because that’s how everyone uses it. It serves no other purpose.

        • nick056

          >>>>>At this point, “neoliberalism” is like “separate but equal” or “states’ rights.”

          Oh my. It’s certainly used incorrectly as a term of abuse, but saying “‘neoliberalism’ is like the legal and cultural theories underpinning white supremacy” kind of posits the accused neoliberals as victims of invidious discrimination.

          • SpiderDan

            You removed the colon I originally wrote at the end of that quote explaining the metaphor, substituted a period, then pretended that there was no metaphor. Don’t do that.

            I would say that “neoliberalism” is like “final solution” but I imagine you would accuse me of comparing the left to either Nazis or German Jews. The point is that the term itself has taken on a meaning outside of the mere definition of the words within.

            • nick056

              Yes, if I were republishing your comment somewhere else, eliding the rest of what you said could be misleading. But since there is literally nobody who would read my comment without having just read your comment, I don’t really think you can accuse me of a deceptive paraphrase.

              My point stands. It’s telling for a person to go there — to those two examples in particular — in decrying the current usage of neoliberalism. “States rights” and “separate but equal” are discredited because they were window dressing for racial animus without any coherent, independent rationale at any point in time — that is emphatically not the state of “neoliberalism” as a term of abuse today.

              • SpiderDan

                You are right. “Neoliberalism” is not window dressing for racial animus.

                I am happy to make that clarification to any readers (including, apparently, you) who took that as the meaning of my original comment.

                • nick056

                  Downthread you’re arguing neoliberalism has no actual proponents and here you’re arguing the term is the equivalent to “states rights” and “separate but equal” and is also a content-free epithet without any valid analytical framework as a referent.

                  This ain’t a subtle act you’re putting on.

                • SpiderDan

                  I don’t really understand what you’re accusing me of. Apparently my “act” is too subtle even for me!

                  I said it was equivalent to “separate but equal” in that it doesn’t matter if you are literally talking about two things that are distinct but comparable, when you use the term “separate but equal” there is a very specific other meaning that will be necessarily be implied. It is therefore a waste of everyone’s time to use that term UNLESS you actually intend to convey the meaning it has taken on. Better to use a different, non-loaded term instead.

                  Elsewhere in these comments, I also referenced the terms “final solution,” “idiot,” and “moron” as other examples of terms that have a common understanding that is significantly different than the mere definition of the words.

                • SpiderDan

                  On a separate note, I’ll now address something I should have done from the start: your original interpretation that comparing the term “neoliberalism” to the term “separate but equal” or “states’ rights” somehow confers victimhood to the accused is utter nonsense.

                  It doesn’t even make any sense; I can disapprovingly reference neoliberals, separate-but-equal laws, and states’ rights politicians in the same sentence, but by your logic, I am somehow painting all three of them as… the victims of unfair discrimination? Absurd.

    • y10nerd

      The neo-liberalism involved connected to testing has to do with the use of private and market oriented resources to evaluate teachers.

      For example, you can argue that most standardized testing, scoring and then the use of think-tank created evaluation models meant that evaluation was privatized and off-shored by school districts so they wouldn’t have to do their own work.

      I’d also say in general, there’s this broader idea, particularly from a lot of ed reform people, that sees schools through a marketplace prism of choice (which is a very neoliberal idea that the market is more useful for equity than not) rather than as a center for communal reproduction of values and knowledge.

      • SpiderDan

        Ultimately, at this point “neoliberalism” is a lazy epithet and nothing more. If one wants to expend the effort to explain how this particular instance is Real, Literal Neoliberalism and not the lazy epithet that the purity left regularly deploys, I would submit that the same amount of effort would be better spent simply finding a different term to describe the idea you wish to convey.

        • SWIOTI

          Which is funny, because using neoliberalism allowed me to understand the point that Erik was trying to make in one sentence, while that same point took three paragraphs for y10nerd to unpack.

          So I disagree, sir. It is far more than a lazy epithet.

          • SpiderDan

            One of the hallmarks of epithets is that they convey an intended meaning with impressive brevity. So I’m not sure I agree with your premise here.

      • zoomar2

        I suppose the whole movement toward privatization through charters, privatized tests and curriculum though the Pearson and Kaplan monopolies, union busting etc would fall into the Neoliberalism rubric quite well. Add to that the fact that self defined liberals like Rahm, Cuomo, Booker, Obama, Arne Duncan are behind this list of policy deplorables, It walks and quacks like neoliberalism to me. I just never hear the word in education circles when we talk about resisting these things.

  • Paul Thomas

    The fundamental problem with trying to use test scores to evaluate teachers is that the sample size you need to draw any kind of reliable conclusion– several years– is so large that none of the “fire ’em all and let God sort ’em out” reformicon types are willing to wait long enough to see it through. As a result, they are constantly coming up with systems that make no sense and produce irrational results.

    In some ways, it’s rather ironic that the people most in need of due process protections– new teachers for whom there’s insufficient data to fairly evaluate them– are the ones least likely to receive them.

  • applecor

    I think one of the forces pushing toward testing is that it is “objective”, not just in the sense that it is quantifiable, but that there is broad consensus that if the kids do better on a standardized test, that’s better than if they do worse. People of different political views will have wildly different beliefs about what constitutes “good education” once you get away from test performance and into the cultivation of personal traits. As people here are saying, evaluating teachers is tough enough without this dimension. Once you start trying to respond to parents X who want their kid to be “self-reliant” while parents Y want their kid to “learn to cooperate”, the entire measurement project (if not public education generally) becomes kind of hopeless. In that light, I can see the urge to just avoid measurement entirely.

  • numbertwopencil

    I wish it wasn’t so but in some states the teachers union is the only thing holding K-12 together. They are generally the best organized group fighting the day-to-day stupid that is descending on our schools. (Ahem, just an aside but remind me again why the GOP considers teachers union evil while at the same time blessing police unions?)

    • joel_hanes

      Teachers unions typically support Democratic candidates

      Police and correctional unions typically support Republican candidates

      • Paul Thomas

        This is correct. It’s literally all about the dollars.

    • sapient

      At this moment in history, thanks for teachers unions, and any other pushback against anti-education, anti-environment, etc.

    • FlipYrWhig

      Tangentially related: can you imagine the society-wide shitstorm from all corners that would result from imposing some sort of metric to differentiate good cops from bad via value-added metrics?

  • Denverite

    I’m just curious why the parties consented to a magistrate. The only time I ever really see that is when it’s a super technical subject that the magistrate has more experience (like social security appeals) or when the district is such that both sides could get a reeeeeaaallllyyy bad draw if they went the AIII route.

    • Paul Thomas

      Really? I interviewed with a magistrate a few years back when I was coming out of law school, and his clerks gave me the distinct impression that refusing consent was akin to hanging a “kick me” sign on your back, and thus basically no one other than cranks ever did it. Plus, it’s ineffectual anyway. The district judge can still make you litigate before the (now probably somewhat pissed-off) magistrate, and if you don’t like his ruling, well, congrats, you can appeal it to… the district judge. Doesn’t exactly seem worth it.

      The only way refusing consent would really seem to make sense is if you’re trying to win by delay and attrition, and just want things to drag out as much as possible. And most lawyers don’t like to admit to themselves that that’s their strategy, because it’s patently unethical.

      • Pete

        Depends on the type of case. In my particular niche, no one ever (almost literally) consents to trial by magistrate, and no one is ever forced to do so. Although usually the MJ handles discovery disputes and case management up to trial.

        • Drew

          It always seemed like a shit end of the stick type of job. Deal with discovery disputes and case management (yay) and no life tenure. I guess they’re hoping to move up to district judge?

          • Denverite

            The pay is good, the hours aren’t terrible, you’re guaranteed eight years with a strong possibility of renewal, and the promotional opportunities are great.

      • Denverite

        This isn’t my experience. Consenting to magistrate jurisdiction is the exception and not the rule everywhere I’ve practiced. (I’m not saying that’s a good thing — there are some great magistrate’s out there.)

  • And if standardized tests are a lousy way to evaluate students?

  • jpgray

    VAM is bullshit. Blaming teachers is bullshit, though some teachers suck and some are great. This whole debate is bullshit, and I will endeavor to explain why.

    A useful exercise is to consider what determined your own track. To steal a metaphor, when I examine my lazy, directionless schoolboy self, I felt myself to be a curious prisoner, chained in a cave and forced to view the cast shadows of real objects passed around by parents, teachers, and peers – occasionally passed to me.

    I had little idea of what I was doing or why, but I felt compelled to pass for one of my fellow prisoners, to not be too much better nor too far worse, to not disappoint my parents, and above all to not fail in observance of our particular cave customs, lest I bring shame on my cave people. The shadowy whatsits that others treated as important, I treated as important. For me, that meant going to college. It meant not being in jail. It meant getting decent grades.

    You could have dropped Eton on me – absent any changes to the rituals, to what my peers and the general parentage were passing around, it would have had a marginal effect at best. You could have corralled the worst teachers mankind has ever seen and sicced them on me, and, showered with their indifference or ignorance, I would have turned out about the same.

    Think of the brilliant men and women turned out by this country in past centuries from an utterly benighted or nonexistent school system. US Grant is one of the best prose writers ever, to take one example, and his primary-secondary schooling was negligent at best and harmful at worst. Talk about the value of military history – his asides on the causes and morality of the Mexican War, the strange case of poor whites loving slaveowners, etc., are wise and relevant to this very day.

    For me, the shadowy objects of value meant I was going to college. If that’s not getting passed around, NOBODY is going to college, or near enough. If something nasty is being reverently passed about, EVERYBODY will be doing something nasty, or near enough.

    How anyone expects teachers to somehow ruin the benign form of this process or reverse it when it is evil is just a fucking mystery to me. 10,000 battalions of superteachers could be airlifted onto a single shitty high school and they would fail as sure as anything.

    It’s hard not to see this as a massive grift for public dollars when those are the only dollars being spent and invested.

    • Paul Thomas

      I think you may be overgeneralizing. Some folks have internal drive to acquire knowledge; others need to be inspired to it. The latter group shouldn’t just be written off.

      You’re quite right about Grant, though. His description of his feelings at Appomattox has stuck with me for decades:

      “What General Lee’s feelings were I do not know. As he was a man
      of much dignity, with an impassible face, it was impossible to say
      whether he felt inwardly glad that the end had finally come, or
      felt sad over the result, and was too manly to show it. Whatever
      his feelings, they were entirely concealed from my observation; but
      my own feelings, which had been quite jubilant on the receipt of
      his letter, were sad and depressed. I felt like anything rather
      than rejoicing at the downfall of a foe who had fought so long and
      valiantly, and had suffered so much for a cause, though that cause
      was, I believe, one of the worst for which a people ever fought,
      and one for which there was the least excuse.”

      That’s quality writing– he had an amazing clarity of thought. Keep in mind that this passage was written while he was terminally ill with throat cancer, and it’s even more commendable.

      • jpgray

        Definitely overgeneralizing, but…

        Some folks this and that, but ALL folks feel the call of what’s making the rounds, what’s expected, what’s decorous. Teachers offer up a vision on those, but the best and worst are so far down on a kid’s list of important norm-slingers that it’s laughable to expect such a huge lift from them as turning kids’ lives around. The internal drive to conform to the expectations of the nearest and most keenly regarded of one’s group (so almost never teachers) is as close to universal as can be imagined.

        That so many see no path of worth in applying oneself at school does not mean the school is to blame. Think of your go-getterest classmate you had and your worst teacher – would the latter ever put any significant dent in the former? Think of your most determined unlearner fuck this bullshit classmate and your most superlative holiest kid whisperer of a teacher – would the latter ever significantly engage the former? For more normal cases, it’s not that they can’t hurt or can’t help, but no teacher is putting out a delta v in any direction that can wholly knock kids out of an inexorable orbit around their place, peers and parents.

        We’d do better to evaluate the quality of students’ place, peers and parents – break out the VAM and see what pols are hitting their improvement metrics there with their constituents.

  • MikeG

    empowering parents and moving decision-making to the hands of parents

    DeVos Code for “Creationism and Jesus Studies for everyone, because parents (i.e. one noisy and well-connected bible-thumper) want it”.
    Raising a generation of science-denying eager consumers of Tox News and Prosperity Gospel.

  • TheoLib

    Harlequin provided a video of Cathy O’Neil, a former math professor and former quant. I highly recommend her book, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy — a short, interesting, and very informative read about the misuse (often intentional) of non-evidence-based and no-feedback algorithms and of the big data being massively collected and fed into these algorithms. She tackles a number of fields, from VAM … to employers doing credit-card checks on potential employees … to the crazy, online psychological tests given to job applicants (my son had to take one for a minimum wage job at a big-chain bookstore in college). For-profit universities and credit-card companies targeting particular zip codes … and banks avoiding particular zip codes (wink-wink) … And so on. No math required to read the book.

    Also check out her blog, MathBabe, at https://mathbabe.org/.

  • Icis Bokonon

    The best way to evaluate teachers is to index their pay to their pay. Those districts that pay well have the highest performing students, after all. Science!

  • JamesWimberley

    Teaching may be the only profession that does not rely on systematic peer review and mentoring as a prime method of maintaining standards. How often are there two teachers in a classroom, one teaching, the other observing? It should be 10% of the time.

    • DAS

      Peer review can and is used to harrass.

      I dunno how it works in K-12, but everytime I teach a class I’ve not taught before, I get peer reviewed. Also, if I want a promotion or range adjustment, I have to get my teaching peer reviewed.

  • hawqize

    I agree that current models of evaluating teachers are problematic at best and often unfairly reward those teachers with higher achieving students. But for years I’ve had this argument with my teacher wife and many of our teacher friends–the current situation is the fault of teachers and their unions. Virtually every other job/profession has some form of evaluation that attempts to identify the high performers and rewards them accordingly. Teachers have resisted this and fallen back on the “there’s just no way to adequately evaluate us” position. I’m not a teacher and I don’t know what the correct method is, but until teachers accept the notion that taxpayers who pay their salary want some kind of accountability, they’re going to get sub-optimal review mechanisms imposed upon them. It’s way past time for teachers to coalesce and propose something that works.

  • zoomar2

    Modern teaching “Reform” and discussions centered around it make the assumption that there’s a problem with teachers and teaching to begin with that is so bad it requires tearing down the existing system (which is what’s happening,) and micromanaging classrooms, and agonizing over evaluations to find that one bad teacher. I don’t believe it. American kids score lower in STEM subjects than other countries. Thats been the case for over 50 years. Lots of good reasons for that. For one, we educate everybody’s kids here. Other countries don’t. Yet we had (still sort of have for now) the most advance economy in the word with the top colleges and universities for all that time. If the magic formula or combination of formulas existed we’d be doing it. Well, the magic formula does exist. It doesn’t revolve around occasional bad teachers, evaluations, high stakes test centered curriculums. But, small classrooms cost money. social services for at risk kids costs money. Art studios, music, marching bands, sports, clubs, cost money and have been cut to the bone. Real professional development that teachers themselves can choose costs money. Personally, I’m a CTE teacher and there’s nothing available of any use in my area. The whole discussion and resulting policies are the proverbial drunk looking for his keys under the nearest streetlight. It won’t work. Teachers and most administers don’t support it, and if they did, there’s no training or follow up. Plus every year the goalposts move. The policy pronouncements from above change. The only ones who make out are Kaplan, Pearson, and the paid consultants, which, BTW, is where a lot of those bad and burned out teachers end up going.

    • DAS

      One of the problems with micromanaging classrooms is that when you force teachers to stick to the standards and school selected textbooks implementing those standards is that if the standards aren’t good or are poorly implementated, then students will be underprepared. And guess which schools face the most pressure to conform strictly to standards and externally provided curricular materials implementing those standards?

  • Richard Gadsden

    In the UK, we use value-added for measuring school quality, rather than teacher quality.

    So this is based on total improvement of a full year-cohort between 11 and 16 (ie over five years), and it still needs to be averaged over three years of students to get a big enough sample for the numbers to be reliable. Also, we publish the algorithm and the reasoning behind it.

    Measuring teacher quality is like measuring clutch ability in baseball – there is enough signal there that we can identify that the signal exists, but the sample isn’t big enough to work out who is actually good and who isn’t.

  • TheBrett

    I’m not a big fan of standardized testing, but then how do you determine who is a good teacher and who is a bad one? If we’re just going off of “the parents seem happy”, then remember that that’s the big argument the tuition voucher people use as well.

It is main inner container footer text