Home / General / Charter Schools Do Better On Standardized Tests: One of Rheeism’s Many Lies

Charter Schools Do Better On Standardized Tests: One of Rheeism’s Many Lies


Gary Rubinstein does some number crunching from New York school test report that showed declining scores and what he finds exposes more Rheeism lies:

In the Stephanie Simon report she mentions that KIPP Star and Democracy Prep hadn’t done so well with their proficiency rate, but she doesn’t mention how far they had dropped. Out of over 500 schools, which includes about 35 charter schools, of the one hundred largest drops, 22 were charter schools.

The most stunning example is the famed Harlem Village Academy which had 100% passing in 2012, but only 21% passing in 2013 for a 79% drop (you can see that sad dot all the way at the right of the scatter plot). Democracy Prep Harlem Charter, run and staffed by many TFAers, dropped 84% in 2012 to 13% in 2013. KIPP Amp dropped from 79% in 2012 to just 9% in 2013. The Equity Project (TEP) which pays $125,000 for the best teachers had finally gotten some test scores they can brag about with 76% in 2012, but that has now sunk to just 20% in 2013. The Bronx Charter School Of Excellence, which recently received money from a $4.5 million grant to help public schools emulate what they do, dropped from 96% in 2012 to 33% in 2013. So these are the schools that are the red ‘outliers’ hovering near the bottom right of the scatter plot. In general, the average charter school went down by 51 percentage points compared to 34 percentage points for the average public school. The most plausible explanation for charters dropping so much more than public schools is that their test prep methods were not sufficient for the more difficult tests. In other words “you’re busted.”

I just don’t see how the ‘reformers’ can reconcile these statistics with their statement that these lower scores are a good thing since we are now being honest about where we stand. The low scores in general do not decisively prove anything. The cutoff scores for passing were an arbitrary choice by some politicians in Albany. But the evidence that charters are certainly not working the miracles they claim is very clear from this data.

Ouch. It’s all about the data for education reformers, right? Well the data suggests that the promises reformers tell about charter schools aren’t coming true.

  • Facebook
  • Twitter
  • Google+
  • Linkedin
  • Pinterest
  • Yeah, you can see how damaging this is by the frantic spinning by Joel Klein, Bloomberg, Arne Duncan, etc. All of them saying – “well, it’s good that parents aren’t being lied to any more”- without admitting “we were the ones lying to parents, because we needed to show results from our reforms.”

  • Sly

    It gets even more surprising when you account for the fact that charters are exempt from the state rule that prohibits internal grading; traditional public schools are barred from grading the exams of their own students, but charters are not. This exemption was adopted this year, and these are the first battery of exams that utilize external grading.

    The reason why traditional public schools in the state were barred from doing this has more to do with suspicions of cheating; grading your own exams creates a kind of “home field advantage is other, more subtle ways. You could teach a certain course a certain kind of way that an impartial grader may not account for. And when the DoE exempted charters, a bunch of us in NY thought it was just to give a preference to the particular test prep methods of certain kinds (i.e. politically well-connected) charters.

    The fact that results didn’t show this is… well… really shocking. They can’t even win when the game is rigged in their favor.

    • Sly

      To clarify, the rule on external grading was adopted this year, and charters were specifically exempt, and the rule was adopted not just to help weed out cheaters (there’s already a grade auditing process external to the school district), but to create a greater sense of a statewide standard.

      Brain isn’t firing on all cylinders today.

  • joe from Lowell

    staffed by many TFAers

    Well, there you go. No school should be staffed by “many” newbies fresh out of school. New teachers need to be embedded within a larger staff of experienced teachers if they’re going to have any hope of figuring out what they’re doing.

    • Linnaeus

      That’s just commie teachers’ union stuff, right there. Everybody knows you don’t need experience to be a good teacher*.

      *Applies only to teachers and no other profession.

      • Lee Rudolph

        Everybody knows you don’t need experience to be a good teacher*.

        *Applies only to teachers and no other profession

        except for conservative bloggers.

        • Lee Rudolph

          That is to say, conservative pundits and bloggers. Sorry about that.

        • Also elected officials. The less experience the better.

  • c u n d gulag

    Conservatives POV:
    That data obviously has a Liberal bias.

    We demand that we be allowed to use Rovian Math, and then submit the results to the SCOTUS, for their judgement.

    • Lord Jesus Perm

      The data must be UNSKEWED.

      • Davis X. Machina

        There’s a guy in Florida with some experience precisely in the field who’s presently between jobs….

        • c u n d gulag

          Let’s hope he stays that way.

  • jim, some guy in iowa

    Q for the teachers here: how much value should be placed on a standardized test score? I was always able to do well on them, but that was in large part because I knew how to pick the right answer out of multiple choices… not the same thing as actually *knowing* the subject

    • joe from Lowell

      My take: Depends on the level we’re talking about, and how they’re used.

      On the level of the individual student, they’re pretty much worthless.

      On the level of a school system, or a large school, or the grade levels in a large school, comparing standardized test scores year over year can provide useful data. Sort of like how Rasmussen’s polling trends can tell you what’s happening in an election campaign, even if you know you can’t trust the absolute numbers.

    • Sly

      Every teacher teaches to a test. Most of the time, however, those are tests that we create. We know what knowledge and skills we worked to impart upon our students, and design assessments accordingly with the implicit understanding that we are the ones being tested.

      The problem with standardized testing is that it is standardized; it assumes there is one method that works better than all others when it comes to assessing knowledge and skills. The reality is that standardized tests are designed to prioritize the costs of test administration and grading; the tests that are the cheapest to design, implement, and score are the ones that achieve prominence.

      I teach the social sciences; whether or not a multiple choice question is an effective mechanism for any other subject is not for me to judge (though I very seriously doubt it), but, if I had my way, I would never give a single question of that type to a student, let alone a test where they make up 50% or more of a score that determines whether or not they completed my class.

      • Matt McKeon

        In fact, we used to be instructed to write the test first, then teach the unit.

        standardized tests can be a useful tool. My classes have traditionally been on the low end of the totem pole. The school system now makes a stronger effort to serve these students effectively to increase the overall high school performance.

        • Sly

          In fact, we used to be instructed to write the test first, then teach the unit.

          It’s certainly not gone. One of the amusing things about educational fadism is that what people think is all new and insightful is just stuff people we’re doing 20 or 30 years ago that fell out of fashion for one reason or another. The “backwards design” of Wiggins and McTighe wrote about in the late 90s isn’t all that different than Ralph Tyler’s “statement of objectives” from the late 40s.

          When I was getting my degree, one of my professors was a woman who taught during the Eisenhower administration. She told us that we shouldn’t be worried whether or not her methods were outdated, because someone was probably writing a book about them called How to Educate Children in the 21st Century.

          standardized tests can be a useful tool. My classes have traditionally been on the low end of the totem pole. The school system now makes a stronger effort to serve these students effectively to increase the overall high school performance.

          Yeah. A while back NYS dropped the state assessment for 5th grade Social Studies, and over the subsequent years the funding to teach it was continually reduced. So when the state announced a few years ago that it was considering dropping the 8th grade assessment (over costs), organizations like LICSS immediately started lobbying to have it reinstated over funding concerns.

          • joe from Lowell

            The “backwards design” of Wiggins and McTighe

            Brilliant, brilliant book.

    • Joel Patterson

      Hi, Jim, some guy in Iowa!
      My advice is, if we are talking about you as a parent looking at your kids’ test scores, take a look at the work in your kid’s notebook or binder, especially any tests the teacher has graded. Can you follow the logic of the writing/mathematics? Do the sentences seem clear enough and sensible? Can the kid explain it to you?
      It’s nice to have a high number on a state test, but if the kid really is getting the class’ ideas, they should be able to have a conversation about it. You know some one has the idea if they can communicate it to you–that is the basic problem many kids have with standardized tests: the test is written in a way that does not clearly communicate the idea to them. That’s how the distractor answer choices lead them away.
      If the PSAT scores or ACT scores seem low, encourage the kid to get a good teacher recommendation because those scores aren’t everything in the choices of college admissions. Quite often, the kids who graduate with honors had lower SATs/ACTs than their fellow classmates.

    • James E. Powell

      I teach high school English in California. The multiple choice format can only test some parts of a comprehensive education in English language. The short essays that are part of some standardized tests mostly test the ability to write short essays on timed tests. The methods used to score well on such tests, the whole five-paragraph essay approach, are not the same thing as good writing.

      The problem isn’t standardized tests, but the widespread belief that nothing else matters. The test scores is almost the only thing anyone every talks about.

  • Johnny Sack

    I’ve always admired grifters. I gotta come up with some scam-could use some scratch.

    • Linnaeus

      Ah. Thanks to The Sopranos, I now know where you got your nym.

      • Johnny Sack

        Real name is John, from New York, and have a similar temperament. I thought it fit.

  • Porternator

    The logic is ‘See, we weren’t lying last year because look how much we suck this year!’? That is some grade-A turd polishing. And these people get paid to be representatives for their particular organization? Look out Jay Carney!

    On the flip side, 30% passing rate sounds like some people (not students/teachers, but administrators) fucked this exam up pretty bad.

    • Joel Patterson

      It’s all about the questions the test-writer chooses. Choose questions with lots of complex clauses and more kids get thrown off. Start off with a hard question, and kids get mired in that and never get to an easy one at the end.

  • Arne Duncan

    How am I going to pull down 7 figures after I leave DC if there aren’t any large charter school corporations to pay me? Really, use your heads people.

    • LosGatosCA

      Thanks for stating the obvious. Sometimes that’s necessary.

  • kingcatfish

    The NY standardized tests were totally ridiculous this past year. It’s like… the NY Department of Education wanted public schools to look bad or something! I bet we won’t see any closings of public schools due to low test scores and the subsequent opening of charter schools in their place in 2014, no sir.

  • cpinva

    bob somerby has been all over the charter school scam for years, and especially the michelle rhee scam, so none of this is at all surprising. the only legitimate reason for a charter school to exist, and siphon public school dollars, is if it is producing superior results, right from the start. if it isn’t, it should be shut down, the students placed back in their regular schools, and the funds reallocated to the those schools.

  • “Teacherken” at Kos, a diarist I tend to avoid because he’s a pompous windbag who enters everyone’s diaries simply in order to redirect attention to his own writings, has written what he thinks of as an important “open letter” to college professors about just how badly the “teach to the test” thing has hurt the students they will be getting in the fall. Its not very well written, to be honest, and not worth linking to but it is salutory reading–like a cry for help and an anathema from the bottom of a well. The guy has been teaching highschoolers AP history, 160 students at a time in all classes, and he points out that the way the highschool teachers are forced to teach the one thing you can be sure at the college level is that your new students will need tons of remediation.

    • This is not news to us college folk.

      It’s parents and taxpayers who need to know that the system as it’s been “reformed” over the last 15 years is deeply broken.

      • ChrisTS

        Hah! There you go, trying to pretend it isn’t all our fault at the college end. Admit it, you lefty intellectual, you took perfectly decent students and bent their fragile little minds.

    • Gregor Sansa

      Pompous windbags can be worth listening to if they have something worthwhile to say. It’s pretty easy for someone who thinks they’re better than Shakespeare to be both wrong and still worth reading.

    • James E. Powell

      Kind of harsh, no? Teacherken’s pompous windbaggery is about average for the internet.

      Anyway, for anyone interested, a copy of his open letter was published here.

      I do not know what kind of school teacherken has worked in. I work in the very low end schools in Los Angeles. The students are so far from where they need to be to succeed at the college level that talking about testing is just a distraction.

      • No, I’d say Teacherken’s pomposity is far higher than average on the internet. This diary, which links to his essay, is actually worth reading but he not only writes a diary every other day, he enters other people’s diaries to direct everyone’s attention to his diary. He’s a nice guy and on his area of specialization he has some interesting things to say, but I’ve been watching him do this for years at Kos and I mostly just avoid his comments and diaries at this point because he has nothing new or interesting to say and spends most of his time talking about himself.

    • ed

      145 words to damn someone as a windbag? Pot meet kettle.

      • Rigby Reardon

        I don’t really think that damning teacherken as a windbag was actually the point of her comment. You might want to work a bit on your reading comprehension.

  • Isn’t a possible explanation of variation in charter school scores due to smaller school sizes and regression to the mean? That cuts both ways, that charters posted higher-than-average scores in the past not from any substantive superiority, but rather because of their small size allowed a result far from the mean–then the next year, with a new set of students, the results plummet. See http://www.samefacts.com/2013/06/education-policy/why-the-best-performing-schools-are-small-schools-just-like-the-worst-performing-ones/. So the reformers take advantage of statistical outliers when the news is good, but when there’s a regression back (or below) the mean, it’s the system! cf. Kahneman’s story in Thinking Fast and Slow about the performance of the Israeli air force pilots. So we’ve been upending education in this country because we don’t understand statistics?

    • Matt McKeon

      Maybe if there was a standardized test for statistics this problem would be completely solved.

      Standardized high stakes tests, MCAS currently in Mass. do tend to flatten and distort the curriculum, as nontested subjects and activities are squeezed or reduced to make room for test prep.

  • It’s all about the data for education reformers, right?

    Uhhhhh ……. no.

    It’s all about the money.

    I think you are collecting the wrong data to measure their effectiveness.

    Let’s try it again with measuring the portion of educational funding that is going to charter/voucher, etc schools. Tells a different story of growth/improvement, doesn’t it?

    You keep focusing on educational outcomes, you silly twits, and us reformers will keep on cashing those bigger and bigger checks. Maybe someone will actually learn something from this experience (crime pays).

    • If there’s one lesson that folks have learned in America since the early 1990’s is that accountability is for suckers and that the only folks who have to obey the laws or display any conscience whatsoever are the little people.

      Corporations have led the way, Republicans have thrown their full support behind them, and unfortunately neoliberals have decided to join them (in a superficially reluctant way).

      Who knew that when normal people bemoaned their children sitting in front of a teevee set all day long not learning anything useful, predatory entrepreneurs would say to themselves – We can make millions doing the same thing and call it edjucashon!

  • The whole thing just breaks my heart. Children are so, so very smart and curious. And they love to play with each other. They just love it. And the way they are betrayed by society and by their schools (under pressure from these “reformers”) is just obscene. Education should cost a lot. Every child should have the full time attention of one teacher for every ten kids, plus dance, art, computers, sports, and library time. In addition to that every kid should have free tutoring after school, plus free breakfast and lunch and private field trips to museums and theater. And not only should that all cost a lot–because it costs a lot to have the highest caliber people and the best rooms and art supplies–but you have to get up and pay for it all again next year, too, for each new crop of students. This is not rocket science. You can’t do this on the cheap and nor should you try.

    • James E. Powell

      You’ve got an awful lot of ‘shoulds’ in there.

      The big one you don’t mention is that American citizens should care about the education of children other than their own. They may say that they do, but the overwhelming majority of them do not.

      • Actually, I think people are just as naive about the cost of great schools as they are about how much the US spends on foreign aid. The logic of all of this school reform shit is basically the same: how can we perform something like the educational function for the least amount of money–i.e. efficiently. Children don’t need efficiency. They need one on one education, attention, tutoring. If we totalled up all the money we need to spend on children actually getting the education they need, and subtracted it from all the stupid efficiency experts and their grift and tests, we’d do a hell of a lot better for probably the same money.

        • James E. Powell

          I agree with you. But you and I and the handful of people who feel the same way are not going to have any impact on policies.

          The schools where poor peoples’ children go are going to be handed over to corporations. There is no force, political or economic, that is even trying to stop this process.

          Diane Ravitch’s blog posts are not moving the ball. Here in Los Angeles, the best the teachers’ union can do is write sternly worded letters.

          I heard about some big victory by Chicago teachers that was supposed to represent a turning of the tide, but I then I heard that 2000 people were laid off.

      • LosGatosCA

        Prop 13, baby.

        • Lee Rudolph

          For Aimai and me, that would be Proposition 2-and-a-half, though I would guess that her location in the Commonwealth [God save it!] is less crazy about refusing ever, ever to override than mine is.

          • joe from Lowell

            The town I grew up in has never, ever approved an override.

            That’s embarrassing. It’s getting a reputation as the town with the cheap schools.

            I’m sure that $200 bucks a year in lower property taxes will be quite a comfort when it comes time to sell your house.

    • Kyle

      That’s what private schools are for, and the GW Bushes of the world get all of this and a lot more besides. But if everyone had this, what would stop the masses from outcompeting the idiot sons of privilege and challenging their inherited power?

      • JL

        In the neck of the woods that I grew up in, private schools were mostly for religiously indoctrinating children.

        Also for allowing parents to brag about how their kids were surely getting a better education, even while public school kids dominated academic competitions.

  • Ken

    Was there some event that caused the sudden drop in test scores? Like, the charter schools got past some probationary period and now the contracts are locked in for ten years?

    • Rcshowman

      New set of standards (the Common Core State Standards). Most states are giving a transitional year before they start giving year-end assessments centered on these new standards, but NY went ahead and rushed to throw a new test together, and started testing the standards that were brand new to both teachers and students over the last year. Whole lot of acclimating super-quick…with predictable results.

      To move this point on topic–the teachers most likely to suffer the most with a new set of standards that are more rigorous and demanding than the previous NY state standards? Yeah, newbie and TFA teachers.

      • Ni Hao Lao Wai

        This should be added to the post, pretty much EVERY school in NY cratered in test data from the reports I’ve read. This does at the very least show that charter schools aren’t any better than public schools- and the drops for KIPP and Harlem Children’s Academy should be front and center the next time someone tries to make that bullshit argument. IIRC they did worse than the public schools.

      • joe from Lowell

        NY went ahead and rushed to throw a new test together, and started testing the standards that were brand new to both teachers and students over the last year. Whole lot of acclimating super-quick…with predictable results.

        Uh-huh. So next year, when the schools have had some time to acclimate, we’ll be seeing headlines about the super-duper increases in test scores.

        • Rcshowman

          Easy, fella. That’s not what I said. New standards and the hurriedly-constructed assessments explain the cratering. The rigor of the standards means it’ll be a slow crawl out of that crater–no “super-duper increases.” And just as NY students get used to the state-created tests, they’ll migrate to a new test, either in 2014 (targeted for implementation) or 2015 (likely, since that assessment consortium has been very late on all other deadlines).

          • joe from Lowell

            No, it’s what I said.

            They set it up so the scores would inevitably crater this year, and inevitably rise next year, even if there hasn’t been any meaningful change in what kids are learning. Just a single year to get ready and actual teach to the right test will produce a bump in the scores, for exactly the same reason that rushing in the test this year produced a crater.

            The hurried testing sets them up to brag, next year, about their big improvements.

            • Rcshowman

              Apologies. I mistook your cynicism for sarcasm.

            • Joel Patterson

              This is the truth. It’s a bit like Bunny telling his officers to write up every infraction honestly so his numbers stink, then starting up Hamsterdam so his numbers rocket to the top. Everyone wins.
              Schools’ test scores “improve”
              And the newspapers who berated them get to feel like their public berating of the schools was what caused the “improvement.”

              Of course, it will not be clear if all this hullabaloo actually makes students’ education more meaningful or rigorous.

It is main inner container footer text