Hello, reader! If you intend to post a link to this blog on Twitter, be aware that for utterly mysterious reasons, Twitter thinks this blog is spam, and will prevent you from linking to it. Here's a workaround: change the .com in the address to .ca. I call it the "Maple Leaf Loophole." And thanks for sharing!

Saturday, April 6, 2013

The Tests Matter

Here is what is going on right now, in the time before the Common Core Standards have really hit high schools, and before a common assessment has been inflicted on any live children. The non-teachers in education are going: "Just start teaching the right way. Pay no attention to the tests. If you teach right, you don't have to worry about the tests. The tests will take care of themselves." The teachers are saying: "The way I teach is basically fine, anyway, so I'll make whatever adjustments I need to make once I see what they want kids to do on these new tests." I know there are probably some teachers changing their practice, and some non-teachers with half an eye on assessment. I'm painting with a broad brush. Go with it.

This is what I am afraid of: the thing that happened in New York State, starting in 1999. That's when NY changed from Course1/2/3: a decontextualized, integrated curriculum with very predictable though rigorous exams that were none of them a graduation requirement... to Math A/B, standards with more focus on applications and much less predictable tests -- also, kids had to pass the Math A exam to graduate. (This was a huge deal. Regents exams had traditionally been taken by your college-bound academically-oriented students, and suddenly everybody had to take one of them.) The new requirements were supposed to make things tougher, with all the rhetoric that comes with such changes. June 1998:
Yesterday, officials at New York City public schools welcomed the tougher tests, while some education advocates worried about the lack of resources to train teachers to teach for the higher standards.
If it sounds familiar, that's because it's straight from whatever school-reform-article-generating-machine the news has been using for thirty years. Moving on.

Some shit started hitting some fans. October 2000:

Mr. Mills said middle schools ''need to rethink what they are doing'' and quickly figure out how to teach students the skills they need to meet the new standards. He said he had no intention of backing down on the standards, which as of last June required every high school student to pass an English Regents exam to graduate, and by next June will require every high school student to pass a Regents math exam as well.
People started freaking out when they realized that requiring a passing score on an algebra test was going to be a graduation-rate debacle:
Students in the next class, which entered in fall 1997, will have to pass both the English and Math Regents to get their high school diplomas. If the results hold steady, about a quarter of this year's seniors will not be allowed to graduate.
There were protests (May 2001). There were districts trying to opt out (Nov 2001). 

I don't know what happened to all the kids in the early 2000's who were denied a diploma because they couldn't pass the Math A Exam. A bunch of heartbreaking shit, I'm sure. 

In June 2003, there was TESTMAGEDDON. The Math A Regents exam was the straw that broke New York's resolve
Though many districts have not finished tabulating their scores, superintendents, principals and math department heads are reporting preliminary results that some described yesterday as ''abysmal,'' ''disastrous'' and ''outrageous.''
It was not a good test. Post-Course 1/2/3 exams were not good tests, generally: problems that didn't make sense, weird, contrived contexts, a fetishization of goofy vocabulary and notation. Too much content was a huge problem. A test that didn't know whether it was an algebra or geometry test was a huge problem. A test that didn't know what it was measuring -- readiness for higher mathematics courses? Basic skills that should be expected of every graduate? -- was a huge problem. In the end, the test measured nothing but whether or not a kid had passed that test. The accountability movement compelled schools with lower scores to make their math courses all about passing the test. Math A became a de facto curriculum, and a horrible one. 

NY tried to raise the bar. Then, a whole mess of kids ran head-first into the bar and fell on their asses. Then, instead of re-evaluating any of their faulty premises, NY responded by lowering the bar.

On the June 2003 exam, they relented and lowered the cut score

Then, they eased up on subsequent tests

New York State's education commissioner, Richard P. Mills, said Wednesday that the state would loosen the demanding testing requirements it has imposed for high school graduation in recent years, including the standards used to judge math proficiency.
They made the tests easier. Lots easier. Also, the thing happened that took all the respectability out of the historically respected  regents exams: for the tests required for graduation, the score you needed to pass got dramatically lower. They said it was a 65, but after June 2003, you only needed a raw score of around 42% to pass the Math A with a scaled score of 65. (The raw scores in the linked table are not percentages -- they are out of 84 points.)

I wasn't around when this all happened. I didn't start teaching until 2005. And I don't think we're getting exactly a repeat with the Common Core. For one, there does seem to be a coordinated, genuine effort to support teachers in changing their practice, independent of testing. For two, there's a coherence and focus in the CCSS that New York was sorely lacking. But also, there's the whole added wrinkle that tests are trying to fulfill still another purpose: teacher evaluation. The disaster story might not be "so many kids can't graduate", it might be "so many teachers are being rated poorly, even good ones that kids, parents, colleagues respect."

But I still think it serves as a cautionary tale, and I'm still curious about how this is going to play out once the new tests hit a computer lab near you. If they really measure the stated goals of the new standards, they're going to be very different. Because of that, they're going to be perceived as too hard. How the test-writing consortia, DoE, states, districts, etc react to that is going to be really interesting.


  1. I taught math in NYC from 1999-2003. We gave that infamous Math A Regents exam in June 2003, and I remember hearing from my old colleagues later that summer about the aftermath. My first thought was, "boy, now that I'm teaching in NJ I'm so glad I don't have to deal with that stupidity any more!".

    How naive I was then. Because everything that happened in NYS in the early 2000s is going to happen again in the spring of 2015, when the first PARCC Algebra 1 tests roll off the assembly line. My predictions are-
    Year 1: bad scores, lots of teachers with low ratings, lots of students with "below basic" levels of "proficiency", etc. Many teachers leave the profession.
    Year 2: scores go up (slightly), in part b/c teachers are teaching to the test (please folks, can we agree that "standards" = "curriculum" in this country, even though they shouldn't?), and in part b/c the passing scores are lowered slightly. More teachers leave the field.

    By year 5 or so, there are 2 populations of math teachers, by and large: 1) TFA-types, who are only in the job for a year or two before heading Wall Streetward, and 2) those close enough to retiring to hang on until they can earn what still remains of their pensions. (As an aside, I'm not in either of those populations. What that says I don't know.)

    May you live in interesting times, as they say...

  2. Thanks for your perspective, Mike. I was reluctant to make predictions, as some of the negative possible outcomes, like the one you describe, are so grim.

    One additional exacerbating factor: New York had a disaster on their hands when they wanted every kid to show proficiency with some Algebra 1 and Geometry. The CCSSM for high school has every kid completing some Algebra 2 and some trig.

    Interesting times indeed.

  3. >problems that didn't make sense, weird, contrived contexts, a fetishization of goofy vocabulary and notation.

    That's exactly what PARCC and SBAC are promising as evidenced by their sample tasks:

    NYSED's samples are full of oddities, too, like a question with negative numbers in fifth grade, where Common Core squarely places that in Grade 6.

  4. I don't know how things will work in other states, but in our state (Arizona) rather than the PARCC being a high-stakes graduation test (as our previous AIMS had been) it is going to be incorporated into the classes (Algebra I, Geometry, Algebra II) themselves. A set percent (I think that is still getting wrangled on, but something like 20%) of the class grade is mandated to be from the PARCC. So it's being treated like a "common final".

    Because NCLB got the axe I think states can technically do whatever they want.

    Mind you it's still hard to know what the thing will even look like. There are some places on the Internet which maintain it will absolutely be a computer-based test, but I have one source who says there will be a paper version.

  5. I may not be able to fit all I want to say here, but here goes...

    I went to high school in New York when the Regents exams were still respected. In fact, the Regents exam was part of my grade for most of my high school classes, usually counting as a midterm or final. I believe about 80% of students at my high school took Regents exams, and we all (even the slackers) studied for them. We bought old question banks the state published, we made notes, we quizzed each other. I thought they were hard, and I was a good student. We walked into those tests feeling as though we could prepare, even if we didn't feel fully prepared, if you know what I mean.

    There were some differences with the way testing goes today. Our teachers submitted potential questions to the Regents folks so that the tests reflected what was being taught by teachers across the state. And right after the exam there was always a week or so when teachers would protest various questions for various reasons. Sometimes they won and a question got thrown out. No one used the tests to rate teachers, though my high school liked to brag about our passing percentage. Our score was a straight percentage. And if we didn't pass, it was summer school.

    The school I work in now doesn't even have summer school. It has credit recovery for high school students. It's not the same thing as repeating. It's seat time and an end-of-course assessment that only counts in our district. The ECAs are tough, but they aren't the same as the old Regents exams.

    Teachers in my state (I'm not in New York anymore) don't get to submit questions to the state tests. No one publishes old copies of the test, because that would be cheating (and the companies making the tests can't reuse questions that way). Students walk in feeling like they can't really study for standardized tests.

    So, I don't have anything against standardized testing because I feel like those tests made me a more focused student. I feel it can be done well, or at least adequately. But the big business that testing has become makes my stomach turn.

  6. Texas just entered the "raise the bar and quickly lower the bar" phase. In the late 90s, students took basic level reading, science, and math exams in 10th grade as a requirement for graduation. They were stupid easy. In the 2000s, they introduced tests at the 9th, 10th, and 11th grade levels with the 11th grade tests required for graduation. After an adjustment period these weren't so bad. Most students taking the 11th grade math test were in a math class above the standards addressed on the test and had seen all the material for quite some time. Even in underprivileged populations the number of students held up by these tests was not huge.

    Then came STAAR. The idea was way more tests (15 all told before graduation), subject specific (from generic math to Algebra I, Geometry, Algebra II), passing required to move on to the next course (for example failing the Geometry test would require you to take Geometry again AND your next course at the same time, regardless of your grade from the coursework), it counts as 15% of the final grade, and if you failed it, you were required to retest until you passed. Teachers were given little to no expectations as to what to expect from these things. My district drafted benchmark exams with only insight from the published standards (which are nice and vague), no sample test questions were released.

    Naturally, it was a disaster. Scores were horrible (the English tests were a blood soaked battlefield of sadness), passing rates were set incredibly low (Geometry was in the 30-35% range), and 75% of Texas high schools were deemed "unacceptable" according to the new standards.

    The response? Proposals in the legislature to reduce the number of tests from 15 to 5, decreasing the number of core credits to graduate (we were at 4 social studies, 4 science, 4 math, all suggestions drop this to 3/3/3) and the 15% thing got scrapped before they even tried to see what the effect would be.

    I have no idea what they think they've accomplished. None of it showed a focus on better teaching methods.

  7. Interestingly Engage NY and its curriculum has caught the eye of the elementary school teachers in my district(Berkeley, CA). They believe (I haven't done my own research to know if this is true) that NY is way ahead of the nation in CCSS implementation and teacher training. Is this your opinion? Folks from Engage NY were in Berkeley last week training teacher on A Story of Units and the district is awaiting the development of A Story of Ratios with high hopes to adopt it for our middle schools. Is this good news for us?

  8. I have heard good things about Engage NY, and there is certainly alot there. But I haven't spent the time to evaluate it for quality. I am a fan of the progressions documents published by Illustrative Mathematics - they are an important step in translating standards (which really only say what kids should know when they are done) to a more comprehensive picture of what the mathematical story will look like as it unfolds.


Hi! I will have to approve this before it shows up. Cuz yo those spammers are crafty like ice is cold.