Thursday, June 25, 2009

How Math Must Assess: a Post-Mortem

I am writing this now, instead of getting the heck up out of here for the summer like I should be doing, because I don't want to forget anything. In fact, I probably should have written it sooner.

For the uninitiated, read this first. Dan lays down his system.

What I Kept
Each skill considered, numbered, recorded as a separate grade, and tested twice. Unlimited opportunities for remediation, but outside of class. I would typically re-do the test items with the student and troubleshoot her misunderstandings or more trivial errors. Then if she could prove that she knew what she was doing on a different problem without assistance, I changed her grade to 10/10. I still gave assessments at times most teachers would be giving a quiz or a test: typically the middle of a unit and the end of a unit.

What I Changed
I made each question a maximum of 5 points. Sometimes I made half-point deductions, like for rounding errors. After the first assessment of a skill, it was recorded as a grade out of 5 points in my gradebook. After the second assessment, I changed the entry to be out of 10 points. If the student scored a 5 the second time, I made her grade 10/10, no matter what she scored the first time. If not, I recorded the sum of the scores from the two attempts.

I didn't use a stamp. I put a small sticker on each question earning a 5 on the second attempt, and it was up to the student to transfer the sticker to his checklist. Major instructional time saver.

What Worked
I tried this with both Algebra 1 (9th grade) and Algebra 2/Trig (10th and 11th grade). It worked really well for the older students. Every time I was available after school, I could count on anywhere from 2 to 15 kids wanting to remediate something. I have no doubt that spending more time on difficult topics benefitted these kids' enduring retention; the results of their state regents exam (administered almost 2 weeks after classes ended) bear it out. I received an email this afternoon which pretty much made it all worth it. This is the complete text:
I PASSED???????????????????!????!????? with a seventy five?? not just a
64.5?????? i see the numbers on my gradebook but seeing is not always believing.

That girl worked her tail off all year because she had a tangible incentive. If she did well the first time, she wouldn't have to stay after school. And if she didn't, she could change her grade.

What Didn't Work
Dan promises students excited about tests... I didn't see that. They were happy that they weren't stuck with bad grades and felt empowered to do something about it, but I wasn't feeling a big "Yippee!" on test days.

Since we are required to publish our grades on mygradebook.com, which students and parents have access to, the concept checklists were mostly a flop. A handful of hyper-conscientious-types made good use of them, but students were more likely to go print out their scores before coming to remediation. I may do away with the checklists altogether next year. Not sure. I am not good at compelling students to keep a good notebook in general. I need to work on that.

Sometimes my grades felt a little inflated. I'm ok with that.

I think it worked better for older students because they have the maturity to take ownership of their grades and learning, they are invested in their performance because they are starting to worry about college applications, and they follow through by monitoring mygradebook.com and coming to remediation. In practice, for me, the system relies at least partially on student initiative and how much he is motivated by grades. I did not have as much success with the freshmen, as much as I tried to sell what a good deal this was. They are much less likely to stay after school in general, and as far as grades go, they just want to do well enough to keep their parents off their backs. I'm not sure what changes I could make to do better there.

Conclusion
It's not perfect yet, but I'm sold on the idea. Much gratitude to Dan for coming up with and publicizing a better way. I thank you and my kids do too.

Documents
Algebra 2/Trig Concept List 08-09
(I can add to this, just ask.)

19 comments:

unapologetic said...

I think it worked better for older students because they have the maturity to take ownership of their grades and learning

I wish I had classes with more than a handful of students like this. Maybe once the application pressure is off they regress to the mean.

Glenn said...

Kate,
Can you post more about your skills and breakdown for the Alg 2/ Trig class? I gave some serious thought last year about doing this, but ran into mental roadblocks as to the implementation / breakdown.

Thanks!

Kate Nowak said...

Here you go Glenn.

Alison B said...

Kate, thank you so much for including your concept list in these comments. I am hoping to implement this system next year, and have been haunted by the idea that I was doing something horribly wrong because I had so many concepts. If your breakdown is this long, then so can mine be.
--A soon-to-be-second-year teacher

P.S. I love your blog and have been lurking for ages. I think the "voice" you project to your students is perfect and have tried to mimic it on occasion. I hope you're not copyrighting "Check Yo'Self".

Kate Nowak said...

If anything, "check yo'self" should be copyrighted by Sam Shah, he's way cooler than me.

And this concept list could totally be whittled, FO SHIZZ.

Glenn said...

Kate,
Thank you very much!
Glenn

coxmathblog said...

Kate
I really think that this system has a chance to work really well. However, do you think that you would be able to get more results if you were able to provide time for remediation during the regular class period? I know that most classes run anywhere from 45-60 minutes and there already doesn't seem to be enough time simply cover all the curriculum. But, how much time is wasted going over homework that many students don't even do? Could those minutes be banked and then used once per week to allow for tutoring and/or reassessment? Otherwise, how can this system that depends on kids coming in on their own time work for athletes or kids who have other major time constraints outside the school day (ie. family obligations, work, etc.)?

It seems that there has to be a way to make this work inside our alloted time?

Dan said...

In practice, for me, the system relies at least partially on student initiative and how much he is motivated by grades.

Agreed. At various moments in my year, I would actively pursue failing students, hectoring them to come in and remediate but I consider it a feature of the system that I can downscale my involvement as my dwindling sanity demands it. Some days I'll have some kids lined up for retakes and I'll just send 'em off. "Gotta spend this lunch period, y'know, eating." But at least there's a mechanism for student remediation that goes beyond unit-test retakes and (god forbid) some mealy extra-credit assignment.

Kate Nowak said...

Dave - our class periods are 43 minutes long. I don't actually spend much time going over homework. I project the solutions while I'm walking around checking for completion. Meantime the kids are usually asking people around them if they missed some and don't know why. Then I say, "Any Questions?" and go over what they ask about - usually it's only one or two. If nobody asks, I go over one or two anyway. So the whole thing takes maybe 5 minutes.

If I didn't go over homework or assign it in the first place, I wouldn't be giving them the frequent practice, feedback and opportunity to clear up misunderstandings that is the point of homework.

Miss Cal.Q.L8 said...

Kate,
You know I'm a newbie so I still need some help understanding how to do this. How do I assess each concept twice? I'm kinda confused. Going on what Dan says, if I assessed concepts 1, 2, 3 one week and then the next week concepts 2,3, 4 well, then aren't I teaching the same thing two weeks in a row. lol This probably sounds so dumb. But can you explain or give me an example of how lesson planning works with this type of assessment?

Kate Nowak said...

Sure thing - good question. The way assessment normally goes is like maybe

Teach 123
Quiz 123
Teach 456
Test 123456, end of unit

Teach 789 etc etc repeat

My assessments have a bit more of a rolling feel. Even though we're done with the lessons for a unit doesn't mean we're done being tested on it. My classes look more like this:

Teach 123
Quiz 123
Teach 456
Quiz 123456
Teach 789
Quiz 456789
Teach 10,11,12
Quiz 789,10,11,12
etc etc

Except maybe 80% of the class did miserably on #2, so I might adjust:

Teach 123
Quiz 123
Teach 456 & 2 another way
Quiz 123456

I was also known a few times to put a test item on a THIRD time, if many students needed remediation. Completing the third appearance of an item was optional if you already got 8/10 or better, and well-publicized in advance so we could take steps to prepare.

I hope that helps.

Miss Cal.Q.L8 said...

Thanks, that does help. How did you break your concepts down? I'm trying to go by the book but the order of it makes no sense in some places. But looking at Dan's list, it doesn't make sense to me either. I want to plan it the way that makes sense to me to teach it but I don't know how to do it. Overwhelmed again.

Kate Nowak said...

I got a copy of the unit calendars and tests from a few different colleagues for each course I taught. I used these documents to list out a breakdown of skills. Then, I made adjustments in a few areas that didn't make sense to me.

I was lucky that it's long been common practice in my department to use unit calendars - just a sheet of paper for each unit showing the topic and assignment for each day.

If all you have to go by is the book, at least it will give you a place from which to start.

Teaching Goddess said...

I've used the concept test idea before and I liked it to a certain extent. It seemed like it took me forever to grade exams, but I like the idea of limiting the number of concepts tested. I also like the idea of only putting any concept on the test twice. I gave my students 3 chances to get a perfect score. The other thing I liked was that there really wasn't a need for makeup exams when students missed class.

I think I didn't go back to it because I felt that it was so time consuming. I might try again next year though. Thanks!

H. said...

You did this with 118 test items?! And with this being only one of four courses? The sheer volume of work you've done to pull this off is astounding - and inspiring. Enjoy a very well-deserved break!

glynna said...

Couple of questions:

1) How many days do you stay after school and for how long?

2) So if a student earned a 5 the first time around and then earned a 1 the second time around, they earned a 6/10? Or did you let them opt out of the question the second time around because they earned a perfect score on the first round?

3) Was the first round a "B" level question and the second round an "A" level question, as in Dan's system?

4) If one student earned poor grades on both rounds, how did you deal with that?

I'm sure I have a few more questions, but I'll ask them later.

Kate Nowak said...

1) How many days do you stay after school and for how long?

I'm typically available 3 or 4 days a week for an hour. Our last instructional class ends at 2:06, and teachers are required to stay until 3 anyway. So I'm available unless I have a meeting or other commitment in school.

I also have a duty assignment every day for one period in our Learning Support Center. So kids who had lunch or study hall that period would often visit me there.

2) So if a student earned a 5 the first time around and then earned a 1 the second time around, they earned a 6/10? Or did you let them opt out of the question the second time around because they earned a perfect score on the first round?

In that scenario they would get a 6/10. I think it's important that they can demonstrate understanding more than once.

3) Was the first round a "B" level question and the second round an "A" level question, as in Dan's system?

Often I would make the first question a little easier and the second question a little harder. But I didn't codify all my questions as A and B.

4) If one student earned poor grades on both rounds, how did you deal with that?

If the problem was widespread, I would do more work on the concept in class and put the question on a test a third time. If it was one student, I would encourage him/her to come in for remediation outside of class.

Tracie said...

Wow. This is what I want to do. I teach science, so I don’t have skills so much as concepts. How would I assess that? Essay questions? Multiple choice doesn’t seem right. Unless you have five different questions over the concept? We could do some practical assessment. Have you tried anything like that?

Kate Nowak said...

Hi Tracie - I don't think I can answer this. I have pretty bad myopia for teaching math and not much else. Math is more concepts than skills, too, ideally, but the skills are an avenue to the concepts, if that makes sense. I know science has its own challenges - more vocabulary, etc. How do you normally assess understanding? I always see the science teachers running Scantrons through the machine in the staff room, so I assume multiple choice is common with them, at least at my school.