Monday, August 30, 2010

Hot For Teacher Testing

When I decided to leave consulting and become a teacher (note to self: in a lifetime full of errors, bevues, and regrets, this had to be one of the real keepers...) Oregon, and the nation, were in the midst of the fervor for "high-stakes testing".

Part of this was the whole Bush "No Child Left Behind" thing, but a lot of it goes back to the "A Nation At Risk" kid-brain-missile-gap hysteria and the usual need for the usual suspects to Do Something About It, or at least to be visibly seen Doing Something.In our case we had an assessment at 8th Grade, and then another in 10th, called the CIM or "Certificate of Initial Mastery". The theory was that in 12th grade the kiddos would take a CAM or "Certificate of Advanced Mastery" - what my father and mother had taken in high school in New York state in the 1940s and had been called a Regent's Exam.

The idea was the same; to use the test to certify that the kids had learned their lessons, at least the ones the state felt they needed to learn.

Well lots of schools torqued their entire curriculums around to fit this damn thing. For example, because the 10th graders had to take a History CIM the entire freshman year social science was taken up with something called "World Studies A" and "WS-B". And because the CIM started with questions about the Industrial Revolution, we started in September with the Industrial Revolution. And from there on to WW1. And from there on to the Russian Revolution. Why Russia? Who the hell knows? Especially since in 2002 teaching the Russian Revolution was like teaching buggy-whip making. The damn ramshackle Soviet ediface had just come tumbling down - who the hell cared about the Aurora and Kerensky and the Five Year Plan and Lenin.

Though my students loved the hell out of Boney M's "Rasputin":

"Rah Rah Rasputin
Lover of the Russian queen
There was a cat that really was gone
Rah Rah Rasputin
Russia's greatest love machine
It was a shame how he carried on..."


Anyway, have you ever tried to teach history to 14-year-olds? It's like teaching sudoku to a cat. You will only frustrate yourself and the cat doesn't give a shit. Not surprisingly, for most of the kids WS-A and -B were a total wash.

Except maybe for Rasputin. They liked Rasputin.

But whether the little buggers learned history or not wasn't the point. They HAD to get the history fed them in 9th grade so they'd be ready for the Big Dance in 10th. God forbid that a teacher, or a school, or a district, flushed the CIM. You might never get funded for so much as a new Habitrail for the biology classroom again...

But I always looked at it this way; if you make me take a test and my welfare depends on it, I will likely try and do well on the test, or at least as well as I can. If my paycheck, or my standing, or my future employment rests on doing well on the test, I will bust my hump to make that happen.

But if YOUR welfare depends on MY performance?


You'd better be a pretty sweet pal, or have some sort of serious threat to hold over me. Because otherwise, Giacomo?

You could kiss my ass. Why should I bust my butt for you?

So when I read that there was a study that concluded that "There is also little or no evidence for the claim that teachers will be more motivated to improve student learning if teachers are evaluated or monetarily rewarded for student test score gains.", among other things, I had to laugh. This was a surprise?

Teaching is an odd thing. There is tremendous power there. Forget the "classroom discipline", the power game between adolescent and adult that characterizes a hell of a lot of American high school classrooms. No, it's the tension in the relationship between the teacher as possessor of knowledge and the student as seeker of knowledge as old as Plato. The student has to learn as much as the teacher has to teach. The failure of one is the failure of the teacher's power is shot through with faults and weakness.

And it's not really a science and its not really an art. If anything, teaching is a sort of craft, where you learn people like a carpenter learns wood, feeling the grain of them, searching for the places where the gouge will pare away smoothly, where it will bind and crack.

So when you reduce teaching and learning into the kind of test you complete by filling in little ovals with a number 2 pencil...well...let the researchers explain it: "As we show in what follows, research and experience indicate that approaches to teacher evaluation that rely heavily on test scores can lead to narrowing and over-simplifying the curriculum...provide disincentives for teachers to take
on the neediest students (and) also create disincentives for teacher collaboration."

Or, as one of the commentors on this study said over at Crooked Timber: " seems like it would make kids, particularly difficult to teach kids, my adversaries in a sense. If they do not improve, then I get fired. It would be so hard to keep my eye on a student’s well-being in that context and not see them as little performers who hold the key to my future. If I suspect them of being unable to help being underperformers, there’s the risk I would start to resent them."



And y'know what?

I could have told you that twenty years ago, and saved a lot of money I spent on getting a teaching certificate. Because when I was an Army sergeant part of my evaluation was a graded exercise called an ARTEP. Several months before the ARTEP I would gather my squad for a friendly talk.

“We’re about to do this graded field problem” I would explain. “We will be graded as a squad but the grade will only reflect on me. The graders will not listen to my explanation of how many of you are gimps, wheezers, chronic self-abusers, morons, gomers, mouth-breathers, learning disabled products of the union between a Marine and a gorilla, the offspring being, of course, a retarded gorilla. They will not believe that the reason we fucked up were because you oxygen thieves were unable to learn. They will blame it on my being unable to teach you.

Therefore, I will carefully explain everything we will do. I will show you how to do it. I will coach you through it. You will then do it for yourselves, with my direct supervision and correction. Finally we will do it at combat speed.

After that you have my personal assurance that any subsequent failure on your part, however small, will result in your horrible lingering death, probably involving a red-hot poker and one or more of your bodily orifices, or a transfer to a posting on the Korean DMZ, whichever you fear more.”

Now I never failed an ARTEP. But this is, in effect, what high-stakes testing will do for teachers and students; make the student fuckups the teachers' problem.

Mistakes? Mistakes are good. We learn from mistakes. But fuckups? As a teacher I can deal with fuckups.But the fuckups won't like it, and neither will I.

This doesn't seem like a good way to teach, or a good way to learn.

But I have to tell you; I'm not sure if a lot of people really understand how to do these either, and that will be the subject of some of the next posts.

Oh, and just as a note?

The Oregon Department of Public Instruction never fully implemented the CAM and abandoned the CIM several years ago.

But CIM or no CIM I wonder if my students still remember Rasputin?

"Rah Rah Rasputin
Lover of the Russian queen
They put some poison into his wine
Rah Rah Rasputin
Russia's greatest love machine
He drank it all and he said "I feel fine"Oh, those Russians..!


Ael said...

I am a firm believer in metrics.
In order to make good decisions, you need some reliable way to know what is working and what isn't. Then do more of what works and less of what doesn't. Note that "what works" is not the same thing as the "measure of what works".

However, if you tie rewards to the metric itself, then all sorts of bad things happen. People optimize the metric and ignore everything else. That which brings down the metric is dealt with ruthlessly and when possible, people will lie about and cheat the metric itsself

Unless you are lucky enough to be in a situation where the metric *is* what you need to optimize (eg. final score at a sports event), you now are worse off than before. (i.e. you still don't know what works and what doesn't and now people are lying to you).

The solution is in breaking the link between metric and reward. This is harder to do than you would think.

FDChief said...

Ael: And in this case it's even more problematic, since the metric is based on the performance of Group A but the reward (or, more typically, the punishment) is assessed against Person B.

I see this like the problems that sports (since you introduced the metaphor) with doping. So long as the pressure is on and there is a chance of not being caught, athletes will dope. Their livelihood depends on it - they'd be fools not to! If your competitor is cheating and won't get caught and you're not, what's your reward for finishing 25th?

So of COURSE the teachers are going to lie, cheat and steal here. They know the sort of material they are working with; they will do anything to shove the 25% of their class who are the gomers and total fuckups either out the door or through the cheating hole to save their jobs.

My larger point is that I think we're rapidly approaching the point of diminishing returns in industrial education. That is, we've developed a system in which:

1. The top 25% - which would have excelled anyway - will excel.
2. The middle lump, say 50% or so, will struggle through, some more, some less. A great system combined with great parental and social support will succeed with, say, 48 of this 50%. A shitty system and absent or crappy parenting and no societal pressure will struggle to get half of this half over the bar.
3. The bottom 25% are just not physically, mentally, and emotionally designed for classroom-type learning. A handful will succeed, more in the better systems, but even then many will not "get it".

The functional reality is that most people can walk a mile an hour. About half of us can walk a mile in 15-20 minutes. A decent runner, say 10-15% of the population, can run an 8-minute mile. Only a handful can run a mile in less than four minutes.

We're now educating a hell of a lot more people than almost anytime in history. I'm going to suggest in a later post that what we now perceive as "failing" - the statistic that 15% of our 10th grades can't read well - in in fact the hard edge of the physiomental reality that 15% of humans JUST DON'T READ WELL.

And we're in a tough place, because in a technic society, you have to be able to read and figure and write.

And I'm not sure what we can do about that.

Lisa said...


I do agree that teaching is a craft, and the art of apprenticing is not so prevalent today. Sure, there is the few weeks of "observation" tacked onto the end of an education program, but generally, you're sent out there with rules and metrics.

I think NCLB has been a colossal failure. The stress levels of both students and teachers is high. As you mention, it promotes rote and cursory teaching.

Yet ... how to have an excellent education system, or at least, a functional one? Part relies on good teaching (a craft/art reliant upon personal excellence and mastery), part the student's willingness/preparedness. Both of those all-too-human components are being altered daily via environmental/technological impacts.

I don't know how you optimize the teacher or the student pool, but you can't have one succeed without the other and say learning has occurred.

FDChief said...

Lisa: I think that most teachers know what would work to get the student success rate to about 90% or so, and what it would take to measure that success. The problem is that it would cost the heavens and the earth.

What you'd need is almost a presonal teacher for every two or three kids in that bottom 25% group. For some, it would have to be one-on-one. And you'd have to have a very individualized assessment method, ranging from paper tests to verbal viva voce exams to hands-on PEs. Like I say, it would be spendy in a way that would make most politicians - for all they like to talk a good game about not leaving any children behind - collapse in a heap of steaming funk.

So my belief is that, for all we like to vent and fulminate, that bottom 25% will ALWAYS be "left behind", to some extent. How far will depend on the available cash, the public/elite desire for this group to succeed, and the economic tenor of the times.

Lisa said...

(As someone who writes, "The stress levels of both students and teachers is high", I hardly seem qualified to speak on educational matters ... but that was due to a bit of drink, to which I am unaccustomed, ahem ...)

While I do agree with you that, "bottom 25% will ALWAYS be 'left behind', to some extent", it is not this group that causes me the most concern. It is rather our commitment to leveling in the school system which is most disturbing.

We know the dropout rates are skewed to the high- and low-achieving ends of the spectrum. Just as we cannot adequately address the bottom tier, so is it with the upper, in the average classroom.

Sure, pupil/teacher ratio must be reduced, but what of the quality of that educator and that student? You were an excellent instructor, but not all are. Mediocrity is propagated by the system, especially with the mainlining of the underachieving populations.

The middle 50-70% will reliably continue in their middling ways, because is what the system does best.

FDChief said...

Lisa: The highly intelligent, well-supported, and well-prepared students will almost always find muddle through. There is always private school. They will be bored, and a tiny handful will, largely though personal problems that might have caught up with them even in a terrific system, self-destruct. I'm not happy with what public education does with them, but it's more a question of benign neglect than the way the bottom quarter gets hammered.

The problem with teaching is a lot like the problem with learning. Just as with any craft, the number of truly brilliant teachers is going to be in a tiny handful. There's a reason that the great jewelers, cabinetmakers, glassblowers, and goldsmiths are considered artists and treasured as such; there's not a whole lot of them out there.

But industrial education requires a hell of a lot of workers. So you get a lot of people who are, frankly, not really very good craftsmen. They do fine when the timber is straight-grained and the gouges are sharp. But given a blunt instrument and a knotty problem? Ugly.

It's a tough situation. To craft every student to the kind of level that the best schooling used to bring the elite that was once the only significant group of educated people is nearly impossibly expensive. So any mass-education scheme is going to be the intellectual equivalent of a mass-produced cheap pine dresser compared to a hand-crafted Chippendale chest. Which means a lot of mediocrity.

And look at my own situation. I went from a job where I had a genuine professional standing, working with peers who respected me, for a fairly decent middle-class salary to working alone amid 150 hormone-addled adolescents who largely could have cared less for about 3/5th of what I made before.

Add to that the very real possibility that my professional reputation and even employment depended on the whim of those same adolescents?

Is it any wonder I went back to consulting?

Lisa said...

But what of the highly intelligent, not well-supported, not well-prepared? Their situation is as dire as the lower tier, IMHO.

Too true: "The problem with teaching is a lot like the problem with learning. Just as with any craft, the number of truly brilliant teachers is going to be in a tiny handful."

Further, "To craft every student to the kind of level that the best schooling used to bring the elite that was once the only significant group of educated people is nearly impossibly expensive." It IS impossible. Not everyone is cut out for that echelon, nor was our p.s. school system ever intended to achieve it.

I think it is unfortunate we removed technological/clerical tracks from the system. Everyone now believes they must do college, and as you have seen, that means some unfortunates pay up to five times in the CC in an attempt to pass a core class. Tragic.

Public school was only intended by its elite Founders to cultivate a working class that could read manuals and so not lop off their thumbs. I don't know that we have incurred any revolution in thinking or approach since then.

It is no wonder you left, as do so many others.

Lisa said...

FYI -- see Friedman today on flagging student motivation: