What can you say about the President of the United States?

[LKD]

We have these tests here in Alberta. They are called Provincial Achievment Tests for grades 3, 6, 9 and in Grade 12 the final exams are known as Diploma exams.

As was mentioned, what can one test measure? Multiple choice is good and all but there's some things it can't measure. Essay tests measure other things but they also can't measure everything, and they take a HELLUVA long time to mark with anything close to reliability, fairness, and accuracy. How do you use these sorts of tests to measure speaking skills? Presentation skills? It gets dicey.

Plus, many factors outside the teachers' control can come into play. The socioeconomic and cultural makeup of the school and class are huge factors in student success.

The Diploma exams here are mandatory for core academic classes. They count for 50% of the students' final grades. Let me tell you, quite often teachers end up "teaching to the test", not in a dishonest way, but we spend a lot of time looking at released exams from the past and working toward having our students be able to tackle the kinds of tasks that have been thrown at students before.

In a nutshell, it's not that easy, and professionals (and parents!) have been struggling for decades if not centuries trying to determine the best way to instruct and evaluate students. The debate isn't going to end anytime soon.

[Blackthorne TA]

I see; so in the meantime it's best just not to make any attempt at measurement.

No method of performance measurement in any field is perfect, but just about everyone outside the pubilc sector has figured out something, and realizes something is better than nothing.

[LKD]

Le sigh. There are measurements in place -- teacher reviews and such by administration. But they are usually only invoked if the administration is out to hose the teacher over. It's a method of laying a paper trail to intimidate the teacher into leaving. It's quite subjective. But there are efforts to evaluate teachers, and I have to concede some of them are done in good faith.

[Blackthorne TA]

Sorry, I'm always thinking of California specifically since our schools are horrendous yet the teacher's unions will not agree to any proposed objective performance measurement method.

[joacqin]

I am horribly intrigued that me and LKD who live in completely different parts of the world who hold WILDLY different views on just about everything can have such very similar experiences and complaints about the teaching job. It leads me to believe that we are maybe not barking up the wrong tree...

[damedog]

As someone who has just recently come out of school I find it hard to believe you could evaluate the quality of a teacher through a test. My two favorite teachers, and who I thought were the best in the school, didn't always teach by the book or what was strictly in the curriculum. They made you interested and made you think, and that has to count for something. One of them was very friendly to his students and motivated even the most lazy person to participate. There are things you just can't measure in a teacher and it isn't fair to penalize them just because some other teacher sticks to the basics just so their students will do good on a test as opposed to actually learning and caring.

[Aldeth the Foppish Idiot]

Originally Posted by joacqin

pretty soon the teachers will know what is on the test and unless the tests are huge we would quite easily be able to focus solely on teaching for the test.

+1

Geez, I thought BTA was an American citizen. We did EXACTLY what BTA suggests in the US - giving a test at the beginning and end of the year to assess improvement. It's the primary means of measure in "No Child Left Behind". Remember that?

The problem is exactly what joacqin points out - teachers then teach to the test. Students actually learn LESS than before, because subjects not covered in the test are given short shrift during the year.

NCLB tested Reading, Math, and Science. So schools geared their curriculum to those three subjects, and greatly reduced the amount of time spent teaching things like history, social studies, spelling, composition writing, and health, to name just a few. If it's not on the test, it's not taught in the classroom. While I'll be the first to admit that I think Reading Comprehension, Math, and Science are very important, all those other subjects being "left behind" actually reduces what the child learns during the year.

[The Great Snook]

I'm curious as to why "teaching to the test" is so horrible. Doesn't it make sense that the "test" covers the curriculum that the students are supposed to know? Is the concern that electives will get shorted as people focus on the "test"? If so, I'm OK with that. Electives by definition are optional. If someone wants to learn how to draw, they can join an after school art class.

[Aldeth the Foppish Idiot]

History, composition, health, etc. are electives? Where did you go to school?

[Blackthorne TA]

The problem is exactly what joacqin points out - teachers then teach to the test. Students actually learn LESS than before, because subjects not covered in the test are given short shrift during the year.

Then the problem is a test that is not comprehensive and known to the teachers: an obvious flaw. If the teachers don't know what is going to be tested other than what is on the curriculum, then how can they teach to the test?

My two favorite teachers, and who I thought were the best in the school, didn't always teach by the book or what was strictly in the curriculum. They made you interested and made you think, and that has to count for something. One of them was very friendly to his students and motivated even the most lazy person to participate.

Do you not think that a teacher who gets the students to engage and enjoy learning will get them to learn?

[The Great Snook]

Originally Posted by Aldeth the Foppish Idiot

History, composition, health, etc. are electives? Where did you go to school?

That is my point. All of those are required classes so if they are teaching to the test aren't they teaching what you need to know about all of them. I don't see what is being left out by teaching to the test.

[joacqin]

We have national tests in English, Math and Swedish and I am very fond of the one I do in English. It gives me something to lean on in my grading of the students. Although the test is new every year I know exactly what they are asking for and I am teaching to the test. That works fine in English though as the test is divided into four parts: The ability to communicate by speaking and writing and the ability to understand spoken and written English.

Much harder to make a single test to check everything in a history or social science course. The main problem is as has been pointed out before, not all knowledge can be tested in a test. The ability to argue a point? To gather information from different sources, the ability to evaluate different sources. Facts are a small part of school nowadays.

[Blackthorne TA]

Certainly all that is true. But as I said, some method of objective performance measurement is necessary to give incentive, reward good performance and be rid of the dead wood. Even if the method has flaws, it doesn't have to be static; if something is not working make adjustments, but have something other than seniority and tenure.

As an aside, tenure seems to be a bad idea. I've seen a lot of stories about professors doing so much good work to get tenure and then just sit on their laurels once they have it. I don't even undertsand the point of tenure at any level other than a university.

[Montresor]

In my opinion, a teacher should teach the students some basic facts, motivate them to do further reading on their own, and teach them to distinguish between good and bad sources.

The problem is that it is a lot easier to measure whether a teacher has tought his or her students the facts (Grammar: Can they read and write?; Mathematics: Can they perform simple calculations?; etc.) than it is to measure whether they have learned to study more on their own. (Have they read any books outside the school curriculum? Which books? And did they comprehend the books, or did they just read the words?)

Which is why tests too often measure whether the students have learned the curriculum and too rarely whether they have learned to learn more.

Calvin & Hobbes just rock!

[Blades of Vanatar]

Originally Posted by Aldeth the Foppish Idiot

History, composition, health, etc. are electives? Where did you go to school?

They were all part of the standard class regimen where I went to school. Basically we HAD to take English, Mathematics, Science, Social Studies(various History classes) and Gym(health was part of gym, 4 weeks of the year you spent in class learning about health). The "electives" were Languages(French, Spanish or Latin), Art, Music, Computers or a part-time Vocational/Technical program at the county Vo-Tech school. I actually choose the Tech option of "Computer Maintenance".

[Aldeth the Foppish Idiot]

Originally Posted by The Great Snook

That is my point. All of those are required classes so if they are teaching to the test aren't they teaching what you need to know about all of them. I don't see what is being left out by teaching to the test.

Sorry for my delayed response - I was away for the holiday weekend. I think we're disconnecting on some level. Since there are many subjects that are required curriculum, but are being given much less time that they have in the past because they aren't included in the test, those subjects are getting left out.

My wife, who is an elementary school teacher has said as much, and I will use it as an example to illustrate my point. English classes now focus much more heavily on reading comprehension - because it's on the test. You can make a test where the students read a paragraph or two, and then you ask them questions to see if they understood what they read. What they don't do is make time in class to focus on writing, because you can't grade a student's ability to write using a bubble sheet. So unless you think it is fairly unimportant for students to learn how to write well, that's what's being left out.

Or let's take math. 5th graders used to learn things like co-ordinates on a graph, raising integers to some power (like learning how to compute 3 to the 3rd power), and basic algebra (like solving: 12 - x = 7). They don't teach those things anymore. Why? Because they don't cover that on the 5th grade test. 5th grade is almost completely arithmetic. So how is omitting parts of subjects aiding the learning process? Do you feel it's simply more important to know how to do long division as compared to solving algebraic equations? (I honestly don't know the answer to that myself, although I'm inclined to think not. I guess it's good that I know how to do long division, although I can't remember the last time I didn't just use a calculator.)

[T2Bruno]

What we've found in many districts in Illinois is that the tests are very consistent and the teachers KNOW what the questions will be (with reasonable certainty). The lazy teachers teach ONLY what is on the test, not the breadth of the subject. Basically those teachers are cramming a four hour test into 162 days of class. What we lose by teaching to the test is the broad education base to prepare the child for other topics.

You simply cannot have a test question for every history, english, or science bit of knowledge necessary to make someone proficient in those areas. The only real exception is math -- in math you can test to fundamentals which can demonstrate good comprehension.

I'm reasonably certain I could teach a chemistry class and have nearly everyone pass the exams ... but not have anyone ready for the next course. Ultimately, I would be setting my students up for failure; which is exactly what "no child left behind" allows. It was a good idea, but poor in execution.

[LKD]

It's not just a matter of lazy teachers, it can be a matter of teachers whose jobs are on the freaking line. In a world where test results are used as weapons against all teachers, is it any wonder that some teachers, fearful of losing their jobs, will focus on the test results to the exclusion of other things? In some areas teachers have been known to cheat because their livelihoods are on the line.

You cannot treat teachers like you would someone who assembles machinery for a living. In the case of machinery assembly, a worker can take steps and have every unit come out within a 1% tolerance range, or even 100% perfection. The materials and components are not sentient and make no choices -- any problems that arise can be solved by the assembler.

Students are not unthinking machines. They make choices. They are living, breathing, sentient human beings over whom the teachers have very little control. Despite our best efforts, many choose not to learn.

This is not "blaming the victim" or other such horse$!**. It's a simple fact. Especially in today's day and age wherein parents will gleefully come in and tell you "my son doesn't have to do a single damn thing you say!"

Those who take a run at teachers need to step up to the plate and teach for a year. And not options like second languages, art, shop, or phys ed, where the challenging students are more likely to want to be there. No, step into a remedial math class or a language arts class. Step into a history class or a science class that is mandatory for graduation. Step in there for a few months and then tell me that the problems are solely with the teachers.

Now getting back to the issue at hand, teachers need to teach the curriculum, absolutely -- major deviations from it are not acceptable. We had a case here in Alberta in the 80s where a douche named Jim Keegstra was teaching high school Social Studies (Alberta's conglomerated History, PoliSci, Economics, Civics and Current Events course). This A-hole was a Holocaust denier. He was teaching his lying filth to the students. To me, that's grounds for dismissal right there.

But that's not what I'm hearing here. It seems to me that some want student test results to be the end all and be all in terms of teacher evaluation. That doesn't work, and it isn't in the best interests of the students, either.

[Blackthorne TA]

If a teacher can't get their students to learn, then perhaps teaching is not the best profession for them. And, no, I'm not saying it's easy or that I could do it better; I'm sure I couldn't. But then I'm not a teacher.

If I told my management that sorry, I don't know how to meet the requirements the customer gave me, what would they do? Say, hey that's OK, I know it's difficult; just do what you can and here's the normal raise you get for the amount of time you've been with the firm? No, they'd find someone who could do it and replace me; and my career would be in serious jeopardy.

Keeping bad teachers doesn't work and isn't in the best interests of the students either.

[joacqin]

Aw man BTA, you really have no respect or understanding for teachers whatsoever. You must have been traumatized something fierce in your school days.

Your customer gives you impossible requirements and pays you a pittance. It would be so wonderful if you actually did what you suggest to teachers. We would have none left. Fire them all but I get the feeling that you would be ok with that. What is your alternative to school as it is now as you obviously have nothing but disdain for the current system and the people who work in it?

[Blackthorne TA]

No idea what you are talking about joa. I know there are plenty of great teachers who can get their students to learn. I also know there are plenty of bad teachers who keep their job only because it's too difficult to get rid of them and in California at least there is no objective performance measurement to determine who are the good and who are the bad teachers.

I really don't understand why you think I have a problem with good teachers. And again I don't understand why my having a problem with bad teachers keeping their jobs is somehow an assault on all teachers.

[Aldeth the Foppish Idiot]

Originally Posted by Blackthorne TA

Keeping bad teachers doesn't work and isn't in the best interests of the students either.

I don't think anyone would argue with that. I think the issue that LKD and joacqin have repeatedly pointed out (and to a lesser extent T2 and I) is that judging whether a teacher is a good teacher or a bad teacher based on test scores is not a very precise way of going about it. I don't really have a good alternative, but in the few years where I taught on a collegiate level I can tell you that some students just aren't very good at certain courses.

I'll just use my experience in chemistry as an example. Most people who took freshman chemistry weren't planning on being chemists. They were taking it because it was a graduation requirement if you were pre-med, or if you were a nursing major. (Ah... The fond memories of my classes filled with nursing majors. A classroom filled with women between the ages of 18-20. And since I was only 24 at the time, it didn't feel icky yet. But I digress...)

The point being is that they weren't there because they wanted to be there, they were there because they had to be there. And so even though most of the students were of the mind set to just get through it to check the box for graduation, most of them also performed pretty well in the class. I know I handed out more As and Bs than Cs and lower. And yet, out of the 80 or so students I had each semester, invariably there were 2 or 3 that received Ds or failed outright. (With pre-med and nursing majors, since the courses were not elective, getting a D was still considered a failure - you had to retake the course if you got less than a C-.) So if I have 77 students learning the material adequately, and 3 of them not, I don't think that necessarily is the fault of me, the teacher. I mean, the other 77 didn't have a major issue with my teaching technique... And yet, if I my pay and continued employment were based on having everyone pass, you can bet your sweet ass that everyone would have got at least a C-. And THAT would make me a lousy teacher.

[Blackthorne TA]

I think the issue that LKD and joacqin have repeatedly pointed out (and to a lesser extent T2 and I) is that judging whether a teacher is a good teacher or a bad teacher based on test scores is not a very precise way of going about it.

And I have pointed out that NOBODY'S job has a precise way of judging whether they are good or bad, yet every other job in the private sector has some way of doing it. Is it perfect, no, but it DOESN'T HAVE TO BE. You get rid of the worst and reward the best; the rest get the typical raise.

Your examples of people who don't want to learn is mainly immaterial unless you want to claim that every student in a given teacher's class is like that. That's the whole point of statistics: Throw out the outliers and get an average over the entire classroom. If every one of the students does poorly in a given classroom, I would maintain it's the teachers fault. If every one of the students does superbly, I would maintain it's the teachers fault. For an average teacher, I would expect some students to do bad, some to do great and most to do average.

[Aldeth the Foppish Idiot]

Originally Posted by Blackthorne TA

Is it perfect, no, but it DOESN'T HAVE TO BE. You get rid of the worst and reward the best; the rest get the typical raise ... Throw out the outliers and get an average over the entire classroom. If every one of the students does poorly in a given classroom, I would maintain it's the teachers fault. If every one of the students does superbly, I would maintain it's the teachers fault. For an average teacher, I would expect some students to do bad, some to do great and most to do average.

We're throwing OUT the outliers? Aren't those the specific people we're trying to identify? And it's the teacher's fault if everyone does good, and everyone does bad? (I'm assuming if everyone does good you're saying it's the teacher's fault in a good way, despite my having shown how one could easily manipulate such a system.)

Of course, if you're saying if everyone does good it shows that the teacher might be manipulating the system, then we have a real Catch 22 on our hands. The expectation is that some students would perform well and some poorly in a "reversion to the mean" type of way. Of course, that would mean that the classes where everyone did bad had bad teachers, the classes where everyone did good had fake good teachers who were just manipulating the system. So we could say everyone who had an average class had an average teacher, and anything other than that was probably a bad teacher.

Again, how does this identify the good teachers? How are we identifying - much less rewarding - the best?

[Blackthorne TA]

Outliers statistically speaking.

Throw out the outliers, meaning student results. If some student does far more poorly than all the others, it probably means there was something else going on. Same thing if some student does far better than all the rest.

Yes, "fault" was a poor choice of word since that implies a mistake. I suppose the phrase "due to the teacher's skill" for the good case and "due to the teacher's lack of skill" for the bad case would have been better.

So, I didn't mean to imply that the teachers with the best results overall were manipulating the system and shouldn't be rewarded; quite the contrary.

I am assuming of course that there are systems in place to detect and avoid manipulations.