Cheating on Tests, by the Teachers

If you give people enough incentive to cheat, people will cheat:

Of all the forms of academic cheating, none may be as startling as educators tampering with children’s standardized tests. But investigations in Georgia, Indiana, Massachusetts, Nevada, Virginia and elsewhere this year have pointed to cheating by educators. Experts say the phenomenon is increasing as the stakes over standardized testing ratchet higher—including, most recently, taking student progress on tests into consideration in teachers’ performance reviews.

Tags: , ,

Posted on June 21, 2010 at 12:01 PM • 49 Comments

Comments

David Thornley • June 21, 2010 12:10 PM

The book “Freakonomics” had a section on that. It gave an example where an entire class had the same answers on a range of questions. Unfortunately, a fair number of the uniform answers were wrong, suggesting a possible reason why that teacher wanted to cheat.

Clive Robinson • June 21, 2010 12:17 PM

Sadly nothing new here (remember the post about changing the multi choice question answer cards).

In the UK we have government imposed “sats tests” half a dozen times during a childs education. The whole point of these tests is usually counter productive any way, due to “teaching to the test”. The Government only wants them to claim their policies are working or sack staff depending on which way they go.

We hear all the time “how easy” final tests such as A Levels have become. In practice the children are working harder because thanks to Tony Blair you now need a degree to be a filing clerk where as fourty years ago a C pass at O Level in English would have done…

Given the problems with the “economy” it does not seem very productive tying up the majority of adolecents from 16 to 25 in education when they would be doing themselves and the economy a favour if they worked…

Kris • June 21, 2010 12:19 PM

This is hardly surprising. Any metric used as a stand-in to judge value or quality will be subject to abuse as people try to “game” the metric, possibly at the cost of actual value/quality.

This has been well known in CS since they tried using lines of code as a performance measure decades ago only to get bloated code in return.

While the metric in discussion here (test scores) could be improved by having someone else do the grading, it would not solve the problem of “teaching to the exam”. I.e. teachers prep students for the exam without regard for how useful that may be in general application of the subject matter.

The reliance on metrics is an attempt at impartiality as otherwise a subjective evaluation would need to be made of (in this case) the teachers performance. It is an understandable goal, but unfortunately it fails to achieve its purpose. All this metric tells you is what grades were given, not what the pupils actually learned.

HJohn • June 21, 2010 12:30 PM

I didn’t find it startling. What I find startling is that some people found it startling.

Self interest and self preservation is a huge factor even among otherwise decent and well intentioned people.

It happens in the workplace as well. Bosses overrate and underrate employees based on who they want to promote for personal reasons or who they feel most threatened by.

Tragically, though unjustificable, it is easy to rationalize–what’s a couple points here and there if it doesn’t hurt anyone but helps me?

DayOwl • June 21, 2010 12:45 PM

My own conclusion is that the tests themselves are a failure. To hold a teacher responsible for scores, when many elements of learning are beyond their control, invites cheating, an element they can control. Like Clive says, it’s really just a way for administrations/government to pass the buck.

RH • June 21, 2010 12:46 PM

Like several others already commenting, I find it very amusing that this is old news. Half of their examples came from early 2009. A year later, it sounds like exactly Nothing has been learned from the experience.

A human mind in a system of rewards and penalties is a great global optimizer. As my friend says, “Global optimization does little more than show you just how bad your fitness function was.”

In this case, fitness function was a very weak “The more correct bubbles filled in, the better”

Coward • June 21, 2010 12:53 PM

People are quick to criticise SATs, grade inflation, and “teaching to test”, etc. (in the UK).

I’m no expert (not even close in fact), but what is the alternative? Scrap tests altogether? That’s just crazy.”You can’t manage what you don’t measure” as the old adage goes.

Besides, children endure two SATs not “half a dozen”. Hardly back-breaking.

And grade inflation; is it really a problem? If anything, inflated grades are likely to give a pupil more self-esteem. Maybe it is a bad thing – I’m just saying it’s not as black and white and some people seem to make out.

R Cox • June 21, 2010 12:58 PM

For some teacher is has basically become a logic problem externally equivalent to the condemned prisoner. If one cheats and ones scores are high, there is huge amounts of bonus money, and the only thing that can be done is the teacher is fired. More than likely a certificate will not likely be revoked, as it is very difficult to prove the teacher cheated.

If one does not cheat, there is a very good chance that the scores will be bad and the teacher will be fired.

So the reason some cheat here is that the punishment for cheating and not cheating are equivalent, and the only incentive is provided for those that cheat. This is problem may be prevalent more in lower grades where more money is often involved.

This is exacerbated by the ability to give certain teachers certain groups of students known to do worse or better. Since teachers who are given worse kids often are done so as a prelude to getting fired, there is an incentive to cheat to show they are good teachers and save their jobs by showing good test scores.

In fact the only thing that keeps most from cheating is a moral code that says cheating is wrong. I believe it is morally wrong to cheat, and the purpose of exams to give the student honest feedback, so I don’t cheat. Clearly this is the way to minimize cheating. However, society is so fixated on punishment that such a system would never work politically.

HJohn • June 21, 2010 1:00 PM

@Coward: “If anything, inflated grades are likely to give a pupil more self-esteem. Maybe it is a bad thing – I’m just saying it’s not as black and white and some people seem to make out.”

If that’s the case, why not just give every student an A+ for everything?

Yes, grade inflation is a problem. It’s a problem for people who are passed from one grade to the next without the education and then promoted into the real world unprepared for adult hood. The only way tests can protect against this is to accurately grade students and ensure they are learning enough.

The solution may largely be on standardized tests to have teachers proctor exams for students that are not their students. The incentive to cheat is absent in that case.

Don • June 21, 2010 1:03 PM

What else do you expect the teachers to do? The tests are part of a dishonest system to scapegoat the teachers, and give politicians and school boards a pass, so why should they respect them?

Teachers have no control over budgets, schedules, curriculum, the building, the surroundings, the students, or their parents. Yet they are held solely responsible for the outcome.

Here in Austin, there was a failing school. All the teachers were fired, and new ones brought in. The budget, the building, the curriculum, the students, and the neighborhood stayed the same. A couple years later the school is still failing, so a new crop of teachers is set to be fired.

HJohn • June 21, 2010 1:17 PM

@Don: “Teachers have no control over budgets, schedules, curriculum, the building, the surroundings, the students, or their parents. Yet they are held solely responsible for the outcome.”

I think that’s a fair point. Happens all the time in business (IT is well aware of this)… you have someone making well above 100K allocating resources, micromanaging from the top, etc., then when something goes wrong the person making $30K with no power is who gets scapegoated.

Though we like to view teachers as selfless and well meaning, which the vast majority of them are, we have to keep in mind that they are also human and employees, just as anyone else one may hire for another position. If we give them an unreasonable amount of power and hold them to an unreasonable standard, we can expect them to do unreasonable things.

No One • June 21, 2010 1:18 PM

@Coward: “If anything, inflated grades are likely to give a pupil more self-esteem. Maybe it is a bad thing – I’m just saying it’s not as black and white and some people seem to make out.”

There’s no evidence that this form of self-esteem boosting is of benefit to children. There is evidence, however, that training children to manage failure is of benefit in that many “advanced” children who never face difficulty until hard courses at college are really shit-poor students who have been promoted by natural talent and thus have no time or skill-management abilities.

Michael Ash • June 21, 2010 1:21 PM

@Coward

I have not noticed a lack of self esteem to be a significant problem these days. Quite the opposite, in fact: a major problem is that there are huge numbers of people with seriously exaggerated self-esteem, and this massive overestimation of self-worth causes innumerable problems. If we’d actually give kids useful feedback instead of telling them that they’re all wonderful even when they’re clearly not, I think it would help a great deal.

Thomas Jefferson • June 21, 2010 1:26 PM

Virginia has this group of animals called the “Standards Of Learning” tests. Sifting through statewide test results, it would appear that Virginia might have the most academically gifted population of students, attending the very best schools, in the United States — perhaps even the world!

RH • June 21, 2010 1:27 PM

@Coward: Despite your name, thanks for having the guts to challenge others’ opinions.

The issue with all of your questions is that grades and tests boil the value of a person into a single number or letter. Then that letter is used by the rest of society in a nasty feedback loop that decides whether or not you get to do a job or not.

A perfect example: I’ve got a friend who really knows his stuff in engineering. No one would hire him because he goofed off the first year, and his GPA wasn’t in the magic range they accepted.

This, as I said, is a feedback loop. As schools try to attract money/students, they try to show how well they can do in the workforce… by making sure they have the artificial grades to be hired.

They don’t take just “two tests.” That would probably work out okay. They take two tests after being taught only how to pass those tests for 10 years because the teacher’s paychecks depended on it.

As for what to replace it with? That is a question as old as time. Whatever it is, it needs to be more complex than just a number, because students are more complex than a number.

I see potential in the concept of the “Ivy League School.” For at least a hundred years, we’ve had a solution to needing to rate schools… schools had a notoriety. If you graduated with a law degree from Harvard, it was treated with the respect earned by the previous Harvard graduates in that firm. With social networking, its possible that we may be able to support such nuanced ‘grades’ earlier in the schooling curricula.

That being said, its been argued that the whole process of grading becomes pointless when you strip away societies ability to arbitrarily select based on GPA. The idea there is that “yes, you can leave a child behind… they can go to a vocational school and have a WONDERFUL life (currently vocational jobs are matching the wages for many ‘professional’ jobs like engineering).” Once you do that, the schools are for those who want to learn.

HJohn • June 21, 2010 1:33 PM

@Michael Ash: “a major problem is that there are huge numbers of people with seriously exaggerated self-esteem, and this massive overestimation of self-worth causes innumerable problems. ”

Agree 100%. Exaggerating test scores to feed an already inflated ego further feeds an entitlement mentality too.

I think some people confuse self-esteem with self-respect. A person with exaggerated self esteem feels entitled to the grade regardless of it being earned. We sometimes confuse exaggerated self esteem with low self esteem, which is counter productive since it adds fuel to the fire.

That problem is really is low self respect–a person that respects themselves would want to earn the grade, and not receive it as yet another entitlement.

This gets mildly off topic, but I think you make a good point. Someone who feels entitled or riggs the game in their favor often isn’t lacking in their sense of self-worth. Be it a teacher who rigs scores or a student that feels entitled to them.

Michael Ash • June 21, 2010 1:50 PM

@HJohn

Your distinction between self esteem and self respect clarifies the issue greatly. It’s what I’ve wanted to put into words but never quite figured out how. Thank you!

People think the path to success is to tell children that they are excellent. But the true path to success is to give children good and correct feedback on their accomplishments. You must praise them, but only when they have done something praiseworthy. To praise them regardless of what they have done is to teach them that work and accomplishment are meaningless.

HJohn • June 21, 2010 2:02 PM

@Michael Ash at June 21, 2010 1:50 PM

Very true post. I think also sometimes we praise them for the wrong things.

I think of some popular kids in schools. They receive fanfare for being MVP on a sports team, on the honor roll, class president, marching for AIDS/Cancer/etc. Others proclaim them to be–and they regard themselves as–a fine person. The fact that they bully, harrass, humiliate, and misstreat others isn’t considered–if anything, they see it as justified since putting others down and getting laughs from spectators boosts their ego, which they regard as of supreme importance.

Some people lack the capacity for academics or business achievement. It doesn’t mean they are stupid, some people just are not wired for it. Such people should still be praised for kindness, charity, compassion, honesty, etc. I want my daughters to be successful in many areas of life, including academic–but it is more important to me that they are good people.

This solves a lot of problems. A teacher that properly ranks the importance of himself or herself and has self respect will not tamper with a students grades for his/her own benefit, be it to the students advantage or detriment. I’ve personally told bosses before, when owning up to my missteps in my career, that “it is what it is, and I’ll accept the consequences.” I respect myself enough to know I’m not above the rules, I’m no more important than anyone else, and I don’t have to be perfect.

Great post Michael.

Dom De Vitto • June 21, 2010 3:19 PM

I knew someone once, ahem, who had a university lecturer that taught him very particular information processing definitions – to the point of handing out little slips of paper with each definition on.

On the day of the exam, he was genuinely surprised that it was ‘open book’ – and we had all arrived with our books, notes and his little stacks of cut-out paper.

aikimark • June 21, 2010 3:19 PM

Grade inflation was a problem long before NCLB, especially at universities. There are pressures on both students and teachers alike. The cheating incentive in this case seems to be NCLB policies

============
A humorous story from UNC Chapel Hill. In the early 70s, a subversive grad student named Nyle Frank gave everyone in his class an ‘A’. The department said he couldn’t do that and made him regrade the semester, so he failed everyone. This time the students protested and Nyle was ordered to regrade again. He gathered the students and made them call a coin flip. The girls lost the coin flip, so they failed and all the boys got an ‘A’.

papa zita • June 21, 2010 3:53 PM

@aikimark
Rather sexist way of resolution, is it not? Why divide the class that way, rather than having people draw lots? It really isn’t funny, BTW, unless you haven’t finished the story.

RSaunders • June 21, 2010 4:03 PM

Software has long known “You don’t get what you want, you get what you measure.” The question in the minds of a lot of teachers is “Are we measuring the right thing?”.

Teaching is a very complex process with very complex outcomes. The tests don’t measure the complete outcome, because we don’t know how to measure the complete outcome. They measure some parts of the outcome. Good tests, like the SAT, have years and years of calibration data that show their correlation with college graduation rates. Good tests are like credit scores, they have good statistical correlation. However, good correlation is NOT perfect correlation. Your credit score is not your balance sheet. Students have highly variable learning styles, and standardized tests can’t be constructed with perfect correlation to long term outcomes.

My children have math classes where they write “well developed responses”, aka paragraphs, to answer questions about math. The tests contain both problems where they have to get the right answer, aka math, and these other questions. When asked why, their teacher explained “We are including writing in all classes, to give students more practice.” which seemed like a fine reason, but she continued “and the girls get a lot better grades on the writing problems and so they like math class more.” Sure enough, you can get a good grade in math even if you can’t do the math, if we just choose to measure something else.

Politicians tend to like oversimplification. that’s unfortunate, because it causes them to make bad decisions and enforce them with the force of law. Clinton wanted higher home ownership among minorities, and he got it, even when they didn’t have the financial strength to weather a routine economic upset. Thus we have a mortgage meltdown. Bush wanted to force schools to improve, and so we have No Child Left Behind. More money is spent keeping up the appearance that special needs children are “making progress” and we have cuts for the gifted children that might be our next generation of scientists because Bush wanted uniformity at the expense of excellence.

The reason there is cheating is that teachers are, not surprisingly, educated folks. They know that standardized tests are measuring a tiny fraction of the real learning process. They don’t want to replace learning with test coaching, but you can make them if you measure test scores above all else. Some of that coaching may be “cheating”, but most educators involved probably feel the end justifies the means.

Joe • June 21, 2010 6:32 PM

A point that seems to be implied is that any evaluation of a system that involves humans sets up a recursive cycle between the evaluation process and those being evaluated.

After a couple of iterations of the cycle, the system has shifted state and the measure and evaluation process may have lost their meaning.

This will happen with any measure that is an abstraction of what is wanted. Only a measure that is a close approximation of the desired result will work.

The post notes the problem in teaching and programming. It is also a problem with pay-for-performance measures in medicine.

On a more humorous note (to me anyway) I remember a sheet of “final exam questions” when I was in college. It had questions like:

Anthropology 101: One hundred angry aborigines are screaming for your blood. Calm them.

Mechanical Engineering 101: The disassembled parts of .306 hunting rifle are under your desk. In three minutes a hungry tiger will enter the room. Take appropriate action.

Seems much harder to game tests like those.

tobias d. robison • June 21, 2010 7:32 PM

Traditionally, teachers are the umpires who keep students honest when they take tests. The idiots who decided to grade teachers according to student’s test results turned the very people whose job it was to keep tests honest, into people with an incentive to cheat.

Legislators who created this new system did not give a passing thought to funding a new set of umpires to keep the teachers honest. If they had, they might have realized how silly the whole business is.
– tobias robison, tobyr21

askme • June 21, 2010 8:08 PM

OK, it is not really a Heisenberg effect but someone else that noted that we cannot observe the system witout changing it= observer effect.

aikimark • June 21, 2010 8:52 PM

@papa zita

I don’t remember whether the department stepped in to grade it themselves or the students had agreed to abide by the coin flip grade. Nyle left the department some time after this grading incident.

William O. B'Livion • June 21, 2010 9:16 PM

Coward:

“I’m no expert (not even close in fact), but what is the alternative? Scrap tests altogether? That’s just crazy.”You can’t manage what you don’t measure” as the old adage goes.”

How about scrapping federal control (and funding), dis-incenting state control (especially in larger/more populous states) and letting the local school boards actually run the shit. Then let the parents decide what education is good enough for their precious darlings. If the left wing fruitloops in San Mateo and Santa Clara county want to run self-esteem workshops and sex-ed clinics for their first graders, they those kids will grow up to serve hamburgers to the engineers that move in from the east coast.

Not every problem is amenable to top-down management. In fact most work better when solved as locally as is possible.

Gweihir • June 21, 2010 10:20 PM

Not all people will cheat, some are fundamentally honest. Unfortunately, honesty comes with drawbacks such as less career success. I recently read somewhere that the ability to lie (and regular use of it) is a characteristic of a healthy personality. This may explain why the human race has significant problems and why it is not really looking good for the next 100 years or so.

Robert • June 21, 2010 10:36 PM

Ah Cheating on exams, a topic close to my heart…

I remember when I was in Middle school we had a bunch of tests, on which I scored very highly. So the teachers asked me, what specifically I had done to prepare? I told them simply that I cheated, but I gave no details on how. It was incredibly educational to see the teachers brainstorming and trying to figure out exactly what I had done. The sad truth is that the tests were trivial so I just knew all the answers. The punishment was well worth enduring, for the sheer entertainment value, of watching the system try to figure out how I had bettered their security. The process exposed hundreds of obvious flaws…ah the beginnings of a career!

A Nonny Bunny • June 22, 2010 3:39 AM

@Clive Robinson

Given the problems with the “economy”
it does not seem very productive tying
up the majority of adolecents from 16
to 25 in education when they would be
doing themselves and the economy a
favour if they worked…

Somehow, it seems unlikely to me that a less educated populace would be very beneficial in today’s economy. More and more today’s economy relies on mental rather than physical labour. And that’s reflected by the difference in unemployment rate of better and lesser educated people.

It might not seem productive to tie the majority of adolescents up in higher education, but they make up the loss of a few productive years by being more productive (in terms of economic value) in the following years. I really don’t think they would do themselves or the economy a favour by dropping out of the education system early.

averros • June 22, 2010 3:51 AM

The culprit here is the desire of socialists to “fix” the market in education. They may have good intentions, but they do not have a slightest comprehension of the economic calculation problem (first described by von Mises) which dooms any socialist enterprise to failure.

The problem of teachers cheating en mass cannot exist in a free-market education. People pay money to get their children educated – and prefer schools which have good reputation. The reputation of the school is a composite of many factors (such as safety, facilities, etc), but, by far, the most important component is success of its graduates in the real life. This may be not exclusively due to academic performance – the school may help to forge friendships, to help children to develop good work ethics, to teach them healthy lifestyle, etc.

None of that is helped by teachers trying to inflate grades. Bad behavior of this kind produces badly educated graduates – which reduces their real-life success. Which damages reputation of the school, which, in turn, causes parents to reduce estimates of the worth of education in this school. Which directly affects the salaries and job security of the teachers.

Private education thus has negative feedback built into the system – in the form of system of prices, resulting from people doing voluntary exchanges. Attemps to create such feedback in a socialized system are futile – the very act of using force of law to change interactions between people results in disappearance of the information about what they really value.

The socialized education system is, therefore, doomed to produce education nobody really wants or needs at disproportionate cost – simply because no “objective” artificial metric can recover subjective valuations. It stopped being an enterprise in teaching useful skills and became an enterprise in shaking down taxpayers and the captive audience of students and their parents.

From the economic perspective, education is just a form of investment. Good schools will produce good ROIs for their patrons.

Modern public (i.e. socialist) schools and universities typically produce negative ROI – when the “contribution” of taxpayers is accounted. Recently American universities started to produce negative ROI to the students even when the socialized costs are not counted – graduates end up with less additional lieftime earnings than they owe for the tuition with interest. People started to bail out of the system, and employers increasingly ignore academic credentials in favor of twork experience – which puts people who didn’t waste years in a U at an advantage at work – and without debt.

A Nonny Bunny • June 22, 2010 5:10 AM

@averros

Modern public (i.e. socialist) schools
and universities typically produce
negative ROI – when the “contribution”
of taxpayers is accounted.”

So, you’re saying, including taxpayer contribution, a university education typically costs over a million dollar per student? Because that is the average life-time benefit according to http://wpcarey.asu.edu/seid/upload/Value%20Full%20Report_final_october%202005a.pdf

I think your arguments are based more on ideology than reality and fact. We have seen enough damage from “cheats” in free market systems the last few years. Certainly, there will always be a correction — eventually; but that feedback loop takes a long time to feed back.
Also, none of what you say actually requires schools to be free market; it only requires that parents can choose what school to send their children to. Who pays and how is irrelevant to that.

Clive Robinson • June 22, 2010 5:39 AM

@ A Nonny Bunny,

“Somehow, it seems unlikely to me that a les educated populace would be very beneficial in today’s economy”

I did not say that they should not be educated or not edujcated to a high level.

Full time higher education can take away 1/5 of a persons working life. It is generaly a myth that all can make up this loss (they cann’t).

You end up with a stupid situation that even quite lowly jobs now require a degree. Not a relevant degree just any degree…

It is a waste of a persons potential having them chase “any degree” for five years. Outside of science and engineering related employment few people will benifit.

Even in science and engineering few universities offer the real required education (ie fundementals) and most pander to employers short term requirments.

Life long learning is now a requirment in medicine and many other subjects so ultimatly te argument boils down to how do you do it “on the job” or “part time HE”.

Ultimatly for basic skills this boils down to an apprenticeship at the beginning of a working life, “on the job” for specific employer needs and part time further education to maintain concurrency or to progress further/upwards with a career.

Only exceptional students going into a very limited number of jobs actually benifit economicaly from the full time education to 25.

For the rest not just they but the majority of us are deprived of their economic benifit to society not just in the short term but the long term.

All this harm is just so Politicians and their ilk can have a control on eucation that Stalin would have given not just his right arm but his legs as well.

And it is the majority of us that pay for this political stupidity one way or another.

The reality is that only 5-10% of the population need a degree to do their jobs and majority of those don’t need the degree level training at the start of their careers. And as others have noted this system gives a false sense of entitlement that causes problems to employers.

tqft • June 22, 2010 7:43 AM

http://www.couriermail.com.au/news/posters-guide-pupils-in-naplan-exams-claims-parent/story-e6freon6-1225868890657

http://www.couriermail.com.au/news/gillard-promises-probe-into-claims-of-naplan-cheating-at-all-hallows-and-state-high/story-e6freon6-1225868224644

http://www.couriermail.com.au/news/queensland/students-told-to-stay-home-from-naplan-tests/story-e6freoof-1225865351317

and I could go on. Students being asked to stay at home so as not to drag the schools overall score down, “In all, about 1.1 million children around the country will sit the tests, designed to provide benchmarking for national education standards, but which have been criticised by teachers’ unions for allowing unfair comparison of schools through “league tables”.”

DayOwl • June 22, 2010 7:46 AM

The best self-esteem is produced when you teach a child how to achieve something successfully. Showing them how to solve a math problem on their own is far more powerful than giving them a gold star for being in class. Acknowledging what they have truly gotten right also makes a difference–as opposed to only pointing at what they got wrong.

Grade inflation is just a cover for lousy teaching.

M&M • June 22, 2010 8:01 AM

how many people of Da Vinci’s potential were born without Da Vinci’s access to influence,
Obviously great genius only succeeds when allowed to and promoted by a warlord.
Clive speaks of society being deprived of peoples potential, well society has been arranging that for a long time. whenever mankind comes to a fork in the road between progress and regress, there are ten thousand regressives fighting to block progress.
Genius as potential is common, Genius in fruition is uncommon, because most of the stupid people in the world will not and cannot appreciate it. by definition the stupid cannot value the intelligent. They don’t have the means to evaluate it.

Farmer • June 22, 2010 8:54 AM

“It seems blindingly obvious that the person whose pay raises depend on the outcome of a test should not be proctoring said test.”

Mr. Fox, I will be gone today, could you please guard my chickens while I am away?

HJohn • June 22, 2010 9:18 AM

@Scott at June 22, 2010 8:11 AM

Betrand Russell once said that a big problem in the world is that the stupid are so full of confidence, yet wiser people so full of doubts.

In my own life i’ve noticed that the more I learn, the more I realize just how much I don’t know.

HJohn • June 22, 2010 9:19 AM

@HJohn at June 22, 2010 9:18 AM

My coffee-deprived mind strikes again. What I said was accurate, but I didn’t realize that one of the first comments in the article mentioned the same Bertrand Russell quote I did.

MikeA • June 22, 2010 11:14 AM

Side note on the (US) SATs. These are “sold” on a few points, one is that they measure “aptitude” (That’s the ‘A’ in the name), and thus cannot be “gamed”. Another is that they predict academic success in college. When the University of California did their own study and concluded that they were not such a good predictor of academic success, it decided to de-emphasize them. That is, it said that SAT scores would have less weight in admission decision. It then ran into a buzz-saw of criticism from parents who had shelled out a fair bit of money to “test prep outfits”, to game their little darlings into admission slots from which they would later flunk out. So much for both the key premises.

To re-iterate, why is it that everybody seem to know the joke about the drunk looking for his glasses under the lamppost, yet so many insist on “metrics, gimme metrics, I don’t care how they correlate to desired outcome”?

JimFive • June 22, 2010 2:53 PM

@Coward RE: “You can’t fix what you don’t measure”

The problem is using a measurement inappropriately. In this case the test is measuring student aptitude, but the measurement is being used to rank teaching skill. Teachers can’t really do a lot to improve the aptitude of the students (as a whole–certainly a teacher can reach an individual student, but that doesn’t help here). Because of this discrepancy, the only way teachers can be sure of getting a good score is to cheat.

If you really want to measure teaching ability you need to develop a test that measures it.

JimFive

greg • June 23, 2010 4:09 AM

@Clive Robinson

I agree completely. Teaching at uni we have seen the first year class get more and more “dumb”. By dumb i mean they can’t solve basic integral or differential equations. A must for any enginner or Physics major.

My firend an I mused over this for a long time. Are we getting dumber? Are the schools doing a worse job, etc.

Later my friend went into school teaching. Then we saw what was really happening. More that half the first year uni students, would have never come to uni in “our day”. University was becoming an extension to high school.

How many people need to be competent at basic calculus in a Job. I do, an engineer does. But you can be a great mechanic without it.

aikimark • June 23, 2010 6:52 AM

re: dumber students
devolution at work.

Good news…Devo has a new album and is touring again.

Bad news…This might lead us into the world of Idiocracy:
http://www.imdb.com/title/tt0387808

============
re: the Nyle Frank story

I heard from Nyle and he corrected my memory of the story.
” I did teach two sections of Political Science… The Fall, 1970 course received about 36 A’s and 4 B’s (if a student convinced me they put 60 hours into the course, they received an “A”, 45 hours for a “B”, and 30 for a “C”. I believe I told them “What are A’s for if you can’t give them to your friends!”. I had another section in 1972 and tightened it up a bit (75 hours for an “A”, 60 for a “B”, etc)…..cant remember how many received A’s, but it was quite a few.
I didn’t grade by sex, but I was in two English courses as an undergrad at UCLA where the professor, the poet Jack Hirschman, did. He told us “according to the I Ching the male is number one, the female number two….all males will receive A’s, all females B’s….anyone who is dissatisfied with their grade can see me after class about having it changed. I ran into him many years later, and he claimed that he did this to keep the guys out of the Vietnam War.”

markm • June 26, 2010 3:52 PM

“If you really want to measure teaching ability you need to develop a test that measures it.

JimFive”

Wrong measurement. I once had a tenured, union-protected teacher that simply didn’t bother to teach – and hadn’t bothered in 20 years. IF there was an effective test for teacher aptitude, it would probably only make it even harder to get rid of this lazy bastard. It’s much less work to cram for a test than to actually do the work every day.

Of course, I also have encountered many teachers that could not teach no matter how hard they tried – but had degrees and certificates claiming that they could teach. I have seen a certified high school math and science teacher that was incapable of simple algebra. I have seen an elementary teacher that were not as smart as her fourth graders – and the kids knew it! I have seen far too many that just confused the kids more the harder they tried to communicate. To put it bluntly, either the schools of education are conscious scams, or they don’t know what good teaching is. You aren’t going to get that teacher aptitude test, because they don’t know what to test for.

At a deeper level, you cannot effectively control a process just by looking at the inputs, but you have to look at outputs. Otherwise, you’re going to be putting effort into areas that no longer matter, while not even knowing how well you are really doing. And that’s if the process is well understood and everyone is honest. If you’re not sure which inputs are needed, measuring only them will get you nowhere, and if you do know what to measure, inputs are far easier for the dishonest to fake.

In teaching, the inputs include curriculum, teacher training and credentials, teacher aptitude (whatever that is), and all the BS that incompetent principals score teachers on when they bother to do classroom observations. (Of course, they also include things like desks, books, paper, and pencils – that seem to get neglected in following the latest ed school fads. Or hiring three times as many administrators with luxuriously decorated offices.)

Outputs are what the kids learned. That needs to be tested at least once a year, by outsiders rather than those who are being rated, and the results correlated to check the progress of each student, and to compare different schools and school systems. And then fire teachers AND ADMINISTRATORS as needed, while giving better pay and promotions to those that did well.

Clive Robinson • June 27, 2010 8:30 AM

@ markm,

“IF there was an effective test for teacher aptitude…”

It’s not the teaching or the pupils end abbility we need to be measuring.

It’s the difference between starting ability and end ability scaled by each pupils intrinsic ability.

That is how much they have improved within their individual capabilities.

And with a little thought you will realise that those with more limited ability will produce better results than those of higher ability, within any large group.

And this is a fundemental flaw of our one size fits all teaching in oversized classes. Economicaly it’s a real disaster, which is why we have the “oh so politicaly incorrect” streaming of pupils by their ability in some places.

From a societal point of view, with a one size fits all system you have to make a choice of where you focus limited resources. At the top where the best get encoraged to be better or at the bottom where they aspire to be as good as average.

This latter goal being more easily achived if those above average ability are brought down as the average more easily aproximates the bottom.

Untill we know how people learn we will not be able to decide what teaching methodology is best just what appears to work best for the masses.

Which will always be very suboptimal and continue the race for the bottom, thus not what society in general wants or more importantly needs. But that which suits those in charge…

Leave a comment

Login

Allowed HTML <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre> Markdown Extra syntax via https://michelf.ca/projects/php-markdown/extra/

Sidebar photo of Bruce Schneier by Joe MacInnis.