Cheating in Online Classes

Interesting article:

In the case of that student, the professor in the course had tried to prevent cheating by using a testing system that pulled questions at random from a bank of possibilities. The online tests could be taken anywhere and were open-book, but students had only a short window each week in which to take them, which was not long enough for most people to look up the answers on the fly. As the students proceeded, they were told whether each answer was right or wrong.

Mr. Smith figured out that the actual number of possible questions in the test bank was pretty small. If he and his friends got together to take the test jointly, they could paste the questions they saw into the shared Google Doc, along with the right or wrong answers. The schemers would go through the test quickly, one at a time, logging their work as they went. The first student often did poorly, since he had never seen the material before, though he would search an online version of the textbook on Google Books for relevant keywords to make informed guesses. The next student did significantly better, thanks to the cheat sheet, and subsequent test-takers upped their scores even further. They took turns going first. Students in the course were allowed to take each test twice, with the two results averaged into a final score.

“So the grades are bouncing back and forth, but we’re all guaranteed an A in the end,” Mr. Smith told me. “We’re playing the system, and we’re playing the system pretty well.”

Tags: cheating, schools

Posted on June 14, 2012 at 12:27 PM • 39 Comments

Comments

bcs • June 14, 2012 12:41 PM

I wonder what would happen if all the students did questions in sync. Most of the first questions would be wrong but the subsequent questions would be more an more likely to be repeats. This would tend to even out the scores between tests. It might be harder to detect cheats from overall scores but it would be (somewhat) evident in the progression of right/wrong rates.

JF Sebastian • June 14, 2012 1:14 PM

Enforcing larger pools of questions in the questions bank can help with this problem, but questions often get used over and over for multiple terms, so fraternities etc can if so inspired (and they tend to be) to create their own banks of questions for over time. Therefore not only increasing bank sizes but also rotating in new questions regularly (and/or using random banks of questions banks per assignment or exam) seems the best route, or at least a better route to follow for now.

Smokey • June 14, 2012 1:15 PM

We actually had a very similar scheme in one of my courses at university. The pool of questions was much, much larger, however the course existed for many years, and the questions remained almost the same year after year.

We were on about the 8th year, and each year a word document would be passed down through some common friend. On monday nights about 8 of us would meet at my house, each with a laptop, and we would all help eachother by searching thru the answers while one of us took the test. Just like this example, we all took turns going first.

We all got about 95% on the testing portion of the couse. Good times.

Jog4Hoagies • June 14, 2012 1:36 PM

Posted by: bcs at June 14, 2012 12:41 PM

You are correct, if the students are trying not to get caught cheating in an open-book exam that you can take more then once, for an online course.

But if they are trying to get the best overall score, then the way they are going is pretty good.

Tomasz Wegrzanowski • June 14, 2012 2:05 PM

If the test didn’t give instant per-question feedback but delayed it for a few days, it would completely defeat this method of cheating.

You can still have a group of people solving the quiz cooperatively, but they’d actually need to research the answers, or at least have one person with a clue (in which case the entire class can already pay that person to pass the test for them).

nycman • June 14, 2012 2:33 PM

If questions are randomized and from a large bank, but online collaboration is possible, you could get together a group of students, but each student would focus on a portion of the material. So if you had 5 people, each person would only need to study one-fifth of the material. post questions to a forum while taking the test, and the person who studied the relavant chapter would answer them. Better than having to study the entire syllabus. Ultimately all the cheating shows that the credential is worth a lot more than the knowledge the credential is supposed to represent.

bcs • June 14, 2012 2:53 PM

@nycman: The credential is perceived by the students to be worth more than the knowledge.

I wonder how they feel a few semesters down the line or a few years after graduation? If I were setting up an integrated curriculum, I’d focus most of the anti cheating efforts on the later courses. If you can’t graduate without some sr. level class that’s very hard to cheat on and impossible to pass honestly without knowing the materiel from the lower division classes, then the students will learn the materiel before they graduate or flunk out.

Dena Shunra • June 14, 2012 3:26 PM

It seems like we’ve achieved “education theater” – where instead of people with knowledge, we have people with credentials.

This relates, I think, to security theater. In fact, the trend toward this “theater of education” is long enough to imply that security theater was put in place (and is enforced) by people who went through that sort of a system.

Alan Kaminsky • June 14, 2012 3:41 PM

From the article: Although the syllabus clearly forbids academic dishonesty, Mr. Smith argues that the university has put so little into the security of the course that it can’t be very serious about whether the online students are learning anything. Hundreds of students took the course with him, and he never communicated with the professor directly. It all felt sterile, impersonal, he told me. “If they didn’t think students would do this, then they didn’t think it through.”

Despite being a computer science professor and working with high tech every day, I have always viewed online course delivery with deep suspicion. Set aside the ease with which one can get an A without learning anything, as this student demonstrated. The real damning indictment of online courses is their sterility, their impersonality, their lack of direct communication with the teacher.

The answer is not to get into an arms race between online cheating technology and online anti-cheating technology. The answer is to get rid of the online courses. Stick with fertile, personal, face-to-face-taught courses.

When I give an exam in one of my classes (taught face-to-face; I do not teach online), I let the students bring anything they want into the classroom with them — cheat sheets, notes, books, phones, tablets, laptops. I then write exam questions whose answers cannot be looked up in the book or on the Internet. I’ve never had a problem with cheating on exams.

Peter • June 14, 2012 3:48 PM

God, I hate cheaters.

Anton • June 14, 2012 3:51 PM

They get top scores when they do the course on security: How to prevent cheating.

Alan Kaminsky • June 14, 2012 3:59 PM

@Dena: We’ve had “education theater” for a long time, except we’ve called it “grade inflation.”

nick • June 14, 2012 4:02 PM

Getting rid of online classes is a silly solution.

The real solution is professors who are willing to grade papers rather than rely on automatically graded tests. Students and teachers who complain about a “sterile” learning environment simply aren’t putting the effort in. You can make the same complaints about any large class. The lazy teachers will phone it in and the lazy students will not be engaged. Both parties need to make an effort to reach out and communicate. If either is unwilling to do that then system won’t work.

There are ways to test knowledge in online classe that make it hard on cheaters, a quiz that pulls from a small pool of questions and provides instant feedback isn’t the way.

Ramblin Wreck • June 14, 2012 4:49 PM

When I was in college, our physics professors solved this problem by having the system randomly choose different values for variables for each student. (This makes the question bank huge, but without actually needing new questions.) “Cheating” was essentially limited to telling other students how to do the question correctly.

Our professors were also notorious for making the “wrong” answers things you were likely to get if you made some small, common mistake in your calculation…

Daniel • June 14, 2012 5:13 PM

Aren’t multiple-choice tests a way for TEACHERS to cheat? Instead of interacting and communicating with what the students think, they get to game the system.

Nobody should be surprised if students start seeing education in a similar way, and act accordingly.

FWIW, in the country I grew up in, multiple-choice tests were referred to as “American tests”. That says it all.

jeremy • June 14, 2012 5:20 PM

I went to an online college, there were no tests, at all ever! Knowledge was borne out through written essays, marketing projects, documents etc that proved you either understood the material or you didn’t.

Also, if like the professor above; want to abolish online colleges consider finding ways to lower the overall costs of college education, do that and you can solve the education theater issue. further, as a whole the entire collegiate system is a scam that has sold generations down the river. Promising jobs where none exist whole swaths of pharmacists and lawyers are out of work. Instead of the protectionist sentiment put forth change focus towards actual education. When businesses begin to realize this and invest in their people, college will no longer be an issue.

Cool Hand Geek • June 14, 2012 8:05 PM

Then there are the not discussed so far courses like law and education where learning the lingo is critical to passing the exams. As Rick Wolff at rdwolff.com explains it, these are the degrees where one takes very simple concepts that are easily understandable but couched in language that is so impenetrable that if one has a problem, let’s say in law, one is forced to go to the relevant expert in order for him to converse with other people who are in positions of authority who speak the same impenetrable language so that one’s simple problem can only be resolved, often at considerable expense, by those who know the relevant impenetrable language.

Then there are those who are simply stupid, but obtuse enough to appear to be brilliant. Allen Greenspan comes to mind. First he was for greed, then he was against it, and I understand he’s now for it again.

Cool Hand Geek • June 14, 2012 8:08 PM

Pardon me for my first sentence mistake, I meant to say law and economics. I can’t speak for the teaching arts as I am not versed in them.

Hillary • June 14, 2012 9:42 PM

I took one course where a variant on this was encouraged by the professor. He told us how big the question bank was and we were allowed to take the test as many times as we liked in a week. I don’t remember if collaboration was allowed, but I didn’t.

I was the only person who took the time to repeat and get a perfect score. Most of my classmates stopped around 90%. I was doing it for the challenge rather than the grade.

Brian • June 14, 2012 10:04 PM

@Daniel – Multiple choice exams can be much harder on those who have not studied as there is no way to get partial credit. Also I do not understand what you mean by “American Tests” as many of the pinnacle universities in the world are in the U.S. I assure you multiple choice exams are not easy when written correctly.

http://www.usnews.com/education/worlds-best-universities-rankings/top-400-universities-in-the-world

Cheating is not solely limited in taking the exam. After spending 4 days with 4 other people grading 220 physical chemistry finals, we had a few with errors in the grading returned for re-grading. Quite often these re-grades lack merit as the correct answer has been penciled in after the fact. As a common practice we scan all graded exams before returning. But it does not stop there. Students will ask for their current grades. When told they received a 0 for the second quiz or exam or homework set. They will respond they did it and will hand us a copy of the graded quiz with their name on it (that just so happens matches someone else’s work we scanned). All of the students that are caught cheating get anywhere from a quarter to a year suspension. I know there are many people that do cheat that I can not detect, but even if I can detect it, I have to prove it. My word is NOT enough. The University has its own internal Academic Integrity department. They are given the information, the student gets an advocate, and there is a hearing where a board decides if they were cheating. Instructors are not allowed to give vigilante grades and can only fail a person after the committee decides if they were cheating. Catching a cheater in NOT enough. In the linked article, “Mr. Smith” admits to cheating. None of my cheaters have ever admitted to it, and only one has shown remorse after the fact.

This information may not phase you. But this is chemistry. These are the people making your therapeutic drugs. Do you want people who cheated and have no idea what they are doing making compounds you are going swallow? Take two and call me in the morning!

Daniel • June 14, 2012 10:46 PM

@Brian

I never said multiple-choice tests are easy. They can be very difficult indeed. The very best multiple choice tests can even be so construed that they examine understanding rather than memorized knowledge facts – though usually they don’t.

The problem with multiple-choice test is not that they are hard or easy, or that they are easy or not easy to cheat on, but that they are based on the assumption that there is one correct answer, known to the teacher. It takes away the dialog, the ability of a student to make a point, to explain his line of thought.

What I’m saying is that through the institution of multiple-choice tests, the student is cheated out of his learning experience. It may be an appropriate way to “test” whether a book was memorized appropriately, but it is never an adequate conclusion of a process which was supposed to be a thought-stimulating educational exchange between a teacher and a student. it reduces education to technology (as opposed to science).

Imagine that comments to blog posts would have to be given in multiple-choice formats. Oh wait, some newspapers indeed “poll” readers’ “opinions” that way. That’s what our education system has come to.

Cheating is one way to arrive at the results that the educational system is set up to achieve.

I’m not disagreeing with you about the pernicious RESULTS of cheating, quite the contrary. I am, however, pointing out that the ROOTS of the “education theater” are deeper than some people think.

And of course, multiple choice tests have spread around the world. Where I grew up they were referred to as “American tests” by supporters and detractors alike. As a detractor with an international contrastive perspective, I see these tests as symptomatic for the American education theater, which is rapidly going the way of the American health theater. To coin another phrase.

Daniel (a different one) • June 14, 2012 11:30 PM

But why even go through all that trouble. If you can take the test TWICE it’s a simple matter to use a software program that makes a video of the screen (like My Screen Recorder) to record the questions the first time around, look up the answers, and then take the text again. If that fails for some reason well just use a normal video camera; they are cheep enough these days and they would be totally undetectable.

So it seems to me one of the big problems here is allowing the students to take the exam more than once.

aaaa • June 15, 2012 2:13 AM

@Daniel Speaking as a graduate of free online crypto course which was great.

1.) The multiple choice test cheats student out of nothing. It is a test method, one of several. Just because the course tests students using multiple choice tests does not meant there is no interaction in discussion forums, IRC channels or in person. Neither it means that other assignments are done different way.

2.) The multiple choice test can be designed to test understanding. It does not have to test memorization only. It is only about what kind of questions are asked.

3.) Maybe it is my faulty observation, but why does Americans treat everything as absolute something against other thing case? The argument is structured as if you would have a choice between everything done with multiple choice or nothing at all. If every single moment is not filled by discussion, then it is apparently bad because none is. If discussion (or project doing) is good, then every single moment must be filled with discussion (or projects).

Good course may use mix of various methods to determine grades. Multiple choice for one thing, programming assignment for another.

Grey Bird • June 15, 2012 2:18 AM

If the test generating program is written correctly, then video taping taking the test won’t help that much. The next time the student takes the test, the generating program will use different questions from the first time.

MikeO • June 15, 2012 2:51 AM

Ah, but can they recall the first 100 digits of pi from memory…

Åke • June 15, 2012 4:20 AM

Display the questions as flash files, thus disabling copy-paste. Problem solved.

Clive Robinson • June 15, 2012 5:44 AM

@ Hillary,

I took one course where a variant on this was encouraged by the professor. He told us how big the question bank was and we were allowed to take the test as many times as we liked in a week.

Sounds like a reasonable way to get students to actually do some work 😉

Many many years ago before electronic testing had become even realisticaly possible I went to college and studied electronics. One part of the course (testing techniques) was actually considered less interesting than mathmatics by most on the course (even though it’s fundemental to any type of research, design work, etc).

The person teaching the course used the usual techniques of the time to prepare us for the exam including providing previous papers and a couple of mock examinations. The week before the actual exam he surprised us with another paper and went through it showing us how to answer the questions and get us to write them down. He told us to go and practice again with the paper.

Well some of us did and others not. Come the day of the final exam you can imagine the surprise of most people on turning over the exam paper to discover it was the same as the surprise paper he had gone through.

For some of us we zoomed through the paper and left early (we were the “high scores”) having done as we were told. Others who did not bother studying the paper didn’t even finish the paper and failed (ie less than 40% mark).

It taught me an important “life lesson”.

On a more uptodate note, in the UK we used to have Grammar Schools which had the equivalent of a national entrance examination you took when you were 11. It was called “The 11+ Exam”, well it nologer exists as such, however some educational authorities have the equivalent called “The Secondary School Transfer Test”. Well you can imagine my surprise when asking a teacher for “old papers” for my offspring to practice with to be told “It would be unproffessional for me to do so”. When I asked why I was told it was the same test every year…

Now imagine if you were a “private hire teacher” knowing this it would not be difficult to obtain the questions and drill the children of your “fee paying parents” so that they would pass the test with very high if not perfect scores…

So sometimes the idea is good (ie encorages studying) and sometimes bad (ie encorages drilling).

Paul Renault • June 15, 2012 6:54 AM

The ‘learned’ professor has never played the old BBS games Pyroto Mountain, I guess.

Ahh, back to my misspent youth. Back to the SPLIFF format (Standard Pyroto List Information File Format) in the pre-Google research days…

We were discussing the process of automating the question and answer process.

Daniel (a different one) • June 15, 2012 10:23 AM

@Grey Bird.

That’s where statistics don’t meet up with pedagogy. Sure, one can defeat the recording of a multiple choice test by picking say 10 random questions from a pool of 100. The problem is (1) coming up with 100 meaningful questions that cover the material to begin with. 10 questions is a quiz and find me a book chapter (any book chapter) where it’s possible to elucidate 100 meaningful questions from within the context of the course. That takes a great deal of instructor time.

The second pedagogical problem is that as an instructor I want my students to know certain concepts and facts; it isn’t all created equal. So if I create lots of questions and then pick them randomly there is no promise that the questions the student answers will actually relate to what I want them to know.

I’m not opposed to multiple choice tests. I can use statistics to defeat cheating but has the student learned what they need to learn from the class in the process. The instructor needs to think carefully about how that assessment method advances the learning goals of the class.

Someone • June 15, 2012 4:09 PM

If you’re testing several subject areas, then you can adapt that technique by making a pool of questions for each subject and taking a few questions at random from each pool.

Danny • June 15, 2012 5:41 PM

Cheating had a very interesting effect on me – each time before a test in school (junior/high and college) I would create cheating sheets, copying the entire courses on a very small piece of paper with smaller then atoms letters and hiding them under my sleeves – then in the day of the exam, if the questions were from those courses I copied the previous day I never need the papers, while copying I would also learned them.

Neek • June 15, 2012 10:56 PM

This is old, Cisco Academy and others online learing schools have this problem since 2006. Only a few of them implement strict policies like “paper an pen only tests” forbidding the possesion/use of any electronic gadget in classes.

Ralph Cox • June 16, 2012 10:29 AM

There are two security considerations here. One is for the formative exams, tests that help a teacher diagnose content deficiencies in a student, and the other for summative exams, which are designed to determine the level of content knowledge.

On the formative tests, the reasonable security model is an honor system. These test should not overly impact a grade. They exist to help students learn and teachers teach. They exist to clearly model the information and use cases the students are required to master. A student who cheats on these test will be less likely to pass the summative tests.

The summative tests must be more secure. For distance and online courses they must be given in a secure environment. For many classes in our local community college this is what happens. It is sometimes thought that these summative exam be the same content and look and feel as the formative tests.

Many people think this model is flawed because ‘high stake’ testing is unfair. It measures performance on a single day, not learning. I say it is fair because it gives time for the student to learn content, and a student who has learned content will not feel undue stress on the ‘high stakes’ day.

I see it like the oft-overused athlete analogy. An athlete spends most days practicing, and then plays a few games that count, some of which count more than average. If they or the team does well, they get to go to the high stakes games. If they do not do well on that day, they are the losers. Plain and simple. Most athletes know that winning the high stakes games means practicing every day. In high school and college, students mostly get this implicitly, and they don’t try to cheat.

Of course this model of testing also assumes that the purpose of the course is to learn, and not just get a cheaper degree. The thing I see is that increasingly online learning is not used to provide broader opportunities for education, but to provide cheaper degrees that are to be used from promotions where the knowledge is never really going to be applied. In these cases the motivations make cheating not only acceptable, but a rational decision. If one has been working for 20 years, and know the business, and only need the MBA to get a pay rise, where is the rational to work hard at the MBA. If someone will give you the sheet of paper for 10K, why not take it.

Also along this cheap degree line, another big problem is small test banks. On a computer there are many ways to increase the size of test banks using regular expressions and calculated questions. A good test is no easier to implement than good security. In both cases naive people think they don’t need to pay for either, and then learn that it is not as trivial a process as it appears.

hector • June 16, 2012 1:59 PM

I’ve long thought that exams are themselves a form of “education theatre,” since they do not really measure learning. What happens with exams: student crams for exam, stuffs a whole bunch of information into his short-term memory, writes exam, then dumps the information out of his short-term memory. Very little “sticks” in his long-term memory. The result, of course, is most of what is “learned” in university is not remembered once the student has left university, so what really has been learned?

jb • June 17, 2012 9:07 PM

This could be easily defeated by:

Telling the student only their total score once they complete, until after the due date
Giving them the chance to retake once they know their total score

Jon • June 18, 2012 12:47 AM

Just FYI, FCC radio license exams also draw from a large bank of multiple-choice questions, and both the questions and the answers (all the answers, even the wrong ones) are publicly available.

The testing is closed-book and in person. You can take the tests as many times as you want, and they’re scored only pass/fail.

askme233 • June 18, 2012 7:06 AM

@MikeO:

Great reference. I would just memorize the first ten digits and assume the instructors too lazy to check after that.

psicognito • June 18, 2012 9:46 AM

In our comp sci dept we administer two types of multiple-choice tests. The first type is to help with self-learning: the student gets a right-wrong answer immediately and will receive several versions of the same question if he/she gets it wrong initially. The second type of test is for the grade: there is only a final mark and wrong answers are marked negatively. Both tests select questions randomly from the same question pool, and actual numerical quantities are also randomly generated (from sensible ranges, of course).

local tourist • August 12, 2012 6:09 PM

One of the reasons that it is more feasible to cheat while taking online courses is because students can get by with no relationship with their instructor. Therefore, it is a lot harder for an instructor to detect work that sounds worded suspiciously like someone else.

Cheating in Online Classes

Comments

Leave a comment Cancel reply