Remote Proctoring and Surveillance

Interesting article. There are a lot of surveillance and privacy issues at play here.

Posted on April 29, 2015 at 6:12 AM • 27 Comments

Comments

Clive RobinsonApril 29, 2015 6:52 AM

@ Bruce,

Due to a bug in the NY Times system, it forces some mobiles into an endless series of redirects.

Do you have another link to the article? Or further key words that can be used to search for a copy?

Peter TreiApril 29, 2015 9:03 AM

I've taken a number of online courses from Coursera; for a "Verified Certificate" you have to enroll with a photoID, and submit a typing sample biometric, along with a concurrent head shot from a web cam. At each submission of a homework assignment or quiz, you have to resubmit the typing sample, and a new head shot. Some of the exams are single sitting, and timed. Most are not.

For some programming projects, students checked each other's work. Each of us had to check the work of 5 other random students, so there was a lot of overlap.

It's far from airtight, but you'd have to have physical presence and collaboration with your ringer over the entire duration of the course to cheat.

Of course, these don't give college credit, and are open book.

When I took the CISSP certification exam, I had to travel to a testing center (this seems to be a thriving business), and pay to sit at carrel with a single-purpose computer (no Internet access) after being stripped of everything in my pockets, and take the exam under the eyes of proctors. But that was a single (6 hour!) exam, not an ongoing thing.

I'd really like to see a way for people to get credible college degrees online, at reduced costs - most of the online college degrees given by respected institutions gouge on the price, requiring the same or nearly the same tuition as the on-campus offering. This is understandable; they don't want to cut into their traditional model. However, doing so eviscerates the promise of online education.

Like it or not, in the US, getting a piece of paper (the diploma) is more and more the point of tertiary education; many HR departments will summarily filter out non-degreed applicants for STEM positions, while accepting a degree totally unassociated with the work at hand.

This solution seems a little heavy handed, but is it more intrusive than what I went through for the CISSP? Or what people who have proctored exams in college put up with? I'm not sure.

Peter Trei

rdmApril 29, 2015 10:37 AM

This strikes me as missing the point of education.

Education traditionally provides the student with other people's perspectives, contacts with other people who can be helpful later in life, and experiences which can be useful in dealing with practical issues. There's also the certificate one can get ... presumably that's what's being "protected" here.

But the kind of class which can be cheated on by accessing external resources isn't a class worth spending time on.

If the point of the class is merely to regurgitate rote facts, no one is going to care that you took the class, since anyone can look up those facts.

If the point of the class is to teach how to think and solve problems, then it should be the case that there are a huge variety of examples of those problems. So one part of the solution should be to present the students with a wide variety of individualized problems.

Another part of the solution should be to find multiple ways of engaging the student over time. If that's not being done, then there probably shouldn't be any charge for the class because there's no effort involved in teaching it. And if there's no charge, there shouldn't be any worries about certification...

That said, perhaps what's needed here is less emphasis on "teaching" and more emphasis on supporting the student's learning? As I understand it, there's a lot of work that goes on, in educational settings, which is not strictly "teaching".

I know that a large part of my own education benefited more from being able to talk with teachers and from independent study (including various forms of "lab work") than from tests or even classroom instruction (though I did get some benefit there, also). But I also know that my professional employment has never tapped even a fraction of what I learned in school, so there's that, also...

In related news: markets are not efficient. But how many students would have been marked down in a class for claiming that markets are anything but efficient?

So I guess, ultimately, I'm going to have to say that I really don't care one way or the other how this gets resolved (or does not get resolved).

tyrApril 29, 2015 12:56 PM


The first question that arises for me, is what kind of
a student are they trying to produce with this insane
methodology. If they were after communist apparatchik
or Sturmabteilung then it makes sense. The idea that
this enhances someones ability to become educated is
totally ludicrous.

An education is supposed to teach you how to find the
answers to questions without wasting your time at
re-inventing the wheel or being overwhelmed by all
of the available data humans have accumulated over
millenia.

This whole operation seems to be designed to turn you
into a robotic asshole who follows orders without any
questions for the carrot of a piece of worthless paper.

Supposedly it makes the paper more valuable. Why
would any employer who wants vibrant engaged human
beings hire someone who submits to this. You can buy
all of the robots you need and they get cheaper every
day. Trying to turn humans into robots ruins the only
real advantages humans have over a robotic workforce.

All this program does is hammer a square peg into a
round hole while deforming the peg into a caricature.

From a technical aspect it is done to save the hard
work of programming a decent version of the task by
forcing the student (who should be concentrating on
the test) to do all of the work for a bad piece of
software. I'll bet it costs a lot which seems to be
a basal characteristic of software which is crap
sold to people who should never be allowed to decide
what to use. In that it isn't particularly unique.

LMingoApril 29, 2015 1:23 PM

Proctortrack looks vile, as does the Stoplight program referenced in the article. The latter apparently thinks that, if you aren't friending people on Facebook, you are at risk of dropping out of college.

Alan KaminskyApril 29, 2015 3:17 PM

If the flight control software in an airplane was written by someone who learned programming at Coursera, would you get on that plane?

Would you trust yourself to a neurosurgeon who trained at the Internet School of Medicine?

Like "military intelligence" and "business ethics", "online education" is an oxymoron.

A MoraApril 29, 2015 3:28 PM

A friend of mine was going to Southwestern University (Law School) in Los Angeles. The school required all of the students to install similar software on their Laptops, and required them to complete their online tests and assignments from them.

Students found out that the proctoring software conflicted with anti-virus software, so they disabled it. The it being the Antivirus software. On every machine the students used. They prohibited students from re-installing any security software even between tests.

I have never seen as much malware on one computer. It had been slowed to a crawl by Adware and toolbars, tried to dial 976 numbers that had been out of service for a decade, and occasionally sent out bursts of mail on behalf of Nigerian banking experts.

In addition the proctoring software gave backdoor access to the students machine, apparently with no auditing or oversight.

This industry is a classic example of the dark side of the security industry. It exploits students, who have no voice in the process, to the benefit of lazy school officials who are not held accountable for the impact of their actions.

Peter TreiApril 29, 2015 4:27 PM

I find it interesting that most of the commentary is attacking the whole concept of online education, not just the poor implementation described in the article.

FWIW, I have a degree, obtained on-campus, long before online degrees were anything more than science fiction. My degree is in Biochemistry. Yet, I'm now entering my 32nd year as a full time software engineer. I'm partly self taught, partly taught in non-matriculated extension courses (on campus) at Columbia University, taken after graduation elsewhere.

I started taking Coursera courses to round out some areas where, while I had many years of practical experience, I lacked some of the theoretical background. The online courses do that very well, and letting experts define the syllabus means I get a broad overview, not just what I need for my current task.

The proof of the pudding is in the eating; not what paper the chef has on the wall. But to get interviewed for a job, he needs that paper. You can say that's not the point of a college education as many times as you like: it doesn't change the facts on the ground of what HR departments do, and how people respond to that predicament.

pt

albertApril 29, 2015 4:59 PM

OK, I hate to keep bringing this up, but it's all about the money. These 'software proctoring' companies are just trying for of big piece of the pie. Colleges and universities are forever cutting corners. Witness the rise of 'adjunct professors'. Universities are becoming entrenched, top-heavy dinosaurs, with tens of millions of dollars spent on 'executive' salaries. They are, for all practical purposes, corporations. Cut costs, increase income.
.
So they offered Ms. Chao "...a live human proctor for a fee of $40 per exam....". How nice of them. What they need are some super-expensive class action lawsuits, and plenty of bad-mouthing online. These 'companies' are selling snake oil. Maybe they'll get bit.
.
@Alan K
The best programmers I've ever met had no college training. Some didn't even have college degrees. I've changed my opinion of 'military intelligence'. Top level officers are very intelligent and they know their fields inside and out. The US military is efficient and well run, but they take a lot of heat for the politicians who don't know (or don't care) what they're doing. 'Online education' has its purpose: basically recreating what a good high school education did 40 years ago. College graduates today have the same employment prospects that high school grads did back then. That diploma and $1 to $5 will get you a cup of coffee today.
.....
I guess the point is: no one gives a rats ass about education. It's about making money for the providers, getting high-priced vocational training for the students, and a metric (pray it's accurate) for employers.
.
Am I being too cynical?
.
...

Jonathan WilsonApril 29, 2015 6:37 PM

The big question is why so many HR departments will reject someone who has all the skills and experience they are looking for but no college degree in favor of someone who has a lot less skills but has a degree in some totally unrelated field.

Although I suspect the HR departments aren't interested in the degree because of the particular program they followed but because hiring someone with a degree is the only way to get someone who actually has core base skills and knowledge that they should be picking up in high school but need to go to college to pick up thanks to the broken US school system.

daffodilApril 29, 2015 6:47 PM

"... students would have to pay a fee of $37 to use it."

"[The CTO] came up with the idea for the service after he worked on a project for the Transportation Security Administration which involved scanning airport security video footage for abnormal facial expressions."

Sweet, I get to use it for free while I wait for my groping at the TSA checkpoint!

DanApril 29, 2015 6:49 PM

@Alan K

I'd think a majority of military kids use online ed because it's a field that recruit from the young fresh out of high school. They don't have the luxury of devoting full time to work on degrees if not for these tailored programs.

Nick PApril 29, 2015 7:36 PM

@ Jonathan Wilson

Many do it to argue for more H1-B visas. The argument is that they cant find any Americans that meet their "skill" requirements. So, they need cheaper, smarter, foreign labor instead. A bunch were also suppressing wages by no poaching agreements.

Past those cases, I'd guess organizational stupidity is the case.

Peter TreiApril 29, 2015 8:23 PM

Many HR departments find themselves with far more applicants for STEM
positions than they can handle. I suspect the degree requirement is
just a first filter to cut down the volume a bit. It's very crude,
known to have a high rate of both false positives and negatives, but
better than rejecting at random.

I've heard it justified on the grounds that it shows that the
applicant is capable making and carrying out a long term plan,
but that seems a pretty weak argument.

In any case, once you're more then 5 years past graduation, what
you've accomplished since then is actually much more relevant.

The classic four-year residential college experience and degree is a
very nice thing to have, and there's a great deal to be said for all
the arguments presented above. Unfortunately, its price is going up
so quickly that many students attain it only at the cost of crippling
debt (I managed to put two kids through debt free, but my wife and I
are both well paid software engineers (neither with a CS relsted
degree)).

For many other people, the alternative to online courses isn't going to
college: its not getting any higher education at all.

To analogize, bespoke tailored clothes fit better, make you look
better, and usually are of better materials and construction than
of-the-rack items from Old Navy. Everybody used to get their
clothes that way.

But how many people reading this are wearing them right now?

pt

rgaffApril 29, 2015 8:28 PM

Whenever there are enough more applicants than positions, HR departments get in the business of finding any little excuse to weed out applicants, not of looking to include all the qualified people that apply. Otherwise the bosses trying to hire people are overwhelmed.

So if you're looking for a job, and it's hard to find one due to competition with other applicants, try to get around the HR dept if you can figure out a way... Be creative and sneaky if necessary! And yes, these are the situations where people go back to school just for the piece of paper!

Mark JApril 29, 2015 10:15 PM

Our school uses a similar product by a company called Respondus. It requires the user to scan around their desk and room with a webcam to verify no alternate means of accessing the internet. The browser (Lockdown Browser) won't go anywhere but the designated test. The webcam software part watches for eye movements away from the main screen to detect the use of other devices. Very few instructors use the webcam part, but some use the browser, although using the browser itself is pretty much useless if the student is at home and has access to another web-enabled device.

As it's been stated here by others, the best instructors simply design tests in a way that require students to use available resources to answer questions.

Turnitin is actually pretty useful. I've seen papers come back showing they were more than 50% copied from other sources.

Ole JuulApril 29, 2015 11:39 PM

This software requires MS Windows. There are very defensible ideas about why students shouldn't be using that, and why the school is irresponsible in requiring it. In addition, I can't imagine installing anything which doesn't have publicly available source code. Personally, any discussion beyond this much is not practically applicable in this case. The academic discussion as on this blog, however, is most useful.

Peter A.April 30, 2015 4:23 AM

Re: HR departments in large companies.

There's imminent conflict in having a typical HR department and getting the most skilled workforce on board.

Project managers don't have time to go through piles of job applications or do interviews, they need to get the work done. They expect that HR selects the best candidates and are wiling to spare only an hour or two a week to interview prospective employees. On the other hand, the people typically employed in HR departments are not technical people and they don't have domain knowledge relevant to any projects going in the company, do they don't really know what to look for in the job applications for a particular project besides scanning for some keywords - and the applicants soon learn to pepper their applications with.

One solution would be staffing HR depts with technical people who can spot a real and relevant achievements in job applications; but it is not "in fashion" and/or seen as a waste - and few technical people would actually want to take such positions.

This is a hard problem in general and several partial solutions exist but few companies really care - and not all would actually benefit from a higher quality of recruitment process. Startups do - but big dinosaurs often do not: they mold all new hires into the same shape, so it is not important what shape they had before.

kronosApril 30, 2015 8:24 AM

I got my bachelor's degree online and found it to be as good if not better than the physical classes I took in the past. Many of the instructors gave 'open book' tests which I (wrongly) assumed were going to be easy. It took a lot of time going through the book AND listening to the audio/video lectures to get the answers they wanted. They often stressed that 'Google answers will not be accepted' and for the most part that was enforced.

Peter GalbavyApril 30, 2015 9:01 AM

I guess it's a nice way of habituating the next generation to the in-office monitoring they should expect to be under when they go work for megacorp after leaving their on-line college.

bobApril 30, 2015 9:23 AM

@Alan Kaminsky

At the moment, to get a job with us, you must have completed Coursera's "Functional Programming Principles in Scala" Traditional education is out-dated.

albertApril 30, 2015 12:25 PM

Re; HR Departments.
@Peter A. hit it: "...the people typically employed in HR departments are not technical people and they don't have domain knowledge relevant to any projects going in the company..." The bigger the company and/or the more technical the position, the worse the problem. Workers are just a commodity; easily replaceable. There are alternatives to deep-vetting job candidates.
......
Re: H1B visas: Don't believe the propaganda spewed out by the Corporatocracy; it's a load of bull. It's all about money. A PhD programmer from India can come to the US and work for peanuts, compared to an experienced US citizen. The problem is this: after they get enough job experience, they move to 'market price' jobs, displacing the citizens who can't afford to work for below market wages. Getting rid of H1-B visas would reduce unemployment, oh and profits....
......
...

Alan KaminskyApril 30, 2015 1:34 PM

@bob At the moment, to get a job with us, you must have completed Coursera's "Functional Programming Principles in Scala" Traditional education is out-dated.

I really hope your company does not write airplane flight control software or do neurosurgery.

Leave a comment

Allowed HTML: <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre>

Photo of Bruce Schneier by Per Ervland.

Schneier on Security is a personal website. Opinions expressed are not necessarily those of IBM Resilient.