Volkswagen and Cheating Software

Portuguese translation by Ricardo R Hashimoto

For the past six years, Volkswagen has been cheating on the emissions testing for its diesel cars. The cars' computers were able to detect when they were being tested, and temporarily alter how their engines worked so they looked much cleaner than they actually were. When they weren't being tested, they belched out 40 times the pollutants. Their CEO has resigned, and the company will face an expensive recall, enormous fines and worse.

Cheating on regulatory testing has a long history in corporate America. It happens regularly in automobile emissions control and elsewhere. What's important in the VW case is that the cheating was preprogrammed into the algorithm that controlled cars' emissions.

Computers allow people to cheat in ways that are new. Because the cheating is encapsulated in software, the malicious actions can happen at a far remove from the testing itself. Because the software is "smart" in ways that normal objects are not, the cheating can be subtler and harder to detect.

We've already had examples of smartphone manufacturers cheating on processor benchmark testing: detecting when they're being tested and artificially increasing their performance. We're going to see this in other industries.

The Internet of Things is coming. Many industries are moving to add computers to their devices, and that will bring with it new opportunities for manufacturers to cheat. Light bulbs could fool regulators into appearing more energy efficient than they are. Temperature sensors could fool buyers into believing that food has been stored at safer temperatures than it has been. Voting machines could appear to work perfectly -- except during the first Tuesday of November, when they undetectably switch a few percent of votes from one party's candidates to another's.

My worry is that some corporate executives won't interpret the VW story as a cautionary tale involving just punishments for a bad mistake but will see it instead as a demonstration that you can get away with something like that for six years.

And they'll cheat smarter. For all of VW's brazenness, its cheating was obvious once people knew to look for it. Far cleverer would be to make the cheating look like an accident. Overall software quality is so bad that products ship with thousands of programming mistakes.

Most of them don't affect normal operations, which is why your software generally works just fine. Some of them do, which is why your software occasionally fails, and needs constant updates. By making cheating software appear to be a programming mistake, the cheating looks like an accident. And, unfortunately, this type of deniable cheating is easier than people think.

Computer-security experts believe that intelligence agencies have been doing this sort of thing for years, both with the consent of the software developers and surreptitiously.

This problem won't be solved through computer security as we normally think of it. Conventional computer security is designed to prevent outside hackers from breaking into your computers and networks. The car analogue would be security software that prevented an owner from tweaking his own engine to run faster but in the process emit more pollutants. What we need to contend with is a very different threat: malfeasance programmed in at the design stage.

We already know how to protect ourselves against corporate misbehavior. Ronald Reagan once said "trust, but verify" when speaking about the Soviet Union cheating on nuclear treaties. We need to be able to verify the software that controls our lives.

Software verification has two parts: transparency and oversight. Transparency means making the source code available for analysis. The need for this is obvious; it's much easier to hide cheating software if a manufacturer can hide the code.

But transparency doesn't magically reduce cheating or improve software quality, as anyone who uses open-source software knows. It's only the first step. The code must be analyzed. And because software is so complicated, that analysis can't be limited to a once-every-few-years government test. We need private analysis as well.

It was researchers at private labs in the United States and Germany that eventually outed Volkswagen. So transparency can't just mean making the code available to government regulators and their representatives; it needs to mean making the code available to everyone.

Both transparency and oversight are being threatened in the software world. Companies routinely fight making their code public and attempt to muzzle security researchers who find problems, citing the proprietary nature of the software. It's a fair complaint, but the public interests of accuracy and safety need to trump business interests.

Proprietary software is increasingly being used in critical applications: voting machines, medical devices, breathalyzers, electric power distribution, systems that decide whether or not someone can board an airplane. We're ceding more control of our lives to software and algorithms. Transparency is the only way verify that they're not cheating us.

There's no shortage of corporate executives willing to lie and cheat their way to profits. We saw another example of this last week: Stewart Parnell, the former CEO of the now-defunct Peanut Corporation of America, was sentenced to 28 years in prison for knowingly shipping out salmonella-tainted products. That may seem excessive, but nine people died and many more fell ill as a result of his cheating.

Software will only make malfeasance like this easier to commit and harder to prove. Fewer people need to know about the conspiracy. It can be done in advance, nowhere near the testing time or site. And, if the software remains undetected for long enough, it could easily be the case that no one in the company remembers that it's there.

We need better verification of the software that controls our lives, and that means more -- and more public -- transparency.

This essay previously appeared on CNN.com.

EDITED TO ADD: Three more essays.

EDITED TO ADD (10/8): A history of emissions-control cheating devices.

Posted on September 30, 2015 at 9:13 AM • 76 Comments

Comments

AlanSeptember 30, 2015 9:57 AM

Can anyone quote and cite and specific rule(s) VW is alleged to have violated? I ask this simply because so far I haven't seen anyone quote an actual rule. The various governments defined the tests that VW's cars had to pass, the VW designed them so they passed the tests. Unless there is a specific rule prohibiting what they did, that is not "cheating".

Anonymous 1September 30, 2015 10:02 AM

Alan:
Look up Defeat Device, they are highly illegal.

The US EPA actually opposes what Bruce suggests on the ground that it'd make it easier for the aftermarket to violate emissions regulations but I suspect trying to stop that with DRM is going to be just as effective as stopping music and movie piracy (i.e. won't work) and that monitoring the cars on the road for excessive emissions would be far more effective (there are apparently laser based systems capable of doing that).

DJMSeptember 30, 2015 10:21 AM

The links to other essays right at the end have messed up URLs --- looks like some quotes don't match or something like that.

ChuckBSeptember 30, 2015 10:36 AM

Audi's branding ads following its recent diesel powered victories at LeMans were tagged "Truth in Engineering". Need more be said?

TJ WilliamsSeptember 30, 2015 10:55 AM

Some SIGINT agencies could also fool people into believing ECC is crap because quantum computers are "just around the corner".

Michael SierchioSeptember 30, 2015 10:58 AM

This brings to mind another instance of a manufacturer gaming the testing methods, in the older case it was Barclays cigarettes - which, when tested for tar, performed admirably. But the filter had air inlet grooves that were closed when human users smoked.

In the case of VW, you can argue that it was a net with for VW and the consumer (those who bought the cars), but at the cost of everyone who breathes air.

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1747993/

Bob S.September 30, 2015 11:07 AM

"There's no shortage of corporate executives willing to lie and cheat their way to profits."

Amen.

Also, there are plenty of military and government personnel willing to lie and cheat to get elected or increase their power, budgets and domination over the people aka "adversaries".

And, they cooperate with each other.

Another good point: Corporations, even the biggest name brands, are willing to cheat for short term gain, then let the next guy clean up the mess. The VW CEO is gone of course, after making hundreds of millions of pounds in salary. Also, he left with a perpetual one million pounds per year severance package. Not too shabby. And no legal sanctions, either.

We need to start educating the kids right now about how bad it is and give them practical instruction for protecting themselves while using electronic devices. I think a few people are starting to get the message. But, not enough.

Irrelevant aside: This week I found my electric utility company opted me and all of it's customers into targeted tracking and ads by at least 243 companies. The only way out is to "opt out" digging and clicking through the fine print on their web site. Even then, when you opt out n the form they now have your ip address. There is no way to talk to someone in person or on the phone about this. You must use the web form, write a letter or use email. Once again, the email address becomes their property when they get it.

I am still hoping technology may save us. Lately though, it's going the other way.

GuillermoSeptember 30, 2015 11:10 AM

Alan, it is illegal under Section 203 (a)(3)(b) of the Clean Air Act (CAA), 42 U.S.C. Sec. 7522(a)(3)(b). Quoting "prohibits the manufacture, selling, or installation of any device that bypasses, defeats, or renders inoperative a required element of the vehicle’s emissions control system".

K.S.September 30, 2015 11:25 AM

Corporations will simply lobby for more laws making it even harder to test for cheating. If it is illegal to look for these issues, then private sector security researchers can be threatened with lawsuits and chilled this way. Than regulatory capture and consultant revolving door, and you are safe from any kind of scrutiny.

JPRSeptember 30, 2015 11:33 AM

Alan: I'm reasonably sure the various governments defined limits to how much the cars could emit, not "tests that the cars had to pass". The tests were checking to see if the cars were complying, not the situation that they were being tested for.

meSeptember 30, 2015 11:33 AM

I guess things are so bad in the world today that when I saw the story my first thought that this was just the US warning the Germans to toe the line. And the "test" outfit was just another form of Parallel Construction laundering the results of economic surveillance of German manufacturers.

BLSSeptember 30, 2015 11:51 AM

I'd love to go all electric, I just recently got a new car and was giving Tesla a serious look... but the limited range, lack of viable charging options in many places, and of course the rather expensive price tag... all had me liking the theory, but just not ready to pull the trigger on such a vehicle right now.

It's an ideal "other" car. One you drive around locally, but never too far to risk running out of juice. But not many people can afford a $70kUSD+ vehicle simply to drive around town in.

I actually picked up a VW, not a Diesel thankfully, but I likely would have avoided it had this story broke before I purchased.

As for this issue itself. This is where actual diligence comes into play. They set up a routine that companies could count on, and were able to use that reliability to their advantage. Had the various agencies been the only ones testing, this would never have been caught.

Being completely predictable when it comes to testing criteria tends to invite people to exploit that for their benefit.

Peter A.September 30, 2015 12:07 PM

@Amiablechief:

It's not (yet) the time to go electric - current technology is ok-ish for light passenger cars, but not usable for heavy vehicles. Even if it was or a suitable solution was found next year, there's not enough installed power in generation plants - and not enough electric energy distribution capacity. Scaling this up will take decades.

On top of that, there's probably not enough raw materials on Earth to make that many batteries for all those cars... not mentioning the nearly non-existing recycling protocols...

So, we're quite far from switching to electric cars on a large scale. Before that happens, other options may become available, only time could tell.

K-VeikkoSeptember 30, 2015 12:12 PM

> We need better verification of the software that
> controls our lives, and that means more -- and more
> public -- transparency.

This is not a matter of verification, transparency or least a matter of better regulations. Not a matter of means to be applied to "other people" that "they" don't get pissed on. It is a question of morality and freedoms.

Freedom of speech means the freedom to lie, tell lies; to write any computer-code to cheat others. This must be respected so the consumer knows the name of the game and can have a choice to buy or not to buy a car with closed-source codes (or any criteria they choose).

If this freedom to lie is effectively forbidden by any organisation, e.g. the state, it only means that this organisation preserves itself the privilege to tell lies. – And you don't have to go far to find a lot of good reasons to lie; beginning from "national security", going through "protecting" domestic car manufacturers and ending down to small talk with a local cop.

If these state sponsored liars are caught up, the get newer sentenced. Instead a medallion and a good pension is awaiting.

keinerSeptember 30, 2015 12:30 PM

Windows is b*llshit software al large, insecure by design, with "telemetry" and NSA backdoors. At LEAST as criminal as a little NO2 more.

I would like to see mass torts against MS within the next 5 years....

AnuraSeptember 30, 2015 12:36 PM

@Bob S.

I'm hoping that we are heading for a world of post-labor scarcity, where technology and industrialization makes it so that everyone in the world can live in relative comfort without every able-bodied person having to work. When that happens, it becomes a lot easier to convince us to abandon capitalism and move to an economic system that minimizes incentives for screwing over the public.

The reason I believe we will abandon capitalism is because if we don't, then we end up with a society in which everyone who has wealth inherited it, and everyone else makes just enough to get by. It's not a society that can survive; either democratic change or violent revolution will occur when most of the population is forced into a crappy situation for long periods of time.

Money is the driver of corruption, and a democratic, socialist society without aristocracies and without high inequality can avoid the vast majority of that corruption.

BLSSeptember 30, 2015 12:43 PM

@Anura

Money is not the driver of corruption. Money is a stand in.
"Wealth" is the driver. And wealth can be property, food, etc.

Wealth, in whatever form a society places emphasis on, brings with it influence.

And no matter what kind of society you talk about (democratic, socialist, etc), there is the potential for systems to be corrupted and used to benefit a select few.

Communist nations have the same issues with inequality as capitalistic ones, it's simply a matter of who benefits, and to what degree.

I'd love to see this mythical future you speak of that we occasionally see in such fiction as Star Trek, but it's going to be a VERY long time before it's ever going to have a chance to exist.

AnuraSeptember 30, 2015 1:19 PM

@BLS

Post labor scarcity will occur in many of our lifetimes. Robotics and AI are already getting to the point where they will be able to replace human cab and truck drivers. It won't be long before most low-skilled labor can be replaced - assembly line workers exist primarily because of limits in things like dexterity and object recognition for robots, which won't be the case in a couple decades.

In a true democratic socialist society, wealth is significantly limited in the sense we use it today. There is no private property ownership (as in land, structures - this is separate from personal property like art, clothing, cars, etc.), there are no corporations, so you are left with two primary ways to grow wealth: spending less than your income, and interest on your bank account, both of which become fairly unreliable. The vast majority of wealth is thus non-transferrable, and immaterial. After that, you have art and collectables, in which it becomes very difficult to become a millionaire off of when there are very few millionaires.

Countries like the USSR were not democratic, socialist countries, they were socialist dictatorships - those that gained power used their influence to live extravagantly, and everyone else was poor. It's a poor model, and no one is really advocating anything similar.

WaldoSeptember 30, 2015 1:27 PM

Pardon my ignorance, but *why* did VW install this cheat device? Why not have the engine always operate with lower emissions? Is it because with the lower emissions (i.e. during testing when the algorithm kicked in) the performance goes down and then they cannot claim their stated figures?

interested ignoramusSeptember 30, 2015 1:33 PM

How easy is it to make certain that "transparent source code" actually matches what it purports to?

AnuraSeptember 30, 2015 1:35 PM

@Waldo

Correct. They need to meet emissions standards, and claim high fuel efficiency since that is why people buy diesel in the first place, so they make it fuel efficient in most scenarios, and low emission when testing.

mark brownSeptember 30, 2015 1:44 PM

I am responding to the tone of the comments. It makes out that the people high up in corporates and government are the bad guys and we are all victims.

Read a book called Freakonomics. The evidence is that we are all very prone to cheating IF we think that there is almost zero chance of being found out. It is human nature.

Obviously we despise this kind of behaviour, but unfortunately, stealing or cheating from our fellow primates has been imprinted in our DNS over hundreds of thousands of years. You see monkeys stealing a neighbours stash whn they are not looking. It's only social peer pressure that keeps us on the straight and narrow.

Having read this blog, I would say that "firmware cheating" is more of a threat to the general population than AI.

JaySeptember 30, 2015 2:12 PM

So long as settings are stored in lookup tables then adjusting your vehicles performance at will is just a matter of poking the right location with the desired value.

VW used inside knowledge of the tables to implement the tweaks. But now that average owners are aware that their own late model vehicle likely has similar means of adjusting performance characteristics expect to see many DIY hacks that give the owner control over tests, performance and mileage.


MaxSeptember 30, 2015 3:51 PM

Waldo, I'd like to know too. Improving fuel efficiency is an obvious motive, but not the only possibility. The cheat could have been done to fix a long term reliability problem.

J. PetersonSeptember 30, 2015 4:03 PM

A useful model for how to prevent (or at least try to regulate) cheating with embedded software is how the state of Nevada handles electronic gaming machines (slot machines, video poker, etc.). They have very strict requirements for being provided with the source code for all games, and a strict audit procedure when modifications are made.

BoppingAroundSeptember 30, 2015 4:27 PM

[re: basic income] Anura,
OTOH, I find it pretty easy to paint a picture where one would have to forfeit
whatever last shreds of freedom one has for that 'benefit'. Will it all be that rosy?

passerbySeptember 30, 2015 4:33 PM

It's fairly easy to estimate how many people died from the additional NOx emissions.

Perhaps the number is zero. Perhaps not.

The millions in CEO compensation cover the risk of slammer time, don't they?

BooSeptember 30, 2015 4:45 PM

I'm a Golf Diesel driver. Where I come from, most cars are diesel. So now my car lies to me. Just watched my local news and apparently if you get your car re-programmed, it can lower the MPG. The fundamental reason for having the car was for economy. My car is going to an independent garage that specializes in German vehicles, particularly tuning. So I'll get their word on things before I trust VW again.

The whole area of IoT hacking is something I'm gonna enjoy looking into in the coming months and years ahead. Just recently I was talking to a sales guy for a solar panel company on behalf of some family members. As part of the package they were pedaling some fancy schmancy remote monitoring solution that sends 'important data' back to some mothership to ensure that it was 'working'. I told him that I could tell myself if the damn was working and that I would ring them if it wasn't. My point is that I'm not actually anti-IoT .... it's just that I can't trust vendors who are less than transparent as to what on earth is going on with the stuff they've sold us.

steve37September 30, 2015 5:03 PM

In-Use Emissions Testing of Light-Duty Diesel Vehicles in the United States


http://www.theicct.org/sites/default/files/publications/WVU_LDDV_in-use_ICCT_Report_Final_may2014.pdf


Der vorläufige Stand der Dinge in Sachen Diesel-Stickoxide und Volkswage (german)

http://www.heise.de/autos/artikel/Der-vorlaeufige-Stand-der-Dinge-in-Sachen-Diesel-Stickoxide-und-Volkswagen-2832601.html

Wide range of cars emit more pollution in realistic driving tests, data shows

http://www.theguardian.com/environment/2015/sep/30/wide-range-of-cars-emit-more-pollution-in-real-driving-conditions-tests-show

milkshakenSeptember 30, 2015 6:02 PM

Waldo, Max: VW originally licensed clean diesel emission control system from Mercedes but it was way too elaborate for cars like Jetta (diesel exhaust system that has catalyst, exhaust recirculation and and urea-based exhaust control liquid). The system was too expensive for VW - and VW was losing a chunk of profit margin on cars sold in US because of this emission control system. So, within two years they replaced the emission control with their own, simplified system which never quite worked as promised, and they realized the only too late that the way they could meet the emission standards was to dramatically increase the consumption of the urea-based emission control fluid (which would be a big nuisance to the customers). As a stop-gap "solution", they were tweaking the software, gradually in more and more blatant way - and when they were found out in 2014 they still did not have a fix ready, a fix that would work without the cheating software. Basically, they were bullshitting themselves for too long, so they when they promised to fix the problem in 2014, they did not and instead they tweaked the software once more, and pretended as if everything was fine...

Harry JohnstonSeptember 30, 2015 6:20 PM

"Stewart Parnell [...] was sentenced to 28 years in jail [...] That may seem excessive, but nine people died"

Far from excessive, the sentence seems rather inadequate to me. Dzhokhar Tsarnaev only killed three people, and he was sentenced to death.

Sancho_PSeptember 30, 2015 6:34 PM

I don’t know how it was done - and this is a serious restriction to my point.
But I know there is a way to avoid what was reported as to “switch” pollution control between drive and test mode.
No software error, no accident, it couldn’t be “detected”, no one could call that cheating.

The reason is that regulatory standards often do not represent the real world scenario, e.g. when lifetime of energy saving bulbs is defined with the socked facing the floor but most appliances have it reverse (overheating the electronics).

The VW “cheating” points at very poor regulatory standards.

AnuraSeptember 30, 2015 6:34 PM

@BoppingAround

OTOH, I find it pretty easy to paint a picture where one would have to forfeit whatever last shreds of freedom one has for that 'benefit'. Will it all be that rosy?

The dystopia will come first, but it will be temporary. Your freedoms are eroding, even with laissez-faire capitalism (which most libertarian capitalists have a very strange definition of "freedom"). The direction we are heading deeper into is a world of corporate and government secrecy, corruption, and tight tracking of individuals, with great inequality. What replaces that is likely to be a socialist state without corporations and limited-to-no private property (structures, land) ownership; I'm optimistic, but how much it resembles my vision remains to be seen (which, I'm not going to get too detailed on my post-capitalist vision in this thread, since it would be a large wall of text - if you are interested, I can write it up for the squid post). As it stands, I'd wager it will be better than what we have today, and the big-data intrusions into your life will be significantly less.

Alyson BergerSeptember 30, 2015 7:13 PM

VW was greed and stupid. They have mastered Diesel engines, even so that Audi has been winning Les Mans prototype races and championships with their award winning engines.

But the stupidity came in the form of trying to create small diesel engines for small cars. Diesel engines are supposed to be big and strong. But they wanted to grab the moment and increase market share with the idea of the small clean diesel. Since its engineers were not able to deliver, greedy management launched it anyway.

This is a clear case of the marketing department taking over engineering.

Unfortunately and dangerously new technology will allow many people to try to mask results and defraud the public. I agree with you that it will be an incentive for other greedy managers to try to get away with other scenarios in which their engineers are not able to deliver.

Stuart LynneSeptember 30, 2015 7:20 PM

"and temporarily alter how their engines worked so they looked much cleaner than they actually were"

That is incorrectly worded.

While being tested the engines where cleaner. And if not disabled would also run that clean in normal operation.

The engines are capable of running clean. They are not capable of running clean and delivering an acceptable level of efficiency and performance while doing so.

Gas Mileage, Performance, EPA compliance; pick two.

milkshakenSeptember 30, 2015 8:52 PM

I predict that even if VW brings the cars into compliance with US pollution standards, you can expect these trade-offs with your Passat/Jetta: 1) The milage per gallon decrrease will by 10% 2) You will notice worse performance with cold engine - i.e. hick-ups on idle on the first intersection stop, especially if the A/C is running 3) You will notice worse acceleration when driving uphill and with a fully loaded car (i.e. the whole family + stuff) 4) the consumption of the blue urea liquid will dramatically increase - you will need to re-fill it about as often as gas tank every few hundred miles 5) And obviously there will be no more "clean diesel" tax write offs, and the re-sale value of used VW cars will go down by few thousand.

I would expect Audi diesels being easier to fix and far less negatively affected (in terms of value and performance) than small VW four-cylinder diesels

acidtongueSeptember 30, 2015 10:09 PM

This kind of cheating has been, and is probably still being done, by the pharmaceutical companies. They have been caught misleading the public and the physician community time and time again and multi-billion dollar fines have been levied. They seem to see that as just the cost of doing business. Manipulating the software is just a more concealable way of cheating, but the companies don't really seem to care if they are discovered as long at their profit is more than the cost of the fines. Now is the execs responsible got heavy jail sentences then maybe that would be a deterrent.

MikeASeptember 30, 2015 10:43 PM

@J. Peterson

Note that at least one fairly significant hack of slot machines was done by a Gaming Commission inspector. Who watches the watchers?

I agree that careful vetting of the transformation of inspected source code to presumed kosher object code would be a very good idea.

Much like Western Civilization.

CouldntPossiblyCommentOctober 1, 2015 3:15 AM

I'm not sure why we're focused on source code transparency specifically here, beyond the usual w.r.t. security.

The end of the article actually touches on the real root problem - talk of conspiracy, knowingly doing something - exactly how do you prove that a software design was deliberate rather than a bug, or even someone adding what they thought was a feature with unintended side-effects? Transparency of code isn't useful at all for this. You're asking for transparency of corporate operation, of git commit history, of interviewing software developers, decision makers, email threads, meeting minutes, and what people are thinking or were thinking. How ironic that it is implicitly proposed that everyone should have access to everything in a blog that so often decries invasion of privacy and global surveillance.

Catching people commiting crimes is always easier if their entire lives are available for peer review. Are we saying that complete surveillance is ok if they work for a particular company or on a particular product?

Don't get me wrong; transparency is a very nice concept and for critical systems, it probably is reasonable to argue that the source code should be legally available for public peer review. That's just good sense for safety purposes. I'm just pointing out that it's easy to argue for transparency in a case where transparency would have solved a problem, but simultaneously arguing for privacy in a case where privacy is desired creates a tension between the two.

I'd concur with earlier posters - the viable root problem to solve is that government(s) regulation testing (the verification part) is too different from reality & thus detectable or causes software to operate in a different way even just by context. As Feynman famously said 'reality cannot be fooled'. Genuine Quality Assurance testing would have found this. Malfeasance is no different to incompetence - we don't really want either, but both are a given part of human nature that will never be changed. For systems we genuinely need to trust, the usual QA department needs to be independent rather than part of the company. That is the key change needed for safety critical systems (and it's not a huge stretch to argue that fossil fuel emissions is a safety issue).

There's no need for a revolution, or regime change, or transparency of corporate operations. Just actually require independent verification of the behaviour in real world scenarios. The problem we dance around is that that's expensive & requires diligence, something not historically associated with collective-based organisations.

Who do you trust to verify?

BlakeOctober 1, 2015 3:23 AM

Not sure if it's mentioned in any of the articles Bruce linked to, but within last couple of weeks it has come out that VW has not been the only company doing this.

Not intending to condone or minimize what they have done, but among other manufacturers with similar systems we have at least Skoda, SEAT, and Audi. Mercedes Benz has now been accused of it too but they are denying it.

gregsfcOctober 1, 2015 4:25 AM

I know cheating is cheating, but it would be nice to know which pollutant, specifically exceeded the limit. Everything being reported tells me, that it's NOx. And if its NOx, then I say "good for VW" for trying to bring affordable diesel engines to the masses by letting a little bit of a harmless gas out of all their 2.0 TDIs into the atmosphere in North America's biased regulatory environment.

Of course, if it is NOx and only NOx that was exceeded and not something really bad for us to breathe like particulates of matter or hyrdrocarbons, then that explains alot of things. Now we know how VW designed and sold the only light-duty diesel for all 50 states that does not require urea post injection in the tailpipe; it explains how VW and only VW offered a light-duty straight shift diesel; and it explains how VW and only VW kept the premium for a light-duty diesesl affordable, which I have been waiting on for many years in a compact or full-size basic truck, but will not happen due to the EPA's biased regulations on diesel power trains.

Ram recently introduced an Ecodiesel, 3.0 V6; mpg up to 29 highway. Fantastic, except that the starting price for the most basic configuration and trim they'll sale the diesel in is over $38K added to a basic Ram truck price of around $26K. The main driver of the cost is NOx compliance, which forces engineers to design diesel engines, which naturally combust using very little fuel and lots of air and oxygen, which is most basic to their inherent advantage over gas cars, and re-design them to get rid of the NOx that is produced via lean combustion (select catalyst reaction via urea post injection), or, as in the case of Volkswagen, force them to run cycles of rich combustion so that the NOx is not created to begin with (LNT). Of course the latter strategy used by VW is counter to lean combustion and is counter to superior fuel economy provided by well-designed diesel power trains.

And so, in this case, if it's only NOx, due to the fact that NOx is only harmful when it combines with VOCs and other chemicals in our atmosphere that should be regulated out instead of NOx, then the only thing I regret is that VW got caught. Without this unshakable insistence by our regulatory agencies in North America that diesels must be on par with gas cars with respect to NOx (which is not true any where else in the world, but Japan is close), then we'd have fantastic, affordable diesels here in America that are still just as clean-burning overall as gas cars.

Peter GalbavyOctober 1, 2015 5:10 AM

@Blake,

Skoda / SEAT / Audi are all subsidiaries of VW - which in turn implies more collusion than the denials of knowledge by the VW execs would cover

qwertyuiopOctober 1, 2015 5:21 AM

@Blake - Skoda, SEAT, and Audi are all part of the Volkswagen group and use VW engines in some of their cars, so this is all part of the same problem.

CrudeOctober 1, 2015 8:53 AM

then I say "good for VW" for trying to bring affordable diesel engines to the masses by letting a little bit of a harmless gas out of all their 2.0 TDIs into the atmosphere in North America's biased regulatory environment.

Blessed are the ignorant...

Steve FriedlOctober 1, 2015 9:09 AM

This kind of cheating was only possible because mileage and emissions are tested separately: testing them *at the same time* eliminates this kind of arbitrage.

VW reportedly will issue a software fix (via recall) to turn on the pollution features all the time, presumably at a cost of performance or mileage, but California (at least) will require proof of application of the fix before the cars can be registered again [not sure when this kicks in].

I expect many other US states to do the same.

aikimarkOctober 1, 2015 9:56 AM

I have lawyer friends, specializing in DUI cases, who can't give proper defense because both the breathalyzer hardware and software aren't available to them in NC.

albertOctober 1, 2015 12:17 PM

Frankly, I was surprised at the reaction to this VW emissions thing. Clearly, VW hasn't figured out a way to detect a probe in the exhaust pipe. How would the EC system know when the engine is being monitored? It's because the tester is uploading data from the ECS. So we expect to fox to tell us how many chickens are left in the coop? What could possibly go wrong? Why not just probe the exhaust? Anyone with expertise in this?
. .. . .. _ _ _

DasDubOctober 1, 2015 12:43 PM

The high NOx is a fault of eliminating soot. Mostly harmless carbon that will fall to the ground, but not the most pleasant to look at, or an invisible gas with many contradicting studies as to it's danger or lack there of.

These issues are a drop in the bucket in the grander scheme of cars as the older standards are 100x or more that of the newer and unrealistic standards placed on US diesels. I can guarantee that there are as many or more of these older standards cars out there that will not only continue on longer then these new high tech reduced emissions vehicles, but have already put out more emissions in the same period then these controversial cars have in their lifetime. Not the first time the US emissions have driven diesel manufactures to cheat and likely just the beginning of a larger look into the others as well.

@gregsfc
Some of the effected models did include urea treatment.

These systems no matter which kind or brand, DPF/SCR/DEF/LNT/Soot Trap, all require some period of less then peak efficiency, adding extra fuel and/or restricting intake air to raise the Exhaust temps to ridiculously high levels, either for the emissions gear to function or to clean it out as it collects.

Gerd October 1, 2015 2:09 PM

I am German, but I am not really proud of it what I
read here. Its blaming us and destroys a usually
good reputation.

Alan YoderOctober 1, 2015 5:36 PM

I saw a very detailed explanation from an automotive engineer that was over my head. In brief, it mentioned closed loop and open loop operation (these are terms from control theory). The second is when the car emits 40x the pollutants, and occurs when you step on the gas. The take home for me was that VW may have not actually done anything nefarious. Rather, it appeared to me that the emissions test procedures are not thorough enough.

Another AlanOctober 2, 2015 2:03 AM

"cite specific rules"? It's called fraud: claiming one thing and doing another. No other rule matters

JpbqcOctober 2, 2015 12:33 PM

Cheating software is used on a large scale for tax evasion in restaurants and retail stores. Zapper or phantomware software allows businesses to erase or modify sales transaction records in electronic cash registers. Some governments have tackled the problem, for instance in Québec, as reported by this story:


In Quebec, the government made it mandatory in 2011 for most restaurants to outfit cash registers with so-called sales-recording modules, or "black boxes," which record data on sales transactions that businesses are then required to relay regularly to the provincial tax agency. The Quebec government says the black boxes helped generate $160 million in additional tax revenue in the first year.

MarkHOctober 2, 2015 12:42 PM

The first instance I remember of using firmware to break the law in a quotidian piece of equipment (as distinct from machines used in gambling, for example):

In the late 1990s, I recall that some filling-station "pumps" (the apparatus that dispense and meter the fuel) in the northeast of the USA were found to have modified firmware that short-changed the customers by indicating a larger quantity than was actually dispensed.

The hack was made with a little subtlety:

First, the metering was normal up to 5 gallons. This was important because sometimes officials check the accuracy of such pumps by dispensing fuel into a calibrated container, never exceeding 5 gallons in capacity ... so the hack would go undetected by such testing.

Second, the metering was normal after 10 gallons. This was important because customers filling large tanks or containers were more likely to notice a systematic mis-metering: a 1% defect would stand out when filling a large tank.

Between 5 and 10 gallons, the pumps would overstate the incremental quantity by a modest amount -- I don't remember the number, but it was quite a small percentage.

For the average purchase, the overcharge to the customer was only on the order of one percent. But the mark-up for fuel at filling stations is so small, that this was quite profitable to the operators!

This highly illegal firmware was NOT like VW's crime, in that the pump manufacturers were supplying valid firmware. The filling stations were paying to have the criminal firmware installed in their pumps.

MarkHOctober 2, 2015 12:47 PM


On How it Was Done

Many readers of the blog may know this by now, but the VW cheat doesn't rely on an obvious but unreliable clue like connection of an external computer to the ECS (sometimes made during emissions testing).

It is a more sophisticated algorithm that synthesizes multiple inputs, including:

• steering wheel angle (wheel roughly centered and not moving for an extended time)
• range and steadiness of engine RPM
• intake air pressure

The last clue is of particular importance: when a car is in motion, air flows cause large changes in the under-hood air pressure.

On the Significance of How it Was Done

1. The software cheat was a fairly sophisticated engineering project.

2. Its complexity and specificity makes it obvious that it was intentional -- it is implausible that such an algorithm could result from an innocent error.

This second point is an argument in favor of using software transparency as a means to protect against this kind of illegality.

Of course, criminal organizations (a category into which VW sadly must now fall) would work hard to find tricky ways of making their cheats look innocent ... but this isn't a good argument against transparency as a safeguard.

There are universities around the world with computer science and automotive engineering programs. With transparent ECS design, these teams of faculty and students might not be able to prove that a failure to perform is intentional, but it would be much easier and less costly for them to conclude, "this doesn't meet the specs and must be corrected."

And that's the main thing.

On Aftermarket Hacks

Yes, transparency could make it easier to make aftermarket defeat devices. However, after decades of "chipping" cars, only a tiny fraction seem to have ever been modified in this way ... most vehicle owners have no interest in making such mods, and they remain rather costly.

Affordable safeguards are available (for example, incorporating a "code-ROM" crypto signature verification into the ECS processor) that could enable transparency to co-exist with a regime that discourages aftermarket hacking.

Any safeguard can be bypassed, but if done correctly, the bypass will require replacing most of the ECS, and they remain painfully expensive ... which would limit such circumventions to small numbers of vehicles.

SteveOctober 2, 2015 6:14 PM

You know I recall seeing alot of complaints from people who have had smart meters installed from their utility companies followed by significant increase in their gas and electric bills. I don't suppose it's too far fetched to speculate that the software in these meters could be designed to pass calibration testing only to overstate energy consumption in the field. How would one go about testing this?

John StoneOctober 3, 2015 11:06 AM

I was initially not keen on the idea of people receiving compensation for a problem that doesn't actually sound like it affects their car's performance. Was reading some actual data on what VW did though and its the environmental impact that is shocking. The NOx gases you might have heard about actually have a terrible effect on people's health, read about it here at the ICCT.

When I realised we weren't talking about people getting less miles out of their tank, but about premature deaths across our communities, my perspective changed. When you add to that JUST HOW BADLY they understated their vehicles NOx emissions (see graphic here), I think VW deserve everything they get.

Interested to hear what you guys think about this.

Anonymous CowardOctober 3, 2015 6:02 PM

Has Resilient Systems, Inc. taken a position on making their own source code available for analysis?

I searched the web but found nothing pertinent (which may say more about my search abilities than about source code availability).

MarkHOctober 3, 2015 6:23 PM

@John Stone:

These two matters are interrelated: the cars will be recalled in order to bring them into compliance with NOx standards.

Those cars that are made compliant are expected to suffer both reduced horsepower and greater fuel consumption.

Despite what a previous commenter said dismissing the health effects of NOx, we know by calculation that people must have died from pollution emitted by these grossly noncompliant vehicles.

Owners' claims for compensation are expected to have two bases:

1. The recalled cars will have materially degraded performance.

2. Many of the owners selected the particular models partly on the strength of their claimed environmental friendliness.

I don't think VW will be able to avoid making monetary compensation.

SpainGarageCheaterOctober 4, 2015 4:00 PM

www.phys.org
http://phys.org/news/2015-10-spanish-police-probe-garages-emission.html
Spanish police probe garages offering emission cheating

1.)THE EMPEROR HAS NO CLOTHES.

2.)Everyone (almost) knows about the cheating on car and diesel
engines

3.)Spain garages advertise on Internet and local and other
countries about emission cheating package.

4.)In the USA, it's like Prohibition of Alcohol or the Drug War
or the homosexual morals police.

5.)Prosecuting and jailing security researchers result in
only the outlaws having malware or malware knowledge.

6.)Software obsfucobsfucation works especially if the
defense is static and obvious as in case of VW Volkswagon
diesel emission scandal.

HOW WAS AN OLD CAR KEPT RUNNING to pass the NJ
State, USA emissions test?

long time ago, cannot afford to buy new car.
Tune the points, rotor, condensor and timing.
1.)cannot pass the govt meter test on tailpipe
2.)twist rotor cap for timing and lean the carburetor screws.
3.)keep trying to pass
4.)pass
5.)engine idles rough, so out of sight of the govt bureaucrats
adjust the MECHANICAL tuning to other configuration

6.)repeat for algorithm for three years

old engine, burns oil?, smoke and emissions from tailpipe?
make sure in USA to have LEO Law Enforcement DONATION
sticker and letter in volunteer charity for LEO and police and
state troopers available.

no high interest car loans. no need for college class. learned it
all from a book and study by myself on car.

no secrets about Linux and open source.

The secrets are all at the board of directors of VW.
Like the Emperor with no clothes or the three monkeys
of legend, VW sees no evil.

Diesels are used in TRUCKS, cars, vehicles, boats and BIG
ships

JohnOctober 5, 2015 4:24 PM

@MarkH Regarding historic cheating at pumps and VW detection of being tested.

First off, you omitted one of the easiest tests a 2 wheel drive car can do to see if it's being tested. Namely notice that only one pair of wheels is turning and the other pair of wheels are stationary. And since anti-lock braking systems require wheel motion sensors, the detection comes totally free of cost. And it looks like some independent researchers have managed to dyno a VW diesel and fool it into thinking that it wasn't being tested. See https://www.youtube.com/watch?v=GhvI2oeBPtY

Second, I'm rather surprised that the gas station operators even attempted to perform the "lie on how much gas is dispensed from 5 to 10 gallons" cheat since they already have a routine cheat in every state in the United States except Hawaii. Namely, running non-temperature compensated pumps. For gas and diesel deliveries the standard gallon is measured at 60 degrees F and the size is adjusted up and down according to the temperature so that the energy per gallon is constant regardless of what temperature the gasoline is pumped at. But this temperature compensation is only done for deliveries between companies and not done for the retail delivery to regular customers. The fuel industry claims that it would cost too much to do that compensation for the minor benefit of their customers. But interestingly enough, they support temperature compensation in Canada (where the delivery fuel temperature is lower and the companies can get more money from their customers because of temperature compensation). But in the United States where the delivery temperatures are higher (and therefore the fuel companies manage to sell less product for the money spent by customers) they're against upgrading the pumps because it would be too costly.

MarkHOctober 5, 2015 9:49 PM

@John:

Your observation about a simple method of dynamometer detection is eminently sensible. However, none of the articles I saw about "how VW did it" mentioned this as part of the algorithm.

I can speculate that VW designed their cheat to be sufficiently general, that it can work on their all-wheel drive vehicles.
__________________________

As to fuel dispensers that don't compensate for temperature, my guess is that they aren't a real business advantage for filling station operators, for two reasons:

1. The net effect is probably small. After fuel has been long enough in underground tanks, it comes close to soil temperature -- usually not far from 60F. When a customer buys fuel shortly after the underground tank is filled on a hot day, the filling station's margin is a little better than average; when a customer buys fuel soon after the fuel truck arrives on a cold day, the station's margin is a little worse than average.

Filling station operators don't control the weather (except to the extent that their business is part of the AGW chain), and I suppose usually have little control over the timing of fuel deliveries. They really can't "optimize" the temperature effect as a way to improve revenue.

2. Most filling station mark-ups are affected by competition. To the extent that they make extra money (if any) from the lack of retail temperature compensation, their nearby competitors experience a similar effect. So the "temperature profit" (if any) doesn't enable them to drop the price to attract customers (or alternatively, to have a better mark-up at the same price), any more than it does the filling station down the block.

In contrast to the temperature effect, having cheat firmware in the dispenser gives a consistent improvement in margin, and one the competitors won't match unless all of the stations in town are committing the same fraud.
__________________________

Interestingly, wikipedia suggests that retail temperature compensation is more important in Canada than in the US, because fuel delivery distances tend to be longer.

Fuel weight is critical to aircraft performance computations, so aviation fuel metering is often done in a more sophisticated manner. Aviation fuel dispensers may compute weight by measuring temperature (using an assumed specific gravity and coefficient of expansion), or by direct measurement of density (for example, by measuring the sink-depth of a float).

The flight crew of a transport jet needs to know how many tons (or tonnes) are aboard, not how many gallons (or liters).

Pascal BourguignonOctober 15, 2015 7:48 AM

I would promote indeed the interdiction of sale of any software or firmware that is not provided as free(dom) software.

The administration (a Federal Software Administration, FSA, similar to the FDA?) could refuse the approval, if the software contains too many bugs (or obviously, if it contains cheats, backdoors or other malware).

Concerning the size and complexity objection, the work of Alan Kay at VPRI.org shows that it's possible to write clear, concise and efficient software (that even a end-user knowing the domain could validate).

Given that the time spent on the audit and therefore the invoice sent to the corporation seeking the approval for their software/firmware would depend on the size and complexity of their software, they might be motivated to finally develop software as advocated by Alan Kay!

https://vimeo.com/82301919

Fredrik WahlgrenOctober 15, 2015 11:27 AM

Samsung and deniable cheating is mentioned . Samsung could have made an obscure setting that turned turned this feature on and off. That wold have made it easier to claim that they weren't cheating,

Leave a comment

Allowed HTML: <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre>

Photo of Bruce Schneier by Per Ervland.

Schneier on Security is a personal website. Opinions expressed are not necessarily those of Resilient, an IBM Company.