Schneier on Security
A blog covering security and security technology.
« Workshop on Economics of Information Security |
| Clear Shuts Down Operation »
June 25, 2009
It's a sad, horrific story. Homeowner returns to find his house demolished. The demolition company was hired legitimately but there was a mistake and it demolished the wrong house. The demolition company relied on GPS co-ordinates, but requiring street addresses isn't a solution. A typo in the address is just as likely, and it would have demolished the house just as quickly.
The problem is less how the demolishers knew which house to knock down, and more how they confirmed that knowledge. They trusted the paperwork, and the paperwork was wrong. Informality works when everybody knows everybody else. When merchants and customers know each other, government officials and citizens know each other, and people know their neighbours, people know what's going on. In that sort of milieu, if something goes wrong, people notice.
In our modern anonymous world, paperwork is how things get done. Traditionally, signatures, forms, and watermarks all made paperwork official. Forgeries were possible but difficult. Today, there's still paperwork, but for the most part it only exists until the information makes its way into a computer database. Meanwhile, modern technology -- computers, fax machines and desktop publishing software -- has made it easy to forge paperwork. Every case of identity theft has, at its core, a paperwork failure. Fake work orders, purchase orders, and other documents are used to steal computers, equipment, and stock. Occasionally, fake faxes result in people being sprung from prison. Fake boarding passes can get you through airport security. This month hackers officially changed the name of a Swedish man.
A reporter even changed the ownership of the Empire State Building. Sure, it was a stunt, but this is a growing form of crime. Someone pretends to be you -- preferably when you're away on holiday -- and sells your home to someone else, forging your name on the paperwork. You return to find someone else living in your house, someone who thinks he legitimately bought it. In some senses, this isn't new. Paperwork mistakes and fraud have happened ever since there was paperwork. And the problem hasn't been fixed yet for several reasons.
One, our sloppy systems generally work fine, and it's how we get things done with minimum hassle. Most people's houses don't get demolished and most people's names don't get maliciously changed. As common as identity theft is, it doesn't happen to most of us. These stories are news because they are so rare. And in many cases, it's cheaper to pay for the occasional blunder than ensure it never happens.
Two, sometimes the incentives aren't in place for paperwork to be properly authenticated. The people who demolished that family home were just trying to get a job done. The same is true for government officials processing title and name changes. Banks get paid when money is transferred from one account to another, not when they find a paperwork problem. We're all irritated by forms stamped 17 times, and other mysterious bureaucratic processes, but these are actually designed to detect problems.
And three, there's a psychological mismatch: it is easy to fake paperwork, yet for the most part we act as if it has magical properties of authenticity.
What's changed is scale. Fraud can be perpetrated against hundreds of thousands, automatically. Mistakes can affect that many people, too. What we need are laws that penalise people or companies -- criminally or civilly -- who make paperwork errors. This raises the cost of mistakes, making authenticating paperwork more attractive, which changes the incentives of those on the receiving end of the paperwork. And that will cause the market to devise technologies to verify the provenance, accuracy, and integrity of information: telephone verification, addresses and GPS co-ordinates, cryptographic authentication, systems that double- and triple-check, and so on.
We can't reduce society's reliance on paperwork, and we can't eliminate errors based on it. But we can put economic incentives in place for people and companies to authenticate paperwork more.
This essay originally appeared in The Guardian.
Posted on June 25, 2009 at 6:11 AM
• 50 Comments
To receive these entries once a month by e-mail, sign up for the Crypto-Gram Newsletter.
The demolition company did not trust the paperwork, according to the story I saw. They called the real estate office to confirm they had the right place, described the home, and were told to go ahead and destroy it.
A nightmare scenario I see coming soon to a city near you is that government systems are all changing over to "paperless" computerized systems. Now a single typo can cause you to be locked up as a felon and your entire life ruined (while the murderer who is one ahead of you on the city water customer list goes free) - and since there is no longer physical paperwork at the core of the system which can be examined and the error proven, there's no way for you to get out of it.
Concerning this guy's house, the little bit the article contained showed no remorse or even acknowledgement of wrongdoing on the part of the perpetrators. I assume a big lawsuit is on the way? Hopefully he gets at least 10x what someone who spilled coffee on themselves would get.
Furthermore, wouldn't the norm be for a house being destroyed to not have trinkets, curtains and furniture inside? IANA demolition contractor, but from what exposure I do have I would expect that all valuables are scavenged from buildings before they are flattened.
A typo in the address is as likely as in the GPS coords but just very few GPS coords point to houses whereas almost every address does.
A solution is to bind the authentication rules (user profile, GPS coordinate, time stamp, Internet/physical address, etc.) into the content. The content itself authenticates against the users/use. We call it Content-Centric Security.
I've heard it convincingly argued that identity theft involving banks, at least in the UK, is the fault of the banks -- because, before the age of electronic money transfers, *they* were liable if someone tried to withdraw money under false pretenses.
Somehow, when the paperwork stopped being paper, they managed to get this liability transferred to the customer.
I was thinking something like what bob was: wouldn't a company demolishing a house check to see if it looks, you know, empty? It's not only that paperwork gets fobbed up, but that people, having paperwork, then stop using common sense and reality testing.
one word : Tuttle. Or was it Buttle?
Oh, it's older than that:
Lord Melchett: Unhappily, Blackadder, the Lord High Executioner is dead.
Blackadder: Oh woe! Murdered, of course...
Melchett: No, oddly enough no. They usually are, but this one just got careless one night and signed his name on the wrong dotted line. They came for him while he slept.
Blackadder: He should have told them they had the wrong man.
Melchett: Oh, he did, but you see they didn't. They had the right man, and they had the form to prove it.
Blackadder: Hah. Bloody red tape, eh?
Another well written, and informative article. I for one like paper work, but only because _it is_ so easy to forage. I have to admit to changing a few documents in photoshop, things like sick notes that look like they are from my University, bus passes, student cards. All usually because I couldn't get those going through the proper channels, or it was just a huge pain in the ass and much easier to take 10 minutes on photoshop.
When I had double glazing installed in the 1990s I let the men in at 8am and they started cutting my window frames with electric saws. Then after about a minute turned to me and said "This is the right house?".
Nobody's suggesting getting rid of the appeals process (and other related features of our legal system). This nightmare scenario you see is unrealistic.
@PTB - you might be a fraud, but at least you're regular.
Was the house knocked down to make room for a bypass?
@OA: "One, our sloppy systems generally work fine, and it's how we get things done with minimum hassle."
As one of them (dreaded) auditors, I have been amazed at how much resources get wasted on problems that are rare and cost less than the remedies. Many in my profession get tunnel vision on focus on security and accuracy and fail to consider the associated costs of timeliness and productivity. Documents that are stamped by 17 people may have guaranteed accuracy, but the unmeasured cost is what 12-15 of those stampers could have produced or accomplished elsewhere.
Of course, magnitude is a factor. Clearly, the cost of an undected error on a travel reimbursement can be absorbed, while demolishing the wrong house is near irreparable.
I agree with the incentives suggestion, that penalties line up an entities incentives with authentication. I do wish the government would quit leglislating "how" to authenticate and just provide consequences when an entity doesn't--the entity will likely come up with a better and more efficient means than a bureaucrat.
And what is wrong with having a picture of the house, plus the contractor using some common sense (is the house empty, which should be required; also the contractor should have had keys to the house so it could be inspected first since if the utilities were still on, this would be a dange to the workers).
@kashmarek: "And what is wrong with having a picture of the house, plus the contractor using some common sense (is the house empty, which should be required; also the contractor should have had keys to the house so it could be inspected first since if the utilities were still on, this would be a dange to the workers)."
Definitely. Thank goodness there was no person or pets in it, but the simplest of checks could easy determine if a house is inhabited. But, it goes back to what Bruce said... they just weren't concerned with authentication and verification, they just trusted their information and did it.
1) The house was empty. He was not living there at the time. Had not been there for a while, and the utilities were off (presumably shut off by other contractors based on the same false location)
2) A "description" of the house I lived in 20 years ago would have fit approximately 20% of the houses in that neighbourhood. And the lots were small enough that I'd hope the demolition crew had at least three satellites in view to get the required precision.
3) "Auditing" (aka ISO9000) is a joke. A bad, black-humour joke, but a joke nonetheless. All the ones who sign the auditor's pay authorization care about is that there are some processes (typically almost, but not quite, totally unlike the actual work-process) in writing and someone had the acting skills to tell the auditor they were used. Or the auditor was only looking at his pay.
4) I would welcome paperless government if it also meant that legislation was "source-code-controlled" with timestamps and digital signing of changes. that way the staffer who transferred millions of dollars from musicians to record companies by "correcting a typo" would have been caught much sooner. Possibly before the "corrected" law was passed. They'd still escape prosecution and have a cushy new job with the RIAA, but at least we'd have heard about it.
I know a man whose how was "accidentally" demolished by local city government during a prolonged eminent domain dispute. This was around 1970, and the city knocked down enough of the (occupied but not inhabited at that moment, full of furniture and food and clothes) house to condemn the rest and send his family packing with a check.
Before everybody piles on, this is a story told face to face to me by a man who claimed it happened to him, but I have absolutely no way to gauge its veracity.
What's odd is a simple, old-fashioned, traditional measure would prevent things like this: personal contact. When I was growing up, the rule was that it didn't matter what had been arranged before, the crew doing the work would not start until the owner or his authorized representative was present and gave the OK to go to work. They'd also pound on the door and make sure the place was unoccupied. If someone was there and objected to the work, the crew would point the occupants at the owner and tell them to call when they'd sorted it out.
It seems like more and more I'm seeing situations where the "problems" are being caused simply by rushing things, removing the redundancies and cross-checks that normally catch problems before they become problems. That, and we keep removing responsibility for mistakes. In this story it seems like none of the people who caused the demolition feel they're actually responsible for the mistake being made.
Absolutely. One of my jobs as a young LEO ( Law & Order -type paper-pusher w/ perks)... was physically confirming the description of a front door on houses/apartments about to be searched with warrants.
Our department required that someone -not attached to a case- should physically go and verify the street address, Front door (and apt. #/Apt. door) description, and nearest cross-streets. Our search warrants were required to include this description in the application - then a judge would send someone like me to verify that all the "facts" lined up.
The hard part was trying to do this without tipping off the 'subjects'. Definitely a 'social engineering' job requirement.
@HJohn - amen!.
Our law agencies had been sued for kicking down the wrong doors - in one case a child was shot &almost died.
Obviously, the warrant was trash if a mistake was made, possibly the whole case &months of work might be blown.
There should be an incentive - whichever agency/person actually had signed off should be liable for all the costs to replace the building and lost possessions, any interim expenses like hotels & meals, etc. If this really is a rare event, then insurance should be fairly cheap for the responsible party.
@Frank Ch. Eigler: mod +1 funny !
@Chris: "... very few GPS coords point to houses whereas almost every address does."
I wouldn't be so sure of that. Between inaccuracies of up to about 5m in cheap consumer GPS devices (theoretical accuracy is better, but many devices won't bother with the extra corrections), and spacing of houses in many urban locations, this may not be true. In some places any location will be within five meters of one house or another.
Heck, every point in my house is within five meters of one of my neighbours. The front yard is only eight meters across at the street.
"...verify the providence, accuracy, and integrity of information."
I think you'd want the "provenance" of the information.
@ Frank Ch. Eigler
You beat me to it! The homeowner must not be good at Mondays...
On a more serious note; I just purchased my first home. Should this situation happen to me... Yes, there would be a lawsuit and it would be rather large for their 'little mistake'. I'm sure the homeowner's lawyer is going to have a field day with this one - yeah they get all the money back for their possessions, but I can't even fathom the stress the homeowner(s) are feeling right now.
While on holiday in Loch Lomond in Scotland, and the surrounding towns and villages, I spent a great deal of time browsing local shops. In a small co-op food store I saw hundreds of these application papers for UK Citizen ID cards. I took two, and told the staff that I enjoy fraud and crime, and everything that is taken for granted to verify ID is a weakness. I now have two UK ID cards in false names. Isn't life just fantastic. Teachers at school really were right when they said that "you can be anybody you want to be."
@C: "there would be a lawsuit and it would be rather large for their 'little mistake'."
As it should be. The value of the house would be, frankly, just a fraction of the cost to the family. And I don't just meant the very high cost of possessions (furniture, clothes, computer equipment, everything tangible) that could be replaced. Compensation must also consider all the sentimental and irreplaceable assets such as photos, family and ancestry possessions, gifts, etc. Not to mention psychological anguish.
Indeed, I would think the compensation this family is entitled to would be well into the millions. Not only would that be fair, such judgments would surely mean that whether or not the house is the right house would no longer be an externality. That is for certain.
Just read your Guardian Technology article and its sheer lack of any vision or understanding of the problems leave me appalled. The gist is that if you punish the targets of identity fraud and other cyber crimes, you will get an improvement. Not so. The recent heavy fines levied on banks have arguably not resulted in any less fraud, just more and more crazy and ineffectual back-covering checks.
Meanwhile, BT God bless it, continues, day after day to faithfully relay to my computer the same old phishing and " security" messages which could be recognised as spam by a six year old. Perhaps Mr Schneier could spend a bit more time on fixng these problems and less on spouting such drivel in the Guardian
@lars: That still happens in Chicago; the beloved Meigs Field airport in Lake Michigan adjacent downtown Chicago was "knocked down" (actually knocked up is closer - they dug up the runway) on a whim by Mayor Daley back in 2003 (right after midnight on a Sunday morning - prime time for legitimate city maintenance work).
Originally he claimed it was for "security" reasons (shutting down a control tower and making airspace uncontrolled always increases security), then he claimed it was "abandoned" and needed cleaned up (with ~15 private aircraft parked there - people always abandon those, we should outlaw disposable $200k airplanes), eventually he admitted the truth - he was tired of due process and allowing people to have input and wanted it finished (Chicago desperately needed a ~675th city park there for all the underprivileged kids at the adjacent yacht basin).
The Friends of Meigs like to point out this is the "only US infrastructure destroyed by terrorism since 9/11" - http://friendsofmeigs.org
The part that reminds me of present-day Iran or China is where he had a fire truck shine a searchlight to blind the webcam at the Adler Planetarium during the attack. Up until that day it was called the "Meigs Cam" because thousands of people every day would view the takeoffs and landings at the airport, yet ever since that day there has been no mention of there ever having been such; very similar to Googleing Tiananmen square from within China.
"until the owner or his authorized representative was present and gave the OK to go to work."
It wouldn't have made any difference. Someone had the wrong GPS coordinates. The "owner" wouldn't go himself, it's probably owned by a bank. So an "authorized representative" would have to be sent, and how does he find the house? By GPS of course. He has a description, which is probably generic enough to match all the neighbor houses. The utilities had been turned off by other contractors using the same bad information, so really, there was little hope that it would have ended differently.
Those are some pointed words. Go ahead and argue that there has been no improvement after fines. I disagree and I have the data to not only show improvement, but a comparison to other industries with less regulation that now are worse off.
Although I think Patrick is being harsh, and your piece is very well written, I too feel it lacks important substance.
It does not take much to come to the conclusion that with the right amount of incentive, people will have the right amount of incentive to do the right thing. In fact, you have presented a tautology.
Perhaps instead you could suggest a system to find the right amount or the level of incentive (or penalty). Or you at least could mention obstacles that have prevented such a level from being ratified by those with oversight. You tease us with references to economics and psychology, but never seem to engage them and get down to details -- if we "can put economic incentives in place" then why haven't we? If only it were so easy to just point out that there can be a measure of good as good and bad as bad and the world would self-correct.
"But we can put economic incentives in place for people and companies to authenticate paperwork more."
I think the economic penalties following such f**k-ups should be enough for demolition company insurers to be more diligent in the future.
The mapi is not the territory?
It is fascinating to me how often I still see the phrase "on official letterhead" as if that provided some sort of authentication in these days of laser printers.
Ah yes, company letterhead. It's a number of years ago now, but a friend and I, as freelance software developers, needed to open a business account at a bank.
"You need an authorisation on company letterhead." she said.
"There is no company letterhead. There's just the two of us. Anyway, we have a laser printer. What is 'company letterhead' expected to prove?"
"No, you need the letter on company letterhead before you can open the account."
"But we can just go home and knock something up in 5 minutes in Word."
"Oh, I know, but that's the rules."
"Fine, but you know exactly what we're going to do now, don't you?"
So off we go, knock up some "company letterhead" in 5 minutes in Word, print it out, and open the account.
Now this was some time ago, but it wouldn't surprise me if the same was true today.
Agree with many comments, just to add a personal anecdote. When purchasing our house, and several times since then, they confuse it in part (for value, etc.) with a house 1 block away (5700 vs 5600). It's white, ours is beige, it's got prominent latticework on a porch, ours has a small porch etc. And the photo is generally attached to the paperwork.
On MULTIPLE occasions I have pointed out the error based on this, and never am I aware of anyone else noticing the relatively different house photos. No one looks, or (based on working as a graphics guy with IT) no one is capable of looking in the right way. So photo verification is sorta useless.
I'm surprised the Eds at the Grauniad did not mention a recent case in the UK.
A family where away for a few days and when they returned their house had been re-possessed by a building society.
Just a couple of problems they did not have a mortgague with that building society. The house did not match the description on the reposession paperwork, nor did the address...
And just to rub it in the building society representatives where extreamly tardy about getting the house unlocked so that the owner and his family could get back in.
So yes there ought to be significant penalties for such mistakes such as taking away the directors pension fund or their liberty.
However in the UK Corparates are quite literaly getting away with murder of their and other organisations employees (by the thousands) and members of the traverling public. Due to shody work practices and the law insisting on proof of a "directing mind" they get away with fairly small fines, or as in the case of a railway disaster they try to say it was "sabotage/terrorism".
But hey as long as the free market is being efficient why should government ministers care, afterall they are quite happy with their directorships on the boards of these companies...
With regard to your comment on auditors,
"Many in my profession get tunnel vision on focus on security and accuracy and fail to consider the associated costs of timeliness and productivity."
Have you actually asked yourself what the position in the US is on an auditors liability to the directors, shareholders or "customers" of an organisation that has been audited as OK but still suffers loss of data etc?
From what I was reading the other day ( https://financialcryptography.com/mt/archives/001167.html ) there is a court case comming up that could make being an auditor of such a company an expensive liability.
Immunity from liability for mistakes is the real elephant in the room that has enabled these kind of 'accidents.'
Business management should not be able to hide behind unequal consumer contracts, arbitration clauses, or limited liability to walk away from irresponsible behavior.
A demolition contractor isn't going to demolish the wrong house if the rightful owner has the right to take over the business owner's house within 24 hours.
Credit rating agencies and banks aren't going to abet identity theft if their CEOs were personally liable to make good losses to identity theft victims for life, with no escape through bankruptcy.
I have given similar situations some thought.
Even an entity with good controls can suffer a loss, so it becomes a question of the audit's accuracy and reasonableness (and if they are looking for yet another scapegoat).
If an auditor deemed sensitive data to be secure when in fact it was being transmitted wireless and unencrypted, or if they knew it was being transported unencrypted on mobile media, they would be a liability, for example. If it was breached because a user violated policy, that may not reflect on the auditor.
There are many more examples.
@bob, @frans, @jim
See the excellent dystopian short story "Computers Don't Argue" by Gordon R. Dickson, in which a series of paperwork errors starting with a book club purchase lead to the execution of the hapless reader.
Informality does not work when everybody knows everybody else.
"Even an entity with good controls can suffer a loss, so it becomes a question of the audit's accuracy and reasonableness"
Yup my thoughts on it revolved around "zero day" etc.
But it's the not so little weasle word "reasonableness" that used to make me sweat when designing saftey critical systems for billion dollar pluss systems.
As was once pointed out to me "twenty twenty hindsight is a poor measure of judgment but the one by which it is judged".
Oh and a conversation I had with a projects manager one day
PM, What's the risk on this?
ME, There is insufficient data to make a meaningfull judgment.
PM, So are you saying million to one?
ME, No I'm saying there is insufficient data to make a meaningfull judgment.
PM, What's your best guess?
ME, There is insufficient data to make a meaningfull judgment.
PM, Are you saying you can't work it out?
ME, No I'm saying there is insufficient data to make a meaningfull judgment, therfore there is no way to make a judgment with any degree of confidence with the data available.
PM, Well I need a figure to go in the status report what do you want me to put?
ME, How about what I've said "insufficient data"?
PM, When will there be sufficient data?
ME, Not within the expected life expectancy of the system.
And eventualy after a few more rounds he realised I was not going to give him an answer and put in his report "To be determined"...
Suffice it to say every time I see a form for answers to be filled in, a voice in my head starts singing "little boxes, little boxes..."
The solution to this particular problem is trivial, and in use in other fields. The house should have had "condemned" spraypainted all over it, along with the appropriate notices. A week in advance, or more. Not in a locked filing cabinet at the bottom of a dark stairwell with a sign saying beware of the leopard.
If you notice somebody has put up condemnation notices on your house, you have a chance to correct it. Far better than them reporting back to base, "no condemned marker on this house, boss."
This is now done routinely in surgery. I think it was invented by a patient who, hearing horror stories about a hospital cutting off the wrong foot, wrote on his feet with sharpies, saying which to cut and which not to.
Now, hospitals are making patients do this. I suspect the hospital's motive is that if they do make a mistake, they can now put a lot of the blame on the patient who put a mark on the wrong organ. When they ask you to mark on the side they will be operating on, it's a bit of a moment of fear. You ask, "am I sure it was my left side that was hurting before they put me on the pain meds? I'm mostly sure but am I *really* sure?"
"Now, hospitals are making patients...
...put a mark on the wrong organ."
I can just see it now "Mr Smith we would like you to mark up which kidney we are not to remove?"
Mr Smith rolls on his side then cuts a hole in his back and writes "Do Not Remove" on his organ...
Ah the joys of DIY 8)
Many of the fixes that readers have proposed would not have worked, for the simple fact that the house was unoccupied and the owner had not visited it in more than a month. In fact, another person reported to him that the services had been disconnected, and he dismissed it as "vandalism."
But frankly, that does turn this into a movie plot scenario: risk that a house scheduled for demolition should be right across a narrow street from an identical looking house that just happens to not only be unoccupied, but not visited for the month-long process of disconnecting services? Pretty slender.
If any system needs to be fixed here, it is over-reliance on GPS for things that have better solutions. Yes, it is possible to have a typo in a street address. But in many neighbourhoods the spacing between dwellings is less than the typical errors that a handheld GPS produces on a bad day. If you rely on GPS to find houses, then 1 day in N you *will* end up going to the wrong house. Since the street address system is already available for free, if you are going to use GPS then the logical thing to do is reduce the error rate by using both.
By the way, when I first read the subject line for this posting, I thought it was going to be about Clarkson et al.  paper on "fingerprinting" paper documents to make them unforgeable.
There have been previous suggestions on recording the exact pattern of fibres in paper (which is randomised during production and cannot be duplicated by any known process), but this new method can be both registered and verified with a cheap commodity scanner, is robust against against many sorts of modification to the paper including overprinting and some degree of physical damage, and yet forms a strong cryptographic signature of the identity of the sheet of paper.
It thus serves to authenticate articles of paperwork were it is not the information content that must be protected, but the actual article: e.g. currency, lottery tickets, tickets to concerts and sports matches, temporary access badges.
1. W. Clarkson, T. Weyrich, A. Finkelstein, N. Heninger, J.A. Halderman and E. W. Felten, "Fingerprinting Blank Paper Using Commodity Scanners", to appear in Proc. IEEE Symposium on Security and Privacy, May 2009
"What we need are laws that penalise people or companies -- criminally or civilly -- who make paperwork errors."
More laws will not solve the problem, particularly laws that "penalize ... paperwork errors" -- "penalize" implies fines, which invariably accrue to the government, not the injured party.
Rather, the injured party ought to be able to recover fully from such mistakes, including remuneration for inconvenience and anguish. The extreme cost of such mistakes will then result in the responsible parties taking whatever extraordinary measures are necessary to absolutely avoid the consequences of such errors.
Congratulations on having this post selected for the July Carnival of Trust, hosted this month by Adrian Dayton at http://adriandayton.com/2009/07/carnivaloftrust/ .
The Carnival of Trust is a monthly compendium of the best posts loosely related to the subject of trust. Hosted on a rotating basis by thoughtful and critical hosts like Adrian, the limited (10 or so) selections each month are intended to bring good ideas to the Carnival’s readers, and respect (and readership) to the blogs selected. (A history of past Carnivals can be seen at http://trustedadvisor.com/... ).
Again, congratulations on your selection for the July Carnival of Trust.
Charles H. Green
Trusted Advisor Associates
Schneier.com is a personal website. Opinions expressed are not necessarily those of BT.