Schneier on Security

HomeBooksClick Here To Kill Everybody

Endnotes

Introduction: Everything Is Becoming A Computer

1A video shows the driver’s terrified expression: Andy Greenberg (21 Jul 2015), “Hackers remotely kill a Jeep on the highway—with me in it,” Wired, https://www.wired.com/2015/07/hackers-remotely-kill-jeep-highway, https://www.youtube.com/watch?v=MK0SrxBC1xs (video).

1They hacked in through the diagnostics port: Andy Greenberg (1 Aug 2016), “The Jeep hackers are back to prove car hacking can get much worse,” Wired, https://www.wired.com/2016/08/jeep-hackers-return-high-speed-steering-acceleration-hacks.

1They hacked in through the DVD player: Ishtiaq Rouf et al. (12 Aug 2010), “Security and privacy vulnerabilities of in-car wireless networks: A tire pressure monitoring system case study,” 19th USENIX Security Symposium, http://www.winlab.rutgers.edu/~Gruteser/papers/xu_tpms10.pdf.

1through the OnStar navigation system: Jim Finkle and Bernie Woodall (30 Jul 2015), “Researcher says can hack GM’s OnStar app, open vehicle, start engine,” Reuters, http://www.reuters.com/article/us-gm-hacking-idUSKCN0Q42FI20150730.

1and the computers embedded in the tires: Ishtiaq Rouf et al. (12 Aug 2010), “Security and privacy vulnerabilities of in-car wireless networks: A tire pressure monitoring system case study,” 19th USENIX Security Symposium, http://www.winlab.rutgers.edu/~Gruteser/papers/xu_tpms10.pdf.

1via the entertainment system: Kim Zetter (16 Jun 2016), “Feds say that banned researcher commandeered plane,” Wired, https://www.wired.com/2015/05/feds-say-banned-researcher-commandeered-plane.

1through air-to-ground communications systems: Sam Grobart (12 Apr 2013), “Hacking an airplane with only an Android phone,” Bloomberg, http://www.bloomberg.com/news/articles/2013-04-12/hacking-an-airplane-with-only-an-android-phone.

2remote hack of a Boeing 757: Calvin Biesecker (8 Nov 2017), “Boeing 757 testing shows airplanes vulnerable to hacking, DHS says,” Aviation Today, http://www.aviationtoday.com/2017/11/08/boeing-757-testing-shows-airplanes-vulnerable-hacking-dhs-says.

2In 2016, hackers—presumably Russian: Kim Zetter (12 Jun 2017), “The malware used against the Ukrainian power grid is more dangerous than anyone thought,” Vice Motherboard, https://motherboard.vice.com/en_us/article/zmeyg8/ukraine-power-grid-malware-crashoverride-industroyer. Kevin Poulsen (12 Jun 2017), “U.S. power companies warned ‘nightmare’ cyber weapon already causing blackouts,” Daily Beast, https://www.thedailybeast.com/newly-discovered-nightmare-cyber-weapon-is-already-causing-blackouts.

2The CrashOverride attack was different: Kim Zetter (3 Mar 2016), “Inside the cunning, unprecedented hack of Ukraine’s power grid,” Wired, https://www.wired.com/2016/03/inside-cunning-unprecedented-hack-ukraines-power-grid.

2There, the attackers: Jim Finkle (7 Jan 2016), “U.S. firm blames Russian ‘Sandworm’ hackers for Ukraine outage,” Reuters, https://www.reuters.com/article/us-ukraine-cybersecurity-sandworm/u-s-firm-blames-russian-sandworm-hackers-for-ukraine-outage-idUSKBN0UM00N20160108.

2One of the station operators recorded: C&M News (24 Jun 2017), “Watch how hackers took over a Ukrainian power station,” YouTube, https://www.youtube.com/watch?v=8ThgK1WXUgk.

2It had a variety of other “payloads”: Dragos, Inc. (13 Jun 2017), “CRASHOVERRIDE: Analysis of the threat to electric grid operations,” https://dragos.com/blog/crashoverride/CrashOverride-01.pdf.

2And while this weapon was fired: Nicholas Weaver makes this point. Nicholas Weaver (14 Jun 2017), “A cyber-weapon warhead test,” Lawfare, https://www.lawfareblog.com/cyber-weapon-warhead-test.

2In recent years, Russian hackers penetrated: This operation has been named “Dragonfly.” Security Response Attack Investigation Team (20 Oct 2017), “Dragonfly: Western energy sector targeted by sophisticated attack group,” Symantec Corporation, https://www.symantec.com/connect/s/dragonfly-western-energy-sector-targeted-sophisticated-attack-group. Nicole Perlroth and David Sanger (15 Mar 2018), “Cyberattacks put Russian fingers on the switch at power plants, U.S. says,” New York Times, https://www.nytimes.com/2018/03/15/us/politics/russia-cyberattacks.html.

2The hacker wrote a program: Christopher Meyer (8 Feb 2017), “This teen hacked 150,000 printers to show how the Internet of Things is shit,” Vice Motherboard, https://motherboard.vice.com/en_us/article/nzqayz/this-teen-hacked-150000-printers-to-show-how-the-internet-of-things-is-shit.

2Earlier in the same year: Carl Straumsheim (27 Jan 2017), “More anti-Semitic fliers printed at universities,” Inside Higher Ed, https://www.insidehighered.com/quicktakes/2017/01/27/more-anti-semitic-fliers-printed-universities.

3These are still in their infancy: Jennifer Kite-Powell (29 Oct 2014), “3D printed virus to attack cancer cells,” Forbes, https://www.forbes.com/sites/jenniferhicks/2014/10/29/3d-printed-virus-to-attack-cancer-cells/#7a8dbddb104b. Katie Collins (16 Oct 2014), “Meet the biologist hacking 3D printed cancer-fighting viruses,” Wired UK, https://www.wired.co.uk/article/andrew-hessel-autodesk.

4Modern pacemakers: University of the Basque Country (28 Jan 2015), “Pacemakers with Internet connection, a not-so-distant goal,” Science Daily, https://www.sciencedaily.com/releases/2015/01/150128113715.htm.

4insulin pumps: Brooke McAdams and Ali Rizvi (4 Jan 2016), “An overview of insulin pumps and glucose sensors for the generalist,” Journal of Clinical Medicine 5, no. 1, http://www.mdpi.com/2077-0383/5/1/5. Tim Vanderveen (27 May 2014), “From smart pumps to intelligent infusion systems: The promise of interoperability,” Patient Safety and Quality Healthcare, http://psqh.com/may-june-2014/from-smart-pumps-to-intelligent-infusion-systems-the-promise-of-interoperability.

4Pills are becoming smart: Pam Belluck (13 Nov 2017), “First digital pill approved to worries about biomedical ‘Big Brother,’” New York Times, https://www.nytimes.com/2017/11/13/health/digital-pill-fda.html.

4Smart contact lenses will: Diego Barretino (25 Jul 2017), “Smart contact lenses and eye implants will give doctors medical insights,” IEEE Spectrum, https://spectrum.ieee.org/biomedical/devices/smart-contact-lenses-and-eye-implants-will-give-doctors-medical-insights.

4Fitness trackers are smart: Brendan Borrell (29 Jun 2017), “Precise devices: Fitness trackers are more accurate than ever,” Consumer Reports, https://www.consumerreports.org/fitness-trackers/precise-devices-fitness-trackers-are-more-accurate-than-ever.

4a smart collar for your dog: Anthony Cuthbertson (12 Apr 2016), “This smart collar turns your pet into a living Tamagotchi,” Newsweek, http://www.newsweek.com/smart-collar-pet-kyon-tamagotchi-gps-dog-446754.

4a smart toy for your cat: Owen Williams (21 Feb 2016), “All I want for Christmas is LG’s adorable cat toy,” Next Web, http://thenextweb.com/gadgets/2016/02/21/all-i-want-for-christmas-is-lgs-adorable-cat-toy.

4a smart pen: Livescribe, Inc. (accessed 24 Apr 2018), “Livescribe Smartpens,” http://www.livescribe.com/en-us/smartpen.

4a smart toothbrush: Brandon Griggs (22 Feb 2014), “‘Smart’ toothbrush grades your brushing habits,” CNN, http://www.cnn.com/2014/01/09/tech/innovation/smart-toothbrush-kolibree. Sarmistha Acharya (23 Feb 2016), “MWC 2016: Oral-B unveils smart toothbrush that uses mobile camera to help you brush your teeth,” International Business Times, http://www.ibtimes.co.uk/mwc-2016-oral-b-unveils-smart-toothbrush-that-uses-mobile-camera-help-you-brush-better-1545414.

4a smart coffee cup: Diana Budds (9 Nov 2017), “A smart coffee cup? It’s more useful than it sounds,” Fast Company, https://www.fastcodesign.com/90150019/the-perfect-smart-coffee-cup-is-here.

4a smart sex toy: Phoebe Luckhurst (3 Aug 2017), “These sex toys and smart hook-up apps will make your summer hotter than ever,” Evening Standard, https://www.standard.co.uk/lifestyle/london-life/these-sex-toys-and-smart-apps-will-make-your-summer-hotter-than-ever-a3603056.html.

4a smart Barbie doll: Samuel Gibbs (13 Mar 2015), “Privacy fears over ‘smart’ Barbie that can listen to your kids,” Guardian, https://www.theguardian.com/technology/2015/mar/13/smart-barbie-that-can-listen-to-your-kids-privacy-fears-mattel.

4a smart tape measure: Stanley (accessed 24 Apr 2018), “Smart Measure Pro,” http://www.stanleytools.com/explore/stanley-mobile-apps/stanley-smart-measure-pro.

4a smart sensor for your plants: April Glaser (26 Apr 2016), “Dig gardening? Plant some connected tech this spring,” Wired, https://www.wired.com/2016/04/connected-gardening-tech-iot.

4a smart motorcycle helmet: Samar Warsi (26 Dec 2017), “A motorcycle helmet will call an ambulance and text your family if you have an accident,” Vice Motherboard, https://motherboard.vice.com/en_us/article/a37bwp/smart-motorcycle-helmet-helli-will-call-ambulance-skully-pakistan.

4smart thermostats: Christopher Snow (14 Mar 2017), “Everyone’s buying a smart thermostat—here’s how to pick one,” USA Today, https://www.usatoday.com/story/tech/reviewedcom/2017/03/14/smart-thermostats-are-2017s-hottest-home-gadgetheres-how-to-pick-the-right-one-for-you/99125582.

4smart power outlets: Kashmir Hill and Surya Mattu (7 Feb 2018), “The house that spied on me,” Gizmodo, https://gizmodo.com/the-house-that-spied-on-me-1822429852.

4a smart bathroom scale: Rose Kennedy (14 Aug 2017), “Want a scale that tells more than your weight? Smart scales are it,” Atlanta Journal-Constitution, http://www.ajc.com/news/health-med-fit-science/want-scale-that-tells-more-than-your-weight-smart-scales-are/XHpLELYnLgn8cQtBtsay6J.

4a smart toilet: Alina Bradford (1 Feb 2016), “Why smart toilets might actually be worth the upgrade,” CNET, http://www.cnet.com/how-to/smart-toilets-make-your-bathroom-high-tech.

4smart light bulbs: Alex Colon and Timothy Torres (30 May 2017), “The best smart light bulbs of 2017,” PC Magazine, https://www.pcmag.com/article2/0,2817,2483488,00.asp.

4a smart door lock: Eugene Kim and Christina Farr (10 Oct 2017), “Amazon is exploring ways to deliver items to your car trunk and the inside of your home,” CNBC, https://www.cnbc.com/2017/10/10/amazon-is-in-talks-with-phrame-and-is-working-on-a-smart-doorbell.html.

4a smart bed: Adam Gabbatt (5 Jan 2017), “Don’t lose your snooze: The technology that’s promising a better night’s sleep,” Guardian, https://www.theguardian.com/technology/2017/jan/05/sleep-technology-ces-2017-las-vegas-new-products.

4Cities are starting to embed smart sensors: Matt Hamblen (1 Oct 2015), “Just what IS a smart city?” Computerworld, https://www.computerworld.com/article/2986403/internet-of-things/just-what-is-a-smart-city.html.

4Smart billboards will recognize you: Tim Johnson (20 Sep 2017), “Smart billboards are checking you out—and making judgments,” Miami Herald, http://www.miamiherald.com/news/nation-world/national/article174197441.html.

5Those spatial metaphors don’t make sense: This is why I am still using the uppercase “Internet” in this book, even though most style guides now prefer lowercase. One of the premises of this book is that the Internet is a singular connected network—that any part of it can affect any other part of it—and needs to be viewed in this way to properly talk about security.

5“the network of physical objects”: Gartner (accessed 24 Apr 2018), “Internet of Things,” Gartner IT Glossary, https://www.gartner.com/it-glossary/internet-of-things.

5In 2017, there were 8.4 billion things: Gartner (7 Feb 2017), “Gartner says 8.4 billion connected ‘things’ will be in use in 2017, up 31 percent from 2016,” https://www.gartner.com/newsroom/id/3598917.

5By 2020, there are likely to be: Tony Danova (2 Oct 2013), “Morgan Stanley: 75 billion devices will be connected to the Internet of Things by 2020,” Business Insider, http://www.businessinsider.com/75-billion-devices-will-be-connected-to-the-internet-by-2020-2013-10. Peter Brown (25 Jan 2017), “20 billion connected Internet of Things devices in 2017, IHS Markit says,” Electronics 360, http://electronics360.globalspec.com/article/8032/20-billion-connected-internet-of-things-devices-in-2017-ihs-markit-says. Julia Boorstin (1 Feb 2016), “An Internet of Things that will number ten billions,” CNBC, https://www.cnbc.com/2016/02/01/an-internet-of-things-that-will-number-ten-billions.html. Statista (2018), “Internet of Things (IoT) connected devices installed base worldwide from 2015 to 2025 (in billions),” https://www.statista.com/statistics/471264/iot-number-of-connected-devices-worldwide.

6your T-shirt someday will: Michael Sawh (26 Sep 2017), “The best smart clothing: From biometric shirts to contactless payment jackets,” Wareable, https://www.wareable.com/smart-clothing/best-smart-clothing.

6“The ‘Smart Everything’ Trend”: J. R. Raphael (7 Jan 2016), “The ‘smart’-everything trend has officially turned stupid,” Computerworld, http://www.computerworld.com/article/3019713/internet-of-things/smart-everything-trend.html.

7It’s an Internet that senses: Something that senses, plans, and acts is the classic definition of a robot. Robin R. Murphy (2000), “Robotic paradigms,” in Introduction to AI Robotics, MIT Press, https://books.google.com/books/about/?id=RVlnL_X6FrwC.

8Or, for short, the Internet+: In 2016, I tried calling this the “World-Sized Web.” “Internet+” is a better term. Bruce Schneier (2 Feb 2016), “The Internet of Things will be the world’s biggest robot,” Forbes, https://www.forbes.com/sites/bruceschneier/2016/02/02/the-internet-of-things-will-be-the-worlds-biggest-robot.

10It can be a hard argument to make: Even the conservative Economist published an editorial in 2017 supporting both regulation and liabilities for IoT devices. Economist (8 Apr 2017), “How to manage the computer-security threat,” https://www.economist.com/news/leaders/21720279-incentives-software-firms-take-security-seriously-are-too-weak-how-manage.

11Although this is not a book about: This is an excellent book on that topic: Alexander Klimburg (2017), The Darkening Web: The War for Cyberspace, Penguin, https://books.google.com/books/about/?id=kytBvgAACAAJ.

12“we’re facing 21st-century issues”: Cambridge Cyber Security Summit (4 Oct 2017), “Transparency, communication and conflict,” CNBC, https://www.cnbc.com/video/2017/10/09/cambridge-cyber-security-summit-transparency-communication-and-conflict.html.

Part I: The Trends

15In 2017, a hacker bragged: Ankit Anubhav (20 Jul 2017), “IoT thermostat bug allows hackers to turn up the heat,” NewSky Security, https://.newskysecurity.com/iot-thermostat-bug-allows-hackers-to-turn-up-the-heat-948e554e5e8b.

15Separately, a group of researchers: Lorenzo Franceschi-Bicchierai (7 Aug 2016), “Hackers make the first-ever ransomware for smart thermostats,” Vice Motherboard, https://motherboard.vice.com/en_us/article/aekj9j/internet-of-things-ransomware-smart-thermostat.

15But next time might be my brand: No, I’m not telling you what brand I have.

16crashing airplanes: Kim Zetter (26 May 2015), “Is it possible for passengers to hack commercial aircraft?” Wired, http://www.wired.com/2015/05/possible-passengers-hack-commercial-aircraft. Gerald L. Dillingham, Gregory C. Wilshusen, and Nabajyoti Barkakati (14 Apr 2015), “Air traffic control: FAA needs a more comprehensive approach to address cybersecurity as agency transitions to NextGen,” GAO-15-370, US Government Accountability Office, http://www.gao.gov/assets/670/669627.pdf.

16disabling cars: Andy Greenberg (21 Jul 2015), “Hackers remotely kill a Jeep on the highway—with me in it,” Wired, https://www.wired.com/2015/07/hackers-remotely-kill-jeep-highway, https://www.youtube.com/watch?v=MK0SrxBC1xs (video).

16tinkering with medical devices: Liviu Arsene (20 Nov 2014), “Hacking vulnerable medical equipment puts millions at risk,” Information Week, http://www.informationweek.com/partner-perspectives/bitdefender/hacking-vulnerable-medical-equipment-puts-millions-at-risk/a/d-id/1319873.

16We’re worried about being GPS-hacked: David Hambling (10 Aug 2017), “Ships fooled in GPS spoofing attack suggest Russian cyberweapon,” New Scientist, https://www.newscientist.com/article/2143499-ships-fooled-in-gps-spoofing-attack-suggest-russian-cyberweapon.

16With smart homes, attacks can mean: Colin Neagle (2 Apr 2015), “Smart home hacking is easier than you think,” Network World, http://www.networkworld.com/article/2905053/security0/smart-home-hacking-is-easier-than-you-think.html.

16about 600 million people in the world do: Ad blockers represent the largest consumer boycott in human history. Sean Blanchfield (1 Feb 2017), “The state of the blocked web: 2017 global adblock report,” PageFair, https://pagefair.com/downloads/2017/01/PageFair-2017-Adblock-Report.pdf.

16some sites now employ ad-blocker blockers: Kate Murphy (20 Feb 2016), “The ad blocking wars,” New York Times, https://www.nytimes.com/2016/02/21/opinion/sunday/the-ad-blocking-wars.html.

16Spam is an arms race: Pedro H. Calais Guerra et al. (13–14 Jul 2010), “Exploring the spam arms race to characterize spam evolution,” Electronic Messaging, Anti-Abuse and Spam Conference (CEAS 2010), https://honeytarg.cert.br/spampots/papers/spampots-ceas10.pdf.

17“skimmers” to steal card information and PINs: Alfred Ng (1 Oct 2017), “Credit card thieves are getting smarter. You can, too,” CNET, https://www.cnet.com/news/credit-card-skimmers-thieves-are-getting-smarter-you-can-too.

17remote attacks against ATMs over the Internet: David Sancho, Numaan Huq, and Massimiliano Michenzi (2017), “Cashing in on ATM malware: A comprehensive look at various attack types,” Trend Micro, https://documents.trendmicro.com/assets/white_papers/wp-cashing-in-on-atm-malware.pdf.

1. Computers Are Still Hard to Secure

19“The only truly secure system”: Quoted in A. K. Dewdney (1 Mar 1989), “Computer recreations: Of worms, viruses and core war,” Scientific American, http://corewar.co.uk/dewdney/1989-03.htm.

19Rod Beckstrom summarized it this way: Rod Beckstrom (2 Nov 2011), “Statement to the London Conference on Cyberspace, Internet Corporation for Assigned Names and Numbers (ICANN),” https://www.icann.org/en/system/files/files/beckstrom-speech-cybersecurity-london-02nov11-en.pdf.

19“Security is a process, not a product”: Bruce Schneier (1 Apr 2000), “The process of security,” Information Security, https://www.schneier.com/essays/archives/2000/04/the_process_of_secur.html.

20I play Pokémon Go on my phone: Mystic, level 40. I managed to catch them all for about a week in August 2017 between when I caught a Farfetch’d in Seoul and when Mewto was released in Yokohama, and again from November 2017 after I caught my first Mewto to before Generation 3 was released in December. I travel a lot, and was able to catch all the regional Pokémon in their original regions. Still, I think it will be a while before I catch all of the Generation 3 regionals.

20We occasionally lose important data: In late 2017, I had to quickly replace my iPhone. As part of the process, I enabled iCloud and tried to back up my phone data. I’m not sure how, but iCloud managed to delete 20 years of calendar history. I don’t know what I would have done if I hadn’t had a recent backup.

20Some of them are inherent: Roger A. Grimes (8 Jul 2014), “5 reasons why software bugs still plague us,” CSO, https://www.csoonline.com/article/2608330/security/5-reasons-why-software-bugs-still-plague-us.html. David Heinemeier Hansson (7 Mar 2016), “Software has bugs. This is normal,” Signal v. Noise, https://m.signalvnoise.com/software-has-bugs-this-is-normal-f64761a262ca.

20Microsoft spent the decade after 2002: In 2002, Bill Gates sent his landmark “trustworthy computing” memo to all employees. In that same year, Windows development shut down completely so that every employee could take security training. The company’s first Security Development Lifecycle security tools appeared in 2004. Abhishek Baxi (10 Mar 2014), “From a Bill Gates memo to an industry practice: The story of Security Development Lifecycle,” Windows Central, https://www.windowscentral.com/bill-gates-memo-industry-practice-story-security-development-cycle.

20Apple is known for its quality software: To be fair, the company had some pretty significant bugs in 2017. Adrian Kingsley-Hughes (19 Dec 2017), “Apple seems to have forgotten about the whole ‘it just works’ thing,” ZDNet, http://www.zdnet.com/article/apple-seems-to-have-forgotten-about-the-whole-it-just-works-thing.

20And NASA had a famous quality control process: National Research Council (1996), “Case study: NASA space shuttle flight control software,” in Statistical Software Engineering, National Academies Press, https://www.nap.edu/read/5018/chapter/4.

21NASA still has crazily conservative: Martha Wetherholt (1 Sep 2015), “NASA’s approach to software assurance,” Crosstalk, http://static1.1.sqspcdn.com/static/f/702523/26502332/1441086732177/201509-Wetherholt.pdf.

21An example is something called: Peter Bright (25 Aug 2015), “How security flaws work: The buffer overflow,” Ars Technica, https://arstechnica.com/information-technology/2015/08/how-security-flaws-work-the-buffer-overflow.

21We don’t know what percentage: Eric Rescorla (1 Jan 2005), “Is finding security holes a good idea?” IEEE Security & Privacy 3, no. 1, https://dl.acm.org/citation.cfm?id=1048817. Andy Ozment and Stuart Schechter (1 Jul 2006), “Milk or wine: Does software security improve with age?” in Proceedings of the 15th USENIX Security Symposium, https://www.microsoft.com/en-us/research/publication/milk-or-wine-does-software-security-improve-with-age.

21It remained undiscovered for two years: Heather Kelly (9 Apr 2014), “The ‘Heartbleed’ security flaw that affects most of the Internet,” CNN, https://www.cnn.com/2014/04/08/tech/web/heartbleed-openssl/index.html.

21The Spectre and Meltdown vulnerabilities: Andy Greenberg (7 Jan 2018), “Triple Meltdown: How so many researchers found a 20-year-old chip flaw at the same time,” Wired, https://www.wired.com/story/meltdown-spectre-bug-collision-intel-chip-flaw-discovery.

21Keeping IoT devices cheap means: Sandy Clark et al. (6–10 Dec 2010), “Familiarity breeds contempt: The honeymoon effect and the role of legacy code in zero-day vulnerabilities,” in Proceedings of the 26th Annual Computer Security Applications Conference, https://dl.acm.org/citation.cfm?id=1920299.

22In April 2010, for about 18 minutes: Nate Anderson (17 Nov 2010), “How China swallowed 15% of ’Net traffic for 18 minutes,” Ars Technica, https://arstechnica.com/information-technology/2010/11/how-china-swallowed-15-of-net-traffic-for-18-minutes.

22Because there’s no authentication: Some meager security features have been added by some large networks, but the document that defines BGP explicitly states: “Security issues are not discussed in this document.” Yakov Rekhter and Tony Li (Mar 1995), “A Border Gateway Protocol 4 (BGP-4),” Network Working Group, Internet Engineering Task Force, https://tools.ietf.org/html/rfc1771.

22We know from documents disclosed: Axel Arnbak and Sharon Goldberg (30 Jun 2014), “Loopholes for circumventing the Constitution: Unrestrained bulk surveillance on Americans by collecting network traffic abroad,” Michigan Telecommunications and Technology Law Review 21, no. 2, https://repository.law.umich.edu/cgi/viewcontent.cgi?article=1204&context=mttlr. Sharon Goldberg (22 Jun 2017), “Surveillance without borders: The ‘traffic shaping’ loophole and why it matters,” Century Foundation, https://tcf.org/content/report/surveillance-without-borders-the-traffic-shaping-loophole-and-why-it-matters.

22In 2013, one company reported: Jim Cowie (19 Nov 2013), “The new threat: Targeted Internet traffic misdirection,” Vantage Point, Oracle + Dyn, https://dyn.com/blog/mitm-internet-hijacking.

22In 2014, the Turkish government: Jim Cowie (19 Nov 2013), “The new threat: Targeted Internet traffic misdirection,” Vantage Point, Oracle + Dyn, https://dyn.com/blog/mitm-internet-hijacking.

22In 2017, traffic to and from: Dan Goodin (13 Dec 2017), “‘Suspicious’ event routes traffic for big-name sites through Russia,” Ars Technica, https://arstechnica.com/information-technology/2017/12/suspicious-event-routes-traffic-for-big-name-sites-through-russia.

22a 2008 talk at the DefCon hackers conference: Dan Goodin (27 Aug 2008), “Hijacking huge chunks of the internet: A new How To,” Register, https://www.theregister.co.uk/2008/08/27/bgp_exploit_revealed.

23“It’s not that we didn’t think about security”: Craig Timberg (30 May 2015), “A flaw in the design,” Washington Post, http://www.washingtonpost.com/sf/business/2015/05/30/net-of-insecurity-part-1.

23“It is highly desirable that Internet carriers”: Brian E. Carpenter, ed. (Jun 1996), “Architectural principles of the Internet,” Network Working Group, Internet Engineering Task Force, https://www.ietf.org/rfc/rfc1958.txt.

24It makes little sense: Tyler Moore (2010), “The economics of cybersecurity: Principles and policy options,” International Journal of Critical Infrastructure Protection, https://tylermoore.utulsa.edu/ijcip10.pdf.

24And as with BGP, it’s been 20 years: In 2017, the switchover was again delayed. Internet Corporation for Assigned Names and Numbers (27 Sep 2017), “KSK rollover postponed,” https://www.icann.org/news/announcement-2017-09-27-en.

25a Canon Pixma printer: Michael Jordon (12 Sep 2014), “Hacking Canon Pixma printers: Doomed encryption,” Context Information Security, https://www.contextis.com/blog/hacking-canon-pixma-printers-doomed-encryption.

25a Honeywell Prestige thermostat: Ralph Kinney (25 May 2017), “Will it run Doom? Smart thermostat running classic FPS game Doom,” Zareview, https://www.zareview.com/will-run-doom-smart-thermostat-running-classic-fps-game-doom.

25a Kodak digital camera: JJ (1 Mar 2010), “The DoomBox,” Dashfest, http://www.dashfest.com/?p=113.

25Even the best DRM systems: Kyle Orland (19 Oct 2017), “Denuvo’s DRM now being cracked within hours of release,” Ars Technica, https://arstechnica.com/gaming/2017/10/denuvos-drm-ins-now-being-cracked-within-hours-of-release.

26It’s an old term from cryptography: Seth Schoen (17 Mar 2016), “Thinking about the term ‘backdoor,’” Electronic Frontier Foundation, https://www.eff.org/deeplinks/2016/03/thinking-about-term-backdoor.

26When the FBI demands: Bruce Schneier (18 Feb 2016), “Why you should side with Apple, not the FBI, in the San Bernardino iPhone case,” Washington Post, https://www.washingtonpost.com/posteverything/wp/2016/02/18/why-you-should-side-with-apple-not-the-fbi-in-the-san-bernardino-iphone-case.

26When researchers spot a hard-coded: Dan Goodin (12 Jan 2016), “Et tu, Fortinet? Hard-coded password raises new backdoor eavesdropping fears,” Ars Technica, https://arstechnica.com/information-technology/2016/01/et-tu-fortinet-hard-coded-password-raises-new-backdoor-eavesdropping-fears.

26All computers can be dragooned: Maria Korolov (6 Dec 2017), “What is a botnet? And why they aren’t going away anytime soon,” CSO, https://www.csoonline.com/article/3240364/hacking/what-is-a-botnet-and-why-they-arent-going-away-anytime-soon.html.

26But today, in computers and on the Internet: This has been true since the beginning of computer security. Here’s a quote from a 1979 journal: “Few if any contemporary computer security controls have prevented a tiger team from easily accessing any information sought.” Basically, the attackers always win. Roger R. Schell (Jan–Feb 1979), “Computer security: The Achilles’ heel of the electronic Air Force?” Air University Review 30, no. 2 (reprinted in Air & Space Power Journal, Jan–Feb 2013), http://insct.syr.edu/wp-content/uploads/2015/05/Schell_Achilles_Heel.pdf.

27Complexity is the worst enemy of security: Bruce Schneier (19 Nov 1999), “A plea for simplicity: You can’t secure what you don’t understand,” Information Security, https://www.schneier.com/essays/archives/1999/11/a_plea_for_simplicit.html.

27And our billions of computers: David McCandless (24 Sep 2015), “How many lines of code does it take?” Information Is Beautiful, http://www.informationisbeautiful.net/visualizations/million-lines-of-code.

27Computer security experts like to: Lily Hay Newman (12 Mar 2017), “Hacker lexicon: What is an attack surface?” Wired, https://www.wired.com/2017/03/hacker-lexicon-attack-surface.

27Users regularly fail to change: Robert McMillan (17 Sep 2017), “An unexpected security problem in the cloud,” Wall Street Journal, https://www.wsj.com/articles/an-unexpected-security-problem-in-the-cloud-1505700061.

27In 2017, Stanford University blamed: Elena Kadavny (1 Dec 2017), “Thousands of records exposed in Stanford data breaches,” Palo Alto Online, https://www.paloaltoonline.com/news/2017/12/01/thousands-of-records-exposed-in-stanford-data-breaches.

28“If we were to score cyber”: Dan Geer (6 Aug 2014), “Cybersecurity as realpolitik,” Black Hat 2014, http://geer.tinho.net/geer.blackhat.6viii14.txt.

28Murder is easy, too: Aside from those social systems, our internal psychology and moral values mostly keep us from murdering others.

28The criminals gained access: Elizabeth A. Harris et al. (17 Jan 2014), “A sneaky path into Target customers’ wallets,” New York Times, https://www.nytimes.com/2014/01/18/business/a-sneaky-path-into-target-customers-wallets.html.

29So when Dyn went down: Catalin Cimpanu (30 Mar 2017), “New Mirai botnet slams U.S. college with 54-hour DDoS attack,” Bleeping Computer, https://www.bleepingcomputer.com/news/security/new-mirai-botnet-slams-us-college-with-54-hour-ddos-attack. Manos Antonakakis et al. (8 Aug 2017), “Understanding the Mirai botnet,” in Proceedings of the 26th USENIX Security Symposium, https://www.usenix.org/system/files/conference/usenixsecurity17/sec17-antonakakis.pdf.

29In 2017, hackers penetrated: Alex Schiffer (21 Jul 2017), “How a fish tank helped hack a casino,” Washington Post, https://www.washingtonpost.com/news/innovations/wp/2017/07/21/how-a-fish-tank-helped-hack-a-casino.

29And two: it’s possible that: This essay describes an interaction between the way Gmail and Netflix interpret e-mail addresses that results in an insecurity: James Fisher (7 Apr 2018), “The dots do matter: How to scam a Gmail user,” Jameshfisher.com, https://jameshfisher.com/2018/04/07/the-dots-do-matter-how-to-scam-a-gmail-user.html.

29In 2012, someone compromised: Mat Honan (6 Aug 2012), “How Apple and Amazon security flaws led to my epic hacking,” Wired, https://www.wired.com/2012/08/apple-amazon-mat-honan-hacking. Mat Honan (17 Aug 2012), “How I resurrected my digital life after an epic hacking,” Wired, https://www.wired.com/2012/08/mat-honan-data-recovery.

29A vulnerability in Samsung: Pedro Venda (18 Aug 2015), “Hacking DefCon 23’s IoT Village Samsung fridge,” Pen Test Partners, http://www.pentestpartners.com/blog/hacking-defcon-23s-iot-village-samsung-fridge. John Leyden (25 Aug 2015), “Samsung smart fridge leaves Gmail logins open to attack,” Register, http://www.theregister.co.uk/2015/08/24/smart_fridge_security_fubar.

29The gyroscope on your iPhone: Yan Michalevsky, Gabi Nakibly, and Dan Boneh (20–22 Aug 2014), “Gyrophone: Recognizing speech from gyroscope signals,” in Proceedings of the 23rd USENIX Security Symposium, https://crypto.stanford.edu/gyrophone.

29The antivirus software sold by Kaspersky: Dan Goodin (10 Oct 2017), “How Kaspersky AV reportedly was caught helping Russian hackers steal NSA secrets,” Ars Technica, https://arstechnica.com/information-technology/2017/10/russian-hackers-reportedly-used-kaspersky-av-to-search-for-nsa-secrets.

30The attacker who created the Mirai botnet: Catalin Cimpanu (30 Mar 2017), “New Mirai botnet slams U.S. college with 54-hour DDoS attack,” Bleeping Computer, https://www.bleepingcomputer.com/news/security/new-mirai-botnet-slams-us-college-with-54-hour-ddos-attack.

30They can hire ransomware-as-a-service: Tara Seals (18 May 2016), “Enormous malware as a service infrastructure fuels ransomware epidemic,” Infosecurity Magazine, https://www.infosecurity-magazine.com/news/enormous-malware-as-a-service.

30European companies like HackingTeam: Aaron Sankin (9 Jul 2015), “Forget Hacking Team—many other companies sell surveillance tech to repressive regimes,” Daily Dot, https://www.dailydot.com/layer8/hacking-team-competitors.

30The malware was created by: US Department of Justice (28 Nov 2017), “Canadian hacker who conspired with and aided Russian FSB officers pleads guilty,” https://www.justice.gov/opa/pr/canadian-hacker-who-conspired-and-aided-russian-fsb-officers-pleads-guilty.

31“Class break” is a concept from computer security: Bruce Schneier (3 Jan 2017), “Class breaks,” Schneier on Security, https://www.schneier.com/blog/archives/2017/01/class_breaks.html.

31A cryptographic flaw forced the government: Dan Goodin (6 Nov 2017), “Flaw crippling millions of crypto keys is worse than first disclosed,” Ars Technica, https://arstechnica.com/information-technology/2017/11/flaw-crippling-millions-of-crypto-keys-is-worse-than-first-disclosed.

31According to a 2011 DHS study: US Department of Homeland Security (Nov 2012), “National risk estimate: Risks to U.S. critical infrastructure from global positioning system disruptions,” https://www.hsdl.org/?abstract&did=739832.

32In 2012, this happened to Onity: Andy Greenberg (26 Nov 2012), “Security flaw in common keycard locks exploited in string of hotel room break-ins,” Forbes, https://www.forbes.com/sites/andygreenberg/2012/11/26/security-flaw-in-common-keycard-locks-exploited-in-string-of-hotel-room-break-ins.

32It took months for Onity to realize: Andy Greenberg (6 Dec 2012), “Lock firm Onity starts to shell out for security fixes to hotels’ hackable locks,” Forbes, https://www.forbes.com/sites/andygreenberg/2012/12/06/lock-firm-onity-starts-to-shell-out-for-security-fixes-to-hotels-hackable-locks. Andy Greenberg (15 May 2013), “Hotel lock hack still being used in burglaries months after lock firm’s fix,” Forbes, https://www.forbes.com/sites/andygreenberg/2013/05/15/hotel-lock-hack-still-being-used-in-burglaries-months-after-lock-firms-fix. Andy Greenberg (1 Aug 2017), “The hotel room hacker,” Wired, https://www.wired.com/2017/08/the-hotel-hacker.

32In 1976, cryptography experts estimated: Whitfield Diffie and Martin E. Hellman (1 Jun 1977), “Exhaustive cryptanalysis of the NBS Data Encryption Standard,” Computer, https://www-ee.stanford.edu/~hellman/publications/27.pdf.

32In my 1995 book Applied Cryptography: Bruce Schneier (1995), Applied Cryptography, 2nd edition, Wiley.

32In 1998, the Electronic Frontier Foundation: Electronic Frontier Foundation (1998), Cracking DES: Secrets of Encryption Research, Wiretap Politics, and Chip Design, O’Reilly & Associates.

32Fast-forward a half decade: Stephanie K. Pell and Christopher Soghoian (29 Dec 2014), “Your secret Stingray’s no secret anymore: The vanishing government monopoly over cell phone surveillance and its impact on national security and consumer privacy,” Harvard Journal of Law and Technology 28, no. 1, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2437678.

33Fast-forward another half decade: Kim Zetter (31 Jul 2010), “Hacker spoofs cell phone tower to intercept calls,” Wired, https://www.wired.com/2010/07/intercepting-cell-phone-calls.

33The result is passwords: My essay about how to choose a secure password: Bruce Schneier (25 Feb 2014), “Choosing a secure password,” Boing Boing, https://boingboing.net/2014/02/25/choosing-a-secure-password.html.

33In the 1970s, IBM mathematicians: Don Coppersmith (May 1994), “The Data Encryption Standard (DES) and its strength against attacks,” IBM Journal of Research and Development 38, no. 3, http://simson.net/ref/1994/coppersmith94.pdf.

33The NSA classified IBM’s discovery: Eli Biham and Adi Shamir (1990), “Differential cryptanalysis of DES-like cryptosystems,” Journal of Cryptology 4, no. 1, https://link.springer.com/article/10.1007/BF00630563.

2. Patching Is Failing as a Security Paradigm

34“Move fast and break things”: In 2014, Facebook changed its motto. Samantha Murphy (30 Apr 2014), “Facebook changes its ‘Move fast and break things’ motto,” Mashable, http://mashable.com/2014/04/30/facebooks-new-mantra-move-fast-with-stability/#ebhnHppqdPq9.

36“responsible disclosure”: Stephen A. Shepherd (22 Apr 2003), “How do we define responsible disclosure?” SANS Institute, https://www.sans.org/reading-room/whitepapers/threats/define-responsible-disclosure-932.

36Google has an entire team: Andy Greenberg (16 Jul 2014), “Meet ‘Project Zero,’ Google’s secret team of bug-hunting hackers,” Wired, https://www.wired.com/2014/07/google-project-zero. Robert Hackett (23 Jun 2017), “Google’s elite hacker SWAT team vs. everyone,” Fortune, http://fortune.com/2017/06/23/google-project-zero-hacker-swat-team.

36Despite the seemingly endless stream: Andy Ozment and Stuart Schechter (1 Jul 2006), “Milk or wine: Does software security improve with age?” in Proceedings of the 15th USENIX Security Symposium, https://www.microsoft.com/en-us/research/publication/milk-or-wine-does-software-security-improve-with-age.

37Some people don’t patch: Malwarebytes (4 Oct 2017), “PUP reconsideration information: How do we identify potentially unwanted software?” https://www.malwarebytes.com/pup. Chris Hutton (1 Aug 2014), “12 downloads that sneak unwanted software into your PC,” Tom’s Guide, https://www.tomsguide.com/us/top-downloads-unwanted-software,news-19249.html.

37Equifax was hacked because: Cyrus Farivar (15 Sep 2017), “Equifax CIO, CSO ‘retire’ in wake of huge security breach,” Ars Technica, https://arstechnica.com/tech-policy/2017/09/equifax-cio-cso-retire-in-wake-of-huge-security-breach.

37The Amnesia IoT botnet made use of: John Leyden (7 Apr 2017), “‘Amnesia’ IoT botnet feasts on year-old unpatched vulnerability,” Register, https://www.theregister.co.uk/2017/04/07/amnesia_iot_botnet.

37Sometimes, ISPs have the ability: Fredric Paul (7 Sep 2017), “Fixing, upgrading and patching IoT devices can be a real nightmare,” Network World, https://www.networkworld.com/article/3222651/internet-of-things/fixing-upgrading-and-patching-iot-devices-can-be-a-real-nightmare.html.

37Right now, the only way: Lucian Constantin (17 Feb 2016), “Hard-coded password exposes up to 46,000 video surveillance DVRs to hacking,” PC World, https://www.pcworld.com/article/3034265/hard-coded-password-exposes-up-to-46000-video-surveillance-dvrs-to-hacking.html.

37In 2010, a security researcher analyzed: Craig Heffner (6 Jul 2010), “How to hack millions of routers,” DefCon 18, https://www.defcon.org/images/defcon-18/dc-18-presentations/Heffner/DEFCON-18-Heffner-Routers.pdf. Craig Heffner (5 Oct 2010), “DEFCON 18: How to hack millions of routers,” YouTube, http://www.youtube.com/watch?v=stnJiPBIM6o.

37Things haven’t improved since then: Jennifer Valentino-DeVries (18 Jan 2016), “Rarely patched software bugs in home routers cripple security,” Wall Street Journal, https://www.wsj.com/articles/rarely-patched-software-bugs-in-home-routers-cripple-security-1453136285.

37The malware DNSChanger attacks: Elinor Mills (17 Jun 2008), “New DNSChanger Trojan variant targets routers,” CNET, http://news.cnet.com/8301-10784_3-9970972-7.html.

37In Brazil in 2012, 4.5 million DSL routers: Graham Cluley (1 Oct 2012), “How millions of DSL modems were hacked in Brazil, to pay for Rio prostitutes,” Naked Security, http://nakedsecurity.sophos.com/2012/10/01/hacked-routers-brazil-vb2012.

37In 2013, a Linux worm targeted: Dan Goodin (27 Nov 2013), “New Linux worm targets routers, cameras, ‘Internet of things’ devices,” Ars Technica, http://arstechnica.com/security/2013/11/new-linux-worm-targets-routers-cameras-Internet-of-things-devices.

37In 2016, the Mirai botnet used: Robinson Meyer (21 Oct 2016), “How a bunch of hacked DVR machines took down Twitter and Reddit,” Atlantic, https://www.theatlantic.com/technology/archive/2016/10/how-a-bunch-of-hacked-dvr-machines-took-down-twitter-and-reddit/505073.

37it exploited such rookie security mistakes: Manos Antonakakis et al. (8 Aug 2017), “Understanding the Mirai botnet,” in Proceedings of the 26th USENIX Security Symposium, https://www.usenix.org/system/files/conference/usenixsecurity17/sec17-antonakakis.pdf.

37In 2015, Chrysler recalled: Andy Greenberg (24 Jul 2016), “After Jeep hack, Chrysler recalls 1.4m vehicles for bug fix,” Wired, https://www.wired.com/2015/07/jeep-hack-chrysler-recalls-1-4m-vehicles-bug-fix.

38In 2017, Abbott Labs told: Dan Goodin (30 Aug 2017), “465k patients told to visit doctor to patch critical pacemaker vulnerability,” Ars Technica, https://www.arstechnica.com/information-technology/2017/08/465k-patients-need-a-firmware-update-to-prevent-serious-pacemaker-hacks.

38Kindle does the same thing: Kyree Leary (27 Apr 2017), “How to update your Kindle and Kindle Fire devices,” Digital Trends, https://www.digitaltrends.com/mobile/how-to-update-your-kindle.

38One 2016 survey found: Flexera Software (13 Mar 2017), Vulnerability Review 2017, https://www.flexera.com/enterprise/resources/research/vulnerability-review.

38Android users, for example: Alex Dobie (16 Sep 2012), “Why you’ll never have the latest version of Android,” Android Central, http://www.androidcentral.com/why-you-ll-never-have-latest-version-android.

38The result is that about half: Gregg Keizer (23 Mar 2017), “Google: Half of Android devices haven’t been patched in a year or more,” Computerworld, https://www.computerworld.com/article/3184400/android/google-half-of-android-devices-havent-been-patched-in-a-year-or-more.html.

38In 2014, an iOS patch left: Adrian Kingsley-Hughes (24 Sep 2014), “Apple pulls iOS 8.0.1 update, after killing cell service, Touch ID,” ZDNet, http://www.zdnet.com/article/apple-pulls-ios-8-0-1-update-after-killing-cell-service-touch-id.

38In 2017, a flawed patch: Dan Goodin (14 Aug 2017), “Update gone wrong leaves 500 smart locks inoperable,” Ars Technica, https://www.arstechnica.com/information-technology/2017/08/500-smart-locks-arent-so-smart-anymore-thanks-to-botched-update.

38In 2018, in response to: Mathew J. Schwartz (9 Jan 2018), “Microsoft pauses Windows security updates to AMD devices,” Data Breach Today, https://www.databreachtoday.com/microsoft-pauses-windows-security-updates-to-amd-devices-a-10567.

38There are more examples: Larry Seltzer (15 Dec 2014), “Microsoft update blunders going out of control,” ZDNet, http://www.zdnet.com/article/has-microsoft-stopped-testing-their-updates.

39Maintaining lots of different: Microsoft currently only supports the four most recent versions of Windows. Microsoft Corporation (accessed 24 Apr 2018), “Windows lifecycle fact sheet,” https://support.microsoft.com/en-us/help/13853/windows-lifecycle-fact-sheet.

40Some of the organizations affected: Brian Barrett (14 Jun 2017), “If you still use Windows XP, prepare for the worst,” Wired, https://www.wired.com/2017/05/still-use-windows-xp-prepare-worst.

40About 140 million computers: Jeff Parsons (15 May 2017), “This is how many computers are still running Windows XP,” Mirror, https://www.mirror.co.uk/tech/how-many-computers-still-running-10425650.

40including most ATMs: David Sancho, Numaan Huq, and Massimiliano Michenzi (2017), “Cashing in on ATM malware: A comprehensive look at various attack types,” Trend Micro, https://documents.trendmicro.com/assets/white_papers/wp-cashing-in-on-atm-malware.pdf.

40A popular shipboard satellite communications system: Catalin Cimpanu (26 Oct 2017), “Backdoor account found in popular ship satellite communications system,” Bleeping Computer, https://www.bleepingcomputer.com/news/security/backdoor-account-found-in-popular-ship-satellite-communications-system.

40For an airplane, it can cost: Lucian Armasu (13 Nov 2017), “Boeing 757 hacked by DHS in cybersecurity test,” Tom’s Hardware, http://www.tomshardware.com/news/boeing-757-remote-hack-test,35911.html.

41“false and misleading”: Dan Goodin (30 Aug 2017), “465k patients told to visit doctor to patch critical pacemaker vulnerability,” Ars Technica, https://arstechnica.com/information-technology/2017/08/465k-patients-need-a-firmware-update-to-prevent-serious-pacemaker-hacks.

41The FBI arrested Dmitry Sklyarov: Electronic Frontier Foundation (1 Jul 2011; last updated 7 Aug 2012), “US v. ElcomSoft Sklyarov,” https://www.eff.org/cases/us-v-elcomsoft-sklyarov.

41Also in 2001, HP used the law: John Leyden (31 Jul 2002), “HP invokes DMCA to quash Tru64 bug report,” Register, https://www.theregister.co.uk/2002/07/31/hp_invokes_dmca_to_quash. Declan McCullagh (2 Aug 2002), “HP backs down on copyright warning,” CNET, https://www.cnet.com/news/hp-backs-down-on-copyright-warning.

41In 2011, Activision used it to shut down: Electronic Frontier Foundation (1 Mar 2013), “Unintended consequences: Fifteen years under the DMCA,” https://www.eff.org/pages/unintended-consequences-fifteen-years-under-dmca.

42In 2016, the Library of Congress: Charlie Osborne (31 Oct 2016), “US DMCA rules updated to give security experts legal backing to research,” ZDNet, http://www.zdnet.com/article/us-dmca-rules-updated-to-give-security-experts-legal-backing-to-research.

42it’s a narrow exemption that’s temporary: Maria A. Pallante (Oct 2015), “Section 1201 rulemaking: Sixth triennial proceeding to determine exemptions to the prohibition on circumvention,” United States Copyright Office, https://www.copyright.gov/1201/2015/registers-recommendation.pdf.

42In 2008, the Boston MBTA used: Kim Zetter (9 Sep 2008), “DefCon: Boston subway officials sue to stop talk on fare card hacks,” Wired, https://www.wired.com/2008/08/injunction-requ.

42In 2013, Volkswagen sued: Chris Perkins (14 Aug 2015), “Volkswagen suppressed a paper about car hacking for 2 years,” Mashable, http://mashable.com/2015/08/14/volkswagen-suppress-car-vulnerability.

42And in 2016, the Internet security company FireEye: Kim Zetter (11 Sep 2016), “A bizarre twist in the debate over vulnerability disclosures,” Wired, https://www.wired.com/2015/09/fireeye-enrw-injunction-bizarre-twist-in-the-debate-over-vulnerability-disclosures.

42If you’re a young academic: Electronic Frontier Foundation (21 Jul 2016), “EFF lawsuit takes on DMCA section 1201: Research and technology restrictions violate the First Amendment,” https://www.eff.org/press/releases/eff-lawsuit-takes-dmca-section-1201-research-and-technology-restrictions-violate.

42“Waterfall” is the term used: Winston Royce (25–28 Aug 1970), “Managing the development of large software systems,” 1970 WESCON Technical Papers 26, https://books.google.com/books?id=9U1GAQAAIAAJ.

42“Agile” describes the newer model: Agile Alliance (accessed 24 Apr 2018), “Agile 101,” https://www.agilealliance.org/agile101.

43We need to integrate the two paradigms: There has been some work integrating security into agile development practices. Information Security Forum (Oct 2017), “Embedding Security into Agile Development: Ten Principles for Rapid Development,” unpublished draft.

3. Knowing Who’s Who on the Internet Is Getting Harder

44“On the Internet, no one knows you’re a dog”: Glenn Fleishman (14 Dec 2000), “Cartoon captures spirit of the Internet,” New York Times, http://www.nytimes.com/2000/12/14/technology/cartoon-captures-spirit-of-the-internet.html.

44“Remember when, on the Internet”: Kaamran Hafeez (23 Feb 2015), “Cartoon: ‘Remember when, on the Internet, nobody knew who you were?’” New Yorker, http://www.kaamranhafeez.com/product/remember-internet-nobody-knew-new-yorker-cartoon.

45Tailored Access Operations (TAO) group: It’s now called the Computer Network Operations group.

45In a nutshell, he said: Rob Joyce (28 Jan 2016), “Disrupting nation state hackers,” USENIX Enigma 2016, https://www.youtube.com/watch?v=bDJb8WOJYdA (video), https://www.usenix.org/sites/default/files/conference/protected-files/enigma_slides_joyce.pdf (slides).

45It’s how the Chinese hackers breached: Brendan I. Koerner (23 Oct 2016), “Inside the cyberattack that shocked the U.S. government,” Wired, https://www.wired.com/2016/10/inside-cyberattack-shocked-us-government.

45The 2014 criminal attack against Target Corporation: Brian Krebs (5 Feb 2014), “Target hackers broke in via HVAC company,” Krebs on Security, https://krebsonsecurity.com/2014/02/target-hackers-broke-in-via-hvac-company.

45From 2011 to 2014, Iranian hackers stole: Jim Finkle (29 May 2014), “Iranian hackers use fake Facebook accounts to spy on U.S., others,” Reuters, http://www.reuters.com/article/iran-hackers/iranian-hackers-use-fake-facebook-accounts-to-spy-on-u-s-others-idUSL1N0OE2CU20140529.

45The 2015 hacktivist who broke into: Lorenzo Franceschi-Bicchierai (15 Apr 2016), “The vigilante who hacked Hacking Team explains how he did it,” Vice Motherboard, https://motherboard.vice.com/en_us/article/3dad3n/the-vigilante-who-hacked-hacking-team-explains-how-he-did-it.

45And the 2016 Russian attacks against: David E. Sanger and Nick Corasanti (14 Jun 2016), “D.N.C. says Russian hackers penetrated its files, including dossier on Donald Trump,” New York Times, https://www.nytimes.com/2016/06/15/us/politics/russian-hackers-dnc-trump.html.

45One survey found that 80% of breaches: Andras Cser (8 Jul 2016), “The Forrester Wave: Privileged identity management, Q3 2016,” Forrester, https://www.beyondtrust.com/wp-content/uploads/forrester-wave-for-privilege-identity-management-2016.pdf.

45Google looked at Gmail users: Kurt Thomas and Angelika Moscicki (9 Nov 2017), “New research: Understanding the root cause of account takeover,” Google Security Blog, https://security.googleblog.com/2017/11/new-research-understanding-root-cause.html.

46They guess the answers to the “secret questions”: Bruce Schneier (9 Feb 2005), “The curse of the secret question,” Schneier on Security, https://www.schneier.com/essays/archives/2005/02/the_curse_of_the_sec.html.

46After receiving bad advice: Eric Lipton, David E. Sanger, and Scott Shane (13 Dec 2016), “The perfect weapon: How Russian cyberpower invaded the U.S.,” New York Times, https://www.nytimes.com/2016/12/13/us/politics/russia-hack-election-dnc.html.

46Google found and disabled the worm: Alex Johnson (4 May 2017), “Massive phishing attack targets Gmail users,” NBC News, https://www.nbcnews.com/tech/security/massive-phishing-attack-targets-millions-gmail-users-n754501.

46An example of something you are is biometrics: Nary Subramanian (1 Jan 2011), “Biometric authentication,” in Encyclopedia of Cryptography and Security, Springer, https://link-springer-com/content/pdf/10.1007%2F978-1-4419-5906-5_775.pdf.

46These are things you carry with you: Robert Zuccherato (1 Jan 2011), “Authentication token,” in Encyclopedia of Cryptography and Security, Springer, https://link-springer-com.ezproxy.cul.columbia.edu/referencework/10.1007%2F978-1-4419-5906-5.

47Using two of them together: J. R. Raphael (30 Nov 2017), “What is two-factor authentication (2FA)? How to enable it and why you should,” CSO, https://www.csoonline.com/article/3239144/password-security/what-is-two-factor-authentication-2fa-how-to-enable-it-and-why-you-should.html.

47This, of course, isn’t perfect either: Andy Greenberg (26 Jun 2016), “So hey you should stop using texts for two-factor authentication,” Wired, https://www.wired.com/2016/06/hey-stop-using-texts-two-factor-authentication.

47Sprint, T-Mobile, Verizon, and AT&T: Steve Dent (8 Sep 2017), “U.S. carriers partner on a better mobile authentication system,” Engadget, https://www.engadget.com/2017/09/08/mobile-authentication-taskforce-att-verizon-tmobile-sprint.

47Among other security protections: Dario Salice (17 Oct 2017), “Google’s strongest security, for those who need it most,” Keyword, https://www.blog.google/topics/safety-security/googles-strongest-security-those-who-need-it-most.

47Sticky-note passwords regularly show up: Here’s one example from 2018: Kif Leswing (16 Jan 2018), “A password for the Hawaii emergency agency was hiding in a public photo, written on a Post-it note,” Business Insider, http://www.businessinsider.com/hawaii-emergency-agency-password-discovered-in-photo-sparks-security-criticism-2018-1.

48Your smartphone has evolved into: Gary Robbins (23 Apr 2017), “The Internet of Things lets you control the world with a smartphone,” San Diego Union Tribune, http://www.sandiegouniontribune.com/sd-me-connected-home-20170423-story.html.

48A hacker can convince a cell provider: Steven Melendez (18 Jul 2017), “How to steal a phone number and everything linked to it,” Fast Company, https://www.fastcompany.com/40432975/how-to-steal-a-phone-number-and-everything-linked-to-it.

48They’ll reset bank accounts: Alex Perekalin (19 May 2017), “Why two-factor authentication is not enough,” Kaspersky Daily, https://www.kaspersky.com/blog/ss7-attack-intercepts-sms/16877. Nathaniel Popper (21 Aug 2017), “Identity thieves hijack cellphone accounts to go after virtual currency,” New York Times, https://www.nytimes.com/2017/08/21/business/dealbook/phone-hack-bitcoin-virtual-currency.html.

49This is called a man-in-the-middle attack: Rapid7 (9 Aug 2017), “Man-in-the-middle (MITM) attacks,” Rapid7 Fundamentals, https://www.rapid7.com/fundamentals/man-in-the-middle-attacks.

49A credit card issuer might flag: Gartner (accessed 24 Apr 2018), “Reviews for online fraud detection,” https://www.gartner.com/reviews/market/OnlineFraudDetectionSystems.

50This was one of the techniques: David Kushner (26 Feb 2013), “The real story of Stuxnet,” IEEE Spectrum, https://spectrum.ieee.org/telecom/security/the-real-story-of-stuxnet.

50For years, though, hackers have been: Dan Goodin (3 Nov 2017), “Stuxnet-style code signing is more widespread than anyone thought,” Ars Technica, https://arstechnica.com/information-technology/2017/11/evasive-code-signed-malware-flourished-before-stuxnet-and-still-does. Doowon Kim, Bum Jun Kwon, and Tudor Dumitras (1 Nov 2017), “Certified malware: Measuring breaches of trust in the Windows code-signing PKI,” ACM Conference on Computer and Communications Security (ACM CCS ’17), http://www.umiacs.umd.edu/~tdumitra/papers/CCS-2017.pdf.

51Facebook has a “real name” policy: Amanda Holpuch (15 Dec 2015), “Facebook adjusts controversial ‘real name’ policy in wake of criticism,” Guardian, https://www.theguardian.com/us-news/2015/dec/15/facebook-change-controversial-real-name-policy.

51Google requires a phone number: Eric Griffith (3 Dec 2017), “How to create an anonymous email account,” PC Magazine, https://www.pcmag.com/article2/0,2817,2476288,00.asp.

52He was found by a dogged FBI agent: Nate Anderson and Cyrus Farivar (3 Oct 2013), “How the feds took down the Dread Pirate Roberts,” Ars Technica, https://arstechnica.com/tech-policy/2013/10/how-the-feds-took-down-the-dread-pirate-roberts.

52Pedophiles have been identified and arrested: Joseph Cox (15 Jun 2016), “How the feds use Photoshop to track down pedophiles,” Vice Motherboard, https://motherboard.vice.com/en_us/article/8q8594/enhance-enhance-enhance-how-the-feds-use-photoshop-to-track-down-pedophiles. Tom Kelly (27 Oct 2007), “Ashbourne Interpol officer’s role in paedophile suspect hunt,” Heath Chronicle, http://www.meathchronicle.ie/news/roundup/articles/2007/03/11/1025-ashbourne-interpol-officers-role-in-paedophile-suspect-hunt.

52A Belarusian who ran: Dan Goodin (5 Dec 2017), “Mastermind behind sophisticated, massive botnet outs himself,” Ars Technica, https://arstechnica.com/tech-policy/2017/12/mastermind-behind-massive-botnet-tracked-down-by-sloppy-opsec.

52The Texas hacker Higinio O. Ochoa III: John Leyden (13 Apr 2012), “FBI track alleged Anon from unsanitised busty babe pic,” Register, https://www.theregister.co.uk/2012/04/13/fbi_track_anon_from_iphone_photo.

53“made significant advances in”: Leon E. Panetta (11 Oct 2012), “Remarks by Secretary Panetta on cybersecurity to the Business Executives for National Security, New York City,” US Department of Defense, http://archive.defense.gov/transcripts/transcript.aspx?transcriptid=5136.

53Other US government officials: Andy Greenberg (8 Apr 2010), “Security guru Richard Clarke talks cyberwar,” Forbes, http://www.forbes.com/2010/04/08/cyberwar-obama-korea-technology-security-clarke.html.

53“It’s amazing the amount of lawyers”: Kim Zetter (29 Jan 2016), “NSA hacker chief explains how to keep him out of your system,” Wired, https://www.wired.com/2016/01/nsa-hacker-chief-explains-how-to-keep-him-out-of-your-system.

535 Chinese for hacking: US Department of Justice (19 May 2014), “U.S. charges five Chinese military hackers for cyber espionage against U.S. corporations and a labor organization for commercial advantage,” https://www.justice.gov/opa/pr/us-charges-five-chinese-military-hackers-cyber-espionage-against-us-corporations-and-labor.

5313 Russians for interfering: Matt Apuzzo and Sharon LaFraniere (16 Feb 2018), “13 Russians indicted as Mueller reveals effort to aid Trump campaign,” New York Times, https://www.nytimes.com/2018/02/16/us/politics/russians-indicted-mueller-election-interference.html.

53Unless attribution is followed by: Benjamin Edwards et al. (11 Jan 2017), “Strategic aspects of cyberattack, attribution, and blame,” Proceedings of the National Academy of Sciences of the United States of America 114, no. 11, http://www.pnas.org/content/pnas/114/11/2825.full.pdf.

54The main points are these: William R. Detlefsen (23 May 2015), “Cyber attacks, attribution, and deterrence: Three case studies,” School of Advanced Military Studies, US Army Command and General Staff College, http://www.dtic.mil/dtic/tr/fulltext/u2/1001276.pdf. Benjamin Edwards et al. (11 Jan 2017), “Strategic aspects of cyberattack, attribution, and blame,” Proceedings of the National Academy of Sciences of the United States of America 114, no. 11, http://www.pnas.org/content/114/11/2825.full.pdf. Delbert Tran (16 Aug 2017), “The law of attribution,” Cyber Conflict Project, Yale University, https://law.yale.edu/system/files/area/center/global/document/2017.05.10_-_law_of_attribution.pdf.

54I was on the wrong side of this debate: Bruce Schneier (11 Dec 2014), “Comments on the Sony hack,” Schneier on Security, https://www.schneier.com/blog/archives/2014/12/comments_on_the.html.

54It wasn’t until the New York Times reported: David E. Sanger and Martin Fackler (18 Jan 2015), “N.S.A. breached North Korean networks before Sony attack, officials say,” New York Times, https://www.nytimes.com/2015/01/19/world/asia/nsa-tapped-into-north-korean-networks-before-sony-attack-officials-say.html.

55Right now, Russia doesn’t do much: When Russia attacked the 2018 Winter Olympics in South Korea, it tried to blame North Korea. Ellen Nakashima (24 Feb 2018), “Russian spies hacked the Olympics and tried to make it look like North Korea did it, U.S. officials say,” Washington Post, https://www.washingtonpost.com/world/national-security/russian-spies-hacked-the-olympics-and-tried-to-make-it-look-like-north-korea-did-it-us-officials-say/2018/02/24/44b5468e-18f2-11e8-92c9-376b4fe57ff7_story.html.

4. Everyone Favors Insecurity

56The FBI wants you to have security: I’ll talk about this in Chapter 11, but here’s just one recent example: Cyrus Farivar (7 Mar 2018), “FBI again calls for magical solution to break into encrypted phones,” Ars Technica, https://arstechnica.com/tech-policy/2018/03/fbi-again-calls-for-magical-solution-to-break-into-encrypted-phones.

57“surveillance capitalism”: Shoshana Zuboff (17 Apr 2015), “Big other: Surveillance capitalism and the prospects of an information civilization,” Journal of Information Technology 30, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2594754.

57Companies are trying to figure out: Aaron Taube (24 Jan 2014), “Apple wants to use your heart rate and facial expressions to figure out what mood you’re in,” Business Insider, http://www.businessinsider.com/apples-mood-based-ad-targeting-patent-2014-1. Andrew McStay (4 Aug 2015), “Now advertising billboards can read your emotions . . . and that’s just the start,” Conversation, http://theconversation.com/now-advertising-billboards-can-read-your-emotions-and-thats-just-the-start-45519.

57They’re trying to determine what you’re paying attention to: Andrew McStay (27 Jun 2017), “Tech firms want to detect your emotions and expressions, but people don’t like it,” Conversation, https://theconversation.com/tech-firms-want-to-detect-your-emotions-and-expressions-but-people-dont-like-it-80153. Nick Whigham (13 May 2017), “Glitch in digital pizza advert goes viral, shows disturbing future of facial recognition tech,” News.com.au, http://www.news.com.au/technology/innovation/design/glitch-in-digital-pizza-advert-goes-viral-shows-disturbing-future-of-facial-recognition-tech/news-story/3b43904b6dd5444a279fd3cd6f8551db.

57They’re trying to learn what images you respond to: Pamela Paul (10 Dec 2010), “Flattery will get an ad nowhere,” New York Times, http://www.nytimes.com/2010/12/12/fashion/12Studied.html.

58No one knows how many online: Paul Boutin (30 May 2016), “The secretive world of selling data about you,” Newsweek, http://www.newsweek.com/secretive-world-selling-data-about-you-464789.

58That list would include any apps: Keith Collins (21 Nov 2017), “Google collects Android users’ locations even when location services are disabled,” Quartz, https://qz.com/1131515/google-collects-android-users-locations-even-when-location-services-are-disabled. Arsalan Mosenia et al. (15 Sep 2017), “PinMe: Tracking a smartphone user around the world,” IEEE Transactions on Multi-Scale Computing Systems vol. PP, no. 99, http://ieeexplore.ieee.org/document/8038870. Christopher Loran (13 Dec 2017), “How you can be tracked even with your GPS turned off,” Android Authority, https://www.androidauthority.com/tracked-gps-off-822865.

58In 2013, researchers discovered: Jialiu Lin et al. (5–8 Sep 2012), “Expectation and purpose: Understanding users’ mental models of mobile app privacy through crowdsourcing,” in Proceedings of the 2012 International Conference on Ubiquitous Computing, ACM, https://www.winlab.rutgers.edu/~janne/privacyasexpectations-ubicomp12-final.pdf.

58Any Wi-Fi networks your phone connects to: Retailers are tracking customers using their cell phones’ Wi-Fi as they walk around in stores. Stephanie Clifford and Quentin Hardy (14 Jul 2013), “Attention, shoppers: Store is tracking your cell,” New York Times, http://www.nytimes.com/2013/07/15/business/attention-shopper-stores-are-tracking-your-cell.html.

58The company Alphonso provides apps: Sapna Maheshwari (28 Dec 2017), “That game on your phone may be tracking what you’re watching on TV,” New York Times, https://www.nytimes.com/2017/12/28/business/media/alphonso-app-tracking.html.

58Facebook has a patent on using: Ben Chen and Facebook Corporation (22 Mar 2016), “Systems and methods for utilizing wireless communications to suggest connections for a user,” US Patent 9,294,991, https://patents.justia.comm/patent/9294991.

58Did an automatic license plate scanner: Catherine Crump et al. (17 Jul 2013), “You are being tracked: How license plate readers are being used to record Americans’ movements,” American Civil Liberties Union, https://www.aclu.org/files/assets/071613-aclu-alprreport-opt-v05.pdf.

58Surveillance companies know a lot about us: Dylan Curren (30 Mar 2018), “Are you ready? Here’s all the data Facebook and Google have on you,” Guardian, https://www.theguardian.com/commentisfree/2018/mar/28/all-the-data-facebook-google-has-on-you-privacy.

58We never lie to our search engines: Settings like Chrome’s “incognito mode” or Firefox’s “private browsing” keep the browser from saving your browsing history. It does not prevent any websites you visit from tracking you.

59Already, all new Toyota cars track speed: Hans Greimel (6 Oct 2015), “Toyota unveils new self-driving safety tech, targets 2020 autonomous drive,” Automotive News, http://www.autonews.com/article/20151006/OEM06/151009894/toyota-unveils-new-self-driving-safety-tech-targets-2020-autonomous.

59In 2015, John Deere told: Dana Bartholomew (2015), “Long comment regarding a proposed exemption under 17 U.S.C. 1201,” Deere and Company, https://copyright.gov/1201/2015/comments-032715/class%2021/John_Deere_Class21_1201_2014.pdf.

60Apple censored apps that tracked: Stuart Dredge (30 Sep 2015), “Apple removed drone-strike apps from App Store due to ‘objectionable content,’” Guardian, https://www.theguardian.com/technology/2015/sep/30/apple-removing-drone-strikes-app. Lorenzo Franceschi-Bicchierai (28 Mar 2017), “Apple just banned the app that tracks U.S. drone strikes again,” Vice Motherboard, https://motherboard.vice.com/en_us/article/538kan/apple-just-banned-the-app-that-tracks-us-drone-strikes-again.

60“content that ridicules public figures”: Jason Grigsby (19 Apr 2010), “Apple’s policy on satire: 16 apps rejected for ‘ridiculing public figures,’” Cloudfour, https://cloudfour.com/thinks/apples-policy-on-satire-16-rejected-apps.

60in 2017, Apple removed security apps: Telegraph Reporters (31 Jul 2017), “Apple removes VPN apps used to evade China’s internet censorship,” Telegraph, http://www.telegraph.co.uk/technology/2017/07/31/apple-removes-vpn-apps-used-evade-chinas-internet-censorship.

60Google has also banned an app: AdNauseam (5 Jan 2017), “AdNauseam banned from the Google Web Store,” https://adnauseam.io/free-adnauseam.html.

61“Some of us have pledged our allegiance”: Bruce Schneier (26 Nov 2012), “When it comes to security, we’re back to feudalism,” Wired, https://www.wired.com/2012/11/feudal-security.

61Companies owning fleets of autonomous cars: Judith Donath (16 Nov 2017), “UberFREE: The ultimate advertising experience,” Medium, https://medium.com/@judithd/the-future-of-self-driving-cars-and-of-advertising-will-be-promoted-rides-free-transportation-b5f7acd702d4.

62Because the machines use software: After years of refusing to allow consumers to use refillable pods, Keurig now allows consumers to use any coffee they want, as long as they buy a special add-on. Alex Hern (11 May 2015), “Keurig takes steps towards abandoning coffee-pod DRM,” Guardian, https://www.theguardian.com/technology/2015/may/11/keurig-takes-steps-towards-abandoning-coffee-pod-drm.

62HP printers no longer allow: Brian Barrett (23 Sep 2016), “HP has added DRM to its ink cartridges. Not even kidding (updated),” Wired, https://www.wired.com/2016/09/hp-printer-drm.

62And while some companies have overreached: Electronic Frontier Foundation (last updated 31 Aug 2004), Chamberlain Group Inc. v. Skylink Technologies Inc., https://www.eff.org/cases/chamberlain-group-inc-v-skylink-technologies-inc. Tech Law Journal (31 Aug 2004), “Federal Circuit rejects anti-circumvention claim in garage door opener case,” http://www.techlawjournal.com/topstories/2004/20040831.asp. US Supreme Court (25 Mar 2014), “Opinion,” Lexmark International, Inc. v. Static Control Components, Inc., No. 12–873, https://www.supremecourt.gov/opinions/13pdf/12-873_3dq3.pdf.

63The data is owned by the companies: Hugo Campos (24 Mar 2015), “The heart of the matter,” Slate, http://www.slate.com/articles/technology/future_tense/2015/03/patients_should_be_allowed_to_access_data_generated_by_implanted_devices.html.

63Similarly, people have been hacking: Darren Murph (6 Apr 2007), “Mileage maniacs hack Toyota’s Prius for 116 mpg,” Engadget, https://www.engadget.com/2007/04/06/mileage-maniacs-hack-toyotas-prius-for-116-mpg.

63There are hacks and cheat codes: Jeremy Hoag (13 Mar 2012), “Hack your ride: Cheat codes and workarounds for your car’s tech annoyances,” Lifehacker, http://lifehacker.com/5893227/hack-your-ride-cheat-codes-and-workarounds-for-your-cars-tech-annoyances.

63It’s no different with automobile: Michelle V. Rafter (22 Jul 2014), “Decoding what’s in your car’s black box,” Edmunds, https://www.edmunds.com/car-technology/car-black-box-recorders-capture-crash-data.html.

63Police and insurance companies: Peter Hall (7 Jun 2014), “Car black box data can be used as evidence,” Morning Call, http://www.mcall.com/mc-car-black-box-data-can-be-used-as-evidence-story.html.

63A California law allowing: Brian Heaton (27 Mar 2014), “Expert: California car data privacy bill ‘unworkable,’” Government Technology, http://www.govtech.com/transportation/Expert-California-Car-Data-Privacy-Bill-Unworkable.html.

63And John Deere tractor owners: Jason Koebler (21 Mar 2017), “Why American farmers are hacking their tractors with Ukrainian firmware,” Vice Motherboard, https://motherboard.vice.com/en_us/article/xykkkd/why-american-farmers-are-hacking-their-tractors-with-ukrainian-firmware.

63For example, some people are hacking: Jerome Radcliffe (4 Aug 2011), “Hacking medical devices for fun and insulin: Breaking the human SCADA system,” Black Hat 2011, https://media.blackhat.com/bh-us-11/Radcliffe/BH_US_11_Radcliffe_Hacking_Medical_Devices_WP.pdf. Chuck Seegert (8 Oct 2014), “Hackers develop DIY remote-monitoring for diabetes,” Med Device Online, http://www.meddeviceonline.com/doc/hackers-develop-diy-remote-monitoring-for-diabetes-0001.

64had used it to spy on journalists: John Scott-Railton et al. (19 Jun 2017), “Reckless exploit: Mexican journalists, lawyers, and a child targeted with NSO spyware,” Citizen Lab, https://citizenlab.ca/2017/06/reckless-exploit-mexico-nso.

64dissidents, political opponents: John Scott-Railton et al. (29 Jun 2017), “Reckless redux: Senior Mexican legislators and politicians targeted with NSO spyware,” Citizen Lab, https://citizenlab.ca/2017/06/more-mexican-nso-targets.

64international investigators: John Scott-Railton et al. (10 Jul 2017), “Reckless III: Investigation into Mexican mass disappearance targeted with NSO spyware,” Citizen Lab, https://citizenlab.ca/2017/07/mexico-disappearances-nso.

64lawyers: John Scott-Railton et al. (2 Aug 2017), “Reckless IV: Lawyers for murdered Mexican women’s families targeted with NSO spyware,” Citizen Lab, https://citizenlab.ca/2017/08/lawyers-murdered-women-nso-group.

64anti-corruption groups: John Scott-Railton et al. (30 Aug 2017), “Reckless V: Director of Mexican anti-corruption group targeted with NSO group’s spyware,” Citizen Lab, https://citizenlab.ca/2017/08/nso-spyware-mexico-corruption.

64and people who supported a tax on soft drinks: John Scott-Railton et al. (11 Feb 2017), “Bitter sweet: Supporters of Mexico’s soda tax targeted with NSO exploit links,” Citizen Lab, https://citizenlab.ca/2017/02/bittersweet-nso-mexico-spyware.

64The products of FinFisher: Bill Marczak et al. (15 Oct 2015), “Pay no attention to the server behind the proxy: Mapping FinFisher’s continuing proliferation,” Citizen Lab, https://citizenlab.ca/2015/10/mapping-finfishers-continuing-proliferation.

65And it does—through bribery: Glenn Greenwald (2014), No Place to Hide: Edward Snowden, the NSA, and the U.S. Surveillance State, Metropolitan Books, https://books.google.com/books/?id=AvFzAgAAQBAJ.

65collecting cell phone location data: The cooperation of the telecommunications industry is essential for many of the NSA’s collection programs. Mieke Eoyang (6 Apr 2016), “Beyond privacy and security: The role of the telecommunications industry in electronic surveillance,” Aegis Paper Series No. 1603, Hoover Institution, https://www.hoover.org/research/beyond-privacy-security-role-telecommunications-industry-electronic-surveillance-0.

65Similarly, Russia gets bulk access: Andrei Soldatov and Irina Borogan (8 Sep 2015), “Inside the Red Web: Russia’s back door onto the internet—extract,” Guardian, https://www.theguardian.com/world/2015/sep/08/red-web-book-russia-internet.

65Instead, they buy surveillance: Aaron Sankin (9 Jul 2015), “Forget Hacking Team—Many other companies sell surveillance tech to repressive regimes,” Daily Dot, https://www.dailydot.com/layer8/hacking-team-competitors.

65They even have a conference, called ISS World: Patrick Howell O’Neill (20 Jun 2017), “ISS World: The traveling spyware roadshow for dictatorships and democracies,” CyberScoop, https://www.cyberscoop.com/iss-world-wiretappers-ball-nso-group-ahmed-mansoor.

66Moonlight Maze in 1999: Juan Andres Guerrero-Saade et al. (Apr 2017), “Penquin’s moonlit maze: The dawn of nation-state digital espionage,” Kaspersky Lab, https://securelist.com/files/2017/04/Penquins_Moonlit_Maze_PDF_eng.pdf.

66Titan Rain in the early 2000s: Richard Norton-Taylor (4 Sep 2007), “Titan Rain: How Chinese hackers targeted Whitehall,” Guardian, https://www.theguardian.com/technology/2007/sep/04/news.internet.

66Buckshot Yankee in 2008: Ellen Nakashima (8 Dec 2011), “Cyber-intruder sparks response, debate,” Washington Post, https://www.washingtonpost.com/national/national-security/cyber-intruder-sparks-response-debate/2011/12/06/gIQAxLuFgO_story.html.

66Over the years, China has stolen: Caitlin Dewey (28 May 2013), “The U.S. weapons systems that experts say were hacked by the Chinese,” Washington Post, https://www.washingtonpost.com/news/worldviews/wp/2013/05/28/the-u-s-weapons-systems-that-experts-say-were-hacked-by-the-chinese.

66In 2010, China hacked into Google: Kim Zetter (12 Jan 2010), “Google to stop censoring search results in China after hack attack,” Wired, https://www.wired.com/2010/01/google-censorship-china.

66In 2015, we learned that China: Robert Windrem (10 Aug 2015), “China read emails of top U.S. officials,” NBC News, https://www.nbcnews.com/news/us-news/china-read-emails-top-us-officials-n406046.

66Also in 2015, the Chinese hacked: Brendan I. Koerner (23 Oct 2016), “Inside the cyberattack that shocked the U.S. government,” Wired, https://www.wired.com/2016/10/inside-cyberattack-shocked-us-government. Evan Perez (24 Aug 2017), “FBI arrests Chinese national connected to malware used in OPM data breach,” CNN, http://www.cnn.com/2017/08/24/politics/fbi-arrests-chinese-national-in-opm-data-breach/index.html.

66from Russia: Kaspersky Lab Global Research and Analysis Team (30 Aug 2017), “Introducing White Bear,” SecureList, https://securelist.com/introducing-whitebear/81638.

66China: British Broadcasting Corporation (29 Mar 2009), “Major cyber spy network uncovered,” BBC News, http://news.bbc.co.uk/1/hi/world/americas/7970471.stm.

66the US: Boldizsár Bencsáth et al. (14 Oct 2011), “Duqu: A Stuxnet-like malware found in the wild,” Laboratory of Cryptography and System Security, Budapest University of Technology and Economics, http://www.crysys.hu/publications/files/bencsathPBF11duqu.pdf.

66the US and Israel together: Ellen Nakashima, Greg Miller, and Julie Tate (19 Jun 2012), “U.S., Israel developed Flame computer virus to slow Iranian nuclear efforts, officials say,” Washington Post, https://www.washingtonpost.com/world/national-security/us-israel-developed-computer-virus-to-slow-iranian-nuclear-efforts-officials-say/2012/06/19/gJQA6xBPoV_story.html.

66Spain, and several unidentified countries: Fahmida Y. Rashid (11 Feb 2014), “The Mask hack ‘beyond anything we’ve seen so far,’” PC Magazine, http://securitywatch.pcmag.com/hacking/320622-the-mask-hack-beyond-anything-we-ve-seen-so-far. Brian Donohue (11 Feb 2014), “The Mask: Unveiling the world’s most sophisticated APT campaign,” Kaspersky Lab Daily, https://www.kaspersky.com/blog/the-mask-unveiling-the-worlds-most-sophisticated-apt-campaign/3723. Dan Goodin (8 Aug 2016), “Researchers crack open unusually advanced malware that hid for 5 years,” Ars Technica, https://arstechnica.com/information-technology/2016/08/researchers-crack-open-unusually-advanced-malware-that-hid-for-5-years.

66In 2017, North Korea hacked: Choe Sang-Hun (10 Oct 2017), “North Korean hackers stole U.S.-South Korean military plans, lawmaker says,” New York Times, https://www.nytimes.com/2017/10/10/world/asia/north-korea-hack-war-plans.html.

66China, for example, has stolen: Barack Obama and Xi Jinping (25 Sep 2015), “Remarks by President Obama and President Xi of the People’s Republic of China in joint press conference,” White House Office of the Press Secretary, https://obamawhitehouse.archives.gov/the-press-office/2015/09/25/remarks-president-obama-and-president-xi-peoples-republic-china-joint.

66China does seem to have toned down: Joseph Menn and Jim Finkle (20 Jun 2016), “Chinese economic cyber-espionage plummets in U.S.: Experts,” Reuters, http://www.reuters.com/article/us-cyber-spying-china/chinese-economic-cyber-espionage-plummets-in-u-s-experts-idUSKCN0Z700D.

66Just as the NSA spied on: Josh Dawsey, Emily Stephenson, and Andrea Peterson (5 Oct 2017), “John Kelly’s personal cellphone was compromised, White House believes,” Politico, https://www.politico.com/story/2017/10/05/john-kelly-cell-phone-compromised-243514.

66“You have to kind of salute the Chinese”: Mike Levine (25 Jun 2015), “China is ‘leading suspect’ in massive hack of US government networks,” ABC News, http://abcnews.go.com/US/china-leading-suspect-massive-hack-us-government-networks/story?id=32036222.

67One: its budget is significantly larger: The NSA’s budget is classified, but estimated to be around $11 billion. No other country even comes close. Scott Shane (29 Aug 2013), “New leaked document outlines U.S. spending on intelligence agencies,” New York Times, http://www.nytimes.com/2013/08/30/us/politics/leaked-document-outlines-us-spending-on-intelligence.html. Michael Holt (4 Oct 2015), “Top 15 global intelligence agencies with biggest budgets in the world have tripled since 2009–2016,” LinkedIn, https://www.linkedin.com/pulse/top-15-global-intelligence-agencies-biggest-budgets-world-holt.

67Three: the physical location: Anne Edmundson et al. (10 Mar 2017), “RAN: Routing around nation-states,” Princeton University, https://www.cs.princeton.edu/~jrex/papers/ran17.pdf.

67China leads the way: Kiyo Dorrer (31 Mar 2017), “Hello, Big Brother: How China controls its citizens through social media,” Deutsche Welle, http://www.dw.com/en/hello-big-brother-how-china-controls-its-citizens-through-social-media/a-38243388. Maya Wang (18 Aug 2017), “China’s dystopian push to revolutionize surveillance,” Human Rights Watch, https://www.hrw.org/news/2017/08/18/chinas-dystopian-push-revolutionize-surveillance.

67The government’s goal is not so much: Gary King, Jennifer Pan, and Margaret E. Roberts (May 2013), “How censorship in China allows government criticism but silences collective expression,” American Political Science Review 107, no. 2, https://gking.harvard.edu/files/censored.pdf.

67The Great Firewall of China: The system can be subverted, but combined with China’s surveillance and enforcement regime and the resultant self-censorship, it’s very effective. Oliver August (23 Oct 2007), “The Great Firewall: China’s misguided—and futile—attempt to control what happens online,” Wired, https://www.wired.com/2007/10/ff-chinafirewall.

67Each citizen will be given a score: Josh Chin and Gillian Wong (28 Nov 2016), “China’s new tool for social control: A credit rating for everything,” Wall Street Journal, https://www.wsj.com/articles/chinas-new-tool-for-social-control-a-credit-rating-for-everything-1480351590.

68France and Germany censor Nazi speech: Matthew Lasar (22 Jun 2011), “Nazi hunting: How France first ‘civilized’ the internet,” Ars Technica, https://arstechnica.com/tech-policy/2011/06/how-france-proved-that-the-internet-is-not-global. Anthony Faiola (6 Jan 2016), “Germany springs to action over hate speech against migrants,” Washington Post, https://www.washingtonpost.com/world/europe/germany-springs-to-action-over-hate-speech-against-migrants/2016/01/06/6031218e-b315-11e5-8abc-d09392edc612_story.html.

68Some say cyberwar is coming: Richard Clarke and Robert K. Knake (Apr 2010), Cyber War: The Next Threat to National Security and What to Do about It, Harper Collins, https://books.google.com/books?id=rNRlR4RGkecC.

68Some say cyberwar is here: David E. Sanger (2018), The Perfect Weapon: War, Sabotage, and Fear in the Cyber Age, Crown, https://books.google.com/books?id=htc7DwAAQBAJ.

68Some say cyberwar is everywhere: Fred Kaplan (2016), Dark Territory: The Secret History of Cyber War, Simon & Schuster, https://books.google.com/books?id=q1AJCgAAQBAJ.

68In truth, “cyberwar” is a term: Probably the best consensus definition is in the Tallinn Manual. NATO Cooperative Cyber Defence Centre of Excellence (Feb 2017), Tallinn Manual 2.0 on the International Law Applicable to Cyber Operations, 2nd edition, Cambridge University Press, http://www.cambridge.org/us/academic/subjects/law/humanitarian-law/tallinn-manual-20-international-law-applicable-cyber-operations-2nd-edition.

68Stuxnet, discovered in 2010: David Kushner (26 Feb 2013), “The real story of Stuxnet,” IEEE Spectrum, https://spectrum.ieee.org/telecom/security/the-real-story-of-stuxnet. Ralph Langner (1 Nov 2013), “To kill a centrifuge,” Langner Group, https://www.langner.com/wp-content/uploads/2017/03/to-kill-a-centrifuge.pdf. Kim Zetter (2015), Countdown to Zero Day: Stuxnet and the Launch of the World’s First Digital Weapon, Crown Books, https://books.google.com/books?id=1l2YAwAAQBAJ.

68Targets are not limited to: These are often known as SCADA systems. Alex Hern (17 Oct 2013), “U.S. power plants ‘vulnerable to hacking,’” Guardian, https://www.theguardian.com/technology/2013/oct/17/us-power-plants-hacking. Jack Wiles et al. (23 Aug 2008), Techno Security’s Guide to Securing SCADA, Ingress, https://books.google.com/books?id=sHtIdWn1gnAC.

69In 2007, Israel attacked: David A. Fulghum, Robert Wall, and Douglas Barrie (5 Nov 2007), “Details about Israel’s high-tech strike on Syria,” Aviation Week Network, http://aviationweek.com/awin/details-about-israel-s-high-tech-strike-syria.

69In 2008, Russia coordinated: John Markoff (13 Aug 2008), “Before the gunfire, cyberattacks,” New York Times, http://www.nytimes.com/2008/08/13/technology/13cyber.html.

69The US conducted a series: Alan D. Campen, ed. (1992), The First Information War: The Story of Communications, Computers, and Intelligence Systems in the Persian Gulf War, AFCEA International Press, https://archive.org/details/firstinformation00camp.

69In 2016, President Obama acknowledged: Barack Obama (13 Apr 2016), “Statement by the president on progress in the fight against ISIL,” White House Office of the Press Secretary, https://obamawhitehouse.archives.gov/the-press-office/2016/04/13/statement-president-progress-fight-against-isil.

69In 2017, we learned about a group: This operation has been named “Dragonfly.” Security Response Attack Investigation Team (20 Oct 2017), “Dragonfly: Western energy sector targeted by sophisticated attack group,” Symantec Corporation, https://www.symantec.com/connect/s/dragonfly-western-energy-sector-targeted-sophisticated-attack-group.

69In 2016, the Iranians did the same thing: Joseph Berger (25 Mar 2016), “A dam, small and unsung, is caught up in an Iranian hacking case,” New York Times, http://www.nytimes.com/2016/03/26/nyregion/rye-brook-dam-caught-in-computer-hacking-case.html.

69Experts surmise that these operations: United States Computer Emergency Readiness Team (20 Oct 2017), “Alert (TA17-293A): Advanced persistent threat activity targeting energy and other critical infrastructure sectors,” https://www.us-cert.gov/ncas/alerts/TA17-293A.

69“preparing the battlefield”: Seymour M. Hersh (7 Jul 2008), “Preparing the battlefield,” New Yorker, https://www.newyorker.com/magazine/2008/07/07/preparing-the-battlefield.

69It’s not just the stronger powers: Kertu Ruus (2008), “Cyber war I: Estonia attacked from Russia,” European Affairs 9, no. 1–2, http://www.europeaninstitute.org/index.php/component/content/article?id=67:cyber-war-i-estonia-attacked-from-russia.

69Iran attacked Las Vegas’s Sands Hotel: Benjamin Elgin and Michael Riley (12 Dec 2014), “Now at the Sands Casino: An Iranian hacker in every server,” Bloomberg, http://www.businessweek.com/articles/2014-12-11/iranian-hackers-hit-sheldon-adelsons-sands-casino-in-las-vegas.

69These include the US, the UK: The industry name for these kinds of attackers is APT: advanced persistent threat.

69They are the elite few: Ben Buchanan (Jan 2017), “The legend of sophistication in cyber operations,” Harvard Kennedy School Belfer Center for Science and International Affairs, https://www.belfercenter.org/publication/legend-sophistication-cyber-operations.

70Both of these tiers of countries: Scott DePasquale and Michael Daly (12 Oct 2016), “The growing threat of cyber mercenaries,” Politico, https://www.politico.com/agenda/story/2016/10/the-growing-threat-of-cyber-mercenaries-000221.

70If an isolated and heavily sanctioned country: David E. Sanger, David D. Kirkpatrick, and Nicole Perlroth (15 Oct 2017), “The world once laughed at North Korean cyberpower. No more,” New York Times, https://www.nytimes.com/2017/10/15/world/asia/north-korea-hacking-cyber-sony.html.

70The 2007 document didn’t mention: John D. Negroponte (11 Jan 2007), “Annual threat assessment of the Director of National Intelligence,” Office of the Director of National Intelligence, http://www.au.af.mil/au/awc/awcgate/dni/threat_assessment_11jan07.pdf.

70Even in the 2009 report, “the growing cyber”: Dennis C. Blair (12 Feb 2009), “Annual threat assessment of the intelligence community for the Senate Select Committee on Intelligence,” Office of the Director of National Intelligence, https://www.dni.gov/files/documents/Newsroom/Testimonies/20090212_testimony.pdf.

70By 2010, cyber threats were the first: Dennis C. Blair (2 Feb 2010), “Annual threat assessment of the U.S. intelligence community for the Senate Select Committee on Intelligence,” Office of the Director of National Intelligence, https://www.dni.gov/files/documents/Newsroom/Testimonies/20100202_testimony.pdf.

70“Our adversaries are becoming more adept”: Daniel R. Coats (11 May 2017), “Statement for the record: Worldwide threat assessment of the US intelligence community: Senate Select Committee on Intelligence,” Office of the Director of National Intelligence, https://www.dni.gov/files/documents/Newsroom/Testimonies/SSCI%20Unclassified%20SFR%20-%20Final.pdf.

70Similarly, the Munich Security Conference: Toomas Hendrik Ilves (31 Jan 2014), “Rebooting trust? Freedom vs. security in cyberspace,” Office of the President, Republic of Estonia, https://vp2006-2016.president.ee/en/official-duties/speeches/9796-qrebooting-trust-freedom-vs-security-in-cyberspaceq.

71Even a well-targeted cyberweapon like Stuxnet: Jarrad Shearer (13 Jul 2010; updated 26 Sep 2017), “W32.Stuxnet,” Symantec, https://www.symantec.com/security_response/writeup.jsp?docid=2010-071400-3123-99.

71In 2017, the global shipping giant Maersk: Iain Thomson (28 Jun 2017), “Everything you need to know about the Petya, er, NotPetya nasty trashing PCs worldwide,” Register, https://www.theregister.co.uk/2017/06/28/petya_notpetya_ransomware. Josh Fruhlinger (17 Oct 2017), “Petya ransomware and NotPetya: What you need to know now,” CSO, https://www.csoonline.com/article/3233210/ransomware/petya-ransomware-and-notpetya-malware-what-you-need-to-know-now.html. Nicholas Weaver (28 Jun 2017), “Thoughts on the NotPetya ransomware attack,” Lawfare, https://lawfareblog.com/thoughts-notpetya-ransomware-attack. Ellen Nakashima (12 Jan 2018), “Russian military was behind ‘Notpetya’ cyberattack in Ukraine, CIA concludes,” Washington Post, https://www.washingtonpost.com/world/national-security/russian-military-was-behind-notpetya-cyberattack-in-ukraine-cia-concludes/2018/01/12/048d8506-f7ca-11e7-b34a-b85626af34ef_story.html.

71when Iran attacked the Saudi: Nicole Perlroth (23 Oct 2012), “In cyberattack on Saudi firm, U.S. sees Iran firing back,” New York Times, http://www.nytimes.com/2012/10/24/business/global/cyberattack-on-saudi-oil-firm-disquiets-us.html.

71when North Korea used WannaCry: David E. Sanger and William J. Broad (4 Mar 2017), “Trump inherits a secret cyberwar against North Korean missiles,” New York Times, https://www.nytimes.com/2017/03/04/world/asia/north-korea-missile-program-sabotage.html.

71In 2012, a senior Russian general: Mark Galeotti (6 Jul 2014), “The ‘Gerasimov Doctrine’ and Russian non-linear war,” In Moscow’s Shadows, https://inmoscowsshadows.wordpress.com/2014/07/06/the-gerasimov-doctrine-and-russian-non-linear-war. Henry Foy (15 Sep 2017), “Valery Gerasimov, the general with a doctrine for Russia,” Financial Times, https://www.ft.com/content/7e14a438-989b-11e7-a652-cde3f882dd7b.

71There are cyberattacks that will be: David E. Sanger and Elisabeth Bumiller (31 May 2011), “Pentagon to consider cyberattacks acts of war,” New York Times, http://www.nytimes.com/2011/06/01/us/politics/01cyber.html.

71a state that political scientist Lucas Kello calls “unpeace”: Lucas Kello (2017), The Virtual Weapon and International Order, Yale University Press, https://yalebooks.yale.edu/book/9780300220230/virtual-weapon-and-international-order.

71The US responded to the North Korean attack: Carol Morello and Greg Miller (2 Jan 2015), “U.S. imposes sanctions on N. Korea following attack on Sony,” Washington Post, https://www.washingtonpost.com/world/national-security/us-imposes-sanctions-on-n-korea-following-attack-on-sony/2015/01/02/3e5423ae-92af-11e4-a900-9960214d4cd7_story.html.

71The US responded to Russian hacking: Lauren Gambino and Sabrina Siddiqui (30 Dec 2016), “Obama expels 35 Russian diplomats in retaliation for US election hacking,” Guardian, https://www.theguardian.com/us-news/2016/dec/29/barack-obama-sanctions-russia-election-hack.

72Cyber policy expert Jason Healey developed: Jason Healey (2011), “The spectrum of national responsibility for cyberattacks,” Brown Journal of World Affairs 18, no. 1, https://www.brown.edu/initiatives/journal-world-affairs/sites/brown.edu.initiatives.journal-world-affairs/files/private/articles/18.1_Healey.pdf.

72Here again, the operations had: David E. Sanger and William J. Broad (4 Mar 2017), “Trump inherits a secret cyberwar against North Korean missiles,” New York Times, https://www.nytimes.com/2017/03/04/world/asia/north-korea-missile-program-sabotage.html.

72Cyberweapons were used: Nadiya Kostyuk and Yuri M. Zhukov (10 Nov 2017), “Invisible digital front: Can cyber attacks shape battlefield events?” Journal of Conflict Resolution, http://journals.sagepub.com/doi/pdf/10.1177/0022002717737138.

72This means that a nation finding itself: Robert Axelrod and Rum Iliev (28 Jan 2014), “Timing of cyber conflict,” Proceedings of the National Academy of Sciences of the United States of America 111, no. 4, http://www.pnas.org/content/111/4/1298.

72While that intellectual-property theft: Caitlin Dewey (28 May 2013), “The U.S. weapons systems that experts say were hacked by the Chinese,” Washington Post, https://www.washingtonpost.com/news/worldviews/wp/2013/05/28/the-u-s-weapons-systems-that-experts-say-were-hacked-by-the-chinese. Marcus Weisgerber (23 Sep 2015), “China’s copycat jet raises questions about F-35,” Defense One, http://www.defenseone.com/threats/2015/09/more-questions-f-35-after-new-specs-chinas-copycat/121859. Justin Ling (24 Mar 2016), “Man who sold F-35 secrets to China pleads guilty,” Vice News, https://news.vice.com/article/man-who-sold-f-35-secrets-to-china-pleads-guilty.

73Countries are also getting more brazen: The Council on Foreign Relations is trying to track all of them. Adam Segal (6 Nov 2017), “Tracking state-sponsored cyber operations,” Council on Foreign Relations, https://www.cfr.org/blog/tracking-state-sponsored-cyber-operations.

73Attack is not only easier than defense: To be fair, just because attack is easier than defense doesn’t mean that offensive cyberspace operations are easier than defensive ones. Rebecca Slayton (1 Feb 2017), “What is the cyber offense-defense balance? Conceptions, causes, and assessment,” International Security 41, no. 3, https://www.mitpressjournals.org/doi/abs/10.1162/ISEC_a_00267?journalCode=isec.

73“I think both China and the United States”: Gideon Rachman (5 Jan 2017), “Axis of power,” New World, BBC Radio 4, http://www.bbc.co.uk/programmes/b086tfbh.

73“We have better cyber rocks to throw”: This quote is attributed to several people, but this is the earliest citation I could find: Fred Kaplan (12 Dec 2016), “How the U.S. could respond to Russia’s hacking,” Slate, http://www.slate.com/articles/news_and_politics/war_stories/2016/12/the_u_s_response_to_russia_s_hacking_has_consequences_for_the_future_of.html.

74In early 2018, the Indiana hospital Hancock Health: Charlie Osborne (17 Jan 2018), “US hospital pays $55,000 to hackers after ransomware attack,” ZDNet, http://www.zdnet.com/article/us-hospital-pays-55000-to-ransomware-operators.

74Ransomware is increasingly common: Brian Krebs (16 Sep 2016), “Ransomware getting more targeted, expensive,” Krebs on Security, https://krebsonsecurity.com/2016/09/ransomware-getting-more-targeted-expensive.

74Kaspersky Lab reported: Kaspersky Lab (28 Nov 2016), “Story of the year: The ransomware revolution,” Kaspersky Security Bulletin 2016, https://media.kaspersky.com/en/business-security/kaspersky-story-of-the-year-ransomware-revolution.pdf.

74Symantec found that average ransom amounts: Symantec Corporation (19 Jul 2016), “Ransomware and businesses 2016,” https://www.symantec.com/content/en/us/enterprise/media/security_response/whitepapers/ISTR2016_Ransomware_and_Businesses.pdf. Symantec Corporation (26 Apr 2017), “Alarming increase in targeted attacks aimed at politically motivated sabotage and subversion,” https://www.symantec.com/about/newsroom/press-releases/2017/symantec_0426_01.

74Carbon Black reported that total sales: Carbon Black (9 Oct 2017), “The ransomware economy,” https://cdn.www.carbonblack.com/wp-content/uploads/2017/10/Carbon-Black-Ransomware-Economy-Report-101117.pdf.

75All in all, it’s a billion-dollar business: Herb Weisman (9 Jan 2017), “Ransomware: Now a billion dollar a year crime and growing,” NBC News, https://www.nbcnews.com/tech/security/ransomware-now-billion-dollar-year-crime-growing-n704646. Symantec Corporation (19 Jul 2016), “Ransomware and businesses 2016,” http://www.symantec.com/content/en/us/enterprise/media/security_response/whitepapers/ISTR2016_Ransomware_and_Businesses.pdf.

75$500 billion: Luke Graham (7 Feb 2017), “Cybercrime costs the global economy $450 billion: CEO,” CNBC, https://www.cnbc.com/2017/02/07/cybercrime-costs-the-global-economy-450-billion-ceo.html.

75$3 trillion: Steve Morgan (22 Aug 2016), “Cybercrime damages expected to cost the world $6 trillion by 2021,” CSO, https://www.csoonline.com/article/3110467/security/cybercrime-damages-expected-to-cost-the-world-6-trillion-by-2021.html.

75Additional losses due to intellectual-property theft: Dennis C. Blair et al. (22 Feb 2017), “Update to the IP Commission Report: The theft of American intellectual property: Reassessments of the challenge and United States Policy,” National Bureau of Asian Research, http://www.ipcommission.org/report/IP_Commission_Report_Update_2017.pdf.

75A thief pretends to be: Federal Bureau of Investigation (14 Jun 2016), “Business e-mail compromise: The 3.1 billion dollar scam,” https://www.ic3.gov/media/2016/160614.aspx. Brian Krebs (23 Jun 2016), “FBI: Extortion, CEO fraud among top online fraud complaints in 2016,” Krebs on Security, https://krebsonsecurity.com/2017/06/fbi-extortion-ceo-fraud-among-top-online-fraud-complaints-in-2016.

75Or to divert the proceeds: Kenneth R. Harney (31 Mar 2016), “Scary new scam could swipe all your closing money,” Chicago Tribune, http://www.chicagotribune.com/classified/realestate/ct-re-0403-kenneth-harney-column-20160331-column.html.

75Turns out that the answer is: plenty: Brian Krebs (12 Oct 2012), “The scrap value of a hacked PC, revisited,” Krebs on Security, https://krebsonsecurity.com/2012/10/the-scrap-value-of-a-hacked-pc-revisited.

75Botnets can be used for all sorts of things: Dan Goodin (2 Feb 2018), “Cryptocurrency botnets are rendering some companies unable to operate,” Ars Technica, https://arstechnica.com/information-technology/2018/02/cryptocurrency-botnets-generate-millions-but-exact-huge-cost-on-victims.

75Hackers use bots to commit click fraud: White Ops (20 Dec 2016), “The Methbot operation,” https://www.whiteops.com/hubfs/Resources/WO_Methbot_Operation_WP.pdf.

76“The CaaS model provides easy access”: Rob Wainwright et al. (15 Mar 2017), “European Union serious and organized crime threat assessment: Crime in the age of technology,” Europol, https://www.europol.europa.eu/activities-services/main-reports/european-union-serious-and-organised-crime-threat-assessment-2017.

76They sell hacking tools: Nicolas Rapp and Robert Hackett (25 Oct 2017), “A hacker’s tool kit,” Fortune, http://fortune.com/2017/10/25/cybercrime-spyware-marketplace. Dan Goodin (1 Feb 2018), “New IoT botnet offers DDoSes of once-unimaginable sizes for $20,” Ars Technica, https://arstechnica.com/information-technology/2018/02/for-sale-ddoses-guaranteed-to-take-down-gaming-servers-just-20.

76North Korea is particularly egregious: Dorothy Denning (20 Feb 2018), “North Korea’s growing criminal cyberthreat,” Conversation, https://theconversation.com/north-koreas-growing-criminal-cyberthreat-89423.

76It employs hackers to raise money: Sam Kim (7 Feb 2018), “Inside North Korea’s hacker army,” Bloomberg, https://www.bloomberg.com/news/features/2018-02-07/inside-kim-jong-un-s-hacker-army.

76it stole $81 million from Bangladesh Bank: Kim Zetter (17 Jun 2016), “That insane, $81M Bangladesh bank heist? Here’s what we know,” Wired, https://www.wired.com/2016/05/insane-81m-bangladesh-bank-heist-heres-know.

76We’ve seen webcams, DVRs: Brian Krebs (16 Oct 2016), “Hacked cameras, DVRs powered today’s massive internet outage,” Krebs on Security, https://krebsonsecurity.com/2016/10/hacked-cameras-dvrs-powered-todays-massive-internet-outage.

76We’ve seen home appliances: Proofpoint (16 Jan 2014), “Your fridge is full of spam: Proof of an IoT-driven attack,” https://www.proofpoint.com/us/threat-insight/post/Your-Fridge-is-Full-of-SPAM. Dan Goodin (17 Jan 2014), “Is your refrigerator really part of a massive spam-sending botnet?” Ars Technica, https://arstechnica.com/information-technology/2014/01/is-your-refrigerator-really-part-of-a-massive-spam-sending-botnet.

76Attackers have bricked IoT devices: Pierluigi Paganini (12 Apr 2017), “The rise of the IoT botnet: Beyond the Mirai bot,” InfoSec Institute, http://resources.infosecinstitute.com/rise-iot-botnet-beyond-mirai-bot.

76Dick Cheney’s heart defibrillator: Dana Ford (24 Aug 2013), “Cheney’s defibrillator was modified to prevent hacking,” CNN, http://www.cnn.com/2013/10/20/us/dick-cheney-gupta-interview/index.html.

76In 2017, a man sent a tweet: David Kravets (17 Mar 2017), “Man accused of sending a seizure-inducing tweet charged with cyberstalking,” Ars Technica, https://arstechnica.com/tech-policy/2017/03/man-arrested-for-allegedly-sending-newsweek-writer-a-seizure-inducing-tweet.

77Also in 2017, WikiLeaks published information: Steve Overly (8 Mar 2017), “What we know about car hacking, the CIA and those WikiLeaks claims,” Washington Post, https://www.washingtonpost.com/news/innovations/wp/2017/03/08/what-we-know-about-car-hacking-the-cia-and-those-wikileaks-claims.

77Hackers have demonstrated ransomware: Lorenzo Franceschi-Bicchierai (7 Aug 2016), “Hackers make the first-ever ransomware for smart thermostats,” Vice Motherboard, https://motherboard.vice.com/en_us/article/aekj9j/Internet-of-things-ransomware-smart-thermostat.

77In 2017, an Austrian hotel: David Z. Morris (29 Jan 2017), “Hackers hijack hotel’s smart locks, demand ransom,” Fortune, http://fortune.com/2017/01/29/hackers-hijack-hotels-smart-locks.

77In 2017, the NotPetya ransomware: Russell Brandom (12 May 2017), “UK hospitals hit with massive ransomware attack,” Verge, https://www.theverge.com/2017/5/12/15630354/nhs-hospitals-ransomware-hack-wannacry-bitcoin. April Glaser (27 Jun 2017), “U.S. hospitals have been hit by the global ransomware attack,” Recode, https://www.recode.net/2017/6/27/15881666/global-eu-cyber-attack-us-hackers-nsa-hospitals.

77delay surgeries: Denis Campbell and Haroon Siddique (15 May 2017), “Operations cancelled as Hunt accused of ignoring cyber-attack warnings,” Guardian, https://www.theguardian.com/technology/2017/may/15/warning-of-nhs-cyber-attack-was-not-acted-on-cybersecurity.

77route incoming emergency patients elsewhere: ITV (16 May 2017), “NHS cyber attack: Hospitals no longer diverting patients,” http://www.itv.com/news/2017-05-16/nhs-cyber-attack-hospitals-no-longer-diverting-patients.

77We saw the harbinger of this trend: Sean Gallagher (25 Oct 2016), “How one rent-a-botnet army of cameras, DVRs caused Internet chaos,” Ars Technica, https://arstechnica.com/information-technology/2016/10/inside-the-machine-uprising-how-cameras-dvrs-took-down-parts-of-the-internet.

5. Risks Are Becoming Catastrophic

78You’ll see it called the “CIA triad”: Mike Gault (20 Dec 2016), “The CIA secret to cybersecurity that no one seems to get,” Wired, https://www.wired.com/2015/12/the-cia-secret-to-cybersecurity-that-no-one-seems-to-get.

78theft of celebrity photos from Apple’s iCloud: Jon Blistein (15 Mar 2016), “Hacker pleads guilty to stealing celebrity nude photos,” Rolling Stone, https://www.rollingstone.com/movies/news/hacker-pleads-guilty-to-stealing-celebrity-nude-photos-20160315.

78breach of the Ashley Madison adultery site: Nate Lord (27 Jul 2017), “A timeline of the Ashley Madison hack,” Digital Guardian, https://digitalguardian.com/blog/timeline-ashley-madison-hack.

78Russians hacked the Democratic National Committee: Eric Lipton, David E. Sanger, and Scott Shane (13 Dec 2016), “The perfect weapon: How Russian cyberpower invaded the U.S.,” New York Times, https://www.nytimes.com/2016/12/13/us/politics/russia-hack-election-dnc.html.

78stole 150 million personal records from Equifax: Stacy Cowley (2 Oct 2017), “2.5 million more people potentially exposed in Equifax breach,” New York Times, https://www.nytimes.com/2017/10/02/business/equifax-breach.html.

79Office of Personnel Management data breach: Brendan I. Koerner (23 Oct 2016), “Inside the cyberattack that shocked the U.S. government,” Wired, https://www.wired.com/2016/10/inside-cyberattack-shocked-us-government. Evan Perez (24 Aug 2017), “FBI arrests Chinese national connected to malware used in OPM data breach,” CNN, http://www.cnn.com/2017/08/24/politics/fbi-arrests-chinese-national-in-opm-data-breach/index.html.

79One way of thinking about this: Ross Anderson uses this language in his writings. Eireann Leverett, Richard Clayton, and Ross Anderson (6 Jun 2017), “Standardization and certification of the ‘Internet of Things,’” Institute for Consumer Policy, https://www.conpolicy.de/en/news-detail/standardization-and-certification-of-the-internet-of-things.

79In 2007, the Idaho National Laboratory: Kim Zetter (26 Sep 2007), “Simulated cyberattack shows hackers blasting away at the power grid,” Wired, https://www.wired.com/2007/09/simulated-cyber.

79In 2015, someone hacked into an unnamed steel mill: Kim Zetter (1 Jan 2015), “A cyberattack has caused confirmed physical damage for the second time ever,” Wired, https://www.wired.com/2015/01/german-steel-mill-hack-destruction.

79in 2016, the Department of Justice indicted an Iranian hacker: Joseph Berger (25 Mar 2016), “A dam, small and unsung, is caught up in an Iranian hacking case,” New York Times, http://www.nytimes.com/2016/03/26/nyregion/rye-brook-dam-caught-in-computer-hacking-case.html.

80“Accidents and, thus, potential catastrophes”: Charles Perrow (1999), Normal Accidents: Living with High-Risk Technologies, Princeton University Press, https://www.amazon.com/Normal-Accidents-Living-High-Risk-Technologies/dp/0691004129.

80In 2015, an 18-year-old outfitted a drone: Michael Martinez, John Newsome, and Rene Marsh (21 Jul 2015), “Handgun-firing drone appears legal in video, but FAA, police probe further,” CNN, http://www.cnn.com/2015/07/21/us/gun-drone-connecticut/index.html.

80Someone could also take control: Jordan Golson (2 Aug 2016), “Jeep hackers at it again, this time taking control of steering and braking systems,” Verge, https://www.theverge.com/2016/8/2/12353186/car-hack-jeep-cherokee-vulnerability-miller-valasek.

80hack a hospital drug pump: Kim Zetter (8 Jun 2015), “Hacker can send fatal dose to hospital drug pumps,” Wired, https://www.wired.com/2015/06/hackers-can-send-fatal-doses-hospital-drug-pumps.

80So are airplanes: Kim Zetter (26 May 2015), “Is it possible for passengers to hack commercial aircraft?” Wired, https://www.wired.com/2015/05/possible-passengers-hack-commercial-aircraft. Anthony Cuthbertson (20 Dec 2016), “Hackers expose security flaws with major airlines,” Newsweek, http://www.newsweek.com/hackers-hijack-planes-flight-system-flaw-534071.

80commercial ships: Jack Morse (18 Jul 2017), “Remotely hacking ships shouldn’t be this easy, and yet . . .” Mashable, http://mashable.com/2017/07/18/hacking-boats-is-fun-and-easy.

80electronic road signs: Jill Scharr (6 Jun 2014), “Hacking an electronic highway sign is way too easy,” Tom’s Guide, https://www.tomsguide.com/us/highway-signs-easily-hacked,news-18915.html.

80tornado sirens: Robert McMillan (12 Apr 2017), “Tornado-siren false alarm shows radio-hacking risk,” Wall Street Journal, https://www.wsj.com/articles/tornado-siren-false-alarm-shows-radio-hacking-risk-1492042082.

80Nuclear weapons systems are almost: John Denley (28 Sep 2017), “No nuclear weapon is safe from cyberattacks,” Wired, https://www.wired.co.uk/article/no-nuclear-weapon-is-safe-from-cyberattacks.

80Satellites, too: Gregory Falco (Mar 2018), “The Vacuum of Space Cyber Security,” Cyber Security Project, Harvard Kennedy School Belfer Center for Science and International Affairs, unpublished draft.

80Attacks against the integrity of data: Neal A. Pollar, Adam Segal, and Matthew G. DeVost (16 Jan 2018), “Trust war: Dangerous trends in cyber conflict,” War on the Rocks, https://warontherocks.com/2018/01/trust-war-dangerous-trends-cyber-conflict.

80In 2016, Russian government hackers: Rick Maese and Matt Bonesteel (9 Dec 2016), “World Anti-Doping Agency report details scope of massive Russian scheme,” Washington Post, https://www.washingtonpost.com/news/early-lead/wp/2016/12/09/wada-report-details-scope-of-massive-russian-doping-scheme.

80In 2017, hackers—possibly hired: Karen DeYoung and Ellen Nakashima (16 Jul 2016), “UAE orchestrated hacking of Qatari government sites, sparking regional upheaval, according to U.S. intelligence officials,” Washington Post, https://www.washingtonpost.com/world/national-security/uae-hacked-qatari-government-sites-sparking-regional-upheaval-according-to-us-intelligence-officials/2017/07/16/00c46e54-698f-11e7-8eb5-cbccc2e7bfbf_story.html.

80There is evidence that the Russians: Nicole Perlroth, Michael Wines, and Matthew Rosenberg (1 Sep 2017), “Russian election hacking efforts, wider than previously known, draw little scrutiny,” New York Times, https://www.nytimes.com/2017/09/01/us/politics/russia-election-hacking.html.

81“Most of the public discussion”: James R. Clapper (26 Feb 2015), “Statement for the record: Worldwide threat assessment of the US intelligence community: Senate Armed Services Committee,” Office of the Director of National Intelligence, http://www.dni.gov/files/documents/Unclassified_2015_ATA_SFR_-_SASC_FINAL.pdf.

81then–director of national intelligence James Clapper: Ashley Carman (11 Sep 2015), “‘Information integrity’ among top cyber priorities for U.S. gov’t, Clapper says,” SC Magazine, http://www.scmagazine.com/intelligence-committee-hosts-cybersecurity-hearing/article/438202.

81then–NSA director Mike Rogers: Katie Bo Williams (27 Sep 2015), “Officials worried hackers will change your data, not steal it,” Hill, http://thehill.com/policy/cybersecurity/254977-officials-worried-hackers-will-change-your-data-not-steal-it.

81Future cyber operations will almost certainly: James R. Clapper (9 Feb 2016), “Statement for the record: Worldwide threat assessment of the US intelligence community: Senate Armed Services Committee,” Office of the Director of National Intelligence, https://www.dni.gov/files/documents/SASC_Unclassified_2016_ATA_SFR_FINAL.pdf.

81Between 2014 and 2016: Shaun Waterman (20 Jul 2016), “Bank regulators briefed on Treasury-led cyber drill,” Fed Scoop, https://www.fedscoop.com/us-treasury-cybersecurity-drill-july-2016.

81and then established a program: Telis Demos (3 Dec 2017), “Banks build line of defense for doomsday cyberattack,” Wall Street Journal, https://www.wsj.com/articles/banks-build-line-of-defense-for-doomsday-cyberattack-1512302401.

82The machine-learning algorithm modifies: Ben Buchanan and Taylor Miller (Jun 2017), “Machine Learning for Policymakers: What It Is and Why It Matters,” Cyber Security Project, Harvard Kennedy School Belfer Center for Science and International Affairs, https://www.belfercenter.org/sites/default/files/files/publication/MachineLearningforPolicymakers.pdf.

82They categorize photographs and translate text: Sam Wong (30 Nov 2016), “Google Translate AI invents its own language to translate with,” New Scientist, https://www.newscientist.com/article/2114748-google-translate-ai-invents-its-own-language-to-translate-with. Cade Metz (9 May 2017), “Facebook’s new AI could lead to translations that actually make sense,” Wired, https://www.wired.com/2017/05/facebook-open-sources-neural-networks-speed-translations.

82They play Go as well as a master: Elizabeth Gibney (17 Jan 2016), “Google AI algorithm masters ancient game of Go,” Nature 529, http://www.nature.com/news/google-ai-algorithm-masters-ancient-game-of-go-1.19234.

82read X-rays and diagnose cancers: Andre Esteva et al. (25 Jan 2017), “Dermatologist-level classification of skin cancer with deep neural networks,” Nature 542, https://www.nature.com/nature/journal/v542/n7639/full/nature21056.html.

82inform bail, sentencing, and parole decisions: Julia Angwin et al. (23 May 2016), “Machine bias,” ProPublica, https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing.

82They analyze speech to assess suicide risk: Peter Holley (26 Sep 2017), “Teenage suicide is extremely difficult to predict. That’s why some experts are turning to machines for help,” Washington Post, https://www.washingtonpost.com/amphtml/news/innovations/wp/2017/09/25/teenage-suicide-is-extremely-difficult-to-predict-thats-why-some-experts-are-turning-to-machines-for-help.

82analyze faces to predict homosexuality: To be fair, there are a lot of questions about this research. Yilun Wang and Michal Kosinski (15 Feb 2017; last updated 16 Oct 2017), “Deep neural networks are more accurate than humans at detecting sexual orientation from facial images,” Open Science Framework, https://osf.io/zn79k.

82predicting the quality of fine Bordeaux wine: Orley Ashenfelter (29 May 2008), “Predicting the quality and prices of Bordeaux wine,” Economic Journal, http://onlinelibrary.wiley.com/doi/10.1111/j.1468-0297.2008.02148.x/abstract.

82hiring blue-collar employees: Mitchell Hoffman, Lisa Kahn, and Danielle Li (Nov 2015), “Discretion in hiring,” National Bureau of Economic Research, https://www.nber.org/papers/w21709.pdf.

82deciding whether to punt in football: Adam Himmelsbach (18 Aug 2012), “Punting less can be rewarding, but coaches aren’t risking jobs on it,” New York Times, http://www.nytimes.com/2012/08/19/sports/football/calculating-footballs-risk-of-not-punting-on-fourth-down.html.

82Machine learning is used to detect: Sally Adee (17 Aug 2016), “Scammer AI can tailor clickbait to you for phishing attacks,” New Scientist, https://www.newscientist.com/article/2101483-scammer-ai-can-tailor-clickbait-to-you-for-phishing-attacks.

82For example, Deep Patient: Riccardo Miotto, Brian A. Kidd, and Joel T. Dudley (17 May 2016), “Deep Patient: An unsupervised representation to predict the future of patients from the electronic health records,” Scientific Reports 6, no. 26094, https://www.nature.com/articles/srep26094.

83But although the system works: Will Knight (11 Apr 2017), “The dark secret at the heart of AI,” MIT Technology Review, https://www.technologyreview.com/s/604087/the-dark-secret-at-the-heart-of-ai.

83A 2014 book, Autonomous Technologies: William Messner, ed. (2014), Autonomous Technologies: Applications That Matter, SAE International, http://books.sae.org/jpf-auv-004.

84One research project focused on: Anh Nguyen, Jason Yosinski, and Jeff Clune (2 Apr 2015), “Deep neural networks are easily fooled: High confidence predictions for unrecognizable images,” in Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR ’15), https://arxiv.org/abs/1412.1897.

84A related research project was able: Christian Szegedy et al. (19 Feb 2014), “Intriguing properties of neural networks,” in Conference Proceedings: International Conference on Learning Representations (ICLR) 2014, https://arxiv.org/abs/1312.6199.

84Yet another project tricked an algorithm: Andrew Ilyas et al. (20 Dec 2017), “Partial information attacks on real-world AI,” LabSix, http://www.labsix.org/partial-information-adversarial-examples.

85Like the Microsoft chatbot Tay: James Vincent (24 Mar 2016), “Twitter taught Microsoft’s AI chatbot to be a racist asshole in less than a day,” Verge, https://www.theverge.com/2016/3/24/11297050/tay-microsoft-chatbot-racist.

85In 2017, Dow Jones accidentally: Timothy B. Lee (10 Oct 2017), “Dow Jones posts fake story claiming Google was buying Apple,” Ars Technica, https://arstechnica.com/tech-policy/2017/10/dow-jones-posts-fake-story-claiming-google-was-buying-apple.

85Within minutes, a trillion dollars: Bob Pisani (21 Apr 2015), “What caused the flash crash? DFTC, DOJ weigh in,” CNBC, https://www.cnbc.com/2015/04/21/what-caused-the-flash-crash-cftc-doj-weigh-in.html.

85in 2013, hackers broke into the Associated Press’s: Edmund Lee (24 Apr 2013), “AP Twitter account hacked in market-moving attack,” Bloomberg, https://www.bloomberg.com/news/articles/2013-04-23/dow-jones-drops-recovers-after-false-report-on-ap-twitter-page.

85We should also expect autonomous: George Dvorsky (11 Sep 2017), “Hackers have already started to weaponize artificial intelligence,” Gizmodo, https://gizmodo.com/hackers-have-already-started-to-weaponize-artificial-in-1797688425.

85The Cyber Grand Challenge was similar: Cade Metz (6 Jul 2016), “DARPA goes full Tron with its grand battle of the hack bots,” Wired, https://www.wired.com/2016/07/__trashed-19.

85One program found: Matthew Braga (16 Jun 2016), “In the future, we’ll leave software bug hunting to the machines,” Vice Motherboard, https://motherboard.vice.com/en_us/article/mg73a8/cyber-grand-challenge. Cade Metz (5 Aug 2016), “Hackers don’t have to be human anymore. This bot battle proves it,” Wired, https://www.wired.com/2016/08/security-bots-show-hacking-isnt-just-humans.

85In a later contest that had both: Sharon Gaudin (5 Aug 2016), “‘Mayhem’ takes first in DARPA hacking challenge,” Computerworld, https://www.computerworld.com/article/3104891/security/mayhem-takes-first-in-darpas-all-computer-hacking-challenge.html.

85Attackers will use software to: Kevin Townsend (29 Nov 2016), “How machine learning will help attackers,” Security Week, http://www.securityweek.com/how-machine-learning-will-help-attackers.

85Most security experts expect: Cylance (1 Aug 2017), “Black Hat attendees see AI as double-edged sword,” https://www.cylance.com/en_us/blog/black-hat-attendees-see-ai-as-double-edged-sword.html.

86“Artificial intelligence and machine learning”: Greg Allen and Taniel Chan (13 Jul 2017), “Artificial intelligence and national security,” Harvard Kennedy School Belfer Center for Science and International Affairs, https://www.belfercenter.org/sites/default/files/files/publication/AI%20NatSec%20-%20final.pdf.

86in robots to remotely take control of them: Matt Burgess (22 Aug 2017), “Ethical hackers have turned this robot into a stabbing machine,” Wired, https://www.wired.co.uk/article/hacked-robots-pepper-nao-alpha-2-stab-screwdriver.

86in teleoperated surgical robots: Tamara Bonaci et al. (17 Apr 2015), “To make a robot secure: An experimental analysis of cyber security threats against teleoperated surgical robotics,” ArXiv 1504.04339v1, https://arxiv.org/pdf/1504.04339v1.pdf. Darlene Storm (27 Apr 2015), “Researchers hijack teleoperated surgical robot: Remote surgery hacking threats,” Computerworld, https://www.computerworld.com/article/2914741/cybercrime-hacking/researchers-hijack-teleoperated-surgical-robot-remote-surgery-hacking-threats.html.

86and industrial robots: Thomas Fox-Brewster (3 May 2017), “Catastrophe warning: Watch an industrial robot get hacked,” Forbes, https://www.forbes.com/sites/thomasbrewster/2017/05/03/researchers-hack-industrial-robot-making-a-drone-rotor.

86Autonomous military systems deserve: Paul Scharre (24 Apr 2017), Army of None: Autonomous Weapons and the Future of War, W. W. Norton, https://books.google.com/books?id=sjMsDwAAQBAJ.

86The US Department of Defense defines: Heather Roff (9 Feb 2016), “Distinguishing autonomous from automatic weapons,” Bulletin of the Atomic Scientists, http://thebulletin.org/autonomous-weapons-civilian-safety-and-regulation-versus-prohibition/distinguishing-autonomous-automatic-weapons.

86If they are autonomous: Paul Scharre (29 Feb 2016), “Autonomous weapons and operational risk,” Center for a New American Security, https://www.cnas.org/publications/reports/autonomous-weapons-and-operational-risk.

86Technologists Bill Gates, Elon Musk, and Stephen Hawking: Michael Sainato (19 Aug 2015), “Stephen Hawking, Elon Musk, and Bill Gates warn about artificial intelligence,” Observer, http://observer.com/2015/08/stephen-hawking-elon-musk-and-bill-gates-warn-about-artificial-intelligence.

86The risks might be remote: Stuart Russell et al. (11 Jan 2015), “An open letter: Research priorities for robust and beneficial artificial intelligence,” Future of Life Institute, https://futureoflife.org/ai-open-letter.

86I am less worried about AI: These two essays talk about that: Ted Chiang (18 Dec 2017), “Silicon Valley is turning into its own worst fear,” BuzzFeed, https://www.buzzfeed.com/tedchiang/the-real-danger-to-civilization-isnt-ai-its-runaway. Charlie Stross (Jan 2018), “Dude, you broke the future!” Charlie’s Diary, http://www.antipope.org/charlie/-static/2018/01/dude-you-broke-the-future.html.

87“Long before we see such machines arising”: Rodney Brooks (7 Sep 2017), “The seven deadly sins of predicting the future of AI,” http://rodneybrooks.com/the-seven-deadly-sins-of-predicting-the-future-of-ai.

87For example, there is widespread suspicion: Sean Gallagher (15 Nov 2016), “Chinese company installed secret backdoor on hundreds of thousands of phones,” Ars Technica, https://arstechnica.com/information-technology/2016/11/chinese-company-installed-secret-backdoor-on-hundreds-of-thousands-of-phones.

87computer security products from Kaspersky Lab: Cyrus Farivar (11 Jul 2017), “Kaspersky under scrutiny after Bloomberg story claims close links to FSB,” Ars Technica, https://arstechnica.com/information-technology/2017/07/kaspersky-denies-inappropriate-ties-with-russian-govt-after-bloomberg-story.

87In 2018, US intelligence officials: Selena Larson (14 Feb 2018), “The FBI, CIA and NSA say Americans shouldn’t use Huawei phones,” CNN, http://money.cnn.com/2018/02/14/technology/huawei-intelligence-chiefs/index.html.

87Back in 1997, the Israeli company Check Point: Emily G. Cohen (7 Jul 1997), “Check Point response to Mossad rumor,” Firewalls Mailing List, Great Circle Associates, http://old.greatcircle.com/firewalls/mhonarc/firewalls.199707/msg00223.html.

87In the US, the NSA secretly installed: Julia Angwin et al. (15 Aug 2015), “AT&T helped U.S. spy on Internet on a vast scale, New York Times, https://www.nytimes.com/2015/08/16/us/politics/att-helped-nsa-spy-on-an-array-of-internet-traffic.html.

87They demonstrate the vulnerability: Arnd Weber et al. (22 Mar 2018), “Sovereignty in information technology: Security, safety and fair market access by openness and control of the supply chain,” Karlsruher Institut für Technologie, http://www.itas.kit.edu/pub/v/2018/weua18a.pdf.

88Adding a backdoor onto a computer chip: Georg T. Becker et al. (Jan 2014), “Stealthy dopant-level hardware Trojans: Extended version,” Journal of Cryptographic Engineering 4, https://link.springer.com/article/10.1007/s13389-013-0068-0.

88China demands to see source code: Paul Mozur (28 Jan 2015), “New rules in China upset Western tech companies,” New York Times, https://www.nytimes.com/2015/01/29/technology/in-china-new-cybersecurity-rules-perturb-western-tech-companies.html.

88So does the US: Zack Whittaker (17 Mar 2016), “U.S. government pushed tech firms to hand over source code,” ZDNet, http://www.zdnet.com/article/us-government-pushed-tech-firms-to-hand-over-source-code.

88Kaspersky offered to let any government: John Leyden (23 Oct 2017), “‘We’ve nothing to hide’: Kaspersky Lab offers to open up source code,” Register, https://www.theregister.co.uk/2017/10/23/kaspersky_source_code_review.

88In 2017, HP Enterprise faced criticism: Joel Schectman, Dustin Volz, and Jack Stubbs (2 Oct 2017), “HP Enterprise let Russia scrutinize cyberdefense system used by Pentagon,” Reuters, https://www.reuters.com/article/us-usa-cyber-russia-hpe-specialreport/special-report-hp-enterprise-let-russia-scrutinize-cyberdefense-system-used-by-pentagon-idUSKCN1C716M.

88According to NSA documents: Whether they were successful or not was deliberately withheld by the New York Times, citing national security concerns. My guess is that they were successful. David E. Sanger and Nicole Perlroth (23 Mar 2014), “N.S.A. breached Chinese servers seen as security threat,” New York Times, https://www.nytimes.com/2014/03/23/world/asia/nsa-breached-chinese-servers-seen-as-spy-peril.html.

88We know from the Snowden documents: The one document we have shows the NSA intercepting devices “bound for the Syrian Telecommunications Establishment (STE) to be used as part of their internet backbone.” Chief (name redacted), Access and Target Development (S3261) (Jun 2010), “Stealthy techniques can crack some of SIGINT’s hardest targets,” SID Today, http://www.spiegel.de/media/media-35669.pdf. Sean Gallagher (14 May 2014), “Photos of an NSA ‘upgrade’ factory show Cisco router getting implant,” Ars Technica, https://arstechnica.com/tech-policy/2014/05/photos-of-an-nsa-upgrade-factory-show-cisco-router-getting-implant.

88That was done without Cisco’s knowledge: Darren Pauli (18 Mar 2015), “Cisco posts kit to empty houses to dodge NSA chop shops,” Register, https://www.theregister.co.uk/2015/03/18/want_to_dodge_nsa_supply_chain_taps_ask_cisco_for_a_dead_drop.

88in Juniper firewalls: Kim Zetter (19 Dec 2015), “Secret code found in Juniper’s firewalls shows risk of government backdoors,” Wired, https://www.wired.com/2015/12/juniper-networks-hidden-backdoors-show-the-risk-of-government-backdoors.

88and D-Link routers: Jeremy Kirk (14 Oct 2013), “Backdoor found in D-Link router firmware code,” InfoWorld, http://www.infoworld.com/article/2612384/network-router/backdoor-found-in-d-link-router-firmware-code.html.

88One report said that 4.2 million fake apps: Gio Benitez (7 Nov 2017), “How to protect yourself from downloading fake apps and getting hacked,” ABC News, http://abcnews.go.com/US/protect-downloading-fake-apps-hacked/story?id=50972286.

88This included a fake WhatsApp app: Lorenzo Franceschi-Bicchierai (3 Nov 2017), “More than 1 million people downloaded a fake WhatsApp Android app,” Vice Motherboard, https://motherboard.vice.com/en_us/article/evbakk/fake-whatsapp-android-app-1-million-downloads.

88Hackers linked to China compromised: Lucian Constantin (18 Sep 2017), “Malware-infected CCleaner installer distributed to users via official servers for a month,” Vice Motherboard, https://motherboard.vice.com/en_us/article/a3kgpa/ccleaner-backdoor-malware-hack. Thomas Fox-Brewster (21 Sep 2017), “Avast: The 2.3M CCleaner hack was a sophisticated assault on the tech industry,” Forbes, https://www.forbes.com/sites/thomasbrewster/2017/09/21/avast-ccleaner-attacks-target-tech-industry.

89Unknown hackers corrupted: Andy Greenberg (7 Jul 2017), “The Petya plague exposes the threat of evil software updates,” Wired, https://www.wired.com/story/petya-plague-automatic-software-updates.

89Another group used fake antivirus updates: Joseph Graziano (21 Nov 2013), “Fake AV software updates are distributing malware,” Symantec Corporation, https://www.symantec.com/connect/s/fake-av-software-updates-are-distributing-malware.

89Researchers demonstrated how to hack: Omer Shwartz et al. (14 Aug 2017), “Shattered trust: When replacement smartphone components attack,” in Proceedings of the 11th USENIX Workshop on Offensive Technologies (WOOT 17), https://www.usenix.org/conference/woot17/workshop-program/presentation/shwartz.

89And there are enough similar attacks: Mike Murphy (18 Dec 2017), “Think twice about buying internet-connected devices off eBay,” Quartz, https://qz.com/1156059/dont-buy-second-hand-internet-connected-iot-devices-from-sites-like-ebay-ebay.

89In 2018, the African Union discovered: Aaron Maasho (29 Jan 2018), “China denies report it hacked African Union headquarters,” Reuters, https://www.reuters.com/article/us-africanunion-summit-china/china-denies-report-it-hacked-african-union-headquarters-idUSKBN1FI2I5.

89I am reminded of the US embassy: Elaine Sciolino (15 Nov 1988), “The bugged embassy case: What went wrong,” New York Times, http://www.nytimes.com/1988/11/15/world/the-bugged-embassy-case-what-went-wrong.html.

89“An aggressor nation or extremist group”: Elisabeth Bumiller and Thom Shanker (11 Oct 2012), “Panetta warns of dire threat of cyberattack,” New York Times, http://www.nytimes.com/2012/10/12/world/panetta-warns-of-dire-threat-of-cyberattack.html.

90“Cyber threats also pose an increasing risk”: Daniel R. Coats (11 May 2017), “Statement for the record: Worldwide threat assessment of the US intelligence community: Senate Select Committee on Intelligence,” Office of the Director of National Intelligence, https://www.dni.gov/files/documents/Newsroom/Testimonies/SSCI%20Unclassified%20SFR%20-%20Final.pdf.

90“The potential for surprise in the cyber realm”: Daniel R. Coats (13 Feb 2018), “Statement for the record: Worldwide threat assessment of the US intelligence community,” Office of the Director of National Intelligence, https://www.dni.gov/files/documents/Newsroom/Testimonies/2018-ATA—Unclassified-SSCI.pdf.

90In 2015, Lloyd’s of London developed: Simon Ruffle et al. (6 Jul 2015), “Business blackout: The insurance implications of a cyber attack on the U.S. power grid,” Lloyd’s Cambridge Centre for Risk Studies, https://www.lloyds.com/news-and-insight/risk-insight/library/society-and-security/business-blackout.

90Someone with a gun can do more damage: Stephen Paddock is an example of this. Alex Horton (3 Oct 2017), “The Las Vegas shooter modified a dozen rifles to shoot like automatic weapons,” Washington Post, https://www.washingtonpost.com/news/checkpoint/wp/2017/10/02/video-from-las-vegas-suggests-automatic-gunfire-heres-what-makes-machine-guns-different.

91That gun-carrying drone will become: ReprapAlgarve (23 Sep 2016), “DIY 3D printed assassination drone,” YouTube, https://www.youtube.com/watch?v=N3mdUjT6C5w.

91Liberal democracies are more vulnerable: Jack Goldsmith and Stuart Russell (forthcoming), “Strengths Become Vulnerabilities: How a Digital World Disadvantages the United States in Its International Relations,” Aegis Series Paper, Hoover Working Group on National Security, Technology, and Law.

92“Our economy is more digitalized”: Barack Obama (16 Dec 2016), “Press conference by the president,” White House Office of the Press Secretary, https://obamawhitehouse.archives.gov/the-press-office/2016/12/16/press-conference-president.

92This asymmetry makes deterrence more difficult: Joseph Nye has written extensively about deterrence in cyberspace. Joseph S. Nye Jr. (1 Feb 2017), “Deterrence and dissuasion in cyberspace,” International Security 41, no. 3, https://www.mitpressjournals.org/doi/pdf/10.1162/ISEC_a_00266.

92The technologies that we most feared: Rochelle F. H. Bohaty (12 Jan 2008), “Dangerously vulnerable,” Chemical & Engineering News, http://pubs.acs.org/cen/email/html/cen_87_i02_8702gov2.html.

92Cyberweapons have been invoked: Also, biological attacks and cyberattacks are both much harder to attribute than the others, making them even scarier.

93Electromagnetic pulse weapons are: Peter Vincent Pry (8 May 2014), “Electromagnetic pulse: Threat to critical infrastructure,” Testimony before the Subcommittee on Cybersecurity, Infrastructure Protection and Security Technologies, House Committee on Homeland Security, http://docs.house.gov/meetings/HM/HM08/20140508/102200/HHRG-113-HM08-Wstate-PryP-20140508.pdf. William R. Graham and Peter Vincent Pry (12 Oct 2017), “North Korea nuclear EMP attack: An existential threat,” U.S. House of Representatives Committee on Homeland Security, Subcommittee on Oversight and Management Efficiency Hearing, http://docs.house.gov/meetings/HM/HM09/20171012/106467/HHRG-115-HM09-Wstate-PryP-20171012.pdf.

93I’m sure that future technological developments: The term “weapon of mass destruction” is now being used for pretty much everything. The FBI referred to the Boston Marathon bombers’ pressure-cooker bombs as weapons of mass destruction. Federal Bureau of Investigation (accessed 24 Apr 2018), “Weapons of mass destruction,” http://www.fbi.gov/about-us/investigate/terrorism/wmd/wmd_faqs. Brian Palmer (31 Mar 2010), “When did IEDs become WMD?” Slate, http://www.slate.com/articles/news_and_politics/explainer/2010/03/when_did_ieds_become_wmd.html.

93“Terrorists—to include the Islamic State”: Daniel R. Coats (11 May 2017), “Statement for the record: Worldwide threat assessment of the US intelligence community: Senate Select Committee on Intelligence,” Office of the Director of National Intelligence, https://www.dni.gov/files/documents/Newsroom/Testimonies/SSCI%20Unclassified%20SFR%20-%20Final.pdf.

93“If there was even a 1 percent chance”: Ron Suskind (2006), The One Percent Doctrine: Deep inside America’s Pursuit of Its Enemies since 9/11, Simon & Schuster, https://www.amazon.com/dp/B000NY12N2/ref=dp-kindle-redirect?_encoding=UTF8&btkr=1.

94I have long thought: James Barron (15 Aug 2003), “The blackout of 2003,” New York Times, http://www.nytimes.com/2003/08/15/nyregion/blackout-2003-overview-power-surge-blacks-northeast-hitting-cities-8-states.html.

94It wasn’t deliberate by any stretch: US-CERT National Cyber Awareness System (Dec 2003), “2003 CERT Advisories,” Carnegie Mellon Software Engineering Institute, https://www.cert.org/historical/advisories/CA-2003-20.cfm.

94The official report on the blackout: Paul F. Barber et al. (13 Jul 2004), “Technical analysis of the August 13, 2003 blackout,” North American Electric Reliability Council, http://www.nerc.com/docs/docs/blackout/NERC_Final_Blackout_Report_07_13_04.pdf. U.S.-Canada Power System Outage Task Force (1 Apr 2004), “Final report on the August 14, 2003 blackout in the United States and Canada: Causes and recommendations,” https://energy.gov/sites/prod/files/oeprod/DocumentsandMedia/BlackoutFinal-Web.pdf.

94Similarly, the authors of the Mirai botnet: Brian Krebs (18 Jan 2017), “Who is Anna-Senpai, the Mirai worm author?” Krebs on Security, https://krebsonsecurity.com/2017/01/who-is-anna-senpai-the-mirai-worm-author.

94In fact, three college students wrote: Garrett M. Graff (13 Dec 2017), “How a dorm room Minecraft scam brought down the Internet,” Wired, https://www.wired.com/story/mirai-botnet-minecraft-scam-brought-down-the-internet.

94But it erased all data on over 30,000 hard drives: Parmy Olson (9 Nov 2012), “The day a computer virus came close to plugging Gulf Oil,” Forbes, https://www.forbes.com/sites/parmyolson/2012/11/09/the-day-a-computer-virus-came-close-to-plugging-gulf-oil.

94The shipping giant Maersk was hit: Iain Thomson (16 Aug 2017), “NotPetya ransomware attack cost us $300m—shipping giant Maersk,” Register, https://www.theregister.co.uk/2017/08/16/notpetya_ransomware_attack_cost_us_300m_says_shipping_giant_maersk.

95To this we can add mass murder: Elton Hobson (24 Nov 2017), “Powerful video warns of the danger of autonomous ‘slaughterbot’ drone swarms,” Global News, https://globalnews.ca/news/3880186/powerful-video-warns-of-the-danger-of-autonomous-slaughterbot-drone-swarms.

95malicious code received from space aliens: Michael Hippke and John G. Learned (6 Feb 2018), “Interstellar communication. IX. Message decontamination is possible,” ArXiv 1802.02180v1, https://arxiv.org/pdf/1802.02180.pdf.

95all the things we haven’t thought of yet: I’ve heard the term “BRINE” used as an acronym to refer to “biology, robotics, information, nanotechnology, and energy.” James Kadtke and Linton Wells II (4 Sep 2014), “Policy challenges of accelerating technological change: Security policy and strategy implications of parallel scientific revolutions,” Center for Technology and National Security Policy, National Defense University, http://ctnsp.dodlive.mil/files/2014/09/DTP106.pdf.

95They put less money into: Bruce Russett et al. (Dec 1994), “Did Americans’ expectations of nuclear war reduce their savings?” International Studies Quarterly 38, http://www.jstor.org/discover/10.2307/2600866?uid=3739256&uid=2&uid=4&sid=21103807505461.

95Some people decided not to have children: William R. Beardslee (Mar–Apr 1983), “Adolescents and the threat of nuclear war: The evolution of a perspective,” Yale Journal of Biology and Medicine 56, http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2589708/pdf/yjbm00104-0020.pdf.

95Over the years, there were plenty: Union of Concerned Scientists (20 Apr 2015), “Close calls with nuclear weapons,” http://www.ucsusa.org/sites/default/files/attach/2015/04/Close%20Calls%20with%20Nuclear%20Weapons.pdf. Future of Life Institute (1 Feb 2016), “Accidental nuclear war: A timeline,” https://futureoflife.org/background/nuclear-close-calls-a-timeline.

95The Cuban Missile Crisis is probably: Benjamin Schwarz (1 Jan 2013), “The real Cuban missile crisis,” Atlantic, https://www.theatlantic.com/magazine/archive/2013/01/the-real-cuban-missile-crisis/309190.

95although the 1983 false alarm is a close second: Sewell Chan (18 Sep 2017), “Stanislav Petrov, Soviet officer who helped avert nuclear war,” New York Times, https://www.nytimes.com/2017/09/18/world/europe/stanislav-petrov-nuclear-war-dead.html.

95although much less damaging than: Laura Geggel (9 Feb 2016), “The odds of dying,” Live Science, https://www.livescience.com/3780-odds-dying.html.

95But instead of regarding it as: As amazing as it seems today, immediately after 9/11 people actually believed that terrorist attacks of that magnitude would happen every few months. Pew Research Center (Apr 2013), “Apr 18–21 2013, omnibus, final topline, N=1,002,” Pew Research Center, http://www.people-press.org/files/legacy-questionnaires/4-23-13%20topline%20for%20release.pdf.

95the Boston Marathon bombings: Adam Gabbatt (23 Apr 2013), “Boston Marathon bombing injury toll rises to 264,” Guardian, http://www.theguardian.com/world/2013/apr/23/boston-marathon-injured-toll-rise.

95Bathtubs, home appliances, and deer: National Safety Council (accessed 24 Apr 2018), “What are the odds of dying from . . . ,” http://www.nsc.org/learn/safety-knowledge/Pages/injury-facts-chart.aspx (text, chart), http://injuryfacts.nsc.org/all-injuries/preventable-death-overview/odds-of-dying (graphic). Kevin Gipson and Adam Suchy (Sep 2011), “Instability of televisions, furniture, and appliances: Estimated and reported fatalities, 2011 report,” Consumer Product Safety Commission, https://web.archive.org/web/20111007090947/http://www.cpsc.gov/library/foia/foia11/os/tipover2011.pdf.

95But while we seem to be coming out of: John Mueller and Mark G. Stewart (1 Jul 2012), “The terrorism delusion: America’s overwrought response to September 11,” International Security 37, no. 1, https://politicalscience.osu.edu/faculty/jmueller/absisfin.pdf.

95In general, people are very bad at: Daniel Gilbert (2 Jul 2006), “If only gay sex caused global warming,” Los Angeles Times, http://articles.latimes.com/2006/jul/02/opinion/op-gilbert2. Bruce Schneier (13 Jun 2008), “The psychology of security,” AfricaCrypt 2008, https://www.schneier.com/academic/archives/2008/01/the_psychology_of_se.html.

96I coined the term in 2005: Bruce Schneier (9 Sep 2006), “Terrorists don’t do movie plots,” Wired, http://www.wired.com/2005/09/terrorists-dont-do-movie-plots.

96One: we are a species of storytellers: Bruce Schneier (31 Jul 2012), “Drawing the wrong lessons from horrific events,” CNN, http://www.cnn.com/2012/07/31/opinion/schneier-aurora-aftermath/index.html.

96And two: it makes no sense: Bruce Schneier (Nov 2009), “Beyond security theater,” New Internationalist, https://www.schneier.com/essays/archives/2009/11/beyond_security_thea.html.

Part II: The Solutions

100Today, spam still constitutes: Statista (Oct 2017), “Global spam volume as percentage of total e-mail traffic from January 2014 to September 2017, by month,” https://www.statista.com/statistics/420391/spam-email-traffic-share.

100but 99.99% of it is blocked: Jordan Robertson (19 Jan 2016), “E-mail spam goes artisanal,” Bloomberg, https://www.bloomberg.com/news/articles/2016-01-19/e-mail-spam-goes-artisanal.

100The EU’s Payment Services Directives: Steven J. Murdoch (3 Oct 2017), “Liability for push payment fraud pushed onto the victims,” Bentham’s Gaze, https://www.benthamsgaze.org/2017/10/03/liability-for-push-payment-fraud-pushed-onto-the-victims. Steven J. Murdoch and Ross Anderson (9 Nov 2014), “Security protocols and evidence: Where many payment systems fail,” FC 2014: International Conference on Financial Cryptography and Data Security, https://link.springer.com/chapter/10.1007/978-3-662-45472-5_2.

100Amazingly, the UK may make this: Patrick Jenkins and Sam Jones (25 May 2016), “Bank customers may cover cost of fraud under new UK proposals,” Financial Times, https://www.ft.com/content/e335211c-2105-11e6-aa98-db1e01fabc0c.

100And similarly, in the US: Federal Trade Commission (Aug 2012), “Lost or stolen credit, ATM, and debit cards,” https://www.consumer.ftc.gov/articles/0213-lost-or-stolen-credit-atm-and-debit-cards.

101“security is a tax on the honest”: Bruce Schneier (2012), Liars and Outliers: Enabling the Trust That Society Needs to Thrive, Wiley, http://www.wiley.com/WileyCDA/WileyTitle/productCd-1118143302.html.

101“guard labor”: Arjun Jayadev and Samuel Bowles (Apr 2006), “Guard labor,” Journal of Development Economics 79, no. 2, http://www.sciencedirect.com/science/article/pii/S0304387806000125.

101The tech analyst firm Gartner estimates: Gartner (16 Aug 2017), “Gartner says worldwide information security spending will grow 7 percent to reach $86.4 billion in 2017,” https://www.gartner.com/newsroom/id/3784965.

101If we want more security: Allison Gatlin (8 Feb 2016), “Cisco, IBM, Dell M&A brawl may whack Symantec, Palo Alto, Fortinet,” Investor’s Business Daily, https://www.investors.com/news/technology/cisco-ibm-dell-ma-brawl-whacks-symantec-palo-alto-fortinet.

102A 2017 Ponemon Institute report concluded: Ponemon Institute (20 Jun 2017) “2017 cost of data breach study,” http://info.resilientsystems.com/hubfs/IBM_Resilient_Branded_Content/White_Papers/2017_Global_CODB_Report_Final.pdf.

102A Symantec report estimated: Symantec Corporation (23 Jan 2018), “2017 Norton cyber security insights report: Global results,” https://www.symantec.com/content/dam/symantec/docs/about/2017-ncsir-global-results-en.pdf.

103“We found that resulting values are”: I was a member of the steering committee for this research project. Paul Dreyer et al. (14 Jan 2018), “Estimating the global cost of cyber risk,” RAND Corporation, https://www.rand.org/pubs/research_reports/RR2299.html.

6. What a Secure Internet+ Looks Like

105“disconcerting lack of regard”: Finn Lützow-Holm Myrstad (1 Dec 2016), “#Toyfail: An analysis of consumer and privacy issues in three internet-connected toys,” Forbrukerrådet, https://consumermediallc.files.wordpress.com/2016/12/toyfail_report_desember2016.pdf.

106Germany banned My Friend Cayla: Philip Oltermann (17 Feb 2017), “German parents told to destroy doll that can spy on children,” Guardian, https://www.theguardian.com/world/2017/feb/17/german-parents-told-to-destroy-my-friend-cayla-doll-spy-on-children.

106Mattel’s Hello Barbie had: Samuel Gibbs (26 Nov 2015), “Hackers can hijack Wi-Fi Hello Barbie to spy on your children,” Guardian, https://www.theguardian.com/technology/2015/nov/26/hackers-can-hijack-wi-fi-hello-barbie-to-spy-on-your-children.

106In 2017, the consumer credit-reporting agency Equifax: Tara Siegel Bernard et al. (7 Sep 2017), “Equifax says cyberattack may have affected 143 million in the U.S.,” New York Times, https://www.nytimes.com/2017/09/07/business/equifax-cyberattack.html. Stacy Cowley (2 Oct 2017), “2.5 million more people potentially exposed in Equifax breach,” New York Times, https://www.nytimes.com/2017/10/02/business/equifax-breach.html.

106The attackers used a critical vulnerability: Lukasz Lenart (9 Mar 2017), “S2-045: Possible remote code execution when performing file upload based on Jakarta Multipart parser,” Apache Struts 2 Documentation, https://cwiki.apache.org/confluence/display/WW/S2-045. Dan Goodin (9 Mar 2017), “Critical vulnerability under ‘massive’ attack imperils high-impact sites,” Ars Technica, https://arstechnica.com/information-technology/2017/03/critical-vulnerability-under-massive-attack-imperils-high-impact-sites.

106Equifax had been notified by Apache: Dan Goodin (2 Oct 2017), “A series of delays and major errors led to massive Equifax breach,” Ars Technica, https://arstechnica.com/information-technology/2017/10/a-series-of-delays-and-major-errors-led-to-massive-equifax-breach.

106but didn’t get around to installing: Cyrus Farivar (15 Sep 2017), “Equifax CIO, CSO ‘retire’ in wake of huge security breach,” Ars Technica, https://arstechnica.com/tech-policy/2017/09/equifax-cio-cso-retire-in-wake-of-huge-security-breach.

106The company’s insecurity was incredible: James Scott (20 Sep 2017), “Equifax: America’s in-credible insecurity,” Institute for Critical Infrastructure Technology, http://icitech.org/wp-content/uploads/2017/09/ICIT-Analysis-Equifax-Americas-In-Credible-Insecurity-Part-One.pdf.

106“laughably bad”: Bruce Schneier (1 Nov 2017), “Testimony and statement for the record: Hearing on ‘securing consumers’ credit data in the age of digital commerce’ before the Subcommittee on Digital Commerce and Consumer Protection Committee on Energy and Commerce, United States House of Representatives,” http://docs.house.gov/meetings/IF/IF17/20171101/106567/HHRG-115-IF17-Wstate-SchneierB-20171101.pdf.

106Equifax had a history of security failures: Thomas Fox-Brewster (8 Sep 2017), “A brief history of Equifax security fails,” Forbes, https://www.forbes.com/sites/thomasbrewster/2017/09/08/equifax-data-breach-history.

106“security by design”: Here’s one example of what that means: Open Web Application Security Project (last modified 3 Aug 2016), “Security by design principles,” https://www.owasp.org/index.php/Security_by_Design_Principles.

109Those principles, and some of the items: Jonathan Zittrain et al. (Feb 2018), “‘Don’t Panic’ Meets the Internet of Things: Recommendations for a Responsible Future,” Berklett Cybersecurity Project, Berkman Center for Internet and Society at Harvard University, unpublished draft.

109While researching for this book: Bruce Schneier (9 Feb 2017), “Security and privacy guidelines for the Internet of Things,” Schneier on Security, https://www.schneier.com/blog/archives/2017/02/security_and_pr.html.

110anonymizing data is much harder: Latanya Sweeney has done some amazing work reidentifying anonymized data. Here are some examples: Latanya Sweeney (accessed 24 Apr 2018), “Research accomplishments of Latanya Sweeney, Ph.D.: Policy and law: Identifiability of de-identified data,” http://latanyasweeney.org/work/identifiability.html.

110Much of this data will be in the cloud: This is not uniformly believed. For example: Debra Littlejohn Shinder (27 Jul 2016), “From mainframe to cloud: It’s technology déjà vu all over again,” TechTalk, https://techtalk.gfi.com/from-mainframe-to-cloud-its-technology-deja-vu-all-over-again.

111At a high level, we expect accuracy: Software and Information Industry Association (15 Sep 2017), “Principles for ethical data use,” SIAA Issue Brief, http://www.siia.net/Portals/0/pdf/Policy/Principles%20for%20Ethical%20Data%20Use%20SIIA%20Issue%20Brief.pdf?ver=2017-09-15-130746-523. Erica Kochi et al. (12 Mar 2018), “How to prevent discriminatory outcomes in machine learning,” Global Future Council on Human Rights 2016–2018, World Economic Forum, http://www3.weforum.org/docs/WEF_40065_White_Paper_How_to_Prevent_Discriminatory_Outcomes_in_Machine_Learning.pdf.

111Some machine-learning algorithms have: Will Knight (11 Apr 2017), “The dark secret at the heart of AI,” MIT Technology Review, https://www.technologyreview.com/s/604087/the-dark-secret-at-the-heart-of-ai.

112Think of them as black boxes: For more on secret algorithms, I recommend this book: Frank Pasquale (2015), The Black Box Society: The Secret Algorithms That Control Money and Information, Harvard University Press, http://www.hup.harvard.edu/catalog.php?isbn=9780674368279.

112Even if an algorithm can’t be made public: Larry Hardesty (27 Oct 2016), “Making computers explain themselves,” MIT News, http://news.mit.edu/2016/making-computers-explain-themselves-machine-learning-1028. Sara Castellanos and Steven Norton (10 Aug 2017), “Inside DARPA’s push to make artificial intelligence explain itself,” Wall Street Journal, https://s.wsj.com/cio/2017/08/10/inside-darpas-push-to-make-artificial-intelligence-explain-itself. Matthew Hutson (31 May 2017), “Q&A: Should artificial intelligence be legally required to explain itself?” Science, http://www.sciencemag.org/news/2017/05/qa-should-artificial-intelligence-be-legally-required-explain-itself.

112That is, we can demand: The EU’s General Data Protection Regulation includes some form of a “right to an explanation.” Experts are still arguing about how extensive that right is. Bryce Goodman and Seth Flaxman (28 Jun 2016), “European Union regulations on algorithmic decision-making and a ‘right to explanation,’” 2016 ICML Workshop on Human Interpretability in Machine Learning, https://arxiv.org/abs/1606.08813. Sandra Wachter, Brent Mittelstadt, and Luciano Floridi (24 Jan 2017), “Why a right to explanation of automated decision-making does not exist in the General Data Protection Regulation,” International Data Privacy Law 2017, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2903469.

112Because of the way machine learning works: Will Knight (11 Apr 2017), “The dark secret at the heart of AI,” MIT Technology Review, https://www.technologyreview.com/s/604087/the-dark-secret-at-the-heart-of-ai.

112and requiring them often reduces the accuracy: Cliff Kuang (21 Nov 2017), “Can A.I. be taught to explain itself?” New York Times Magazine, https://www.nytimes.com/2017/11/21/magazine/can-ai-be-taught-to-explain-itself.html.

112So maybe what we really want: Nicholas Diakopoulos et al. (17 Nov 2016), “Principles for accountable algorithms and a social impact statement for algorithms,” Fairness, Accountability, and Transparency in Machine Learning, https://www.fatml.org/resources/principles-for-accountable-algorithms.

112Or contestability: Tad Hirsch (9 Sep 2017), “Designing contestability: Interaction design, machine learning, and mental health,” 2017 Conference on Designing Interactive Systems, https://dl.acm.org/citation.cfm?doid=3064663.3064703.

112Maybe all we need is auditability: Christian Sandvig et al. (22 May 2014), “Auditing algorithms: Research methods for detecting discrimination on Internet platforms,” 64th Annual Meeting of the International Communication Association, http://www-personal.umich.edu/~csandvig/research/Auditing%20Algorithms%20—%20Sandvig%20—%20ICA%202014%20Data%20and%20Discrimination%20Preconference.pdf. Philip Adler et al. (23 Feb 2016), “Auditing black-box models for indirect influence,” 2016 IEEE 16th International Conference on Data Mining (ICDM), http://ieeexplore.ieee.org/document/7837824.

112After all, what we want to know: Julia Angwin et al. (23 May 2016), “Machine bias,” ProPublica, https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing.

113A 2011 report calculated that: Melissa E. Hathaway and John E. Savage (9 Mar 2012), “Stewardship of cyberspace: Duties for internet service providers,” CyberDialogue 2012, University of Toronto, https://www.belfercenter.org/sites/default/files/legacy/files/cyberdialogue2012_hathaway-savage.pdf.

113This centralization might be bad: Many of the suggestions in this chapter are taken from this report: Melissa E. Hathaway and John E. Savage (9 Mar 2012), “Stewardship of cyberspace: Duties for internet service providers,” CyberDialogue 2012, University of Toronto, https://www.belfercenter.org/sites/default/files/legacy/files/cyberdialogue2012_hathaway-savage.pdf.

113Some ISPs are already blocking: Linda Rosencrance (10 Jun 2008), “3 top ISPs to block access to sources of child porn,” Computerworld, https://www.computerworld.com/article/2535175/networking/3-top-isps-to-block-access-to-sources-of-child-porn.html.

113Certainly, ISPs are in the best position: Engineers are working on a security system where routers can query a centralized database and learn where an IoT device needs to connect and what information it’s allowed to send and receive. It’s called Manufacturer Usage Descriptions. The router can restrict the device’s connectivity to just that, greatly improving security. I’m not claiming that this is the correct way to implement security, but it’s an idea that needs further examination. Eliot Lear, Ralph Droms, and Dan Romascanu (24 Oct 2017), “Manufacturer Usage Description specification,” Internet Engineering Task Force, https://datatracker.ietf.org/doc/draft-ietf-opsawg-mud. Max Pritikin et al. (30 Oct 2017), “Bootstrapping remote secure key infrastructures (BRSKI),” Internet Engineering Task Force, https://datatracker.ietf.org/doc/draft-ietf-anima-bootstrapping-keyinfra.

114This list draws from a paper: Many of the suggestions in this chapter are taken from this report. Melissa E. Hathaway and John E. Savage (9 Mar 2012), “Stewardship of cyberspace: Duties for internet service providers,” CyberDialogue 2012, University of Toronto, https://www.belfercenter.org/sites/default/files/legacy/files/cyberdialogue2012_hathaway-savage.pdf.

114at the time I called it “catastrophic”: Bruce Schneier (9 Apr 2014), “Heartbleed,” Schneier on Security, https://www.schneier.com/blog/archives/2014/04/heartbleed.html.

11417% of the Internet’s web servers: Paul Mutton (8 Apr 2014), “Half a million widely trusted websites vulnerable to Heartbleed bug,” Netcraft, https://news.netcraft.com/archives/2014/04/08/half-a-million-widely-trusted-websites-vulnerable-to-heartbleed-bug.html.

115The result was that the vulnerability: Ben Grubb (11 Apr 2014), “Man who introduced serious ‘Heartbleed’ security flaw denies he inserted it deliberately,” Sydney Morning Herald, http://www.smh.com.au/it-pro/security-it/man-who-introduced-serious-heartbleed-security-flaw-denies-he-inserted-it-deliberately-20140410-zqta1.html. Alex Hern (11 Apr 2014), “Heartbleed: Developer who introduced the error regrets ‘oversight,’” Guardian, https://www.theguardian.com/technology/2014/apr/11/heartbleed-developer-error-regrets-oversight.

115In response to Heartbleed: Steven J. Vaughan-Nichols (28 Apr 2014), “Cash, the Core Infrastructure Initiative, and open source projects,” ZDNet, http://www.zdnet.com/article/cash-the-core-infrastructure-initiative-and-open-source-projects.

115That was okay when the Internet was: Alex McKenzie (5 Dec 2009), “Early sketch of ARPANET’s first four nodes,” Scientific American, https://www.scientificamerican.com/gallery/early-sketch-of-arpanets-first-four-nodes.

115These are companies you have likely never heard of: Yudhanjaya Wijeratne (28 Jun 2016), “The seven companies that really own the Internet,” Icarus Wept, http://icaruswept.com/2016/06/28/who-owns-the-internet.

116They gained access to the pipeline’s control system: Dan Goodin (10 Dec 2014), “Hack said to cause fiery pipeline blast could rewrite history of cyberwar,” Ars Technica, https://arstechnica.com/information-technology/2014/12/hack-said-to-cause-fiery-pipeline-blast-could-rewrite-history-of-cyberwar.

116In 2013, we learned that the NSA had hacked: Simon Romero (9 Sep 2013), “N.S.A. spied on Brazilian oil company, report says,” New York Times, http://www.nytimes.com/2013/09/09/world/americas/nsa-spied-on-brazilian-oil-company-report-says.html.

116In 2017, someone was able to spoof: David Hambling (10 Aug 2017), “Ships fooled in GPS spoofing attack suggest Russian cyberweapon,” New Scientist, https://www.newscientist.com/article/2143499-ships-fooled-in-gps-spoofing-attack-suggest-russian-cyberweapon.

116In the US, a series of documents: Office of Homeland Security (15 Jul 2002), “National strategy for homeland security,” https://www.hsdl.org/?view&did=856. George W. Bush (5 Feb 2003), “The national strategy for the physical protection of critical infrastructures and key assets,” Office of the President of the United States, https://www.hsdl.org/?abstract&did=1041. Homeland Security Council (5 Oct 2007), “National strategy for homeland security,” https://www.dhs.gov/xlibrary/assets/nat_strat_homelandsecurity_2007.pdf. George W. Bush (28 Feb 2003), “Directive on management of domestic incidents,” Office of the Federal Register, https://www.hsdl.org/?view&did=439105. George W. Bush (17 Dec 2003), “Directive on national preparedness,” Office of the Federal Register, https://www.hsdl.org/?view&did=441951.

11616 “critical infrastructure sectors”: Barack Obama (12 Feb 2013), “Directive on critical infrastructure security and resilience,” White House Office, https://www.hsdl.org/?view&did=731087.

117“national security, energy and power”: Donald J. Trump (Dec 2017), “National security strategy of the United States of America,” https://www.whitehouse.gov/wp-content/uploads/2017/12/NSS-Final-12-18-2017-0905.pdf.

117Some people add election systems: Lawrence Norden and Christopher Famighetti (15 Sep 2015), “America’s voting machines at risk,” Brennan Center for Justice, New York University School of Law, https://www.brennancenter.org/publication/americas-voting-machines-risk.

117That statistic comes from a 2002 document: Office of Homeland Security (15 Jul 2002), “National strategy for homeland security,” https://www.hsdl.org/?view&did=856.

117and seems to be a rough guess: One document I found said that only 8% of all utilities are privately owned, but that they generate 75% of the nation’s power. Christopher Bellavita (16 Mar 2009), “85% of what you know about homeland security is probably wrong,” Homeland Security Watch, http://www.hlswatch.com/2009/03/16/85-percent-is-wrong.

117Certainly, it depends on which industry: Midwest Publishing Company (accessed 24 Apr 2018), “Electric utility industry overview,” http://www.midwestpub.com/electricutility_overview.php.

118That we need to secure: Here’s one report: President’s National Infrastructure Advisory Council (14 Aug 2017), “Securing cyber assets: Addressing urgent cyber threats to critical infrastructure,” https://www.dhs.gov/sites/default/files/publications/niac-cyber-study-draft-report-08-15-17-508.pdf.

118“Collect it all”: Glenn Greenwald (15 Jul 2013), “The crux of the NSA story in one phrase: ‘Collect it all,’” Guardian, https://www.theguardian.com/commentisfree/2013/jul/15/crux-nsa-collect-it-all.

119“end-to-end principle”: Jerome H. Saltzer, David P. Reed, and David D. Clark (1 Nov 1984), “End-to-end arguments in system design,” ACM Transactions on Computer Systems 2, no. 4, http://web.mit.edu/Saltzer/www/publications/endtoend/endtoend.pdf.

119And by the way: Tim Wu (6 Dec 2017), “How the FCC’s net neutrality plan breaks with 50 years of history,” Wired, https://www.wired.com/story/how-the-fccs-net-neutrality-plan-breaks-with-50-years-of-history.

7. How We Can Secure the Internet+

121There are four basic ways: ISO 27001 is a good example. International Organization for Standardization (accessed 24 Apr 2018), “ISO/IEC 27000 family: Information security management systems,” http://www.iso.org/iso/home/standards/management-standards/iso27001.htm.

122There’s a distinction in law: Pierre J. Schlag (Dec 1985), “Rules and standards,” UCLA Law Review 33, https://lawweb.colorado.edu/profiles/pubpdfs/schlag/schlagUCLALR.pdf. Julia Black (28 Mar 2007), “Principles based regulation: Risks, challenges and opportunities,” University of Sydney, http://eprints.lse.ac.uk/62814/1/__lse.ac.uk_storage_LIBRARY_Secondary_libfile_shared_repository_Content_Black,%20J_Principles%20based%20regulation_Black_Principles%20based%20regulation_2015.pdf.

122It’s called “outcomes-based regulation”: Cary Coglianese (2016), “Performance-based regulation: Concepts and challenges,” in Francesca Bignami and David Zaring, eds., Comparative Law and Regulation: Understanding the Global Regulatory Process, Edward Elgar Publishing, http://onlinepubs.trb.org/onlinepubs/PBRLit/Coglianese3.pdf.

123Think of the difference between: The 1999 Gramm-Leach-Bliley regulations on financial institutions are a good example. The rules don’t specify what to do. Instead, they specify how to approach the problem and insist that affected institutions establish reasonable safeguards. The result is that those institutions have flexibility in complying, and the regulatory agencies have flexibility in enforcement. The downside is that “reasonable” is often interpreted as “everyone else is doing it,” which results in a herd mentality that can be hard to change. Lorrie Faith Cranor et al. (11 Jun 2013), “Are they actually any different? Comparing thousands of financial institutions’ privacy practices,” Twelfth Workshop on the Economics of Information Security (WEIS 2013), https://www.blaseur.com/papers/financial-final.pdf.

123“Framework for Improving Critical Infrastructure Cybersecurity”: National Institute of Standards and Technology (revised 5 Dec 2017), “Framework for improving critical infrastructure cybersecurity, version 1.1 draft 2,” https://www.nist.gov/sites/default/files/documents/2017/12/05/draft-2_framework-v1-1_without-markup.pdf.

123Unfortunately, the NIST Cybersecurity Framework: Donald J. Trump (11 May 2017), “Presidential executive order on strengthening the cybersecurity of federal networks and critical infrastructure,” Office of the President of the United States, https://www.whitehouse.gov/presidential-actions/presidential-executive-order-strengthening-cybersecurity-federal-networks-critical-infrastructure.

123It also uses a NIST standard: Christina McGhee (21 May 2014), “DoD turns to FedRAMP and cloud brokering,” FCW, https://fcw.com/articles/2014/05/21/drill-down-dod-fedramp-and-cloud-brokering.aspx.

124Equifax’s CEO didn’t get his $5.2 million: Michael Rapaport and Theo Francis (26 Sep 2017), “Equifax says departing CEO won’t get $5.2 million in severance pay,” Wall Street Journal, https://www.wsj.com/articles/equifax-says-departing-ceo-wont-get-5-2-million-in-severance-pay-1506449778. Maria Lamagna (26 Sep 2017), “After breach, Equifax CEO leaves with $18 million pension, and possibly more,” MarketWatch, https://www.marketwatch.com/story/equifax-ceo-leaves-with-18-million-pension-and-maybe-more-2017-09-26.

124His failed bet cost the company: Catalin Cimpanu (11 Nov 2017), “Hack cost Equifax only $87.5 million—for now,” Bleeping Computer, https://www.bleepingcomputer.com/news/business/hack-cost-equifax-only-87-5-million-for-now.

124The Deepwater Horizon disaster cost BP: Nathan Bomey (14 Jul 2016), “BP’s Deepwater Horizon costs total $62B,” USA Today, https://www.usatoday.com/story/money/2016/07/14/bp-deepwater-horizon-costs/87087056.

124We are biased towards preferring: Daniel Kahneman and Amos Tversky (Mar 1979), “Prospect theory: An analysis of decision under risk,” Econometrica 47, no. 2, https://www.princeton.edu/~kahneman/docs/Publications/prospect_theory.pdf.

125This doesn’t mean that no one ever: Bruce Schneier (Jul/Aug 2008), “How the human brain buys security,” IEEE Security & Privacy, https://www.schneier.com/essays/archives/2008/07/how_the_human_brain.html.

125Equifax learned about its 2017 hack in July: Dan Goodin (2 Oct 2017), “A series of delays and major errors led to massive Equifax breach,” Ars Technica, https://arstechnica.com/information-technology/2017/10/a-series-of-delays-and-major-errors-led-to-massive-equifax-breach.

125When Yahoo was hacked in 2014: Jamie Condliffe (15 Dec 2016), “A history of Yahoo hacks,” MIT Technology Review, https://www.technologyreview.com/s/603157/a-history-of-yahoo-hacks.

125Uber, for a year: Andy Greenberg (21 Nov 2017), “Hack brief: Uber paid off hackers to hide a 57-million user data breach,” Wired, https://www.wired.com/story/uber-paid-off-hackers-to-hide-a-57-million-user-data-breach.

125One study found that stock prices: Russell Lange and Eric W. Burger (27 Dec 2017), “Long-term market implications of data breaches, not,” Journal of Information Privacy and Security, http://www.tandfonline.com/doi/full/10.1080/15536548.2017.1394070.

126Something like 90% of the Internet’s infrastructure: Ash Carter (17 Apr 2015), “The Department of Defense cyber strategy,” US Department of Defense, https://www.defense.gov/Portals/1/features/2015/0415_cyber-strategy/Final_2015_DoD_CYBER_STRATEGY_for_web.pdf.

126Author John Greer proposes sending: John Michael Greer (2011), The Wealth of Nature: Economics as if Survival Mattered, New Society Publishers, https://books.google.com/books?id=h3-eVcJImqMC.

127Arthur Andersen was a “Big Five”: Flynn McRoberts et al. (1 Sep 2002), “The fall of Andersen,” Chicago Tribune, http://www.chicagotribune.com/news/chi-0209010315sep01-story.html.

127In 2015, Volkswagen was caught cheating: Megan Gross (3 Mar 2016), “Volkswagen details what top management knew leading up to emissions revelations,” Ars Technica, http://arstechnica.com/cars/2016/03/volkswagen-says-ceo-was-in-fact-briefed-about-emissions-issues-in-2014. Danielle Ivory and Keith Bradsher (8 Oct 2015), “Regulators investigating 2nd VW computer program on emissions,” New York Times, http://www.nytimes.com/2015/10/09/business/international/vw-diesel-emissions-scandal-congressional-hearing.html. Guilbert Gates et al. (8 Oct 2015; revised 28 Apr 2016), “Explaining Volkswagen’s emissions scandal,” New York Times, http://www.nytimes.com/interactive/2015/business/international/vw-diesel-emissions-scandal-explained.html.

127The company was hit with fines and penalties: Jan Schwartz and Victoria Bryan (29 Sep 2017), “VW’s Dieselgate bill hits $30 bln after another charge,” Reuters, https://www.reuters.com/article/legal-uk-volkswagen-emissions/vws-dieselgate-bill-hits-30-bln-after-another-charge-idUSKCN1C4271.

127Note: one VW manager and one engineer: Bill Vlasic (6 Dec 2017), “Volkswagen official gets 7-year term in diesel-emissions cheating,” New York Times, https://www.nytimes.com/2017/12/06/business/oliver-schmidt-volkswagen.html.

128Under current law in the US: Albert Bianchi Jr., Michelle L. Dama, and Adrienne S. Ehrhardt (3 Mar 2017), “Executives and board members could face liability for data breaches,” National Law Review, https://www.natlawreview.com/article/executives-and-board-members-could-face-liability-data-breaches. Joseph B. Crace Jr. (3 Apr 2017), “When does data breach liability extend to the boardroom?” Law 360, https://www.law360.com/articles/907786.

128In the UK, the CEO of TalkTalk: Matt Burgess (1 Feb 2017), “TalkTalk’s chief executive Dido Harding has resigned,” Wired, https://www.wired.co.uk/article/talktalk-dido-harding-resign-quit.

128According to Sarbanes-Oxley: Darren C. Skinner (1 Jun 2006), “Director responsibilities and liability exposure in the era of Sarbanes-Oxley,” Practical Lawyer, https://www.apks.com/en/perspectives/publications/2006/06/director-responsibilities-and-liability-exposure.

128The law’s reality might be much less: Mary Jo White and Andrew J. Ceresney (19 May 2017), “Individual accountability: Not always accomplished through enforcement,” New York Law Journal, http://www.law.com/newyorklawjournal/almID/1202786743746.

128We need to think about doing: Charles Cresson Wood (4 Dec 2016), “Solving the information security & privacy crisis by expanding the scope of top management personal liability,” Journal of Legislation 43, no. 1, http://scholarship.law.nd.edu/jleg/vol43/iss1/5.

129They were able to steal the codes: Earlence Fernandes, Jaeyeon Jung, and Atul Prakash (18 Aug 2016), “Security analysis of emerging smart home applications,” 2016 IEEE Symposium on Security and Privacy, http://ieeexplore.ieee.org/document/7546527.

129If you read SmartThings Inc.’s terms of service: SmartThings Inc. (accessed 24 Apr 2018), “Welcome to SmartThings!” https://www.smartthings.com/terms.

129These are the “terms of service”: This has been called “the biggest lie on the Internet.” Jonathan A. Obar and Anne Oeldorf-Hirsch (24 Aug 2016), “The biggest lie on the Internet: Ignoring the privacy policies and terms of service policies of social networking services,” 44th Research Conference on Communication, Information and Internet Policy 2016 (TPRC 44), https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2757465.

129Not that it would matter if you did: This right has been challenged in court, and today there are some limits to what companies can do in their terms of service. Juliet Moringiello and John Ottaviani (7 May 2016), “Online contracts: We may modify these at any time, right?” Business Law Today, https://www.americanbar.org/publications/blt/2016/05/07_moringiello.html.

129Such agreements force unhappy users: Jessica Silver-Greenberg and Robert Gebeloff (31 Oct 2015), “Arbitration everywhere, stacking the deck of justice,” New York Times, https://www.nytimes.com/2015/11/01/business/dealbook/arbitration-everywhere-stacking-the-deck-of-justice.html.

130Judges have tended to blame hackers: Jane Chong (30 Oct 2013), “We need strict laws if we want more secure software,” New Republic, https://newrepublic.com/article/115402/sad-state-software-liability-law-bad-code-part-4.

130This is why it is so hard to sue: Brenda R. Sharton and David S. Kantrowitz (22 Sep 2017), “Equifax and why it’s so hard to sue a company for losing your personal information,” Harvard Business Review, https://hbr.org/2017/09/equifax-and-why-its-so-hard-to-sue-a-company-for-losing-your-personal-information.

130But they could not prove: Janis Kestenbaum, Rebecca Engrav, and Erin Earl (6 Oct 2017), “4 takeaways from FTC v. D-Link Systems,” Law 360, https://www.law360.com/cybersecurity-privacy/articles/971473.

130The FTC found that LabMD had not: Federal Trade Commission (29 Jul 2016), “In the matter of LabMD, Inc., a corporation: Opinion of the commission,” Docket No. 9357, https://www.ftc.gov/system/files/documents/cases/160729labmd-opinion.pdf.

131The signs are that the court: Craig A. Newman (18 Dec 2017), “LabMD appeal has privacy world waiting,” Lexology, https://www.lexology.com/library/detail.aspx?g=129a4ea7-cc38-4976-94af-3f09e8e280d0.

131The hotel chains’ 2014 class-action lawsuit: Andy Greenberg (15 May 2013), “Hotel lock hack still being used in burglaries months after lock firm’s fix,” Forbes, https://www.forbes.com/sites/andygreenberg/2013/05/15/hotel-lock-hack-still-being-used-in-burglaries-months-after-lock-firms-fix.

131“Public policy demands that responsibility”: Roger J. Traynor (5 Jul 1944), Escola v. Coca Cola Bottling Co. of Fresno, S.F. 16951, Supreme Court of California, https://repository.uchastings.edu/cgi/viewcontent.cgi?article=1150&context=traynor_opinions.

131This is the way wiretap law works: United States Code (2011), “18 U.S. Code §2520—Recovery of civil damages authorized,” in United States Code, 2006 edition, Supp. 5, Title 18—Crimes and Criminal Procedure, https://www.gpo.gov/fdsys/search/pagedetails.action?packageId=USCODE-2011-title18&granuleId=USCODE-2011-title18-partI-chap119-sec2520.

131This is also the way copyright law works: US Copyright Office (Oct 2009; accessed 24 Apr 2018), “504. Remedies for infringement: Damages and profits,” in Copyright Law of the United States (Title 17), Chapter 5: “Copyright Notice, Deposit, and Registration,” https://www.copyright.gov/title17/92chap5.html.

132When connected versions of these things: This article nicely lays out the liability arguments: Donna L. Burden and Hilarie L. Henry (1 Aug 2015), “Security software vendors battle against impending strict products liability,” Product Liability Committee Newsletter, International Association of Defense Counsel, http://www.iadclaw.org/securedocument.aspx?file=1/19/Product_Liability_August_2015.pdf.

132This happened in the 1980s: Greg Reigel et al. (13 Oct 2015), “GARA: The General Aviation Revitalization Act of 1994,” GlobalAir.com, https://.globalair.com/post/GARA-the-General-Aviation-Revitalization-Act-of-1994.aspx.

132Where there is risk of liability: Adam Janofsky (17 Sep 2017), “Insurance grows for cyberattacks,” Wall Street Journal, https://www.wsj.com/articles/insurance-grows-for-cyberattacks-1505700360.

132If we require people who purchase: Paul Christiano (17 Feb 2018), “Liability insurance,” Sideways View, https://sideways-view.com/2018/02/17/liability-insurance.

133Perhaps it would be more accurate: Paul Merrey et al. (12 Jul 2017), “Seizing the cyber insurance opportunity,” KPMG International, https://home.kpmg.com/xx/en/home/insights/2017/06/seizing-the-cyber-insurance-opportunity.html. US House of Representatives (22 Mar 2016), “The role of cyber insurance in risk management,” Hearing before the Subcommittee on Cybersecurity, Infrastructure Protection, and Security Technologies of the Committee on Homeland Security, https://www.gpo.gov/fdsys/pkg/CHRG-114hhrg22625/html/CHRG-114hhrg22625.htm.

133Insurance companies are starting to figure out: Adam Janofsky (17 Sep 2017), “Cyberinsurers look to measure risk,” Wall Street Journal, https://www.wsj.com/articles/cyberinsurers-look-to-measure-risk-1505700301.

133They’re surveillance devices by design: There are some pretty horrendous baby monitor security stories. Craig Silverman (24 Jul 2015), “7 creepy baby monitor stories that will terrify all parents,” BuzzFeed, https://www.buzzfeed.com/craigsilverman/creeps-hack-baby-monitors-and-say-terrifying-thing.

133Many brands are hackable: Carl Franzen (4 Aug 2017), “How to find a hack-proof baby monitor,” Lifehacker, https://offspring.lifehacker.com/how-to-find-a-hack-proof-baby-monitor-1797534985.

133“[Our] technology transmits a secure”: Amazon.com (accessed 24 Apr 2018), “VTech DM111 audio baby monitor with up to 1,000 ft of range, 5-level sound indicator, digitized transmission & belt clip,” https://www.amazon.com/VTech-DM111-Indicator-Digitized-Transmission/dp/B00JEV5UI8/ref=pd_lpo_vtph_75_bs_lp_t_1.

134I couldn’t tell the good from the bad: I found one security assessment of a few brands. Mark Stanislav and Tod Beardsley (29 Sep 2015), “Hacking IoT: A case study on baby monitor exposure and vulnerabilities,” Rapid7, https://www.rapid7.com/docs/Hacking-IoT-A-Case-Study-on-Baby-Monitor-Exposures-and-Vulnerabilities.pdf.

134“lemons market”: George A. Akerlof (1 Aug 1970), “The market for ‘lemons’: Quality uncertainty and the market mechanism,” Quarterly Journal of Economics 84, no. 3, https://academic.oup.com/qje/article-abstract/84/3/488/1896241.

134The result is that insecure products: Bruce Schneier (19 Apr 2007), “How security companies sucker us with lemons,” Wired, https://www.wired.com/2007/04/securitymatters-0419.

134There is nothing like this today: One study estimated it would take average consumers 244 hours/year to read all of the privacy policies they agree to. Aleecia M. McDonald and Lorrie Faith Cranor (1 Oct 2008), “The cost of reading privacy policies,” I/S: A Journal of Law and Policy for the Information Society, 2008 Privacy Year in Review issue, http://lorrie.cranor.org/pubs/readingPolicyCost-authorDraft.pdf.

135“Please be aware that if”: Samsung (accessed 24 Apr 2018), “Samsung local privacy policy—SmartTV supplement,” http://www.samsung.com/hk_en/info/privacy/smarttv.

135in the UK: Samuel Gibbs (24 Jul 2017), “Smart fridges and TVs should carry security rating, police chief says,” Guardian, https://www.theguardian.com/technology/2017/jul/24/smart-tvs-fridges-should-carry-security-rating-police-chief-says.

135the EU: Catherine Stupp (5 Oct 2016), “Commission plans cybersecurity rules for internet-connected machines,” Euractiv, http://www.euractiv.com/section/innovation-industry/news/commission-plans-cybersecurity-rules-for-internet-connected-machines. John E. Dunn (11 Oct 2016), “The EU’s latest idea to secure the Internet of Things? Sticky labels,” Naked Security, https://nakedsecurity.sophos.com/2016/10/11/the-eus-latest-idea-to-secure-the-internet-of-things-sticky-labels.

135Australia: Denham Sadler (23 Oct 2017), “Security ratings for IoT devices?” InnovationAus.com, http://www.innovationaus.com/2017/10/Security-ratings-for-IoT-devices.

136Companies could display a label: US Congress (1 Aug 2017), “S.1691—Internet of Things (IoT) Cybersecurity Improvement Act of 2017,” https://www.congress.gov/bill/115th-congress/senate-bill/1691/actions. Morgan Chalfant (27 Oct 2017), “Dems push for program to secure internet-connected devices,” Hill, http://thehill.com/policy/cybersecurity/357509-dems-push-for-program-to-secure-internet-connected-devices.

136Consumers Union—the organization behind: Consumer Reports (6 Mar 2017), “Consumer Reports launches digital standard to safeguard consumers’ security and privacy in complex marketplace,” https://www.consumerreports.org/media-room/press-releases/2017/03/consumer_reports_launches_digital_standard_to_safeguard_consumers_security_and_privacy_in_complex_marketplace.

136Who Has Your Back? project: Nate Cardozo et al. (Jul 2017), “Who Has Your Back? 2017,” Electronic Frontier Foundation, https://www.eff.org/files/2017/07/08/whohasyourback_2017.pdf.

136Ranking Digital Rights initiative: Rebecca MacKinnon et al. (March 2017), “2017 corporate accountability index,” Ranking Digital Rights, https://rankingdigitalrights.org/index2017/assets/static/download/RDRindex2017report.pdf.

137And we can do much of this: Peter “Mudge” Zatko has some interesting ideas in this area and has set up a cyber underwriters lab to test software security. Kim Zetter (29 Jul 2016), “A famed hacker is grading thousands of programs—and may revolutionize software in the process,” The Intercept, https://theintercept.com/2016/07/29/a-famed-hacker-is-grading-thousands-of-programs-and-may-revolutionize-software-in-the-process.

137In the US, 48 states have: Foley & Lardner LLP (17 Jan 2018), “State data breach notification laws,” https://www.foley.com/state-data-breach-notification-laws.

137There have been several failed attempts: Selena Larson (1 Dec 2017), “Senators introduce data breach disclosure bill,” CNN, http://money.cnn.com/2017/12/01/technology/bill-data-breach-laws/index.html.

138Not only will improved vulnerability disclosure: The results of this have been a mixed bag. For example, we know that while a data breach has short-term effects on the company, it has minimal effect on stock price after two weeks. Russell Lange and Eric W. Burger (27 Dec 2017), “Long-term market implications of data breaches, not,” Journal of Information Privacy and Security, http://www.tandfonline.com/doi/full/10.1080/15536548.2017.1394070.

138“Stop.Think.Connect” campaign: US Department of Homeland Security (accessed 24 Apr 2018), “Stop.Think.Connect,” https://www.dhs.gov/stopthinkconnect.

139Today, a lot of security advice: Bruce Schneier (Sep/Oct 2013), “Security design: Stop trying to fix the user,” IEEE Security & Privacy, https://www.schneier.com/blog/archives/2016/10/security_design.html.

140Existing organizations for software professionals: Here are some examples: IEEE (accessed 24 Apr 2018), “IEEE Computer Society Certification and Credential Program,” https://www.computer.org/web/education/certifications. Association for Computing Machinery (accessed 24 Apr 2018), “Skillsoft Learning Collections,” https://learning.acm.org/e-learning/skillsoft. (ISC)² (accessed 24 Apr 2018), “(ISC)² information security certifications,” https://www.isc2.org/Certifications.

140The International Organization for Standardization (ISO): International Organization for Standardization (accessed 24 Apr 2018), “ISO/IEC 27000 family: Information security management systems,” http://www.iso.org/iso/home/standards/management-standards/iso27001.htm.

141Various reports forecast 1.5 million: Julie Peeler and Angela Messer (17 Apr 2015), “(ISC)² study: Workforce shortfall due to hiring difficulties despite rising salaries, increased budgets and high job satisfaction rate,” (ISC)² Blog, http://.isc2.org/isc2_blog/2015/04/isc-study-workforce-shortfall-due-to-hiring-difficulties-despite-rising-salaries-increased-budgets-a.html. Jeff Kauflin (16 Mar 2017), “The fast-growing job with a huge skills gap: Cyber security,” Forbes, https://www.forbes.com/sites/jeffkauflin/2017/03/16/the-fast-growing-job-with-a-huge-skills-gap-cyber-security. ISACA (Jan 2016), “2016 cybersecurity skills gap,” https://image-store.slidesharecdn.com/be4eaf1a-eea6-4b97-b36e-b62dfc8dcbae-original.jpeg. Steve Morgan (2017), “Cybersecurity jobs report: 2017 edition,” Herjavec Group, https://www.herjavecgroup.com/wp-content/uploads/2017/06/HG-and-CV-The-Cybersecurity-Jobs-Report-2017.pdf.

141“The cybersecurity skills shortage represents”: John Oltsik (14 Nov 2017), “Research confirms the cybersecurity skills shortage is an existential threat,” CSO, https://www.csoonline.com/article/3237049/security/research-confirms-the-cybersecurity-skills-shortage-is-an-existential-threat.html.

142a cyber Manhattan Project: Mark Goodman (21 Jan 2015), “We need a Manhattan project for cyber security,” Wired, https://www.wired.com/2015/01/we-need-a-manhattan-project-for-cyber-security.

142a cyber moonshot: Accenture (2 Oct 2017), “Defining a cyber moon shot,” https://www.accenture.com/t20171004T064630Z__w__/us-en/_acnmedia/PDF-62/Accenture-Defining-Cyber-Moonshot-POV.pdf.

8. Government Is Who Enables Security

144A modern airplane has upwards of: Faye Bowers (29 Oct 1997), “Building a 747: 43 days and 3 million fasteners,” Christian Science Monitor, https://www.csmonitor.com/1997/1029/102997.us.us.2.html.

144182 times in 2017: My average speed was 27 miles per hour. That’s a calm year for me; in 2015, my average speed was 33 miles per hour.

144It wasn’t always like this: This is a good summary: Mark Hansen, Carolyn McAndrews, and Emily Berkeley (Jul 2008), “History of aviation safety oversight in the United States,” DOT/FAA/AR-08-39, National Technical Information Service, http://www.tc.faa.gov/its/worldpac/techrpt/ar0839.pdf.

144The result is that today: The taxi ride to the airport is the most dangerous part of the trip.

145Whenever industry groups write about this: Here’s one example: Coalition for Cybersecurity and Policy and Law (26 Oct 2017), “New whitepaper: Building a national cybersecurity strategy: Voluntary, flexible frameworks,” Center for Responsible Enterprise and Trade, https://create.org/news/new-whitepaper-building-national-cybersecurity-strategy.

145The Federal Aviation Administration has: April Glaser (15 Mar 2017), “Federal privacy laws won’t necessarily protect you from spying drones,” Recode, https://www.recode.net/2017/3/15/14934050/federal-privacy-laws-spying-drones-senate-hearing.

148in 2006, Netflix published 100 million: Katie Hafner (2 Oct 2006), “And if you liked the movie, a Netflix contest may reward you handsomely,” New York Times, http://www.nytimes.com/2006/10/02/technology/02netflix.html.

148Researchers were able to de-anonymize: Arvind Narayanan and Vitaly Shmatikov (18 May 2008), “Robust de-anonymization of large sparse datasets,” 2008 IEEE Symposium on Security and Privacy (SP ’08), https://dl.acm.org/citation.cfm?id=1398064.

148which surprised pretty much everyone: Paul Ohm (13 Aug 2009), “Broken promises of privacy: Responding to the surprising failure of anonymization,” UCLA Law Review 57, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1450006.

148The FTC took action against Netflix: Ryan Singel (12 Mar 2010), “Netflix cancels recommendation contest after privacy lawsuit,” Wired, https://www.wired.com/2010/03/netflix-cancels-contest.

149Today, both the FCC and the SEC: This idea is fleshed out here: Melissa E. Hathaway and John N. Stewart (25 Jul 2014), “Taking control of our cyber future,” Georgetown Journal of International Affairs, https://www.georgetownjournalofinternationalaffairs.org/online-edition/cyber-iv-feature-taking-control-of-our-cyber-future.

149A research group advising the European Commission: Eireann Leverett, Richard Clayton, and Ross Anderson (6 Jun 2017), “Standardization and certification of the ‘Internet of Things,’” Institute for Consumer Policy, https://www.conpolicy.de/en/news-detail/standardization-and-certification-of-the-internet-of-things.

149Ashkan Soltani, former chief technologist: Jedidiah Bracy (7 Apr 2016), “McSweeny, Soltani, and regulating the IoT,” International Association of Privacy Professionals, https://iapp.org/news/a/mcsweeney-soltani-and-regulating-the-iot.

149University of Washington law professor Ryan Calo: Ryan Calo (15 Sep 2014), “The case for a federal robotics commission,” Brookings Institution, https://www.brookings.edu/research/the-case-for-a-federal-robotics-commission.

149And Matthew Scherer of George Mason University: Matthew U. Scherer (Spring 2016), “Regulating artificial intelligence systems: Risks, challenges, competencies, and strategies,” Harvard Journal of Law & Technology 29, no. 2, http://jolt.law.harvard.edu/articles/pdf/v29/29HarvJLTech353.pdf.

149Israel created its National Cyber Bureau: National Cyber Bureau (2 Jun 2013), “Mission of the bureau,” Prime Minister’s Office, http://www.pmo.gov.il/English/PrimeMinistersOffice/DivisionsAndAuthorities/cyber/Pages/default.aspx.

149The UK created the National Cyber Security Centre: National Cyber Security Centre (9 Jun 2017; accessed 24 Apr 2018), “About the NCSC,” https://www.ncsc.gov.uk/information/about-ncsc.

150One: governments tend to regulate industries: Andrew Odlyzko (1 Mar 2009), “Network neutrality, search neutrality, and the never-ending conflict between efficiency and fairness in markets,” Review of Network Economics 8, no. 1, https://www.degruyter.com/view/j/rne.2009.8.issue-1/rne.2009.8.1.1169/rne.2009.8.1.1169.xml.

151The agency doesn’t conduct the testing itself: Food and Drug Administration (accessed 24 Apr 2018), “The FDA’s role in medical device cybersecurity,” https://www.fda.gov/downloads/MedicalDevices/DigitalHealth/UCM544684.pdf.

151Rules for privacy of patients’ medical data: Charles Ornstein (17 Nov 2015), “Federal privacy law lags far behind personal-health technologies,” Washington Post, https://www.washingtonpost.com/news/to-your-health/wp/2015/11/17/federal-privacy-law-lags-far-behind-personal-health-technologies.

151And sometimes the FDA fights back: Russell Brandom (25 Nov 2013), “Body blow: How 23andMe brought down the FDA’s wrath,” Verge, https://www.theverge.com/2013/11/25/5144928/how-23andme-brought-down-fda-wrath-personal-genetics-wojcicki. Gina Kolata (6 Apr 2017), “F.D.A. will allow 23andMe to sell genetic tests for disease risk to consumers,” New York Times, https://www.nytimes.com/2017/04/06/health/fda-genetic-tests-23andme.html.

151In 2015, the FTC sued Wyndham Hotels: Electronic Privacy Information Center (24 Aug 2015), “FTC v. Wyndham,” https://epic.org/amicus/ftc/wyndham.

152The Federal Court of Appeals sided with: Federal Trade Commission (9 Dec 2015), “Wyndham settles FTC charges it unfairly placed consumers’ payment card information at risk,” https://www.ftc.gov/news-events/press-releases/2015/12/wyndham-settles-ftc-charges-it-unfairly-placed-consumers-payment.

152It took 13 years for Facebook: Josh Constine (27 Jun 2017), “Facebook now has 2 billion monthly users . . . and responsibility,” TechCrunch, https://techcrunch.com/2017/06/27/facebook-2-billion-users.

153The law makes an important distinction: Eric R. Hinz (1 Nov 2012), “A distinctionless distinction: Why the RCS/ECS distinction in the Stored Communications Act does not work,” Notre Dame Law Review 88, no. 1, https://scholarship.law.nd.edu/cgi/viewcontent.cgi?referer=&httpsredir=1&article=1115&context=ndlr.

153The logic behind that old law: David Kravets (21 Oct 2011), “Aging ‘privacy’ law leaves cloud email open to cops,” Wired, https://www.wired.com/2011/10/ecpa-turns-twenty-five.

154The big tech companies are spending: Olivia Solon and Sabrina Siddiqui (3 Sep 2017), “Forget Wall Street: Silicon Valley is the new political power in Washington,” Guardian, https://www.theguardian.com/technology/2017/sep/03/silicon-valley-politics-lobbying-washington.

154Google alone spent $6 million: Jonathan Taplin (30 Jul 2017), “Why is Google spending record sums on lobbying Washington?” Guardian, https://www.theguardian.com/technology/2017/jul/30/google-silicon-valley-corporate-lobbying-washington-dc-politics.

154One is the way developers of fitness devices: Alex Ruoff (29 Jul 2016), “Fitness trackers, wellness apps won’t be regulated by FDA,” Bureau of National Affairs, https://www.bna.com/fitness-trackers-wellness-n73014445597. Food and Drug Administration, Center for Devices and Radiological Health (29 Jul 2016), “General wellness: Policy for low risk devices, guidance for industry and Food and Drug Administration staff,” Federal Register, https://www.federalregister.gov/documents/2016/07/29/2016-17902/general-wellness-policy-for-low-risk-devices-guidance-for-industry-and-food-and-drug-administration.

154Data brokers have performed similar: Brian Fung (29 Mar 2017), “What to expect now that Internet providers can collect and sell your Web browser history,” Washington Post, https://www.washingtonpost.com/news/the-switch/wp/2017/03/29/what-to-expect-now-that-internet-providers-can-collect-and-sell-your-web-browser-history.

154“Power interprets regulation as damage”: Yochai Benkler and Julie Cohen (17 Nov 2017), “Networks 2” (conference session), After the Digital Tornado Conference, Wharton School, University of Pennsylvania, http://digitaltornado.net. Supernova Group (19 Nov 2017), “After the Tornado 05: Networks 2,” YouTube, https://www.youtube.com/watch?v=pCGZ8tIrrIU.

154the CAN-SPAM Act that didn’t stop spam: It made things worse, since it superseded stronger state laws and took away individuals’ ability to bring lawsuits. Brian Krebs (2 Jul 2017), “Is it time to can the CAN-SPAM Act?” Krebs on Security, https://krebsonsecurity.com/2017/07/is-it-time-to-can-the-can-spam-act.

154legal action against robocallers: Mitchell J. Katz (13 Jan 2017), “FTC announces crackdown on two massive illegal robocall operations,” Federal Trade Commission, https://www.ftc.gov/news-events/press-releases/2017/01/ftc-announces-crackdown-two-massive-illegal-robocall-operations. Mike Snider (22 Jun 2017), “FCC hits robocaller with agency’s largest-ever fine of $120 million,” USA Today, https://www.usatoday.com/story/tech/news/2017/06/22/fcc-hits-robocaller-agencys-largest-ever-fine-120-million/103102546.

154“do not call” list violators: Mitchell J. Katz (6 Jun 2017), “FTC and DOJ case results in historic decision awarding $280 million in civil penalties against Dish Network and strong injunctive relief for Do Not Call violations,” Federal Trade Commission, https://www.ftc.gov/news-events/press-releases/2017/06/ftc-doj-case-results-historic-decision-awarding-280-million-civil.

154deceptive telco advertisers: Mitchell J. Katz (11 Mar 2015), “FTC charges DIRECTV with deceptively advertising the cost of its satellite television service,” Federal Trade Commission, https://www.ftc.gov/news-events/press-releases/2015/03/ftc-charges-directv-deceptively-advertising-cost-its-satellite.

154excessive data collection by toys: Cecilia Kang (8 Jan 2018), “Toymaker VTech settles charges of violating child privacy law,” New York Times, https://www.nytimes.com/2018/01/08/business/vtech-child-privacy.html.

154and televisions: Juliana Gruenwald Henderson (6 Feb 2017), “VIZIO to pay $2.2 million to FTC, state of New Jersey to settle charges it collected viewing histories on 11 million smart televisions without users’ consent,” Federal Trade Commission, https://www.ftc.gov/news-events/press-releases/2017/02/vizio-pay-22-million-ftc-state-new-jersey-settle-charges-it.

155This way of thinking will become: There are conflicting takes on this in the computer security field. Adam Thierer (11 Mar 2012), “Avoiding a precautionary principle for the Internet,” Forbes, https://www.forbes.com/sites/adamthierer/2012/03/11/avoiding-a-precautionary-principle-for-the-internet. Andy Stirling (8 Jul 2013), “Why the precautionary principle matters,” Guardian, https://www.theguardian.com/science/political-science/2013/jul/08/precautionary-principle-science-policy.

155We don’t want to—and can’t: Kevin Kelly has written about how to be deliberate in deciding which technologies society should use, and how to roll them out. Kevin Kelly (2010), What Technology Wants, Viking, https://books.google.com/books?id=_ToftPd4R8UC.

156International cooperation is coming: It’s starting. This arrest was made by Spanish police, with support from the FBI; Romanian, Belarusian, and Taiwanese authorities; and several cybersecurity companies. Micah Singleton (26 Mar 2018), “Europol arrests suspects in bank heists that stole $1.2 billion using malware,” Verge, https://www.theverge.com/2018/3/26/17165300/europol-arrest-suspect-bank-heists-1-2-billion-cryptocurrency-malware.

156There are other hacker havens: Noah Rayman (7 Aug 2014), “The world’s top 5 cybercrime hotspots,” Time, http://time.com/3087768/the-worlds-5-cybercrime-hotspots.

157Some states, like North Korea: Christine Kim (27 Jul 2017), “North Korea hacking increasingly focused on making money more than espionage: South Korea study,” Reuters, https://www.reuters.com/article/us-northkorea-cybercrime/north-korea-hacking-increasingly-focused-on-making-money-more-than-espionage-south-korea-study-idUSKBN1AD0BO.

157The treaty provides a framework: Council of Europe (accessed 24 Apr 2018), “Details of Treaty No. 185: Convention on Cybercrime,” https://www.coe.int/en/web/conventions/full-list/-/conventions/treaty/185.

157At the extreme, large and powerful countries: Bruce Sterling (22 Dec 2015), “Respecting Chinese and Russian cyber-sovereignty in the formerly global internet,” Wired, https://www.wired.com/beyond-the-beyond/2015/12/respecting-chinese-and-russian-cyber-sovereignty-in-the-formerly-global-internet. Andrea Limbago (13 Dec 2016), “The global push for cyber sovereignty is the beginning of cyber fascism,” Hill, http://thehill.com/s/congress-blog/technology/310382-the-global-push-for-cyber-sovereignty-is-the-beginning-of. Vladimir Mikheev (22 Mar 2017), “Why do Beijing and Moscow embrace cyber sovereignty?” Russia beyond the Headlines, https://www.rbth.com/opinion/2017/03/22/why-do-beijing-and-moscow-embrace-cyber-sovereignty_725018.

157Political scientist Joseph Nye believes: Joseph S. Nye (forthcoming), “Normative restraints on cyber conflict,” Cyber Security.

158The UN had its GGE: United Nations General Assembly (24 Jun 2013), “Report of the Group of Governmental Experts on Developments in the Field of Information and Telecommunications in the Context of International Security,” Resolution A/68/98, http://www.un.org/ga/search/view_doc.asp?symbol=A/68/98.

158These were immediately blocked: Stefan Soesanto and Fosca D’Incau (15 Aug 2017), “The UN GGE is dead: Time to fall forward,” European Council on Foreign Relations, http://www.ecfr.eu/article/commentary_time_to_fall_forward_on_cyber_governance.

158Cyberweapons are easy to hide: Ariel Rabkin (3 Mar 2015), “Cyber-arms cannot be controlled by treaties,” American Enterprise Institute, https://www.aei.org/publication/cyber-arms-cannot-be-controlled-by-treaties.

158In a 2014 report, cyber policy expert: Jason Healey (Apr 2014), “Risk nexus: Beyond data breaches: Global interconnections of cyber risk,” Atlantic Council, http://publications.atlanticcouncil.org/cyberrisks/blog/risk-nexus-september-2015-overcome-by-cyber-risks.pdf.

158In the same year, Matt Thomlinson: Matt Thomlinson (31 Jan 2014), “Microsoft announces Brussels Transparency Center at Munich Security Conference,” Microsoft on the Issues, https://s.microsoft.com/on-the-issues/2014/01/31/microsoft-announces-brussels-transparency-center-at-munich-security-conference.

158Microsoft’s president and chief legal officer: Brad Smith (14 Feb 2017), “The need for a Digital Geneva Convention,” Microsoft on the Issues, https://s.microsoft.com/on-the-issues/2017/02/14/need-digital-geneva-convention.

158Google has its own proposal: Kent Walker (31 Oct 2017), “Digital security and due process: Modernizing cross-border government access standards for the cloud era,” Google, https://.google/documents/2/CrossBorderLawEnforcementRequestsWhitePaper_2.pdf.

9. How Governments Can Prioritize Defense over Offense

160“defense dominant” strategy: Jason Healey (Jan 2017), “A nonstate strategy for saving cyberspace,” Atlantic Council Strategy Paper No. 8, Atlantic Council, http://www.atlanticcouncil.org/images/publications/AC_StrategyPapers_No8_Saving_Cyberspace_WEB.pdf.

160The NSA has two missions: John Ferris (1 Mar 2010), “Signals intelligence in war and power politics, 1914–2010,” in The Oxford Handbook of National Security Intelligence, Oxford, http://www.oxfordhandbooks.com/view/10.1093/oxfordhb/9780195375886.001.0001/oxfordhb-9780195375886-e-0010.

162to criminals on the black market: Dancho Danchev (2 Nov 2008), “Black market for zero day vulnerabilities still thriving,” ZDNet, http://www.zdnet.com/blog/security/black-market-for-zero-day-vulnerabilities-still-thriving/2108. Dan Patterson (9 Jan 2017), “Gallery: The top zero day Dark Web markets,” TechRepublic, https://www.techrepublic.com/pictures/gallery-the-top-zero-day-dark-web-markets.

162and to governments: Andy Greenberg (21 Mar 2012), “Meet the hackers who sell spies the tools to crack your PC (and get paid six-figure fees),” Forbes, http://www.forbes.com/sites/andygreenberg/2012/03/21/meet-the-hackers-who-sell-spies-the-tools-to-crack-your-pc-and-get-paid-six-figure-fees.

162Companies like Azimuth sell: Joseph Cox and Lorenzo Franceschi-Bicchierai (7 Feb 2018), “How a tiny startup became the most important hacking shop you’ve never heard of,” Vice Motherboard, https://motherboard.vice.com/en_us/article/8xdayg/iphone-zero-days-inside-azimuth-security.

162And while vendors offer bounties: Adam Segal (19 Sep 2016), “Using incentives to shape the zero-day market,” Council on Foreign Relations, https://www.cfr.org/report/using-incentives-shape-zero-day-market.

162the not-for-profit Tor Project: Tor Project (last updated 20 Sep 2017), “Policy [re Tor bug bounties],” Hacker One, Inc., https://hackerone.com/torproject.

162the cyberweapons manufacturer Zerodium: Zerodium (13 Sep 2017; expired 1 Dec 2017), “Tor browser zero-day exploits bounty (expired),” https://zerodium.com/tor.html.

163“Every offensive weapon is”: Jack Goldsmith (12 Apr 2014), “Cyber paradox: Every offensive weapon is a (potential) chink in our defense—and vice versa,” Lawfare, http://www.lawfareblog.com/2014/04/cyber-paradox-every-offensive-weapon-is-a-potential-chink-in-our-defense-and-vice-versa.

163Many people have weighed in: Joel Brenner (14 Apr 2014), “The policy tension on zero-days will not go away,” Lawfare, http://www.lawfareblog.com/2014/04/the-policy-tension-on-zero-days-will-not-go-away.

163Activist and author Cory Doctorow: Cory Doctorow (11 Mar 2014), “If GCHQ wants to improve national security it must fix our technology,” Guardian, http://www.theguardian.com/technology/2014/mar/11/gchq-national-security-technology.

163I have said similar things: Bruce Schneier (20 Feb 2014), “It’s time to break up the NSA,” CNN, http://edition.cnn.com/2014/02/20/opinion/schneier-nsa-too-big/index.html.

163Computer security expert Dan Geer: Dan Geer (3 Apr 2013), “Three policies,” http://geer.tinho.net/three.policies.2013Apr03Wed.PDF.

163Both Microsoft’s Brad Smith: Brad Smith (14 May 2017), “The need for urgent collective action to keep people safe online: Lessons from last week’s cyberattack,” Microsoft on the Issues, https://s.microsoft.com/on-the-issues/2017/05/14/need-urgent-collective-action-keep-people-safe-online-lessons-last-weeks-cyberattack.

163and Mozilla: Heather West (7 Mar 2017), “Mozilla statement on CIA/WikiLeaks,” Open Policy & Advocacy, https://.mozilla.org/netpolicy/2017/03/07/mozilla-statement-on-cia-wikileaks. Jochai Ben-Avie (3 Oct 2017), “Vulnerability disclosure should be part of new EU cybersecurity strategy,” Open Policy & Advocacy, https://.mozilla.org/netpolicy/2017/10/03/vulnerability-disclosure-should-be-in-new-eu-cybersecurity-strategy.

163“We recommend that the National Security Council”: Richard A. Clarke et al. (12 Dec 2013), “Liberty and security in a changing world,” President’s Review Group on Intelligence and Communications Technologies, https://obamawhitehouse.archives.gov/sites/default/files/docs/2013-12-12_rg_final_report.pdf.

163If we give up our own offensive: Both the NSA and the FBI have made that argument. David E. Sanger (28 Apr 2014), “White House details thinking on cybersecurity flaws,” New York Times, http://www.nytimes.com/2014/04/29/us/white-house-details-thinking-on-cybersecurity-gaps.html.

163“The idea that these problems”: Rick Ledgett (7 Aug 2017), “No, the U.S. government should not disclose all vulnerabilities in its possession,” Lawfare, https://www.lawfareblog.com/no-us-government-should-not-disclose-all-vulnerabilities-its-possession.

164Some are what the NSA calls “NOBUS”: Andrea Peterson (4 Oct 2013), “Why everyone is left less secure when the NSA doesn’t help fix security flaws,” Washington Post, https://www.washingtonpost.com/news/the-switch/wp/2013/10/04/why-everyone-is-left-less-secure-when-the-nsa-doesnt-help-fix-security-flaws.

164If a vulnerability is NOBUS: Lily Hay Newman (16 Jun 2017), “Why governments won’t let go of secret software bugs,” Wired, https://www.wired.com/2017/05/governments-wont-let-go-secret-software-bugs.

164In 2014, then–White House cybersecurity coordinator: Michael Daniel (28 Apr 2014), “Heartbleed: Understanding when we disclose cyber vulnerabilities,” Office of the President of the United States, http://www.whitehouse.gov/blog/2014/04/28/heartbleed-understanding-when-we-disclose-cyber-vulnerabilities.

164In 2016, the official, heavily redacted: Andrew Crocker (19 Jan 2016), “EFF pries more information on zero days from the government’s grasp,” Electronic Frontier Foundation, https://www.eff.org/deeplinks/2016/01/eff-pries-more-transparency-zero-days-governments-grasp.

164In 2017, new cybersecurity coordinator: [Office of the President of the United States] (15 Nov 2017), “Vulnerabilities equities policy and process for the United States government,” https://www.whitehouse.gov/sites/whitehouse.gov/files/images/External%20-%20Unclassified%20VEP%20Charter%20FINAL.PDF. Rob Joyce (15 Nov 2017), “Improving and making the vulnerability equities process transparent is the right thing to do,” Wayback Machine, https://web.archive.org/web/20171115151504/https://www.whitehouse.gov/blog/2017/11/15/improving-and-making-vulnerability-equities-process-transparent-right-thing-do.

164For example, ETERNALBLUE: Ellen Nakashima and Craig Timberg (16 May 2017), “NSA officials worried about the day its potent hacking tool would get loose. Then it did,” Washington Post, https://www.washingtonpost.com/business/technology/nsa-officials-worried-about-the-day-its-potent-hacking-tool-would-get-loose-then-it-did/2017/05/16/50670b16-3978-11e7-a058-ddbb23c75d82_story.html.

165Any process that allows such a serious: The NSA eventually disclosed the vulnerability, but that was after the Russians stole it. Dan Goodin (17 May 2017), “Fearing Shadow Brokers leak, NSA reported critical flaw to Microsoft,” Ars Technica, https://arstechnica.com/information-technology/2017/05/fearing-shadow-brokers-leak-nsa-reported-critical-flaw-to-microsoft.

165Vulnerabilities are independently discovered: Andy Greenberg (7 Jan 2018), “Triple Meltdown: How so many researchers found a 20-year-old chip flaw at the same time,” Wired, https://www.wired.com/story/meltdown-spectre-bug-collision-intel-chip-flaw-discovery.

165This implies that if the US government: In 2017, I tried to estimate the annual rate of rediscovery, using available data sets, and found it to be between 11% and 22%. Independently, a group of researchers from the RAND Corporation tried to estimate it as well, using different assumptions and a different data set; they found the rate to be less than 6%. We’re all blind folks touching different parts of the elephant. We each extrapolate from our own tiny pieces of data. Clearly we’re not going to learn much about the NSA’s capabilities this way. Trey Herr, Bruce Schneier, and Christopher Morris (7 Mar 2017), “Taking stock: Estimating vulnerability recovery,” Belfer Cyber Security Project White Paper Series, Harvard Kennedy School Belfer Center for Science and International Affairs, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2928758. Lillian Ablon and Timothy Bogart (9 Mar 2017), “Zero days, thousands of nights: The life and times of zero-day vulnerabilities and their exploits,” RAND Corporation, https://www.rand.org/pubs/research_reports/RR1751.html.

165Plus, NOBUS doesn’t take into account: Scott Shane, Matthew Rosenberg, and Andrew W. Lehren (7 Mar 2017), “WikiLeaks releases trove of alleged C.I.A. hacking documents,” New York Times, https://www.nytimes.com/2017/03/07/world/europe/wikileaks-cia-hacking.html.https://www.nytimes.com/2017/11/12/us/nsa-shadow-brokers.html. Scott Shane, Nicole Perlroth, and David E. Sanger (12 Nov 2017), “Security breach and spilled secrets have shaken the N.S.A. to its core,” New York Times, https://www.nytimes.com/2017/11/12/us/nsa-shadow-brokers.html.

165These included some pretty nasty: Bruce Schneier (28 Jul 2017), “Zero-day vulnerabilities against Windows in the NSA tools released by the Shadow Brokers,” Schneier on Security, https://www.schneier.com/blog/archives/2017/07/zero-day_vulner.html.

165Maybe nobody else could have: Dan Goodin (16 Apr 2017), “Mysterious Microsoft patch killed 0-days released by NSA-leaking Shadow Brokers,” Ars Technica, https://arstechnica.co.uk/information-technology/2017/04/purported-shadow-brokers-0days-were-in-fact-killed-by-mysterious-patch.

165In 2015, we learned that: National Security Agency/Central Security Service (30 Oct 2015), “Discovering IT problems, developing solutions, sharing expertise,” https://www.nsa.gov/news-features/news-stories/2015/discovering-solving-sharing-it-solutions.shtml.

165“Every year the government only keeps”: Jason Healey (1 Nov 2016), “The U.S. government and zero-day vulnerabilities: From pre-Heartbleed to the Shadow Brokers,” Columbia Journal of International Affairs, https://jia.sipa.columbia.edu/online-articles/healey_vulnerability_equities_process.

166It’s clear to many observers: Bruce Schneier (19 May 2014), “Should U.S. hackers fix cybersecurity holes or exploit them?” Atlantic, https://www.schneier.com/essays/archives/2014/05/should_us_hackers_fi.html. Ari Schwartz and Rob Knake (1 Jun 2016), “Government’s role in vulnerability disclosure: Creating a permanent and accountable vulnerability equities process,” Harvard Kennedy School Belfer Center for Science and International Affairs, https://www.belfercenter.org/publication/governments-role-vulnerability-disclosure-creating-permanent-and-accountable. Jason Healey (1 Nov 2016), “The U.S. government and zero-day vulnerabilities: From pre-Heartbleed to the Shadow Brokers,” Columbia Journal of International Affairs, https://jia.sipa.columbia.edu/online-articles/healey_vulnerability_equities_process.

166Instead, it’s making us much less secure: Oren J. Falkowitz (10 Jan 2017), “U.S. cyber policy makes Americans vulnerable to our own government,” Time, http://time.com/4625798/donald-trump-cyber-policy.

167The NSA participated in the process: John Gilmore (6 Sep 2013), “Re: [Cryptography] opening discussion: Speculation on ‘BULLRUN,’” Mail Archive, https://www.mail-archive.com/cryptography@metzdowd.com/msg12325.html.

167“devastating effect” on security: Niels Ferguson and Bruce Schneier (Dec 2003), “A cryptographic evaluation of IPsec,” Counterpane Internet Security, https://www.schneier.com/wp-content/uploads/2016/02/paper-ipsec.pdf.

167A second example: in the secret: Elad Barkan, Eli Biham, and Nathan Keller (17 Sep 2003), “Instant ciphertext-only cryptanalysis of GSM encrypted communication,” http://cryptome.org/gsm-crack-bbk.pdf.

167Both of these were probably part: Nicole Perlroth, Jeff Larson, and Scott Shane (5 Sep 2013), “Secret documents reveal N.S.A. campaign against encryption,” New York Times, http://www.nytimes.com/interactive/2013/09/05/us/documents-reveal-nsa-campaign-against-encryption.html. Nicole Perlroth, Jeff Larson, and Scott Shane (5 Sep 2013), “N.S.A. able to foil basic safeguards of privacy on web,” New York Times, http://www.nytimes.com/2013/09/06/us/nsa-foils-much-internet-encryption.html. Julian Ball, Julian Borger, and Glenn Greenwald (6 Sep 2013), “Revealed: How US and UK spy agencies defeat internet privacy and security,” Guardian, https://www.theguardian.com/world/2013/sep/05/nsa-gchq-encryption-codes-security.

168CALEA, the Communications Assistance for Law Enforcement Act: Albert Gidari (22 Feb 2016), “More CALEA and why it trumps the FBI’s All Writs Act order,” Center for Internet and Society, Stanford Law School, http://cyberlaw.stanford.edu/blog/2016/02/more-calea-and-why-it-trumps-fbis-all-writs-act-order.

168AmberJack is another: InfoSec Institute (8 Jan 2016), “Cellphone surveillance: The secret arsenal,” http://resources.infosecinstitute.com/cellphone-surveillance-the-secret-arsenal.

168This enables collection of identification: Joel Hruska (17 Jun 2014), “Stingray, the fake cell phone tower cops and carriers use to track your every move,” Extreme Tech, http://www.extremetech.com/mobile/184597-stingray-the-fake-cell-phone-tower-cops-and-providers-use-to-track-your-every-move.

168Only a few years ago, the FBI: Kim Zetter (19 Jun 2014), “Emails show feds asking Florida cops to deceive judges,” Wired, http://www.wired.com/2014/06/feds-told-cops-to-deceive-courts-about-stingray.

168When it seemed possible that local police: Nathan Freed Wessler (3 Jun 2014), “U.S. marshals seize local cops’ cell phone tracking files in extraordinary attempt to keep information from public,” American Civil Liberties Union, https://www.aclu.org/blog/national-security-technology-and-liberty/us-marshals-seize-local-cops-cell-phone-tracking-files.

169As recently as 2015, St. Louis police: Robert Patrick (19 Apr 2015), “Controversial secret phone tracker figured in dropped St. Louis case,” St. Louis Post-Dispatch, http://www.stltoday.com/news/local/crime-and-courts/controversial-secret-phone-tracker-figured-in-dropped-st-louis-case/article_fbb82630-aa7f-5200-b221-a7f90252b2d0.html. Cyrus Farivar (29 Apr 2015), “Robbery suspect pulls guilty plea after stingray disclosure, case dropped,” Ars Technica, http://arstechnica.com/tech-policy/2015/04/29/alleged-getaway-driver-challenges-stingray-use-robbery-case-dropped.

169What was once a secret NSA: Stephanie K. Pell and Christopher Soghoian (29 Dec 2014), “Your secret Stingray’s no secret anymore: The vanishing government monopoly over cell phone surveillance and its impact on national security and consumer privacy,” Harvard Journal of Law and Technology 28, no. 1, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2437678.

169In 2010, hackers were demonstrating: Kim Zetter (21 Jul 2010), “Hacker spoofs cell phone tower to intercept calls,” Wired, http://www.wired.com/2010/07/intercepting-cell-phone-calls.

169By 2014, dozens of IMSI-catchers: Ashkan Soltani and Craig Timberg (17 Sep 2014), “Tech firm tries to pull back curtain on surveillance efforts in Washington,” Washington Post, http://www.washingtonpost.com/world/national-security/researchers-try-to-pull-back-curtain-on-surveillance-efforts-in-washington/2014/09/17/f8c1f590-3e81-11e4-b03f-de718edeb92f_story.html.

169Now, you can browse the Chinese: A Mr. Mark Lazarte sells a PKI 1640 IMSI-catcher for $1,800. It seems to be made in Guangdong, China. Mark Lazarte (accessed 24 Apr 2018), “IMSI catcher,” Alibaba, https://www.alibaba.com/product-detail/IMSI-catcher_135958750.html.

169We know from the Snowden documents: Charlie Savage et al. (4 Jun 2015), “Hunting for hackers, NSA secretly expands Internet spying at U.S. border,” New York Times, https://www.nytimes.com/2015/06/05/us/hunting-for-hackers-nsa-secretly-expands-internet-spying-at-us-border.html.

170That same phone-switch wiretapping capability: Vassilis Prevelakis and Diomidis Spinellis (29 Jun 2007), “The Athens affair,” IEEE Spectrum, https://spectrum.ieee.org/telecom/security/the-athens-affair.

170CALEA inadvertently caused vulnerabilities: Tom Cross (3 Feb 2010), “Exploiting lawful intercept to wiretap the Internet,” Black Hat DC 2010, http://www.blackhat.com/presentations/bh-dc-10/Cross_Tom/BlackHat-DC-2010-Cross-Attacking-LawfulI-Intercept-wp.pdf.

170“when the NSA tested CALEA-compliant switches”: Quoted in Susan Landau (1 Mar 2016), “Testimony for House Judiciary Committee hearing on ‘The encryption tightrope: Rebalancing Americans’ security and privacy,” https://judiciary.house.gov/wp-content/uploads/2016/02/Landau-Written-Testimony.pdf.

170Even former NSA and CIA director Michael Hayden: Andrea Peterson (4 Oct 2013), “Why everyone is left less secure when the NSA doesn’t help fix security flaws,” Washington Post, https://www.washingtonpost.com/news/the-switch/wp/2013/10/04/why-everyone-is-left-less-secure-when-the-nsa-doesnt-help-fix-security-flaws.

170“the NOBUS comfort zone is”: Michael V. Hayden (17 May 2017), “The equities decision: Deciding when to exploit or defend,” Chertoff Group, http://www.chertoffgroup.com/point-of-view/109-the-chertoff-group-point-of-view/665-the-equities-decision-deciding-when-to-exploit-or-defend.

172I, and many security technologists: Harold Abelson et al. (7 Jul 2015), “Keys under doormats: Mandating insecurity by requiring government access to all data and communications,” MIT CSAIL Technical Report 2015-026, MIT Computer Science and Artificial Intelligence Laboratory, https://dspace.mit.edu/handle/1721.1/97690.

173the UK’s National Cyber Security Centre: I have heard it referred to as GCHQ’s London branch.

173Unfortunately, in 2016, the NSA underwent: Ellen Nakashima (2 Feb 2016), “National Security Agency plans major reorganization,” Washington Post, https://www.washingtonpost.com/world/national-security/national-security-agency-plans-major-reorganization/2016/02/02/2a66555e-c960-11e5-a7b2-5a2f824b02c9_story.html.

173If the NSA is ever to be trusted: Nicholas Weaver makes this point well. Nicholas Weaver (10 Feb 2016), “Trust and the NSA reorganization,” Lawfare, https://www.lawfareblog.com/trust-and-nsa-reorganization.

174Because it was an iPhone 5C: Samantha Masunaga (2 Oct 2017), “FBI doesn’t have to say who unlocked San Bernardino shooter’s iPhone, judge rules,” Los Angeles Times, http://beta.latimes.com/business/la-fi-tn-fbi-iphone-20171002-story.html.

174Apple resisted the FBI’s demand: Arash Khamooshi (3 Mar 2016), “Breaking down Apple’s iPhone fight with the U.S. government,” New York Times, https://www.nytimes.com/interactive/2016/03/03/technology/apple-iphone-fbi-fight-explained.html.

174Eventually, the FBI got some unidentified: Thomas Fox-Brewster (26 Feb 2018), “The feds can now (probably) unlock every iPhone model in existence,” Forbes, https://www.forbes.com/sites/thomasbrewster/2018/02/26/government-can-access-any-apple-iphone-cellebrite. Sean Gallagher (28 Feb 2018), “Cellebrite can unlock any iPhone (for some values of ‘any’),” Ars Technica, https://arstechnica.com/information-technology/2018/02/cellebrite-can-unlock-any-iphone-for-some-values-of-any.

174to break into the phone without Apple’s help: Matt Zapotosky (28 Mar 2016), “FBI has accessed San Bernardino shooter’s phone without Apple help,” Washington Post, https://www.washingtonpost.com/world/national-security/fbi-has-accessed-san-bernardino-shooters-phone-without-apples-help/2016/03/28/e593a0e2-f52b-11e5-9804-537defcc3cf6_story.html. David Kravets (1 Oct 2017), “FBI may keep secret the name of vendor that cracked terrorist’s iPhone,” Ars Technica, https://arstechnica.com/tech-policy/2017/10/fbi-does-not-have-to-disclose-payments-to-vendor-for-iphone-cracking-tool.

174“Don’t Panic”: Jonathan Zittrain et al. (Feb 2016), “Don’t panic: Making progress on the ‘going dark’ debate,” Berkman Center for Internet and Society, Harvard University, https://cyber.harvard.edu/pubrelease/dont-panic/Dont_Panic_Making_Progress_on_Going_Dark_Debate.pdf.

175All the agency’s current employees: Susan Landau (2017), Listening In: Cybersecurity in an Insecure Age, Yale University Press, https://books.google.com/books?id=QZ47DwAAQBAJ.

175“The FBI will need an investigative center”: Susan Landau (1 Mar 2016), “Testimony for House Judiciary Committee hearing on ‘The encryption tightrope: Rebalancing Americans’ security and privacy,’” https://judiciary.house.gov/wp-content/uploads/2016/02/Landau-Written-Testimony.pdf.

175In addition to better computer forensics: Steven M. Bellovin et al. (19 Aug 2014), “Lawful hacking: Using existing vulnerabilities for wiretapping on the Internet,” Northwestern Journal of Technology and Intellectual Property 12, no. 1, https://www.ssrn.com/abstract=2312107.

176If the FBI is going to attract: They’re trying. Federal Bureau of Investigation (29 Dec 2014), “Most wanted talent: Seeking tech experts to become cyber special agents,” https://www.fbi.gov/news/stories/fbi-seeking-tech-experts-to-become-cyber-special-agents.

176The reality always falls short: Neil Robinson and Emma Disley (10 Sep 2010), “Incentives and challenges for information sharing in the context of network and information security,” European Network and Information Security Agency, https://www.enisa.europa.eu/publications/incentives-and-barriers-to-information-sharing/at_download/fullReport.

176This is rational: Lawrence A. Gordon, Martin P. Loeb, and William Lucyshyn (Feb 2003), “Sharing information on computer systems security: An economic analysis,” Journal of Accounting and Public Policy 22, no. 6, http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.598.6498&rep=rep1&type=pdf.

177The FAA maintains an anonymous database: US Department of Homeland Security (10 Sep 2015), “Enhancing resilience through cyber incident data sharing and analysis,” https://www.dhs.gov/sites/default/files/publications/Data%20Categories%20White%20Paper%20-%20508%20compliant.pdf.

177Another idea is to create: Jonathan Bair et al. (forthcoming), “That was close! Reward reporting of cybersecurity ‘near misses,’” Colorado Technology Law Journal 16, no. 2, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3081216.

177This NCSB would investigate: Neil Robinson (19 Jun 2012), “The case for a cyber-security safety board: A global view on risk,” RAND Blog, https://www.rand.org/blog/2012/06/the-case-for-a-cyber-security-safety-board-a-global.html.

177the NTSB’s annual “Most Wanted List”: National Transportation Safety Board (accessed 24 Apr 2018), “2017–2018 most wanted list,” https://www.ntsb.gov/safety/mwl/Pages/default.aspx.

177the most critical changes needed: Ben Rothke (19 Feb 2015), “It’s time for a National Cybersecurity Safety Board (NCSB),” CSO, https://www.csoonline.com/article/2886326/security-awareness/it-s-time-for-a-national-cybersecurity-safety-board-ncsb.html.

177Nongovernmental networks like: Sean Michael Kerner (27 Oct 2017), “Cyber Threat Alliance adds new members to security sharing group,’ eWeek, http://www.eweek.com/security/cyber-threat-alliance-adds-new-members-to-security-sharing-group.

178We also can’t expect corporations: The US indicted five members of the Chinese People’s Liberation Army for these hacks in 2014. Michael S. Schmidt and David E. Sanger (19 May 2014), “5 in China army face U.S. charges of cyberattacks,” New York Times, https://www.nytimes.com/2014/05/20/us/us-to-charge-chinese-workers-with-cyberspying.html.

178We shouldn’t expect the Democratic and Republican: Nicole Gaouette (10 Jan 2017), “FBI’s Comey: Republicans also hacked by Russia,” CNN, http://www.cnn.com/2017/01/10/politics/comey-republicans-hacked-russia/index.html.

178Is it a Cyber National Guard?: In 2017, Representative Will Hurd proposed this. Frank Konkel (21 Jun 2017), “Lawmaker: Cyber National Guard could fill federal workforce gaps,” Nextgov, http://www.nextgov.com/cybersecurity/2017/06/lawmaker-cyber-national-guard-could-fill-federal-workforce-gaps/138851.

179Estonia has a volunteer Cyber Defence Unit: Monica M. Ruiz (9 Jan 2018), “Is Estonia’s approach to cyber defense feasible in the United States?” War on the Rocks, https://warontherocks.com/2018/01/estonias-approach-cyber-defense-feasible-united-states.

10. Plan B: What’s Likely to Happen

180Despite some very strong words: Martin Matishak (1 Jan 2018), “After Equifax breach, anger but no action in Congress,” Politico, https://www.politico.com/story/2018/01/01/equifax-data-breach-congress-action-319631.

180Even a bill imposing the tiniest: Robert McLean (15 Sep 2017), “Elizabeth Warren’s Equifax bill would make credit freezes free,” CNN, http://money.cnn.com/2017/09/15/pf/warren-schatz-equifax/index.html.

180The only thing Congress did: Devin Coldewey (24 Oct 2017), “Congress votes to disallow consumers from suing Equifax and other companies with arbitration agreements,” TechCrunch, https://techcrunch.com/2017/10/24/congress-votes-to-disallow-consumers-from-suing-equifax-and-other-companies-with-arbitration-agreements/amp.

180Story two: The 2017 Internet of Things: Mark R. Warner (1 Aug 2017), “Senators introduce bipartisan legislation to improve cybersecurity of ‘Internet of things’ (IoT) devices,” https://www.warner.senate.gov/public/index.cfm/2017/8/enators-introduce-bipartisan-legislation-to-improve-cybersecurity-of-internet-of-things-iot-devices.

180Story three: In 2016, President Obama: Barack Obama (9 Feb 2016), “Presidential executive order: Commission on Enhancing National Cybersecurity,” Office of the President of the United States, https://www.whitehouse.gov/the-press-office/2016/02/09/executive-order-commission-enhancing-national-cybersecurity.

181At the end of that year: Thomas E. Donilon et al. (1 Dec 2016), “Report on securing and growing the digital economy,” Commission on Enhancing National Cybersecurity, https://www.nist.gov/sites/default/files/documents/2016/12/02/cybersecurity-commission-report-final-post.pdf.

181It’s almost two years later: Donald J. Trump (11 May 2017), “Presidential executive order on strengthening the cybersecurity of federal networks and critical infrastructure,” Office of the President of the United States, https://www.whitehouse.gov/presidential-actions/presidential-executive-order-strengthening-cybersecurity-federal-networks-critical-infrastructure.

181No agency has yet followed that policy: Nick Marinos (13 Feb 2018), “Critical infrastructure protection: Additional actions are essential for assessing cybersecurity framework adoption,” GAO-18-211, US Government Accountability Office, https://www.gao.gov/assets/700/690112.pdf.

181The rest of the report has been ignored: You could blame it on the dysfunctional administration, but I don’t believe a different administration would have fared much better.

182Some observers have noted parallels: Economist (8 Apr 2017), “How to manage the computer-security threat,” https://www.economist.com/news/leaders/21720279-incentives-software-firms-take-security-seriously-are-too-weak-how-manage.

182It was the 1965 publication of: Christopher Jensen (26 Nov 2015), “50 years ago, Unsafe at Any Speed shook the auto world,” New York Times, https://www.nytimes.com/2015/11/27/automobiles/50-years-ago-unsafe-at-any-speed-shook-the-auto-world.html.

184The GDPR—General Data Protection Regulation: European Union (27 Apr 2016), “Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation),” Official Journal of the European Union, http://eur-lex.europa.eu/eli/reg/2016/679/oj.

184For example, the GDPR mandates that: This is a good short summary: Cennydd Bowles (12 Jan 2018), “A techie’s rough guide to GDPR,” https://www.cennydd.com/writing/a-techies-rough-guide-to-gdpr.

184The GDPR’s regulations only affect: Mark Scott and Laurens Cerulus (31 Jan 2018), “Europe’s new data protection rules export privacy standards worldwide,” Politico, https://www.politico.eu/article/europe-data-protection-privacy-standards-gdpr-general-protection-data-regulation.

185If companies have to explain This is already happening. In response to the GDPR, PayPal published a list of the 600+ companies it shares customer data with. It has taken the page offline, but the information has been saved. Rebecca Ricks (accessed 24 Apr 2018), “How PayPal shares your data,” https://rebecca-ricks.com/paypal-data.

185Additionally, legislatures worldwide: Mark Scott and Laurens Cerulus (31 Jan 2018), “Europe’s new data protection rules export privacy standards worldwide,” Politico, https://www.politico.eu/article/europe-data-protection-privacy-standards-gdpr-general-protection-data-regulation.

185Organizations are already doing things: Clint Boulton (26 Jan 2017), “U.S. companies spending millions to satisfy Europe’s GDPR,” CIO, https://www.cio.com/article/3161920/privacy/article.html. Nick Ismail (2 May 2017), “Only 43% of organisations are preparing for GDPR,” Information Age, http://www.information-age.com/43-organisations-preparing-gdpr-123465995. Sarah Gordon (18 Jun 2017), “Businesses failing to prepare for EU rules on data protection,” Financial Times, https://www.ft.com/content/28f4eff8-51bf-11e7-a1f2-db19572361bb.

185Fines can be as high as 4%: EUGDPR.org (accessed 24 Apr 2018), “GDPR key changes,” https://www.eugdpr.org/key-changes.html.

185The EU fined Google 2.4 billion euros: Mark Scott (27 Jun 2017), “Google fined record $2.7 billion in E.U. antitrust ruling,” New York Times, https://www.nytimes.com/2017/06/27/technology/eu-google-fine.html. Aoife White and Mark Bergen (29 Aug 2017), “Google to comply with EU search demands to avoid more fines,” Bloomberg, https://www.bloomberg.com/news/articles/2017-08-29/google-faces-tuesday-deadline-as-clock-ticks-toward-new-eu-fines.

185Separately, the EU fined Facebook: Hayley Tsukayama (18 May 2017), “Facebook will pay $122 million in fines to the E.U.,” Washington Post, https://www.washingtonpost.com/news/the-switch/wp/2017/05/18/facebook-will-pay-122-million-in-fines-to-the-eu.

185Under the GDPR, the fine would: Paul Roberts (2 Nov 2017), “Hilton was fined $700K for a data breach. Under GDPR it would be $420M,” Digital Guardian, https://digitalguardian.com/blog/hilton-was-fined-700k-data-breach-under-gdpr-it-would-be-420m.

186“The EU is already the world’s”: Eireann Leverett, Richard Clayton, and Ross Anderson (6 Jun 2017), “Standardization and certification of the ‘Internet of Things,’” Institute for Consumer Policy, https://www.conpolicy.de/en/news-detail/standardization-and-certification-of-the-internet-of-things.

186If European regulations force minimum: In this way, software is similar to textbooks in the US market, where a few states effectively control what is available nationally because of their very onerous demands.

186In April 2018, Facebook announced: Cyrus Farivar (4 Apr 2018), “CEO says Facebook will impose new privacy rules ‘everywhere,’” Ars Technica, https://arstechnica.com/tech-policy/2018/04/ceo-says-facebook-will-impose-new-eu-privacy-rules-everywhere.

186Singapore has the Personal Data Protection Act: Kennedy’s Law LLP (20 Apr 2016), “Personal data privacy principles in Asia Pacific,” http://www.kennedyslaw.com/dataprivacyapacguide2016.

186In 2017, India’s Supreme Court: Wire Staff (24 Aug 2017), “Right to privacy a fundamental right, says Supreme Court in unanimous verdict,” Wire, https://thewire.in/170303/supreme-court-aadhaar-right-to-privacy.

187Singapore passed a new Cybersecurity Act: Bryan Tan (9 Feb 2018), “Singapore finalises new Cybersecurity Act,” Out-Law, https://www.out-law.com/en/articles/2018/february/singapore-finalises-new-cybersecurity-act.

187New Israeli security regulations: Omer Tene (22 Mar 2017), “Israel enacts landmark data security notification regulations,” Privacy Tracker, https://iapp.org/news/a/israel-enacts-landmark-data-security-notification-regulations.

187In 2016, New York fined Trump Hotels: Steve Eder (24 Sep 2016), “Donald Trump’s hotel chain to pay penalty over data breaches,” New York Times, https://www.nytimes.com/2016/09/25/us/politics/trump-hotel-data.html.

187California investigated companies: Adolfo Guzman-Lopez (2 Nov 2016), “California attorney general warns tech companies about mining student data for profit,” Southern California Public Radio, https://www.scpr.org/news/2016/11/02/65908/attorney-general-warns-tech-companies-to-follow-ne.

187In 2017, Massachusetts sued Equifax: Francine McKenna (15 Sep 2017), “Equifax faces its biggest litigation threat from state attorneys general,” MarketWatch, https://www.marketwatch.com/story/equifax-faces-its-biggest-litigation-threat-from-state-attorneys-general-2017-09-15/print.

187Missouri began investigating Google’s: Nitasha Tiku (14 Nov 2017), “State attorneys general are Google’s next headache,” Wired, https://www.wired.com/story/state-attorneys-general-are-googles-next-headache.

187Thirty-two state attorneys general: Maria Armental (6 Sep 2017), “Lenovo reaches $3.5 million settlement over preinstalled adware,” MarketWatch, https://www.marketwatch.com/story/lenovo-reaches-35-million-settlement-with-ftc-over-preinstalled-adware-2017-09-05.

187Even the city of San Diego: Brian Krebs (18 Mar 2018), “San Diego sues Experian over ID theft service,” Krebs on Security, https://krebsonsecurity.com/2018/03/san-diego-sues-experian-over-id-theft-service.

187In 2019, these standards will also apply: Michael Krimminger (25 Mar 2017), “New York cybersecurity regulations for financial institutions enter into effect,” Harvard Law School Forum on Corporate Governance and Financial Regulation, https://corpgov.law.harvard.edu/2017/03/25/new-york-cybersecurity-regulations-for-financial-institutions-enter-into-effect.

187In 2017, California temporarily tabled: Karl D. Belgum (21 Jun 2017), “Internet of Things legislation in California is dead for this year, but it will be back,” Nixon Peabody, http://web20.nixonpeabody.com/dataprivacy/Lists/Posts/Post.aspx?ID=1155.

187Ten other states debated legislation: Eyragon Eidam and Jessica Mulholland (10 Apr 2017), “10 states take Internet privacy matters into their own hands,” Government Technology, http://www.govtech.com/policy/10-States-Take-Internet-Privacy-Matters-Into-Their-Own-Hands.html.

187“Teddy Bears and Toasters” bill: California Legislative Information (accessed 24 Apr 2018), “SB-327 Information privacy: Connected devices,” https://leginfo.legislature.ca.gov/faces/billHistoryClient.xhtml?bill_id=201720180SB327.

188As this book went to press: Alan L. Friel, Linda A. Goldstein, and Holly Al Melton (31 Jan 2018), “AD-ttorneys@law—January 31, 2018,” Baker Hostetler, https://www.bakerlaw.com/alerts/ad-ttorneyslaw-january-31-2018.

188California’s legislature is also considering: Elizabeth Zima (23 Feb 2018), “California wants to govern bots and police user privacy on social media,” Government Technology, http://www.govtech.com/social/California-Wants-to-Govern-bots-and-Police-User-Privacy-on-Social-Media.html.

188There are some things concerned consumers can do: Deborah Gage (15 Sep 2017), “Eight questions to ask before buying an internet-connected device,” Wall Street Journal, https://www.wsj.com/articles/eight-questions-to-ask-before-buying-an-internet-connected-device-1505487931.

188There’s plenty of good advice: Here are two things to get you started: Electronic Frontier Foundation (21 Oct 2014, last updated 21 Sep 2015), “Surveillance self-defense,” https://ssd.eff.org. Motherboard Staff (15 Nov 2017), “The Motherboard guide to not getting hacked,” Vice Motherboard, https://motherboard.vice.com/en_us/article/d3devm/motherboard-guide-to-not-getting-hacked-online-safety-guide.

189Two years previously, the Swedish Transport Agency: Rick Falkvinge (21 Jul 2017), “Worst known governmental leak ever is slowly coming to light: Agency moved nation’s secret data to ‘the cloud,’” Privacy News Online, https://www.privateInternetaccess.com/blog/2017/07/swedish-transport-agency-worst-known-governmental-leak-ever-is-slowly-coming-to-light.

190Do you prefer Apple’s iMessage: For security, use Signal. If having Signal on your phone would be suspicious, use WhatsApp. Micah Lee (22 Jun 2016), “Battle of the secure messaging apps: How Signal beats WhatsApp,” Intercept, https://theintercept.com/2016/06/22/battle-of-the-secure-messaging-apps-how-signal-beats-whatsapp.

190for example, Microsoft’s ongoing battle: Joe Uchill (23 Jun 2017), “DOJ applies to take Microsoft data warrant case to Supreme Court,” Hill, http://thehill.com/policy/cybersecurity/339281-doj-applies-to-take-microsoft-data-warrant-case-to-supreme-court.

191Elsewhere I have argued that: Bruce Schneier (2015), Data and Goliath: The Hidden Battles to Collect Your Data and Control Your World, W. W. Norton, https://books.google.com/books/?id=MwF-BAAAQBAJ.

Where Policy Can Go Wrong

193Luckily, the courts prevented: Ian Urbina (23 Mar 2007), “Court rejects law limiting online pornography,” New York Times, www.nytimes.com/2007/03/23/us/23porn.html.

193Not only does it not prevent: Electronic Frontier Foundation (1 Mar 2013), “Unintended consequences: Fifteen years under the DMCA,” https://www.eff.org/pages/unintended-consequences-fifteen-years-under-dmca.

193“The widespread use of robust”: Louis J. Freeh (9 Sep 1997), “The impact of encryption on public safety: Statement of the Director, Federal Bureau of Investigation, before the Permanent Select Committee on Intelligence, United States House of Representatives,” https://fas.org/irp/congress/1997_hr/h970909f.htm.

193“As the gap between authority and”: Valerie Caproni (17 Feb 2011), “Statement before the House Judiciary Committee, Subcommittee on Crime, Terrorism, and Homeland Security,” Federal Bureau of Investigation, https://archives.fbi.gov/archives/news/testimony/going-dark-lawful-electronic-surveillance-in-the-face-of-new-technologies.

193“We may not be able to identify”: James B. Comey (8 Jul 2015), “Going dark: Encryption, technology, and the balances between public safety and privacy,” Federal Bureau of Investigation, https://www.fbi.gov/news/testimony/going-dark-encryption-technology-and-the-balances-between-public-safety-and-privacy.

194“But the advent of ‘warrant-proof’ encryption”: Rod J. Rosenstein (4 Oct 2017), “Deputy Attorney General Rod J. Rosenstein delivers remarks at the Cambridge Cyber Summit,” US Department of Justice, https://www.justice.gov/opa/speech/deputy-attorney-general-rod-j-rosenstein-delivers-remarks-cambridge-cyber-summit.

194golden age of surveillance: Peter Swire and Kenesa Ahmad are responsible for that term. Peter Swire and Kenesa Ahmad (28 Nov 2011), “‘Going dark’ versus a ‘golden age for surveillance,’” Center for Democracy and Technology, https://cdt.org/blog/%E2%80%98going-dark%E2%80%99-versus-a-%E2%80%98golden-age-for-surveillance%E2%80%99.

194The idea was called “key escrow”: Andi Wilson, Danielle Kehl, and Kevin Bankston (17 Jun 2015), “Doomed to repeat history? Lessons from the crypto wars of the 1990s,” New America Foundation, https://www.newamerica.org/oti/doomed-to-repeat-history-lessons-from-the-crypto-wars-of-the-1990s.

194In the early 2000s, the FBI argued: Federal Bureau of Investigation (3 Jun 1999), “Encryption: Impact on law enforcement,” https://web.archive.org/web/20000815210233/https://www.fbi.gov/library/encrypt/en60399.pdf.

194A decade later, demands devolved: Ellen Nakashima (16 Oct 2014), “FBI director: Tech companies should be required to make devices wiretap-friendly,” Washington Post, https://www.washingtonpost.com/world/national-security/fbi-director-tech-companies-should-be-required-to-make-devices-wire-tap-friendly/2014/10/16/93244408-555c-11e4-892e-602188e70e9c_story.html.

195Rosenstein has given this security-hostile proposal: Rod J. Rosenstein (10 Oct 2017), “Deputy Attorney General Rod J. Rosenstein delivers remarks on encryption at the United States Naval Academy,” US Department of Justice, https://www.justice.gov/opa/speech/deputy-attorney-general-rod-j-rosenstein-delivers-remarks-encryption-united-states-naval.

195UK policy makers are already implying: Bhairav Acharya et al. (28 Jun 2017), “Deciphering the European encryption debate: United Kingdom,” New America, https://www.newamerica.org/oti/policy-papers/deciphering-european-encryption-debate-united-kingdom.

195In 2016, Croatia, France, Germany: Amar Tooer (24 Aug 2016), “France and Germany want Europe to crack down on encryption,” Verge, https://www.theverge.com/2016/8/24/12621834/france-germany-encryption-terorrism-eu-telegram. Catherine Stupp (22 Nov 2016), “Five member states want EU-wide laws on encryption,” Euractiv, https://www.euractiv.com/section/social-europe-jobs/news/five-member-states-want-eu-wide-laws-on-encryption.

195Separately, the EU is considering legislation: Samuel Gibbs (19 Jun 2017), “EU seeks to outlaw ‘backdoors’ in new data privacy proposals,” Guardian, https://www.theguardian.com/technology/2017/jun/19/eu-outlaw-backdoors-new-data-privacy-proposals-uk-government-encrypted-communications-whatsapp.

195Australia is also trying to mandate access: Rachel Baxendale (14 Jul 2017), “Laws could force companies to unlock encrypted messages of terrorists,” Australian, http://www.theaustralian.com.au/national-affairs/laws-could-force-companies-to-unlock-encrypted-messages-of-terrorists/news-story/ed481d29c956dfac9361061a60dcf590.

195In Brazil, courts temporarily shut down: Vinod Sreeharsha (19 Jul 2016), “WhatsApp is briefly shut down in Brazil for a third time,” New York Times, https://www.nytimes.com/2016/07/20/technology/whatsapp-is-briefly-shut-down-in-brazil-for-a-third-time.html.

195Egypt blocked the encrypted: Mariella Moon (20 Dec 2016), “Egypt has blocked encrypted messaging app Signal,” Engadget, https://www.engadget.com/2016/12/20/egypt-blocks-signal.

195And both Russia: Patrick Howell O’Neill (20 Jun 2016), “Russian bill requires encryption backdoors in all messenger apps,” Daily Dot, https://www.dailydot.com/layer8/encryption-backdoor-russia-fsb. Adam Maida (18 Jul 2017), “Online and on all fronts: Russia’s assault on freedom of expression,” Human Rights Watch, https://www.hrw.org/report/2017/07/18/online-and-all-fronts/russias-assault-freedom-expression. Kenneth Rapoza (16 Oct 2017), “Russia fines cryptocurrency world’s preferred messaging app, Telegram,” Forbes, https://www.forbes.com/sites/kenrapoza/2017/10/16/russia-fines-cryptocurrency-worlds-preferred-messaging-app-telegram.

195and China: Benjamin Haas (29 Jul 2017), “China blocks WhatsApp services as censors tighten grip on internet,” Guardian, https://www.theguardian.com/technology/2017/jul/19/china-blocks-whatsapp-services-as-censors-tighten-grip-on-internet.

196If a company like Apple received: Mallory Locklear (23 Oct 2017), “FBI tried and failed to unlock 7,000 encrypted devices,” Engadget, https://www.engadget.com/2017/10/23/fbi-failed-unlock-7-000-encrypted-devices.

196“Any measure that weakens encryption”: Fred Upton et al. (20 Dec 2016), “Encryption working group year-end report,” House Judiciary Committee and House Energy and Commerce Committee Encryption Working Group, US House of Representatives, https://judiciary.house.gov/wp-content/uploads/2016/12/20161220EWGFINALReport.pdf.

196“My personal view is that we should”: Steve Cannane (9 Nov 2017), “Cracking down on encryption could ‘make it easier for hackers’ to penetrate private services,” ABC News Australia, http://www.abc.net.au/news/2017-11-10/former-mi5-chief-says-encryption-cut-could-lead-to-more-hacking/9136746.

196If Apple adds a backdoor: Lily Hay Newman (21 Apr 2017), “Encrypted chat took over. Let’s encrypt calls, too,” Wired, https://www.wired.com/2017/04/encrypted-chat-took-now-encrypted-callings-turn.

197These controls ended when the Internet: Whitfield Diffie and Susan Landau (1 Oct 2001), “The export of cryptography in the 20th century and the 21st,” Sun Microsystems, https://pdfs.semanticscholar.org/1870/af818dd0075bb5e79764427a7c932fe3cfc6.pdf.

197In 2015, then–UK prime minister David Cameron: British Broadcasting Corporation (12 Jan 2015), “David Cameron says new online data laws needed,” BBC News, http://www.bbc.com/news/uk-politics-30778424. Andrew Griffin (12 Jan 2015), “WhatsApp and Snapchat could be banned under new surveillance plans,” Independent, https://www.independent.co.uk/life-style/gadgets-and-tech/news/whatsapp-and-snapchat-could-be-banned-under-new-surveillance-plans-9973035.html.

197Current prime minister Theresa May: Charles Riley (4 Jun 2017), “Theresa May: Internet must be regulated to prevent terrorism,” CNN, http://money.cnn.com/2017/06/04/technology/social-media-terrorism-extremism-london/index.html.

198In 2016, I surveyed the market: Bruce Schneier, Kathleen Seidel, and Saranya Vijayakumar (11 Feb 2016), “A worldwide survey of encryption products,” Publication 2016-2, Berkman Center for Internet & Society, Harvard University, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2731160.

198Keeping those foreign products out: Cory Doctorow (4 Jun 2017), “Theresa May wants to ban crypto: Here’s what that would cost, and here’s why it won’t work anyway,” Boing Boing, https://boingboing.net/2017/06/04/theresa-may-king-canute.html.

198In their attempts to demand backdoors: Daniel Moore and Thomas Rid (Feb 2016), “Cryptopolitik and the Darknet,” Survival 58, no. 1, https://www.tandfonline.com/doi/abs/10.1080/00396338.2016.1142085.

199“We believe that the greater public good”: Mike McConnell, Michael Chertoff, and William Lynn (28 Jul 2015), “Why the fear over ubiquitous data encryption is overblown,” Washington Post, https://www.washingtonpost.com/opinions/the-need-for-ubiquitous-data-encryption/2015/07/28/3d145952-324e-11e5-8353-1215475949f4_story.html.

200Anonymous speech is valuable: Helen Nissenbaum (1 Sep 1998), “The meaning of anonymity in an information age,” Information Society 15, http://www.cs.cornell.edu/~shmat/courses/cs5436/meaning-of-anonymity.pdf.

201The US government collected phone call metadata: The NSA’s bulk collection program ended in 2015. Now, the phone companies save the metadata, and the NSA is able to query the database on demand. This seems like a difference without a difference. Charlie Savage (2 May 2017), “Reined-in NSA still collected 151 million phone records in ’16,” New York Times, https://www.nytimes.com/2017/05/02/us/politics/nsa-phone-records.html.

201Many local governments keep: Catherine Crump et al. (17 Jul 2013), “You are being tracked: How license plate readers are being used to record Americans’ movements,” American Civil Liberties Union, https://www.aclu.org/files/assets/071613-aclu-alprreport-opt-v05.pdf.

201Governments regularly demand access: Fred H. Cate and James X. Dempsey, eds. (2017), Bulk Collection: Systematic Government Access to Private-Sector Data, Oxford University Press, http://www.oxfordscholarship.com/view/10.1093/oso/9780190685515.001.0001/oso-9780190685515.

201Making them a reality would require: Jeanne Guillemin (1 Jul 2006), “Scientists and the history of biological weapons: A brief historical overview of the development of biological weapons in the twentieth century,” EMBO Reports 7, http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1490304.

202No amount of surveillance can stop: Jim Harper (10 Nov 2009), “The search for answers in Fort Hood,” Cato at Liberty, http://www.cato.org/blog/search-answers-fort-hood. Jim Harper (11 Nov 2009), “Fort Hood: Reaction, response, and rejoinder,” Cato at Liberty, http://www.cato.org/blog/fort-hood-reaction-response-rejoinder.

202The failure to anticipate the Boston Marathon bombing: Office of the Inspectors General for the Intelligence Community, Central Intelligence Agency, Department of Justice, and Department of Homeland Security (10 Apr 2014; unclassified summary released 6 Dec 2016), “Summary of information handling and sharing prior to the April 15, 2013 Boston Marathon bombings,” https://www.dni.gov/index.php/who-we-are/organizations/ic-ig/ic-ig-news/1604.

203“active cyber defense”: Irving Lachow (22 Feb 2013), “Active cyber defense: A framework for policymakers,” Center for a New American Security, https://www.cnas.org/publications/reports/active-cyber-defense-a-framework-for-policymakers.

203On the surface this might seem reasonable: Patrick Lin lays out the various arguments well. Patrick Lin (26 Sep 2016), “Ethics of hacking back: Six arguments from armed conflict to zombies,” California Polytechnic State University, Ethics + Emerging Sciences Group, http://ethics.calpoly.edu/hackingback.pdf.

203Vengeance is satisfying: Josephine Wolff (17 Oct 2017), “Attack of the hack back,” Slate, http://www.slate.com/articles/technology/future_tense/2017/10/hacking_back_the_worst_idea_in_cybersecurity_rises_again.html.

204Almost everybody agrees with this: Josephine Wolff (14 Jul 2017), “When companies get hacked, should they be allowed to hack back?” Atlantic, https://www.theatlantic.com/business/archive/2017/07/hacking-back-active-defense/533679.

204Both the FBI and the Justice Department: Jordan Robertson and Michael Riley (30 Dec 2013), “Would the U.S. really crack down on companies that hack back?” Bloomberg, https://www.bloomberg.com/news/2014-12-30/why-would-the-u-s-crack-down-on-companies-that-hack-back-.html.

204A 2017 bill legitimizing some: Tom Graves (13 Oct 2017), “Rep. Tom Graves formally introduces active cyber defense bill,” https://tomgraves.house.gov/news/documentsingle.aspx?DocumentID=398840.

204The main exception seems to be: Stewart A. Baker (8 May 2013), “The attribution revolution: Raising the costs for hackers and their customers: Statement of Stewart A. Baker, Partner, Steptoe & Johnson LLP, before the Judiciary Committee’s Subcommittee on Crime and Terrorism, United States Senate,” https://www.judiciary.senate.gov/imo/media/doc/5-8-13BakerTestimony.pdf. Stewart A. Baker (11 Sep 2013), “Testimony of Stewart A. Baker before the Committee on Homeland Security and Governmental Affairs, United States Senate: The Department of Homeland Security at 10 Years: Examining Challenges and Addressing Emerging Threats,” https://www.hsgac.senate.gov/hearings/the-department-of-homeland-security-at-10-years-examining-challenges-and-achievements-and-addressing-emerging-threats. Stewart A. Baker, Orin Kerr, and Eugene Volokh (2 Nov 2012), “The hackback debate,” Steptoe Cyberblog, https://www.steptoecyberblog.com/2012/11/02/the-hackback-debate. Stewart A. Baker (22 Jul 2016), “The case for limited hackback rights,” Washington Post, https://www.washingtonpost.com/news/volokh-conspiracy/wp/2016/07/22/the-case-for-limited-hackback-rights.

205For example, what Mattel, Disney: Charles Finocchiaro (18 Mar 2013), “Personal factory or catalyst for piracy? The hype, hysteria, and hard realities of consumer 3-D printing,” Cardozo Arts and Entertainment Law Journal 31, http://www.cardozoaelj.com/issues/archive/2012-13. Matthew Adam Susson (Apr 2013), “Watch the world ‘burn’: Copyright, micropatent and the emergence of 3D printing,” Chapman University School of Law, http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2253109.

206I worry that analogous laws: Cory Doctorow (10 Jan 2012), “Lockdown: The coming war on general-purpose computing,” Boing Boing, http://boingboing.net/2012/01/10/lockdown.html. Cory Doctorow (23 Aug 2012), “The coming civil war over general purpose computing,” Boing Boing, http://boingboing.net/2012/08/23/civilwar.html.

206With respect to radios, one solution: Kristen Ann Woyach et al. (23–26 Sep 2008), “Crime and punishment for cognitive radios,” 2008 46th Annual Allerton Conference on Communication, Control, and Computing, http://ieeexplore.ieee.org/document/4797562.

12. Towards a Trusted, Resilient, and Peaceful Internet+

208On the Internet, the universe: There’s a lot to this trend that’s beyond the scope of the book. Jean M. Twenge, W. Keith Campbell, and Nathan T. Carter (9 Sep 2014), “Declines in trust in others and confidence in institutions among American adults and late adolescents, 1972–2012,” Psychological Science 25, no. 10, http://journals.sagepub.com/doi/abs/10.1177/0956797614545133. David Halpern (12 Nov 2015), “Social trust is one of the most important measures that most people have never heard of—and it’s moving,” Behavioural Insights Team, http://www.behaviouralinsights.co.uk/uncategorized/social-trust-is-one-of-the-most-important-measures-that-most-people-have-never-heard-of-and-its-moving. Eric D. Gould and Alexander Hijzen (22 Aug 2016), “Growing apart, losing trust? The impact of inequality on social capital,” International Monetary Fund Working Paper No. 16/176, https://www.imf.org/en/Publications/WP/Issues/2016/12/31/Growing-Apart-Losing-Trust-The-Impact-of-Inequality-on-Social-Capital-44197. Laura D’Olimpio (25 Oct 2016), “Fear, trust, and the social contract: What’s lost in a society on permanent alert,” ABC News, http://www.abc.net.au/news/2016-10-26/fear-trust—social-contract-society-on-permanent-alert/7959304.

208A 2017 survey illustrated that 70%: Kenneth Olmstead (27 Sep 2017), “Most Americans think the government could be monitoring their phone calls and emails,” Pew Research Center, http://www.pewresearch.org/fact-tank/2017/09/27/most-americans-think-the-government-could-be-monitoring-their-phone-calls-and-emails.

208“The success of the digital economy”: Thomas E. Donilon et al. (1 Dec 2016), “Report on securing and growing the digital economy,” Commission on Enhancing National Cybersecurity, https://www.nist.gov/sites/default/files/documents/2016/12/02/cybersecurity-commission-report-final-post.pdf.

209In 2011, I published Liars and Outliers: Bruce Schneier (2012), Liars and Outliers: Enabling the Trust That Society Needs to Thrive, Wiley, http://www.wiley.com/WileyCDA/WileyTitle/productCd-1118143302.html.

209Surveillance capitalism is not sustainable: Tim Hwang and Adi Kamdar (9 Oct 2013), “The theory of peak advertising and the future of the web,” version 1, Working Paper, Nesson Center for Internet Geophysics, http://peakads.org/images/Peak_Ads.pdf.

210In particular, complex systems: Charles Perrow (1999), Normal Accidents: Living with High-Risk Technologies, Princeton University Press, https://www.amazon.com/Normal-Accidents-Living-High-Risk-Technologies/dp/0691004129. Charles Perrow (1 Sep 1999), “Organizing to reduce the vulnerabilities of complexity,” Journal of Contingencies and Crisis Management 7, no. 3, http://onlinelibrary.wiley.com/doi/10.1111/1468-5973.00108/full.

211“Resilience is the capacity to cope with”: Aaron B. Wildavsky (1988), Searching for Safety, Transaction Publishers, https://books.google.com/books?id=rp6U8JsPlM0C.

211I have been talking about resilience: Bruce Schneier (14 Nov 2001), “Resilient security and the Internet,” ICANN Community Meeting on Security and Stability of the Internet Naming and Address Allocation Systems, Los Angeles, California, http://cyber.law.harvard.edu/icann/mdr2001/archive/pres/schneier.html. Black Hat (accessed 24 Apr 2018), “Speakers,” Black Hat Briefings ’01, July 11–12 Las Vegas, https://www.blackhat.com/html/bh-usa-01/bh-usa-01-speakers.html.

211“Good security systems are resilient”: Bruce Schneier (2006), Beyond Fear: Thinking Sensibly about Security in an Uncertain World, Springer, https://books.google.com/books?id=btgLBwAAQBAJ&pg=PA120.

211In 2012, the World Economic Forum: World Economic Forum (7 Jun 2012), “Risk and responsibility in a hyperconnected world: Pathways to global cyber resilience,” https://www.weforum.org/reports/risk-and-responsibility-hyperconnected-world-pathways-global-cyber-resilience.

211“The most resilient societies will likely be”: Gregory Treverton et al. (5 Jan 2017), “Global trends: Paradox of progress,” NIC 2017-001, National Intelligence Council, https://www.dni.gov/files/documents/nic/GT-Full-Report.pdf.

213A 2017 report by the New York Cyber Task Force: Jason Healey (28 Sep 2017), “Building a defensible cyberspace: Report of the New York Cyber Task Force,” Columbia School of International and Public Affairs, http://globalpolicy.columbia.edu/sites/default/files/nyctf_2017-09-28_report.pdf.

213International laws regarding pollution: Jason Healey and Hannah Pitts (1 Oct 2012), “Applying international environmental legal norms to cyber statecraft,” I/S: A Journal of Law and Policy for the Information Society 8, no. 2, http://moritzlaw.osu.edu/students/groups/is/files/2012/02/6.Healey.Pitts_.pdf.

213“Cyber peace is not the absence of attacks”: Scott J. Shackelford (1 Jan 2016), Managing Cyber Attacks in International Law, Business, and Relations: In Search of Cyber Peace, Cambridge University Press, https://books.google.com/books /?id=_q2BAwAAQBAJ.

213“cyber peace must be grounded in”: Heather M. Roff (24 Feb 2016), “Cyber peace: Cybersecurity through the lens of positive peace,” New America Foundation, https://static.newamerica.org/attachments/12554-cyber-peace/FOR%20PRINTING-Cyber_Peace_Roff.2fbbb0b16b69482e8b6312937607ad66.pdf.

Conclusion: Bring Technology and Policy Together

217“A technology that can give you everything”: Dan Geer (6 Aug 2007), “Measuring security,” USENIX Security Symposium, http://geer.tinho.net/measuringsecurity.tutorial.pdf.

218Yet when Deckard—Harrison Ford’s character: Economist Tim Harford recently pointed this out. Tim Harford (8 Jul 2017), “What we get wrong about technology,” FT Magazine, http://timharford.com/2017/08/what-we-get-wrong-about-technology.

218We also tend to overestimate: This “law” was coined by Stanford University computer scientist Roy Amara, who also directs the Institute for the Future. Matt Ridley (12 Nov 2017), “Amara’s law,” Matt Ridley Online, http://www.rationaloptimist.com/blog/amaras-law.

219This will lessen the relative advantages: Bruce Schneier (Mar/Apr 2018), “Artificial intelligence and the attack/defense balance,” IEEE Security & Privacy, https://www.schneier.com/essays/archives/2018/03/artificial_intellige.html.

220“Politics is the art of the possible”: Wikiquote (accessed 8 May 2018), “Otto von Bismarck,” https://en.wikiquote.org/wiki/Otto_von_Bismarck.

220“the lawyers and engineers whose arguments”: Nicholas Bohm, Ian Brown, and Brian Gladman (31 Oct 2000), “Electronic commerce: Who carries the risk of fraud?” Journal of Information, Law & Technology 2000, no. 3, http://www.ernest.net/writing/FraudRiskAllocation.pdf.

221“I think much of the problem we face today”: Toomas Hendrik Ilves (31 Jan 2014), “Rebooting trust? Freedom vs. security in cyberspace,” Office of the President, Republic of Estonia, https://vp2006-2016.president.ee/en/official-duties/speeches/9796-qrebooting-trust-freedom-vs-security-in-cyberspaceq.

222“Well the laws of Australia prevail”: James Titcomb (14 Jul 2017), “Malcolm Turnbull says laws of Australia trump laws of mathematics as tech giants told to hand over encrypted messages,” Telegraph, http://www.telegraph.co.uk/technology/2017/07/14/malcolm-turnbull-says-laws-australia-trump-laws-mathematics.

222She’s probably the best analyst: Here, Sweeney describes research that led to the de-anonymization of medical data belonging to then–Massachusetts governor William Weld. Latanya Sweeney (8 Jan 2001), “Computational disclosure control: A primer on data privacy protection,” http://groups.csail.mit.edu/mac/classes/6.805/articles/privacy/sweeney-thesis-draft.pdf.

222She has also exposed bias in Internet algorithms: Here’s one paper: Latanya Sweeney (Jan 2013), “Discrimination in online ad delivery,” Communications of the Association of Computing Machinery 56, no. 5, https://arxiv.org/abs/1301.6822.

222has made significant contributions to privacy technologies: Latanya Sweeney (2002), “k-Anonymity: A model for protecting privacy,” International Journal on Uncertainty, Fuzziness and Knowledge-Based Systems 10, no. 5, https://dataprivacylab.org/dataprivacy/projects/kanonymity/kanonymity.html.

223writing books and articles: This is her latest book: Susan Landau (2017), Listening In: Cybersecurity in an Insecure Age, Yale University Press, https://books.google.com/books?id=QZ47DwAAQBAJ.

223testifying before Congress on the topic: This is her latest testimony: Susan Landau (1 Mar 2016), “Testimony for House Judiciary Committee hearing on ‘The encryption tightrope: Balancing Americans’ security and privacy,’” https://judiciary.house.gov/wp-content/uploads/2016/02/Landau-Written-Testimony.pdf.

223He’s probably best known: Here’s one paper: Ariel Feldman, J. Alex Halderman, and Edward W. Felten (13 Sep 2006), “Security analysis of the Diebold AccuVote-TS voting machine,” 2007 USENIX/ACCURATE Electronic Voting Technology Workshop, https://citp.princeton.edu/research/voting.

223Through its Speech, Privacy, and Technology project: American Civil Liberties Union (accessed 24 Apr 2018), “About the ACLU’s Project on Speech, Privacy, and Technology,” https://www.aclu.org/other/about-aclus-project-speech-privacy-and-technology.

224Many universities now offer: A discussion of this trend, and a good list of programs, can be found here: Alan Davidson, Maria White, and Alex Fiorille (26 Feb 2018), “Building the future: Educating tomorrow’s leaders in an era of rapid technological change,” New America/Freedman Consulting.

224Internet Policy Research Initiative: Internet Policy Research Initiative (accessed 24 Mar 2018), Massachusetts Institute of Technology, https://internetpolicy.mit.edu.

224Center on Privacy & Technology: Georgetown Law (accessed 24 Apr 2018), “Center on Privacy & Technology,” https://www.law.georgetown.edu/academics/centers-institutes/privacy-technology.

224Digital HKS program: Digital HKS (accessed 24 Apr 2018), Harvard Kennedy School, https://projects.iq.harvard.edu/digitalhks/home.

224We need to create a viable career path: NetGain is a consortium of large foundations that are trying to make this happen. Tom Freedman et al. (10 Feb 2016), “A pivotal moment: Developing a new generation of technologists for the public interest,” NetGain Partnership, https://www.netgainpartnership.org/resources/2018/1/26/a-pivotal-moment.

224A good model can be found: Freedman Consulting (3 Mar 2006), “Here to there: Lessons from public interest law,” unpublished memo.

224In the late 1960s, there were 92: Robert L. Graham (1977), “Balancing the scales of justice: Financing public interest law in America,” Loyola University Chicago Law Journal 8, no. 3, http://lawecommons.luc.edu/luclj/vol8/iss3/10.

224by 2000, there were over a thousand: Laura Beth Nielsen and Catherine R. Albiston (1 Jan 2005), “The organization of public interest practice: 1975–2004,” North Carolina Law Review 84, http://scholarship.law.berkeley.edu/facpubs/1618.

224Today, 20% of the graduating class: Indeed, some consider this number to be embarrassingly low. Pete Davis (26 Oct 2017), “Our bicentennial crisis: A call to action for Harvard Law School’s public interest mission,” Harvard Law Record, http://hlrecord.org/wp-content/uploads/2017/10/OurBicentennialCrisis.pdf.

up to Click Here to Kill Everybody

Sidebar photo of Bruce Schneier by Joe MacInnis.