Data and Goliath Is Finished

Data and Goliath: The Hidden Battles to Collect Your Data and Control Your World is finished. I submitted it to my publisher, Norton, this morning. In a few weeks, I'll get the copyedited manuscript back, and a few weeks after that, it'll go into production. Stacks of printed books will come out the other end in February, and the book will be published on March 9. There's already an Amazon page, but it's still pretty preliminary. And I expect the price to go down.

Books are both a meandering and clarifying process for me, and I figure out what I'm writing about as I write about it. Data and Goliath started out being about security and power in cyberspace, and ended up being about digital surveillance and what to do about it.

This is the table of contents:

Part 1: The World We're Creating
Chapter 1: Data as a By-Product of Computing
Chapter 2: Data as Surveillance
Chapter 3: Analyzing our Data
Chapter 4: The Business of Surveillance
Chapter 5: Government Surveillance and Control
Chapter 6: Consolidation of Institutional Surveillance
Part 2: What's at Stake
Chapter 7: Political Liberty and Justice
Chapter 8: Commercial Fairness and Equality
Chapter 9: Business Competitiveness
Chapter 10: Privacy
Chapter 11: Security
Part 3: What to Do About It
Chapter 12: Principles
Chapter 13: Solutions for Government
Chapter 14: Solutions for Corporations
Chapter 15: Solutions for the Rest of Us
Chapter 16: Social Norms and the Big Data Trade-off

Fundamentally, the issues surrounding mass surveillance are tensions between group interest and self-interest, a topic I covered in depth in Liars and Outliers. We're promised great benefits if we allow all of our data to be collected in one place; at the same time, it can be incredibly personal. I see this tension playing out in many areas: location data, social graphs, medical data, search histories. Figuring out the proper balances between group and self-interests, and ensuring that those balances are maintained, is the fundamental issue of the information age. It's how we are going to be judged by our descendants fifty years from now.

Anyway, the book is done and at the publisher. I'm happy with it; the manuscript is so tight you can bounce a quarter off of it. This is a complicated topic, and I think I distilled it down into 80,000 words that are both understandable by the lay reader and interesting to the policy wonk or technical geek. It's also an important topic, and I hope the book becomes a flash point for discussion and debate.

But that's not for another five months. You might think that's a long time, but in publishing that's incredibly fast. I convinced Norton to go with this schedule by stressing that the book becomes less timely every second it's not published. (An exaggeration, I know, but they bought it.) Now I just hope that nothing major happens between now and then to render the book obsolete.

For now, I want to get back to writing shorter pieces. Writing a book can be all-consuming, and I generally don't have time for anything else. Look at my essays. Last year, I wrote 59 essays. This year so far: 17. That's an effect of writing the book. Now that it's done, expect more essays on news websites and longer posts on this blog. It'll be good to be thinking about something else for a change.

If anyone works for a publication, and wants to write a review, conduct an interview, publish an excerpt, or otherwise help me get the word out about the book, please e-mail me and I will pass you on to Norton's publicity department. I think this book has a real chance of breaking out of my normal security market.

Posted on October 7, 2014 at 6:36 AM • 52 Comments


Bruce SchneierOctober 7, 2014 7:23 AM

Books are fun when you're start. It's all exciting and new and everything is full of promise. Book are also fun when you're finishing. It's all there and good and the end is in sight. It's the middle, when you've been writing since forever and still have forever to go, that's hard.

Z.LozinskiOctober 7, 2014 7:31 AM

Well done. The table of contents looks interesting, and I like the intertwining of the business and government angles. The economic incentives (cost of storage, cost of compute) that created "big data" apply to both groups.

You must be doing something right, these days I order your books as soon as they are announced.

AlanSOctober 7, 2014 7:48 AM

Congratulations. I wonder if the power book is still to be written? There are a lot of chapters on business, corporations and government in your table of contents. I find that my reading about issues of security, surveillance and privacy increasingly leads me into writings on governmentality, liberalism, and economic ideology.

EricOctober 7, 2014 8:03 AM

Congratulations! Glad you made the topic switch to privacy as far too many still feel that as long as they have nothing to hide, there is no reason for concern. The TOC is good, and its' comprehensive topic coverage will help non-professionals and the public alike better understand the issues. E.

T!MOctober 7, 2014 8:14 AM

Congratulations and thanks for writing it!

I will order it offline at my favourite bookstore, so I will not feed the virtual profile of me through amazon :)

It's good to see that you give your insight view within paper books, that makes it more conclusive to me, that you mean what you write about security, awareness, ...

B. BrewskiOctober 7, 2014 8:16 AM

Congratulations on your book, Bruce! Compared to most of us I think you are a very prolific writer:-) Do you have a set amount of time each day for writing?

Also, a question about Chapter 15: Solutions for the Rest of Us. What kind of solutions do you discuss? Is it
A. technical (e.g. what technologies that can be used, or devices that could be built, like some data diode type?)
B. behavioral (e.g. what to do and what not to do, to reduce the data footprint)
C. political (e.g. putting pressure on those in the government etc)
D. something else
E. all of the above?


J-P. ChavanneOctober 7, 2014 8:53 AM

Looking forward to reading it and especially the chapter 13... Are there real solutions out there?

RGP SecurityOctober 7, 2014 9:59 AM

Mr. Schneier,


You are in the front on these important issues. Those chapter names are tantalizing! Chapter 6: Consolidation of Institutional Surveillance. How interesting!


Bruce SchneierOctober 7, 2014 10:44 AM

@ B. Brewski

Chapter 15 is really all of the above, with an emphasis on technical and behavioral solutions. Chapters 13 and 14 are all about legal solutions, and Chapter 16 is about social solutions.

Bruce SchneierOctober 7, 2014 11:01 AM

"Looking forward to reading it and especially the chapter 13... Are there real solutions out there?"

I think so. I don't believe that we have come to the end of the democratic experiment.

KarasevokOctober 7, 2014 11:56 AM

Your book must be very thoughtful, it has been placed on the religious books section on Finnish online bookstore. Are you perhaps considering on starting your own cult? Anyway, I'm really looking forward to book! (and the cult too)

BoppingAroundOctober 7, 2014 12:06 PM

Just curious: what do you use for writing books apart from ideas? Pen and paper? A simple text editor? Something like MS Word? A specially trained squid to write for you? :-)

Clive RobinsonOctober 7, 2014 1:04 PM

@ Bruce,

I don't believe that we have come to the end of the democratic experiment.

Err I hate to disagree with you but the US was never a "democracy", at best a "Representational Democracy" which although it contains the word "democracy" in it is not in any way a democracy. You have to go to one or two of the smaller Cantons in Switzerland to find a democratic process in action and one or two other small places around the world.

So as the US system was broadly based on the English sysytem which is also not democratic, I actually look forward to the start of a "democratic experiment" in both the UK and US with some enthusiasm.

Seriously though it should be possible for "technology" to give us a true democracy, but I see three basic problems,

1, Getting the technology right.
2, Getting the politicos to accept they are nolonger needed.
3, Getting the citizens out of their "lazy ways" that gives a need for politicos.

It's the lazy, ill informed citizenry that is the biggest hurdle, they have to want not just to get rid of "Monkey in a suit" politicos and actually take responsability and time to be sufficiently well informed to make decisions on real issues not what monkey spends most for their vote and decides for them ( or more correctly favours what is most lucrative for the politico not the voter ).

WayneOctober 7, 2014 1:17 PM

Congrats, Bruce! I will order it for Kindle as always. Now that you have finished the book I am looking forward to reading some more of your essays. It has been quite a drought around here without your input.

DanielOctober 7, 2014 1:18 PM

"Books are fun when you're start. It's all exciting and new and everything is full of promise. Book are also fun when you're finishing. It's all there and good and the end is in sight. It's the middle, when you've been writing since forever and still have forever to go, that's hard."

In my middle age, that seems like a pretty darn good summary of life itself.

Clive RobinsonOctober 7, 2014 1:25 PM

@ Bruce,

On a lighter note, you are no doubt aware of the old saw of "Everybody has a book in them" there have been one or two humorous rejoinders to that saying, perhaps you have one or two that spring to mind, having "given forth" to so many books.

My favourite was from a little story Douglas Adams once told me, he was forever missing deadlines for one reason or another and "Last Chance to See" was especialy problematic. He apparently got into an animated phone call with his publishing bod one day and the line was trotted out, to which the rejoinder came back from the exasperated bod at the publishers "But Douglas even an elephant gestates for only eighteen months, so what is holding you back?". Despite asking Douglas declined to tell me what he said in return, other than it caused the publishing bod "to say focefully and loudly to 'Get on with it', but not quite in those words"...

Any way I look forward to reading it when it touches down on UK shores in dead tree format.

WaelOctober 7, 2014 1:35 PM

I am waiting to read this chapter:
"Chapter 12: Principles" -- No surprises there for the usual suspects.

Strange that you still chose this title given what you thought at the time...

"My absolute favorite is Data and Goliath, but there's a problem. Malcolm Gladwell recently published a book with the title of David and Goliath. Normally I wouldn't care, but I published my Liars and Outliers soon after Gladwell published Outliers. Both similarities are coincidences, but aping him twice feels like a bit much"
Then again, Option 4, I listed, was:
4- Gladwell, Sorry dude... low on entropy: Data and Goliath

J. Mansfield-JonesOctober 7, 2014 2:03 PM

This post is horribly timely: I am reading accounts that Adobe Digital Editions 4 phones home with reader library data in cleartext. Ars Technica and The Digital Reader both have descriptions. The dump going back to Adobe apparently includes titles and other information about whatever epubs the victim has - DRM-encumbered or not.

JoshOctober 7, 2014 2:56 PM

@J. Mansfield-Jones

I believe the Amazon Kindle does the same thing. I've read books on my Kindle Fire - that weren't purchased from Amazon - then a few days later Amazon will send me an email asking me to rate that book on their website.

SofakinbdOctober 7, 2014 3:19 PM

Congratulations Bruce! The book should be excellent, just as Liars and Outliers was.

- Sofa

Bruce SchneierOctober 7, 2014 5:05 PM

"Strange that you still chose this title given what you thought at the time..."

After I wrote that, Gladwell e-mailed me out of the blue and told me he thought I should use the title. So I went for it.

He's blurbing the book.

Bruce SchneierOctober 7, 2014 5:25 PM

"Just curious: what do you use for writing books apart from ideas? Pen and paper? A simple text editor? Something like MS Word? A specially trained squid to write for you? :-)"

I use MS Word, a single file for everything. It works, and I don't want to change and try something new -- even thought I'm sure there are many better writing tools out there, like Scrivener.

SkepticalOctober 7, 2014 8:39 PM

This sounds like a really interesting book, and I'm looking forward to reading it.

Congratulations on finishing it - and on the lightning speed with which it is being published!

chris lOctober 7, 2014 9:20 PM


What he doesn't say is that what he types into the single MS Word file is the hex sequence that comprises the encrypted version of the book. All his editing is done in place on the encrypted version (which is why it ended up not going out to blog readers to review). There is no plaintext version until the publisher decrypts it with his public key.

Nick POctober 7, 2014 10:06 PM

Good job. Look forward to reading the book. Also look forward to reading (more of) Liars and Outliars. Personal, national, and world events have kept me perpetually distracted from your other book. What I read was good. Hope I manage to get into a situation where I can just chill, read them both, and do constructive commentary. Of course, I also wouldn't mind being in a situation where my security engineering work took off so massively I still didn't have time for books. Doubt it, so I'm willing to settle for just enjoying your work. ;)

TobiOctober 8, 2014 3:33 AM

Congratulations to your new book! I think i found my next reading.
Btw: The book link on the Norton site is dead.

John Galt IIIOctober 8, 2014 8:59 AM

Your book would be rather timely if it came out today
A hacker acquaintance of mine has tipped me to a huge security and privacy violation on the part of Adobe. That anonymous acquaintance was examining Adobe’s DRm for educational purposes when they noticed that Digital Editions 4, the newest version of Adobe’s Epub app, seemed to be sending an awful lot of data to Adobe’s servers.
Meanwhile, we have an appalling new look at what law enforcement does once it gets inside your smart phone. A woman in Albany is suing DEA because — after she permitted DEA to conduct a consensual search of her phone — DEA then took photos obtained during the search, including one of her wearing only underwear, and made a fake Facebook page for her with them. They even sent a friend request to a fugitive and accepted other friend requests. They also posted pictures of her son and niece, on a site intended to lure those involved in the drug trade.
And they consider this a legitimate law enforcement activity!

WaelOctober 8, 2014 11:23 PM

@ Clive Robinson,
You knew Douglas Adams personally? How was he like? Then again, not knowing who you are, I wouldn't be surprised if he was privilaged knowing you ;)

AnuraOctober 9, 2014 7:00 PM

@chris l

I was being serious about LaTeX; it's nice for writing because you don't have to worry about formatting so much as you have to worry about what the content is. So you can write a book, and then find a template you like. On top of that, some of Bruce's work in the past has been at least somewhat math/reference heavy, which LaTeX shines at.

chris lOctober 10, 2014 2:06 AM


It was too good of a setup to pass up.

I know many people still use LaTeX for writing technical documents, though I've always used MSWord since the mid 90's. Equation editors for word were already pretty good then (though they still have portability issues, even among MSWord users)-- a friend of mine started out writing her finite temperature field theory (tons of complicated equations) thesis in LaTeX and got annoyed with having to "debug" it. She got ahold of one of the equation editor plugins that could export to both LaTeX and Word and knocked out all the equations in a couple days, with a minimum of debugging. I think she pretty much does everything in Word now, too.

Things are probably going to migrate to HTML5/ePub3 in the long run, since eReaders are starting to support them as a way to do technical docs, and since they're basically XML, they can be reformatted at will as well.

LloydOctober 13, 2014 8:44 AM

@Bruce Schneier:

Have you ever held any book signing events?

Do you have any plans for touring for promotion of the book?

I feel this would be a great idea and you would no doubt catch the eye of the press in doing so.

DROctober 14, 2014 7:52 AM

Congratulations Bruce. Looking forward to buy it. Although I'm a bit disappointed as I volunteered to review the draft and never got it.

SynonymousOctober 14, 2014 1:07 PM

Bruce, you need to change the book cover image under "Latest Book" (displayed in the right margin under "Blog Archives").

Paul BayJanuary 15, 2015 1:53 PM

I've ordered copies of Data and Goliath from the Hennepin County Library. I'm encouraging my Anoka, MN book club to pick it for the next book this March. Excellent topic. I've read many of your other books, and blogs. Do you plan on anymore panels at the Mpls Science Fiction convention scenes? (MiniCon and Convergence?)

WaelJanuary 28, 2016 8:34 PM

@Clive Robinson,

Re: Doglas Adams...

Did he ever share with you why he picked 42? I think I know the secret of his choice answer to the ultimate question of life :)

Nick PJanuary 28, 2016 9:43 PM

@ Wael

It was a shitty password recovery question and answer. That's all I'll tell you.

Clive RobinsonJanuary 29, 2016 9:22 AM

@ Wael,

No Douglas did not tell me, i knew better than to ask, he'd got beyond telling people different stories about it. So half jokingly I mentioned to him that "42 is a happy number" and he could tell people that. Which Steven Fry topped by telling him it was also "a super-multiperfect number".

The number of different stories Douglas told about "42" was legion, the thing is it's also a dull sounding number, so it is funny after "the life the universe and everything" build up with the philospher gags to just drop it in as every one just goes "what?"...

WaelJanuary 29, 2016 9:52 PM

@Clive Robinson,

dull sounding number

That's just "racist"! All numbers look alike to me :)

ianfJuly 25, 2016 2:20 PM

@ MODERATOR :: an octet of same old SPAM.

    Would it be too much for you to write a simple cumulative URL-extractor from manually reported spams, then pipe successive posted URLs through that file, and automagically redirect these posts to a SPAM garbage file? It seems this SPAM robot has primed itself on this blog, and will be reposting this stuff forever. Rollover the file once a fortnight or so?

[links deleted by moderator]

ModeratorJuly 25, 2016 3:42 PM

@ianf Thanks for calling out this particular spammer. Some extra defenses are now in place.

Leave a comment

Allowed HTML: <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre>

Photo of Bruce Schneier by Per Ervland.

Schneier on Security is a personal website. Opinions expressed are not necessarily those of IBM Resilient.