Schneier on Security
A blog covering security and security technology.
« The Future of Ephemeral Conversation |
| Government Can Determine Location of Cell Phones without Telco Help »
November 25, 2008
Here Comes Everybody Review
In 1937, Ronald Coase answered one of the most perplexing questions in economics: if markets are so great, why do organizations exist? Why don't people just buy and sell their own services in a market instead? Coase, who won the 1991 Nobel Prize in Economics, answered the question by noting a market's transaction costs: buyers and sellers need to find one another, then reach agreement, and so on. The Coase theorem implies that if these transaction costs are low enough, direct markets of individuals make a whole lot of sense. But if they are too high, it makes more sense to get the job done by an organization that hires people.
Economists have long understood the corollary concept of Coase's ceiling, a point above which organizations collapse under their own weight -- where hiring someone, however competent, means more work for everyone else than the new hire contributes. Software projects often bump their heads against Coase's ceiling: recall Frederick P. Brooks Jr.'s seminal study, The Mythical Man-Month (Addison-Wesley, 1975), which showed how adding another person onto a project can slow progress and increase errors.
What's new is something consultant and social technologist Clay Shirky calls "Coase's Floor," below which we find projects and activities that aren't worth their organizational costs -- things so esoteric, so frivolous, so nonsensical, or just so thoroughly unimportant that no organization, large or small, would ever bother with them. Things that you shake your head at when you see them and think, "That's ridiculous."
Sounds a lot like the Internet, doesn't it? And that's precisely Shirky's point. His new book, Here Comes Everybody: The Power of Organizing Without Organizations, explores a world where organizational costs are close to zero and where ad hoc, loosely connected groups of unpaid amateurs can create an encyclopedia larger than the Britannica and a computer operating system to challenge Microsoft's.
Shirky teaches at New York University's Interactive Telecommunications Program, but this is no academic book. Sacrificing rigor for readability, Here Comes Everybody is an entertaining as well as informative romp through some of the Internet's signal moments -- the Howard Dean phenomenon, Belarusian protests organized on LiveJournal, the lost cellphone of a woman named Ivanna, Meetup.com, flash mobs, Twitter, and more -- which Shirky uses to illustrate his points.
The book is filled with bits of insight and common sense, explaining why young people take better advantage of social tools, how the Internet affects social change, and how most Internet discourse falls somewhere between dinnertime conversation and publishing.
Shirky notes that "most user-generated content isn't 'content' at all, in the sense of being created for general consumption, any more than a phone call between you and a sibling is 'family-generated content.' Most of what gets created on any given day is just the ordinary stuff of life -- gossip, little updates, thinking out loud -- but now it's done in the same medium as professionally produced material. Unlike professionally produced material, however, Internet content can be organized after the fact."
No one coordinates Flickr's 6 million to 8 million users. Yet Flickr had the first photos from the 2005 London Transport bombings, beating the traditional news media. Why? People with cellphone cameras uploaded their photos to Flickr. They coordinated themselves using tools that Flickr provides. This is the sort of impromptu organization the Internet is ideally suited for. Shirky explains how these moments are harbingers of a future that can self-organize without formal hierarchies.
These nonorganizations allow for contributions from a wider group of people. A newspaper has to pay someone to take photos; it can't be bothered to hire someone to stand around London underground stations waiting for a major event. Similarly, Microsoft has to pay a programmer full time, and Encyclopedia Britannica has to pay someone to write articles. But Flickr can make use of a person with just one photo to contribute, Linux can harness the work of a programmer with little time, and Wikipedia benefits if someone corrects just a single typo. These aggregations of millions of actions that were previously below the Coasean floor have enormous potential.
But a flash mob is still a mob. In a world where the Coasean floor is at ground level, all sorts of organizations appear, including ones you might not like: violent political organizations, hate groups, Holocaust deniers, and so on. (Shirky's discussion of teen anorexia support groups makes for very disturbing reading.) This has considerable implications for security, both online and off.
We never realized how much our security could be attributed to distance and inconvenience -- how difficult it is to recruit, organize, coordinate, and communicate without formal organizations. That inadvertent measure of security is now gone. Bad guys, from hacker groups to terrorist groups, will use the same ad hoc organizational technologies that the rest of us do. And while there has been some success in closing down individual Web pages, discussion groups, and blogs, these are just stopgap measures.
In the end, a virtual community is still a community, and it needs to be treated as such. And just as the best way to keep a neighborhood safe is for a policeman to walk around it, the best way to keep a virtual community safe is to have a virtual police presence.
Crime isn't the only danger; there is also isolation. If people can segregate themselves in ever-increasingly specialized groups, then they're less likely to be exposed to alternative ideas. We see a mild form of this in the current political trend of rival political parties having their own news sources, their own narratives, and their own facts. Increased radicalization is another danger lurking below the Coasean floor.
There's no going back, though. We've all figured out that the Internet makes freedom of speech a much harder right to take away. As Shirky demonstrates, Web 2.0 is having the same effect on freedom of assembly. The consequences of this won't be fully seen for years.
Here Comes Everybody covers some of the same ground as Yochai Benkler's Wealth of Networks. But when I had to explain to one of my corporate attorneys how the Internet has changed the nature of public discourse, Shirky's book is the one I recommended.
This essay previously appeared in IEEE Spectrum.
EDITED TO ADD (12/13): Interesting Clay Shirky podcast.
Posted on November 25, 2008 at 7:39 AM
• 30 Comments
To receive these entries once a month by e-mail, sign up for the Crypto-Gram Newsletter.
I'm not convinced that Coase's floor is actually a floor for organisations in the traditional sense of the word: there's corporate stupidity to take into account.
For example, my company recently spent an amazing amount of effort coordinating workshops around the country so we could all be consulted about what our mission statement should be.
I think the theorem is being extended a bit much; I don't conceive a steel furnace in every back yard or an airplane factory.
It applies to simpler things - agriculture particularly.
@ sooth sayer,
"I don't conceive a steel furnace in every back yard or an airplane factory.
Sorry you are most definatly wrong, you need to think modularization.
During the second world war one of the problems with bombing Japan was the lack of centralised organisations to use as valid industrial targets.
Basicaly just about every home had a small part in the production of an item for the war effort often these where small parts for aircraft etc.
It was only where these modular parts were finaly put together was a target worth while.
And even here the Japanese had an advantage. Most of their buildings where likewise modular and made of wooden frames with light panaling. Therefor small aircraft could be put together in different places at different times. It was only the final assembly of putting the wings on to fly it away need be done at a place with sufficient space for it to take off.
Software is like this to the n'th degree. However piece work at home has been a viable production method for a large part of the last century in most countries.
The problem is that when something becomes very useful for productivity or making money, then others what to get a piece of it, make it expensive for you to use it, or take it away completely. As for the internet, costs to use it will likely exceed its economic value over time. For example, self publishing of books means high data volume, which is already being capped or charged very high rates. Software downloading would also be a target. And fees for copyrighted or patented models for using the internet will certainly limit internet usefulness as well.
Bruce's summary of Shirky's notion of Coase's floor is misleading unless you're already familiar with Coase's theory of the firm.
Basically, Coase asked this question:
"We (as economists) believe that centrally-planned command economies are always less efficient at producing some good (let's say a car) than a free market would be.
But, if we look at who makes cars, it's a handful of large corporations, each of which is (internally) a centrally-planned command economy (especially at that time, when something like Ford was a single, vertically-integrated company doing just about everything from mining raw materials to making parts to assembling finished products).
If free markets were more efficient, you'd think that some "free market" approach to making cars would be beating the "large conglomerate". In this case, a "free market" approach would be like having each step of the car making process done by distinct companies (and lots of different firms competing at each step).
So, why haven't the large conglomerates been beaten out by a free-market approach?"
Coase answered the question by introducing the idea of "transaction costs". At the time it was a big deal, even if by now the idea is so familiar it seems really boring and unoriginal.
Essentially, a large firm can out-compete the "free market" approach to making a particular good whenever the "transaction costs" of doing it the "free market" way exceed the potential cost savings from the "free market" approach.
"Transaction costs" in this case mean things like: "cost to find out who's selling what", "cost to find out what I need to buy", "cost to track down sellers of each item and negotiate a purchase", and so forth.
Where all this leads is to Coase's theory of the firm: you'll only see large organizations develop when both of the following points are true:
(a) there's a product P the organization can make whose value exceeds the cost of establishing and maintaining an organization
(b) the cost of creating P in a "free market" approach -- ie, assembling P piecemeal from unaffiliated buyers and sellers -- is sufficiently high (for an individual who might want P) that P is not currently being produced in a "free market" fashion
In plain english, (a) and (b) reduce to: there has to be something (P) that some people want, and it has to be the case that: (b) it's a big pain for someone who wants P to go assemble it on their
own, but (a) it's something that could be produced at an economical price with the right organization in place.
Given the above, an across-the-board reduction in transaction costs has two ways it could impact institutional structure:
- it could bring new goods (P') to market, b/c previously P' was too much of a hassle for any individual to make, and the value of P' was also too high for it to be worth establishing a formal organization.
- it could eliminate commercial viability of existing products P, b/c now the reduced transaction costs mean that an interested individual can go assemble their own (for less than what the organization-produced P can be sold for).
Shirky tends to blur the above two mechanics in his book. When he's talking about the Coasean floor he is usually referring to the first dynamic: super-low-cost organizational tools means that stuff like community parade photos can now be aggregated ("produced"), whereas without those organizing tools the value of stuff like those photos would've not been enough to justify the effort of aggregating them.
He also implicitly uses the second dynamic to describe why, eg, newspapers probably don't have a long future ahead of them, but he never gives this behavior a catchy name and moreover doesn't make a real effort to distinguish it from the related-but-very-different "coasean floor" phenomena.
It's still a good book full of sparkling ideas on most every page.
It is a small piece of irony that Nobel (an individual) never created a prize in economics.
It was an organization (the Sveriges Riksbank) that created "The Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel".
For some reason reading your post made me think of what might be the ideal illustration of Coase's work, personal computers.
When computers first came out, they were esoteric. It required negotiating with huge numbers of suppliers to get parts, and parts were things like individual chips, or unprinted circuit boards. This is where the firm had an advantage, exactly as you describe.
Then computers became more modular, and it became feasible for people to assemble them on their own. This lead to a proliferation of small time computer shops, producing at lower cost than the large firms.
But as the cost of computers goes down, the savings the small shops have on the overhead side are outweighed by their inability to get economies of scale (i.e. employees spending most of the day sitting around waiting for a customer).
@wut Best comment I have seen in a long time. Kudos also to Bruce for having quality commenters!
Well, corporations also exist because they accumulate capital, and their rate of accumulating it is higher than individuals or loose teams can do. Now, there are VC firms, who intend to provide capital to risky new enterprises, capital which (apparently) banks cannot provide. (Although these days, you wonder why they didn't.) In some respects, the capital accumulation power of corporations may be more important than the products they produce. No doubt, they need products to justify accumulating the capital.
@wut - thanks for the most enlightening and educational precis of Coase's economic theory. That was really helpful!
If you liked "Here Comes Everybody", you might want to check out "Everything is Miscellaneous", by Weinberger.
I wasn't entirely enamored of it (it seems to miss out on the bigger questions), but it's a quick and easy read and he's got some interesting ideas in there.
While the usefulness of Wikipedia can be used to argue the point Clay Shirky is making, I'd hardly describe the maintainers of Linux as amateurs.
"I'd hardly describe the maintainers of Linux as amateurs."
In England there is a saying,
"Every man should have a garden shed"
Which basicaly means he should have a place away from close society (family etc) where he can "potter about" that is have a hobby.
For some it is fixing cars and motor bikes others clocks, radios, televisions, computers etc. Others it is engineering such as making fully functional scale model steam trains or boats etc.
All in all these are activities that might otherwise be a proffession, and usually the level of skill shown is considerably greater than a most practitioners in the field who earn their living by it.
So why should software development be any different...
Economics has not realy been applied to the individual as it just does not fit...
@Richard: A professional in a field performs an act for monetary compensation. An amateur in the same field performs the same act for no monetary compensation.
In short, it does not matter how talented or helpful the individual is: if a maintainer of Linux is unpaid, that is an amateur maintainer of Linux.
A economics podcast that I subscribe to just featured an interview with Clay Shirky. It was good to hear him describe his work in his own words. He discusses his book, Coase, the internet, Wikipedia.
It was really great. One of the better podcasts in the series.
Regarding the distinctions between amateurs and hobbyists and the connotations of skill level:
This is a difficult question related to language, my my native tongue (Finnish) the word for a hobbyist is "harrastaja" which derived from "harras" (in this context devout or pious) and in this case refers to someone who has invested considerable time (and possibly other resources) and usually is (at least in the process of becoming) highly skilled. This is not to be confused with "harrastelija" which is derogatory remark of someone who dabbles in the art but is neither skilled or ready to invest the time to become so.
Amateur is more complex, in formal use it usually refers to unpaid but skilled people, in common use I have it seen more used to refer to less-than-skilled people regardless whether they get paid for the skill or not (especially as slander on less-than-skilled "professionals")
But this is slightly off-topic so I stop now.
Indeed. The root of the word amateur is the French word, amor. It's only in relatively recent times it's come to denote someone who doesn't get paid for what he does.
Most people are lazy. A few people are cranks. I can not prove the following but I would say bloggers (which includes Bruce, sorry) are not lazy and are cranks (a description that includes anyone who is strongly motivated about a subject).
The ground level of the Coasean floor includes only those who have a strong motivation and are not lazy. If things continue to get easier, I think the ground level will be determined by the consumer, not the producer, of the information.
Thanks. I didn't have nearly that much space to explain the basics in the review.
Lost in mathematical transactions is the idea of corporations as life forms, and the marketplace as an evolutionary playing field. Corporations have several competitive edges:
Corporations don't die. Corporations exist only to make a profit. Corporations lack a social conscience. Corporations can off-shore their wealth from taxes. Corporations can fund lobbyists to shape the playing field.
Individuals die. Individuals are irrational maximizers. Individuals have a social conscience. Individuals lack advocacy in the halls of power. Individuals cannot PO Box their wealth in Bermuda.
These are some of the reasons why the corporation has become the dominant life form on the planet.
In no way is the post intending as an anti-corporation diatribe. The corporate organism's evolutionary success on this planet has made humanity wealthier and healthier. The corporation is in a sense humanity's golden goose. I am typing on one of its eggs.
The problem is of course: How do you control short-term thinking corporations so that they best serve mankind in the long run? Ultimately, with the continued degradation of the planet's environment, this becomes the most important question all governments must face...
"We've all figured out that the Internet makes freedom of speech a much harder right to take away. As Shirky demonstrates, Web 2.0 is having the same effect on freedom of assembly."
Since I never really understood what Web 2.0 meant I find that this is a very simple and powerful explanation. Thanks
[as long as it is a correct explanation]
> It is a small piece of irony that Nobel (an
> individual) never created a prize in economics.
I would hardly call that "irony" - more like, prescience. Simply because mainstream economics is a pseudoscience, the modern-day Lysenkoism, profoundly politically motivated and just as profoundly logically invalid.
The reason for that is quite simple - virtually all 20th century work relies on the hidden assumption that one can somehow aggregate utility functions of economic actors. This is obvious nonsense (as was shown time and again by the marginalists) - and so the entire edifice of mathematical economics rests on the use of invalid operation. And so, given the fact that from a false statement one can derive anything, the results of economic models are wildly divergent.
The simple truth about economics is that no quantitative modeling is possible as long as the structure of human utility functions is not known. And if is known, then building an artifical intelligence is a piece of cake - because by definition utility function is the complete description of actor's behavior.
The only logically consistent school of economics is Austrian, which treats economic actors as black boxes (axiomatic base of praxeology is truly minimalistic) - but it is unable to give any quantitative results, making it unpopular with the hard-science wannabe crowd. The other reason for its marginal (pun intended) status is that it is unpopular with funding agencies - because one of the fundamental qualitative results of Austrian economics is that all economic interventionism can do in longer term is make things worse. Meaning that the best things these funding agencies can do is to disband themselves.
> The Coase theorem implies that if
> these transaction costs are low
> enough, direct markets of individuals
> make a whole lot of sense. But if they
> are too high, it makes more sense to
> get the job done by an organization
> that hires people.
In many businesses, a new hire is not immediately productive because they do not understand the product, the testing procedures, the customer base, past failures/successes, idiosyncrasies of the company/management/cohorts, etc.
If a job is somehow "purely novel" for a company or very well defined, hiring a consultant to fill the post temporarily often works well -- much better than hiring a full-time employee.
But any large company has far too much specialization and history to attempt to give up their understood coordination among the players.
But as employees tend to come and go over time, the free market is working, just as an undercurrent.
This is true in sports, where if you build a team of top talent, you aren't ensured a great season because they are not able to work together and take advantage of learning that takes place by being together over time.
In response to:
"During the second world war one of the problems with bombing Japan was the lack of centralised organisations to use as valid industrial targets.
Basicaly just about every home had a small part in the production of an item for the war effort often these where small parts for aircraft etc.
It was only where these modular parts were finaly put together was a target worth while.
And even here the Japanese had an advantage. Most of their buildings where likewise modular and made of wooden frames with light panaling. Therefor small aircraft could be put together in different places at different times. It was only the final assembly of putting the wings on to fly it away need be done at a place with sufficient space for it to take off."
WHAT ????@@@??? Are you kidding?
What do you know of Japan during WWII??
"What do you know of Japan during WWII??"
As I was not alive then, only what has been reported historicaly.
However it was reported as being one of the production methods used, by both Japanese and American reports and historical information.
I first came across it as part of a course on "production methods".
And it is actually not much different to the first manufacturing process (for pins) that had one person doing one part of the production process.
Similar piece work practices where carried out in the Northan States to produce guns for the "American civil war" which is sometimes quoted as the reason the north out produced the south in arms and amunition and is contributed to being a major deciding factor in the outcome (crap comms security by the south is also given as a reason).
As a method of production it was only substantialy advanced by the likes of Henry Ford for the Model T car in the early part of the 20th cent.
My main interest in industrial history is in the transition from wind/water power to steam in Britain running upto the Victorian period, not WWII.
However if you have other sources of information I would be interested in hearing about them as it would be of comparative interest.
And yet the internet an environment where the cost of entry and transaction costs are very low is dominated by a few giants (google, amazon, yahoo, e-bay etc.).
averroes: "The only logically consistent school of economics is Austrian, which treats economic actors as black boxes (axiomatic base of praxeology is truly minimalistic) - but it is unable to give any quantitative results, making it unpopular with the hard-science wannabe crowd."
Which makes the Austrian school a pseudo-science as well. It's not a consistent and complete system --- there an infinity of cultural systems that are complete, but most lack power and just go around in circles, making them also just ideological games, divorced from reality.
If you're not a "hard science", your choices are art and propaganda. Of what use is economics as an aesthetic pursuit?
Please, discuss folks who are actually serious about truth (such as those trying to link in ecological and thermodynamic principles) and not just one more school of bullshitters --- particularly one that has worked hand in hand as propagandists for some of the late 20th century's monsters.
"Sorry you are most definately wrong.."
No, the commenter was right. The Chinese tried the backyard approach to steel manufacturing and the results were dramatically low quality steel. A corporate organization has the power of selection and directed evolution is quite a bit more efficient. As to the Flickr revolution, no organization == NoFlickr. Aggregation as a practice is not new. Once again, selection is the key whether it be by index or individual.
Shirky is in the business of selling the web and to do that, he purloins the concepts of cybernetics as if they were social constructs rather than technology enabling society to do what it does faster and in some cases as with Flickr, at higher density. But as James Anderson noted and others have before him, the web has concentrated into a handful of large organizations yet not because as Anderson said, there is a low barrier but because to dominate, there is an incredibly high barrier now, the cost of server farms and intellectual property.
Pundits like Huffington say the web spells the end of Rovian politics and here Shirky says freedom of speech is now harder to take away. Yet it is more likely the exact opposite trend is prevailing. The high penetration of the web means that anyone skilled in semiotic crafts such as emotional vectoring can practice Rovian politics and the right to freedom of expression is easier to shout down than ever it was in a Bush era rally.
I think we are kidding ourselves when we buy into Shirky's book. We want to believe we are empowered by use of a technology and in some limited and trivial ways we are; yet the levers of power have not changed hands or frequency. The illusion of is just a bit more motley and the kids who once listened to Dylan thinking they would change their parents world are watching a shadow play by their children thinking the same. It makes for a good party; what it makes of society is more of the same faster.
I agree with kangaroo. The Austrian school is just as much pseudo-science because it is founded on assumptions about the viability of free markets and price/value rationality of humans that either have not be proven or actually have been disproven by actual science (econometrics). Econometrics is the only branch of economics that deserves a nobel prize. Of course, it is not a complete 'school', because our knowledge is insufficient to build a coherent economic policy on science.
Of course, by saying that econometrics is somehow *both* consistent and complete, "kangaroo", above, proves his own, heh, self-reference...
Schneier.com is a personal website. Opinions expressed are not necessarily those of BT.