Survey of the Dark Web

Interesting paper on the dark web: Daniel Moore & Thomas Rid, "Cryptopolitik and the Darknet," Survival, 2016. (Technical annex here -- requires the Tor browser.) They conclude that it's mostly used for illegal activity.

No surprise, really, but it's good to have actual research to back it up.

Press coverage.

Posted on February 15, 2016 at 6:19 AM • 33 Comments


aziFebruary 15, 2016 7:46 AM

Unfortunately very bad research. The darknet, by it's very nature, is fractured. There is not one darknet, but many. TOR especially cannot be searched nor indexed. In this paper, the authors describe how they've crawled a portion of one publicly accessible TOR web, which happens to contain a lot of illicit content. To base their conclusions on such an limited and non-representative experiment is unscientific at best.
This paper is nothing more than a publicity stunt.

JeroenFebruary 15, 2016 8:25 AM

Since when is Tor only 'the WWW' or 'WWW using portals'? There's many more ways to use Tor. Those use cases are being ignored

BFebruary 15, 2016 8:26 AM

I think the conclusion is better stated as, "a majority of hidden services that the researchers were able to find and classify were for illegal activities." In fact the majority of hidden services they could find were live but did not serve anything; my guess is that these are serving something, but the path is private and functions as a "key" of sorts. Unfortunately the researchers jump to the conclusion that these sites are a demonstration of how unreliable Tor hidden services are and simply assume they are broken, and then draw even further conclusions (e.g. that most hidden services welcome visibility).

It is also misleading to speak about *numbers* of hidden services, as it says nothing about the share of actual users. The researchers have no reliable way to estimate how many users the sites they found are getting; for all they know, those sites are seeing no traffic at all. It is also possible that a single criminal might run many hidden services as part of a risk-management tactic, so the researchers may be inadvertently double-counting actual criminal use.

I read this paper as a political opinion piece more than a scientific result.

zFebruary 15, 2016 8:32 AM

I don't like engaging in this kind of argument. The problem with bickering over how it's used (in the context of the debate about backdoors) is that it implies that if a service or software can be proven to be commonly used for evil, it ought to be backdoored. If you oppose backdoors, this is a counterproductive way to go about it.

Whether it is commonly used for bad stuff or not is not a justification for backdoors. Anything can be used by bad as well as good people. The premise that it ought to be banned/regulated/subverted based solely on how someone thinks it is used is fundamentally flawed. There are problems of precedent, Constitutionality, the slippery slope, the potential for government abuse, and the impact it has on the good people who do use it. This applies to everything the government wants to control, from encryption, to guns, to the Internet, to large sodas (in former mayor Bloomberg's case).

Green SquirrelFebruary 15, 2016 8:40 AM

Hmm. There is an awful lot I would disagree with in this research. Even the bit which states "the obvious" is open to arguments (as shown above).

To be honest, I'd have to grade this report meh.

BallastFebruary 15, 2016 8:56 AM

I went down into the cellar. I turned on the light. There were cockroaches on the floor. I noticed they ran to the dark and not into the light.

GrowingUpUnderSurveillanceFebruary 15, 2016 9:01 AM

Tor onion services: more useful than you think [32c3]

I found this a useful talk that was a good defense and taught me a lot.

I feel that this is more on "The war on Privacy and Tor". Which would be more popular, a damnation article about Tor or a article about the good of Tor?

I find it sad that I am growing up in the 9/11 generation, the one with no privacy, the one that revels itself online, and the one that never was alive to live before. My peers place themselves on social media, I am in the generation growing up in the surveillance state.

Sancho_PFebruary 15, 2016 9:56 AM

Could someone explain to me, please:
When Python and any human visitor can scrape, understand and categorize the criminal content,
where is the connection to encryption, cryptography and the “going dark” debate ?

When will we see our powers going to jail for not doing their work?
If "researchers" can - why don’t LEOs do their job?

From the researcher’s cyber-stunt:
”Tor’s ugly example should loom large in technology debates.”
OK, I confess I don’t really understand the full meaning (ESL) of that sentence but obviously the word “technology” is wrong in this context.

“Tor’s ugly example” is a mirror of our society.
The debate should be about the lack of prosecution in the IT.

keinerFebruary 15, 2016 9:57 AM

"Taylor & Francis"

... is one of the big scientific publishers :-D

btw. afaik resistance against any kind of totalitarian political system is "illegal", right?

So, I guess, a certain level of "illegal" activity is necessary all the time. How about 100% financial transparency for members of parliaments and governments? How about full authorship information for every new law introduced?

honeypotsFebruary 15, 2016 10:22 AM

Any guesses how many of these "illegal" addresses are maintained by law enforcement agencies?

GweihirFebruary 15, 2016 10:57 AM

While the numbers are interesting, the conclusions are not supported by the numbers in any sensible fashion. That makes this very bad science.

Whether it it a political propaganda piece masked as science or whether these people are just bad scientists, I cannot say, but it would be better to not have these "results".

Tor's Beautiful ExampleFebruary 15, 2016 11:22 AM

The conclusion of "ugly Tor" in the research doesn't follow from their own data:

Most Tor users have never visited any hidden website at a *.onion address; hidden services account for around 3–6% of overall Tor traffic.

Further, if we discount the #1 and #2 categories: "None" & "Other" from the list automatically (2,482 and 1021 respectively of the 5,205 total) we are left with drugs at #3 (423; nearly a third of the 'illicit' category).

Drugs and consenting adults are arguably none of the government's business in a supposed liberal democracy - since it is none of their concern what we put into our bodies i.e. the war on some drugs is a joke and has been since it was instituted in the 70s. Thus, I'm not particularly concerned yet.

Next, if we add up the guns, kiddy porn, fraud, violence and extremist stuff in that table, it approximates around 12% of the total hidden services they found.

Thus, if only 3-6% of all 2 million+ regular Tor users use hidden services, we can assume that less than 10,000 are interested in the worst that hidden services have to offer, or around 1 in every 700,000 people on the face of the planet (I'm sure a math head will correct me shortly). Nevertheless, this is not exactly the end of the world, or any reason to ban encryption outright - it's a stat I can certainly live with to preserve liberty.

Sure, the Tor designers could shut down the hidden content provider model as they outline in the paper, but it will just push the scum back into back alleys again and onto alternative networks. That's the physics of the universe we live in.

The cops/feds are just as idealistic in their views and we don't need to throw out the baby with the bathwater nor accept the propaganda regularly dished out regarding the Tor model.

In fact, the VAST majority of users just want to trawl the standard net without the Stasi keeping a file of their browsing habits forever more, or generally profiling them in secret. Something we should expect with a 4th amendment in place.

But, authoritarians will never let an imagined crisis go to waste. It's in their blood.

Ray DillingerFebruary 15, 2016 12:39 PM

The authors are being silly when they suppose that the Tor team could shut down hidden services; if Tor shuts down its hidden services capability, someone will just fork it and open it right back up. Not that it matters.

Honestly I think Tor exists solely for the purposes of providing traffic analysis data to mine, providing the illusion of anonymity to draw out the elements that the listeners are most interested in mining. You have an application that emits identifiable packets, and an Internet where pretty much all packets in flight are analyzed at least enough to tell what application creates them and what application can receive them. HTTP headers aside, Tor packets are COMPLETELY identifiable as Tor packets.

IMO the return on investment for them is that they do real-time traffic analysis and know exactly who is using it. Maybe they can't see into the packets, but they can see the packets and they know where those packets originate.

Add up the facts:

Development of the application is largely funded by known eavesdroppers.

The eavesdroppers are known deploy Tor nodes they themselves operate which often add up to over half the total Tor network. This goes a long way toward correlating requests and responses.

Timing attacks to a fairly large extent also correlate the requests and the responses, and the information from these sources will be integrated on the back end.

The eavesdroppers can originate requests of their own to provoke responses coming back, which facilitates sharply focused timing attacks.

Most of these so-called hidden services, most of the relays, and almost all the users of the hidden services are operated by people whose opsec otherwise sucks rocks, on identifiable machines which they also use for other things.

Most users of Tor services use it with browsers, mail clients, and operating systems that leak like sieves. Most users of Tor use the same machine for related non-Tor traffic which eavesdroppers can further use to correlate Tor.

Seriously, add up the facts, and the eavesdroppers don't need Tor to be cryptographically backdoored to know pretty much everything that goes on there.

camilo-rodriguezFebruary 15, 2016 1:08 PM

If we outlawed end-to-end encryption or Tor, for potentially enabling criminality, we would have to outlaw Privacy as well - which is what unlawful Mass Surveillance has already done de facto - as it is within our privacy that we can decide whether to do wrong or good. It is thus our autonomy and ability to decide that is ultimately at stake. The absolute ideal of Mass Surveillance is the ability to anticipate and predict decisions. In this scenario, privacy and anonymity are only barriers standing in the way for the real target, which is autonomy and ability to think and decide freely.

We can however not ignore the historical predicament in which encryption is being attacked, by who and why. Mass Surveillance has been implemented de facto and constitutes a grave violation at all judicial levels, and it has been implemented with the sole purpose to unlawfully, illegally and illegitimately gain and exert power.

Encryption is not something we necessarily want, encryption the last and only resort against the ongoing and de facto violation of one of our inalienable human rights, which has proven to be effective in undermining and effectively destroying both our political and justice systems.

The only party who have "gone dark" and acted in the protection of unlawful and illegitimate, yet fully legalized secrecy, are the Intelligence agencies and governments, who have already violated and undermined our rights, through the systematic abuse of their authority, which has completely severely eroded on trust as a necessary element of democracy.

The real situation is not whether we should give up privacy for more security and all agree to be treated as potential criminals under 24hr surveillance - which we already are -, the real situation is whether we can - de facto - protect our rights by using end-to-end encryption and en masse, and aim at forcing a change in digital practices that will enable us to safeguard freedom, and recover the shift in the balance of power that the Internet initially empowered us with.

There is no doubt that Privacy is a fundamental right and that safeguarding it is paramount. The topic here is not Privacy vs Security, it is Freedom vs Control, and there is no valid trade off when it comes to privacy, or any other fundamental right. We cannot continue debating the value or necessity of Privacy, anymore than we can continue debating whether torture should be used against terrorists to save lives. Privacy is a Human Right and part of our human dignity, and in this sense, there is no Us vs Them.

The real task is how we protect the right to Privacy and also Privacy itself, and the answer to the second part of this question is, for now, the only simple one: using end-to-end encryption. The hope is that massive use of encryption will become significant leverage to enforce the formal aspect of Privacy, and step up the normative aspect in respect to digital technology and communications, both as users, and with regard to governance and state mechanisms.

camilo-rodriguezFebruary 15, 2016 2:21 PM

"In all, this latest research provides a deeper, empirical basis for discussion around Tor hidden services, and perhaps encryption more generally."

The more general discussion concerns Human Rights, Civil Rights and Liberties, basic Judicial Principles and International Law. Regarding Human Rights, there cannot be any trade-offs, and no "empirical basis" can be considered as compelling enough for us to become flexible or figuring out "acceptable" trade-offs regarding our fundamental rights, related to our dignity as a species.

Rid hopes that the research “will make it more difficult for anybody to just make these wholesale, rather disappointing statements about encryption. We're just beyond that point.”

The "wholesale" statements are ultimately made in defense of Privacy and Anonymity, as Human Rights, and not primarily in defense of end-to-end encryption. End-to-end encryption is the last resort of Privacy and Anonymity in digital communications and. On the other hand, the nature of the Internet and Encryption, makes the installation of back-doors technologically impossible, without creating a built-in vulnerability that cannot be kept secret or made unusable for criminal and other intelligence agencies alike.

It is interesting that the Internet, as a means to communicate and share ideas and experiences without borders, makes the violation of our the essential liberties rights extensive to all of us. The Us vs Them argument falls on its face. The recent victory of Net Neutrality - and much more than just Net Neutrality - in India, shows how the nature of the Internet has made Human and Civil Rights interesting and easier to relate to than before, as well as easier to act on.

The users of Tor do ultimately not reflect the majority of Humans on the planet, nor specific national realities and levels of criminality or corruption, and they don't reflect some sort of ubiquitous emergence of generalized criminality. What the limited use of Tor does reflect, is the unawareness of the great majority regarding the grave implications of unlawful, illegal and illegitimate Mass Surveillance carried out by few governments (led by the usual one invested in Imperial power). What the limited use of Tor also reflects, is the general lack of knowledge regarding end-to-end encryption and the available and free tools to implement it on most of our significant communications.
And what the limited use of Tor ultimately reflects, is the propagandistic use of tainted and biased views and presentations of facts disseminated about Tor, and the invention and circulation of names such as "dark web". There is even a documentary TV series entitled the "Dark Web", and a CSI series dedicated to consolidating Tor as a place exclusively constituted by criminal activity, crawling with dubious hackers, and as a neighborhood you shouldn't ever visit.

This research may have the merit of intending to be more serious regarding the production of results in order to justify foregone conclusions, but regardless of how accurate it may be, it fails to avoid falling into the same false dichotomy of Privacy vs Security. The study doesn't understand that a quantitative argument has no bearing on Human or Civil Rights, and is clearly biased against Tor, as it does no effort to defend or promote its use for good.

Characterizing Tor or end-to-end encryption, the only means to effectively defend against unlawful, illegal and illegitimate Mass Surveillance practiced and imposed de facto by one government over the whole planet, as purely evil or purely noble fails to understand what Tor - or even Privacy and Autonomy - really is. The subsequent failure to promote it or defend it as a tool that can be used for good and most significantly - right now - simply to avoid oppression of criminal governments, reveals the bias of the study. Wanting to make the case of a governemnt and intelligence agency that really "went dark", legalized secrecy - or more likely secured impunity - has no scientific value whatsoever, especially when the conclusion is that we should consider making exceptions to Human Rights and Civil Liberties.

Some Billionaire MayorFebruary 15, 2016 2:34 PM

"it's mostly used for illegal activity"

It is obvious that Congress does not have the necessary backbone to stand up to the internet lobby. That is why my group, Mayors Against Illegal Internet Activity (MAIIA) will be funding efforts to pass reasonable, common-sense, internet-safety regulation at the state and local levels.

NotYouAgainFebruary 15, 2016 2:44 PM

I wanted to go on about what bovine excrement this research is, but all those commentators before me beat me to it. Yeah!

Also, drawing conclusions about terrorism from the use of encryption is just bad logic, and does not solve any of the real world problems.

AntFebruary 15, 2016 3:26 PM

What is classified as 'illegal activity'?

People reclaiming their right to privacy?

The real illegal activity takes place in the clear, like ISIS posting their atrocities and having USA state support and backup, or 'gay marriage' proponents hunting down people for their beliefs.

eFebruary 15, 2016 4:27 PM

Public roads are mostly used for illegal activity too, don't most people break the speed limit? Therefore, they should be banned too, according to some people's logic...

Cyber ContortionistFebruary 15, 2016 5:03 PM

Pay me the right fee and I'll write a paper concluded TOR is the leading cause of STDs, sexually transmitted diseases, AND use of encryption causes impotence.

Isn't the whole point of TOR you can't track it?

JdLFebruary 15, 2016 6:05 PM

From the nakedsecurity link:

The results suggest that the most common uses for websites on Tor hidden services are criminal, including drugs, illicit finance and pornography involving violence, children and animals.

This lumping of consensual activities (drugs, "illicit" finance (i.e. not wanting government thugs to rip off a large chunk of a transaction)) with truly criminal activities (pornography involving violent children with animals, etc.) is typical of authors who have drunk the Statist kool-aid.

PIEFebruary 15, 2016 7:31 PM

Curses! Foiled again! And we were so close to hooking your innocent children on deadly drugs and seducing them into perverted sex. We almost had them with the MySpace, but concerned idiots worldwide read all about it in Time magazine and thwarted our nefarious plan with a highly effective government-orchestrated moral panic.

We shall never rest. Now we shall execute our fiendish designs using Quantum computing! Nyah-hah-hah-hah-hah-hah!

PeterFebruary 15, 2016 8:33 PM

Seems to me the NSA forgot to tell their "friends" @ GCHQ what that TOR-thing really is .
Some years ago a conservative politician in my country heard of TOR and demanded it banned -
For about 12 hours, then she never spoke of it again .
Yep, TOR is mostly used for "illegal" purposes - By US spooks .

GTMFebruary 16, 2016 9:46 AM

I seem to remember a study once that concluded the majority of $100 bills in circulation were used for illegal activity ... regardless, I'm not sure I'm ready to do away with cash.

hermanFebruary 16, 2016 10:10 AM

Thanks to all the newspaper columnists focusing on TOR, other systems like Retroshare can carry on with the good work in peace and quiet...

Norm de PruneFebruary 16, 2016 12:58 PM

@GTM: I think that conclusion was drawn from the fact that many large denomination bills in circulation have been used to snort cocaine and contain trace residue of the the drug.

"On study reported "92% of the bills were positive for cocaine with a mean amount of 28.75+/-139.07 micrograms per bill, a median of 1.37 μg per bill, and a range of 0.01-922.72 μg per bill. Heroin was detected in seven bills in amounts ranging from 0.03 to 168.5 μg per bill: 6-AM and morphine were detected in three bills; methamphetamine and amphetamine in three and one bills, respectively, and PCP was detected in two bills in amounts of 0.78 and 1.87 μg per bill. Codeine was not detected in any of the one-dollar bills analyzed"

albertFebruary 16, 2016 4:29 PM

Here's a blurb on Daniel Moore:

"...Daniel Moore is a cyber-threat intelligence engineer and a PhD candidate in the Department of War Studies at King's College London...."

and Thomas Rid:

"War Studies" says it all. Of course they're going to find bogeymen everywhere; that's their job.

Note to Rid & Moore: Take up Peace Studies instead. Y'all can help make the world a better place. Let LE turn over the rocks.

Is anyone else here sick of these pseudo-'academic studies'?

This sort of drivel is a plentiful waste of time.

. .. . .. --- ....

DougFebruary 16, 2016 7:44 PM

Peter said, "Yep, TOR is mostly used for "illegal" purposes - By US spooks ."

This is logically true because in order to secure transmit thru unsecured wires, the "spooks" need a way to fleet broadcast while hiding the locs of broadcasters located within unsecure territories. A logical method is to hide within the crowd, of other broadcasters. Thus, what's illegal in unsecure jurisdictions becomes the cover, while our law enforcers go after those in secured jurisdictions.

"Illegal" is correctly quotated to show that usage is nefarious but not illegal because it's out of our legal jurisdiction.

Leave a comment

Allowed HTML: <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre>

Photo of Bruce Schneier by Per Ervland.

Schneier on Security is a personal website. Opinions expressed are not necessarily those of Resilient, an IBM Company.