So you’ve heard the terms ‘Deep Web’ and ‘Dark Web’ thrown around in the media of late, and the latest bust of the the notorious ‘AlphaBay Market’ probably piqued your interest in the topic as well. However, much of what’s available in the media glosses over the everyday realities of people who need to use anonymizing software to reach those deep, dark parts of the internet, and focuses solely on some of the unsavory aspects of the so-called ‘Deep Web’. Today, we’ll take a look at what the Deep Web truly is, and how it differs from the ‘Dark Web’, which is what most people often unknowingly refer to when they talk about the Deep Web. However, before we get to that, let’s take a look at the various layers of the World Wide Web, including the part that we access everyday on our PCs, smartphones and tablets.
The Web As We Know It
The web that we know and love includes a lot of your favorite virtual destinations, including popular search engines like Google and Bing, social networks like Facebook and Twitter, e-commerce portals like Amazon and eBay, informative sites like Wikipedia and Quora, and blogs like Politico and Beebom. In fact, no matter where your interests lie, you’ll always find something on the net that will tickle your fancy.
Most of these sites are eminently accessible via your standard web browser, but no matter how prolific your surfing habits are and how voracious your appetite for knowledge is, you’ll very likely never ever be able to soak up the vast sea of knowledge that exists on the web, especially because so much more information is being posted on the internet daily. These websites that everybody with access to a connected device can visit without any preconditions, are said to form the ‘Surface Web’. However, not all of the information being posted on the net is actually meant to be accessible by one and all.
The Hidden Web
Along with all the publicly-accessible sites (of which there are millions), the World Wide Web also includes websites of your e-mail service provider, your bank, your insurance company, your hospital and other such organizations that have a lot of private and confidential information that are, for obvious reasons, not available to anyone and everyone. Similar is the case with streaming services like Netflix and news publications like The Wall Street Journal, whose contents sit behind a paywall and are only accessible to paying subscribers.
You can, of course, visit their websites via your standard web-browsers, but to be able to progress any further, you’ll need to have a valid account that positively IDs you as someone who has the right to access the inner sanctum of those sites and networks. Even then, you’re only allowed to access the web-pages that relate to you, so as long as Google, Yahoo and Microsoft don’t get it horribly wrong, you shouldn’t, in theory, get access to my e-mails and vice-versa.
So by now, you’ve realized that the general public is only allowed to access a tiny fraction of the billion+ websites and 60 trillion web-pages that exist on the World Wide Web. In fact, it is said that Google and Bing’s search bots can only index about 1 in 3,000 pages on the web, leaving the vast majority of the World Wide Web out of their purview and essentially, ‘hidden’ from public view. It is this very section of the internet that’s often referred to as the ‘Deep Web’, and has often been part of an misinformation campaign because of the prevailing confusion about the topic among the general populace.
The single-largest chunk of the Deep Web actually consists of statistics and databases, with U.S. Government agencies, like the National Oceanic and Atmospheric Administration, NASA, the USPTO and the SEC accounting for a large amount of the info. There are also millions of pages that lie on the ‘intranet’, which are internal networks of organizations and are accessible only by people admitted into the system by the administrators.
Just to be clear, the Deep Web is hidden for a good reason, and unlike what many sensationalist reports on the subject have said over the years, it has literally nothing to do with illegal activity. In fact, it’s just as well that the technology exists to hide private stuff from public search, otherwise we’d have to still store all our personal documents in paper files, like our grandparents were doing all those decades ago.
Dark Web Vs. Deep Web: Clearing Up The Confusion
Unfortunately, some sections of the media have often seemed confused about the Deep Web, and what the term actually means. With all the chatter about alleged illegal activities going on in a dark corner of the internet, some bloggers and commentators have, over the past few years, tended to use the terms Deep Web and Dark Web interchangeably.
First off, let’s get this out the way – the Deep Web is more than 99% of the internet, and it’s not synonymous with the Dark Web. And contrary to the assertions of some confused bloggers who’d want you to believe that the people visiting the Deep Web are degenerates beyond redemption, the fact remains that pretty much all of us have visited the Deep Web at some stage or another.
That’s because you’re on the Deep Web every time you check your e-mail, watch Netflix, pay for your latest purchase on Amazon, or pretty much do anything that requires you to access private web-pages that are meant for your eyes only. It really is that simple. Anything that’s not on the surface of the World Wide Web to be accessed en masse is the Deep Web, so knowingly or unknowingly, we’ve all been there. And fairly often at that.
So what’s the confusion about, then? If the Deep Web is really all that innocent, why do sections of the mainstream media keep writing about how dangerous a place the hidden internet is? All that reportage about the drugs and the hitmen-for-hire, all that outrage about human trafficking and child abuse, that can’t all possibly be fake news, can it? The answer is relatively simple, but still there are plenty of layers to the story.
First off, all that illegal stuff is really happening behind the cloak of anonymity, but as alluded to earlier, when the media talks about all the shady stuff going on in the dark underbelly of the internet, for the most part, they are really only referring to the what’s known as the ‘Dark Web’ – the overlay networks that form a minuscule part of the Deep Web and, can only be accessed through specialized software that have been specifically configured for the purpose.
The confusion between Deep Web and Dark Web among sections of the non tech-savvy is understandable to an extent, although, in reality, the Dark Web is only a subset of the Deep Web. Sure, neither can be search-indexed, but large sections of the the Deep Web, unlike the darknets that form the so-called Dark Web, do not need any special censor-resistant software for access. The Dark Web, on the other hand, can only be accessed through various anonymizing platforms, the best-known and most widely-used among which happens to be Tor. Other platforms like I2P and Freenet are also generally referred to as being parts of the Dark Web.
Legal Standing and Controversies
So now that we’ve gone through the basic differences between the Surface Web, the Deep Web and the Dark Web, we should also take a quick look at some of the reasons why the so-called Dark Web often gets a bad rap. Behind the anonymity of the censor-resistant platforms, Dark Web has flourished a thriving trade of all that is illegal and unacceptable in civil society. In their book Cryptopolitik and the Darknet, researchers Daniel Moore and Thomas Rid claimed that around 57% of the Dark Web includes illicit content. From contraband substances to automatic assault weapons, from disturbing imagery of gore and child abuse to hired assassins, you can find just about anything and everything listed on the many thriving marketplaces on the Dark Web, if you know where to look.
Yet, a large part of the darknets manage to stay within the legal and/or ethical boundaries for the most part, and is certainly worth checking out. In case you’re interested, the first place you should look at is, of course, the Tor platform. Tor hosts a vast majority of the websites on the Dark Web, so it is, understandably, the best place to get started if you want to see for yourself what the fuss is all about. You can get a detailed tutorial on how to access the Dark Web in one of our earlier articles, so you should definitely check that out if you want to know more.
If you are worried about the legality of Tor and wondering whether you can get into trouble for using the software, rest assured that merely surfing the Dark Web isn’t going to get you into any legal trouble as long as you stay away from partaking in illegal activities while in there. But then again, that warning is also equally applicable on the regular web, so it really isn’t any different in this case.
Arguments For and Against The Hidden Internet
Even amidst all the disturbing material found in sections of the so-called Dark Web, the fact remains that anonymous communication networks are a boon for many activists, reporters, researchers and whistle-blowers who simply need the anonymity, without which, their lives and livelihoods may be in grave danger. In fact, Tor is the preeminent anonymizing platform in the virtual world, and has been recommended by various human rights organizations as a shield for activists and dissidents fighting oppressive regimes around the world. Reporters sans Frontières (Reporters without Borders) have also described Tor as an “essential survival kit” for its members, while cyber-security researchers are also known to use Tor to test firewalls and provide emergency DNS lookup services in case of DNS failures.
However, all those endorsements still don’t seem to have convinced some world leaders about the importance of online anonymity. Former British Prime Minister, David Cameron, was a vocal critic of end-to-end encryption employed by apps like WhatsApp and Snapchat, and actually even tried to rope in the then-U.S. President, Barack Obama, in trying to pressurize the U.S. tech companies to give government agencies a backdoor, so that they could keep a tab on the activities of whoever they felt was indulging in suspicious behavior. U.S. law enforcement agencies have also expressed deep reservations about encryption software, most notably after the San Bernardino attack a couple of years ago.
Funnily enough, in spite of U.S. law enforcement agencies publicly crying foul against encryption and anonymity in the virtual world, Tor is actually funded in large parts by the U.S. government, although, indirectly. According to a report published in 2013, the Tor project received over $1.8 million that year from two non-profit organizations that are funded directly by the U.S. federal government. While Internetwork News, which donated $555,000 to The Tor Project, is funded by the US State Department, SRI International, which donated $830,000 to the project, is directly funded by the US Department of Defense.
SEE ALSO: VPN: Pros and Cons You Should Consider Before Using It
Deep Web vs Dark Web: Legal, Illegal and Shades of Gray
As is the case with most things in life, the Deep Web is also a combination of the good and the bad. At its darkest corners or the Dark Web, it indeed has some awfully unsavory and repulsive content, but the same can also be said about the regular web, where sites like LiveLeak, 8chan and many others often host some absolutely abhorrent stuff. However, as any sane, freedom-loving person would agree, blanket-bans and government censorship are definitely not the answer to anything in a free and fair society.
The fact also remains that for better or worse, it is simply technically unfeasible to police the internet, all 60 trillion pages of it. It is even tougher for the law enforcement to monitor anonymizing networks like Tor. China tried it with its much-hyped Great Firewall, but mostly failed to rein in opposition activists, with the Tor Project introducing secret entrance nodes called ‘bridges’ that are very difficult to block.
So once you accept that censorship isn’t the answer either ethically or technologically, how do you propose to rein in the drug dealers, gun runners, terrorists and pedophiles from usurping what is supposed to be a free-speech platform for harassed activists and whistle-blowers? Because all said and done, the ability of criminal elements to abuse the system must be measured against the benefits Tor and other anonymizing networks brings its users, many of whom are truly in need of such services. After all, anonymity in the virtual world isn’t a black and white debate. As with most things in life, in this debate, too, there are enough shades of gray; some just happen to be darker than others.
