Deep Web Vs. Dark Web (A Much-Needed Update)

Good morning, fellow dark web explorers (and curiosity-seekers). How are you all?

Early on in the course of writing this blog, I wrote a post entitled Deep Web vs. Dark Web…, and attempted to explain the differences between the two. Nonetheless, I feel like I have a much better understanding of that now than I did back then – I’d like to share this knowledge with you all!

What prompted this, in part, was a video I watched by one of my favorite horror YouTubers, Elder’s Vault, entitled BTV: 5 Disturbing Deep Web Sites You Shouldn’t Visit. While I love Elder’s videos, I need to clarify, better than last time, the differences between “deep web” and “dark web.”

elders_vault_deep_web

Most YouTube videos on the subject seem to say “deep web,” when they’re actually referring to the latter.

In technical terms, the deep web merely refers to web content that isn’t indexed by standard search engines, like Google, Bing, Yahoo!, and even DuckDuckGo (which some people think is a dark web search engine, because it has an onion site).

Search engines use programs called “web robots,” “web crawlers,” or “spiders” to index the content you see in the search results.

Web-Robot-350x220
We are the web robots. We are here to protect you from the terrible secret of Google.

Every website has what’s called a “robots.txt” file which gives instructions to these robots on whether or not they want to index the site. Have you ever done a search, and received a message like this?

SPO site showing in bing

If a site has that message, then technically it’s in the “deep web,” because the admin has given instructions to the web robots not to index it. Examples of things like this might include your email inbox, the confidential parts of your banking site, a private Instagram profile, or information on a company’s private network. Spooky, right? For more information about this, see The Web Robots Pages.

Now, onto the “dark web” (which this blog is about, obviously). The term technically refers to web content built on top of anonymity networks, such as Tor. In fact, people often think of Tor as being synonymous with the dark web, because it has a large number of sites on it and is the most widely used.

tor_browser_ubuntu

Part of the confusion here stems from the fact that Tor is the way that many people first hear of the dark web, so they may just assume that the two are one and the same. Tor is both a network and a browser. The Tor network consists of sites that receive web traffic through Tor, and (in theory) have their IP addresses and locations disguised. You may also know these as .onion sites. – I’ve shared many of these on this blog! Here’s one example: http://cardedlxzxsphu5y.onion/

Now, just because a website is on Tor doesn’t mean that its users or admins can’t be discovered, but the anonymizing software just makes that a little more challenging.

Beyond Tor (as I’ve also mentioned in other posts), there are other similar networks, like I2P, Freenet, GNUnet, anoNet, and ZeroNet. While all of these have the common goal of anonymity, each one works differently, in a technical sense, and each has advantages and disadvantages. Plus, some of these have custom domain names, like .i2p (elgoog.i2p), .ano(ucis.ano), or .bit (Mail.ZeroNetwork.bit). Freenet and GNUnet, on the other hand, use what are called URIs (Uniform Resource Identifiers), like this: 127.0.0.1:8888/freenet/USK@tiYrPDh~fDeH5V7NZjpp~QuubaHwgks88iwlRXXLLWA,yboLMwX1dChz8fWKjmbdtl38HR5uiCOdIUT86ohUyRg,AQACAAE/nerdageddon/1/

See below:

i2p-4
I2P
freenet_nerdageddon
Freenet
gnunet2
GNUnet
anonet_homepage
anoNet
zeronet_431464_full
ZeroNet

Confused yet? In a nutshell, each one of these is a different network, and uses different software. If you click the links I gave you, you can check out their websites and download them for yourself!

One of the major advantages of networks like I2P, Freenet, and ZeroNet is that they’re good for P2P filesharing, whereas Tor is not (doing so can break your anonymity).

freenet_downloads

For instance, on Freenet, you can designate a part of your hard drive for P2P filesharing, which is called a “datastore.” I’ll get into that in a different post, but suffice it to say – the dark web is not just Tor, by any means.

One last thing – just because a site is “creepy,” that doesn’t make it a dark web site either. Numerous people have said that The Cannibal Cafe was a dark website, when, contrary to popular belief, it was a clearnet site (“clearnet” being the opposite of “darknet”).

cannibal_cafe

There are quite a few creepy clearnet sites, as well as non-creepy dark web sites. Is your mind blown yet?

Have additional questions or comments? Feel free to leave them here!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.