Tuesday, November 5, 2024

Google: We See 1 Trillion URLs Daily

Google says the World Wide Web is big. Really big. You just won’t believe how mind-bogglingly big it really is. I mean you think Kim Kardashian’s butt is big but that’s just peanuts to the Web.

Google’s milestone announcement today is perhaps equally as difficult to wrap one’s head around: There are over one trillion unique URLs with unique content on the Internet.

1 trillion.

1,000,000,000,000.

Twelve zeros and a one.

And you thought a billion was a fun number? Remember when we listened in awe to the thought that if you had a billion dollars and spent a dollar per second you wouldn’t run out of money for 31 years or so? A trillion—trillion—would last you 31.6 THOUSAND years.

It’s the type of number that makes your brain say “I quit” and you take it out and use it as a soccer ball.

If you, says WhatReallyHappened.com, went into business when Christ was born and lost a million dollars per day, you wouldn’t be broke until Fall 2737.

So what I’m saying is, it’s a lot.

On the Google Blog, software engineers Jesse Alpert & Nissan Hajaj say the URLs in Google’s index don’t even include duplicate content, or repetitive pages like ones on calendars. And the web is growing by several billion pages per day.

Google was always fast. That’s one of the reasons the search engine was so successful. But Alpert and Hajaj describe just how fast it has become:

“To keep up with this volume of information, our systems have come a long way since the first set of web data Google processed to answer queries. Back then, we did everything in batches: one workstation could compute the PageRank graph on 26 million pages in a couple of hours, and that set of pages would be used as Google’s index for a fixed period of time. Today, Google downloads the web continuously, collecting updated page information and re-processing the entire web-link graph several times per day. This graph of one trillion URLs is similar to a map made up of one trillion intersections. So multiple times every day, we do the computational equivalent of fully exploring every intersection of every road in the United States. Except it’d be a map about 50,000 times as big as the U.S., with 50,000 times as many roads and intersections.”

It gets worse. At this rate it’s supposed to take 300 years to index the world’s information, assuming no new information pops up.

Erm. Uh, um. Soccer, anyone?
 

Related Articles

8 COMMENTS

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Latest Articles