Central Perk: SEOmoz Daily SEO Blog

sâmbătă, 16 octombrie 2010

SEOmoz Daily SEO Blog

Posted: 16 Oct 2010 08:27 AM PDT

As many of you likely noticed, Linkscape updated its index on Thursday night. New data is now available in the SEOmoz Firefox and Google Chrome toolbars, Open Site Explorer, the classic Linkscape tool and many of our other SEO tools.

This update was, sadly, just over 2 weeks off schedule, primarily due to some hardware failures at Amazon's EC2 where we run processing on our large link graph to produce the metrics and views for the API. Although we've encountered issues like this in the past, this was one of the larger failures and meant processing had to be restarted several times to update the index. You might be able to read more about the technical details in the near future on our nascent and geektacular Dev Blog.

Index Stats

Pages: 41,219,038,886 (41 Billion)
Subdomains: 436,693,488 (436 Million)
Root Domains: 99,649,652 (99 Million)
Links: 402,521,240,277 (402 Billion)

I made some graphs showing a few interesting trends over the past few months in the web's adoption/use of certain protocols.

Nofollow Usage over Time

The chart above shows how using rel=nofollow on internal links is slowly becoming less popular (though it's still a majority of use).

Rel=Canonical Use Over Time

This chart's telling us that rel canonical use has barely grown from June to October (as a percent). In this index, 5.42% of the pages we saw used rel=canonical tags. The datapoint from May (when rel=canonical was on 5.50% of pages we saw) is curious, but I suspect it has more to do with which pages we were choosing to crawl and index vs. an actual shift in usage. It's a good reminder, though, that unless we see large, sustained shifts across indices comprised of relatively similar URLs, we shouldn't jump to conclusions.

The next scheduled update for Linkscape is Nov. 12th (see the Linkscape Calendar page on our API Wiki) and we hope to be in much better shape with hitting that deadline.

Linkscape is also undergoing some serious upgrades over the next 3 months. With our web app launched (and regular upgrades on track), 4 of our 10 engineering folks (Phil, Chas, Bryce & Ben) are going to be working to make Linkscape fresher, faster, more comprehensive and higher quality by January. Expect to see some incremental improvements between now and then, which we'll report here on the blog.

Do you like this post? Yes No