luni, 20 august 2012

SERP Crowding & Shrinkage: It's Not Your Imagination

Posted: 19 Aug 2012 08:00 PM PDT

Back in June, I wrote about what I called the “Bigfoot” Update, a major Google ranking fluctuation that seemed to be tied to SERP crowding. Put simply, fewer domains were getting more real estate in the Top 10. Since June, this trend only seems to be continuing. This is a graph of domain diversity from April 4, 2012 through last Thursday (August 16):

Domain Diversity (4/4 - 8/16)

The percentage represents the density of unique subdomains across the entire data set (Top 10 rankings for 1,000 keywords) – the lower the domain diversity, the more crowding in the SERPs. The large drops are: (1) the original Penguin update and (2) the “Bigfoot” update. Crowding continued to worsen until (3), when a slight improvement occurred around 8/14.

The Incredible Shrinking SERPs

As I was digging into (3), I noticed that something else happened around that same time. There was a burst of chatter that people were seeing only 7 results on some SERPs. One of the benefits of the Mozcast data set is that I can go back and run new stats over historical Page 1 rankings. So, I set out to determine if this 7-result phenomenon was real, and if so, if it was new. This is a graph of Top 10 SERPs with less than 10 results since April 4:

SERPS with <10 Results

While, historically, it seems there have been some SERPs with less than 10 results for a while, they ranged from 1-4% of the data set. In just two days, from about August 12-14, that number jumped to over 18%. Mozcast shows Page 1 SERPs with between 7-10 results, but almost the entire spike occurred in the 7-result pages. This is a graph of just the 7-result data:

SERPs with Exactly 7 Results

SERPs with 7 results were an anomaly prior to 8/13, with the system tracking a maximum of one (0. 1%) on any given day. On 8/13, that number jumped to 10.7% and then, the following day, to 18.3%. Almost one-fifth of SERPs tracked by our data now have 7 results.

“George Is Getting Upset!”

Nobody likes shrinkage, and we naturally get upset when someone messes around with our familiar, 10-result page. So, what’s happening here? Here’s a sample SERP, for “pc tools”, with numbered results (1-7):

7-result SERP ("PC Tools")

You’ll notice two things right away: (1) “PC Tools” is a brand, and (2) the #1 result has expanded site-links. Not every SERP affected appears to be branded, though – a search for “krill” (the #1 result is a Wikipedia entry for the crustacean) also returns 7 results, for example.

To maintain the integrity of the Mozcast crawl, I can’t do a public data dump of all of the affected keywords we measured, but spot-checking them reveals expanded site-links in almost all observable cases. While not all keyword phrases were branded, site-links and branded queries are naturally correlated.

“No! Not Six! I Said Seven!”

Sorry, I just wanted an excuse to use this movie clip. While the vast majority of the shrunken SERPs have 7 results, a couple of 6-result pages snuck into the mix. This is a screenshot from a Google result for “pictures of cats”:

6-result SERP ("pictures of cats")

Here, the standard, organic results are preceded by a mega-block of image results. Like the expanded site-links, I can only guess that these are being treated as multiple pieces of SERP “real estate”. In every case, the first result or result-block appears to be counting as more than one position.

What’s the Crowding Connection?

It’s tough to say if the slight decrease in crowding (increase in diversity) is directly related to the explosion in 7-result SERPs. My best guess is that, since many of the 7-result SERPs are branded and branded results seem to have more crowding (anecdotally, at least), cutting them short of a full 10 improved overall diversity slightly. In other words if a 10-result SERP was crowded and three got lopped off, then the remaining 10-result SERPs are counting more and pushing diversity back up a bit. It’s impossible to say if this was intentional or just a side-effect.

Why Was “Flux” Relatively Normal?

If you follow the Google weather on Mozcast, you may be wondering why temperatures were just slightly above average on the two days when SERP shrinkage rolled out. Digging into the data, it appears that the baseline flux for those two days was relatively low. Without the 7-result shift, temperatures on 8/13 would've been closer to 62°F. Combined with the two-day roll-out (split almost evenly across the two days), the introduction of the 7-result pages snuck just below the radar. It's hard to say whether the two-day roll-out was intentional or simply an artifact of our 24-hour data collection.

What Can You Do About It?

In a word: nothing. This isn't an SEO-related change, where an on-page or link-profile tweak might change your SERP back to 10 results. This is an algorithmic volume knob Google can turn and we can't, right or wrong. My best advice is to spot-check the SERPs for your main keywords. Don't just rely on rank-tracking tools - they may tell you that you're in the #8-#10 spot, but they won't tell you whether your SERP cut off after #7. If you're sitting on a lot of #8 keywords, you may find yourself suddenly on Page 2. If that's the case, it could be worth the effort to get back up into the Top 7, especially if the cost of getting from #8 to #7 is relatively low. Of course, this is a recent development, and it's likely Google is testing the waters (and could make a course-correction). My best advice is to pay attention - as part of your regular reporting process, make sure you look at SERPs in the wild, and see what you're up against.

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!