Central Perk: What Does "X% of Queries" Mean?

joi, 25 octombrie 2012

What Does "X% of Queries" Mean?

Posted: 24 Oct 2012 08:00 PM PDT

Over the past few months, we’ve seen a number of public statements from Google regarding algorithm updates that sound something like this:

Tweet from Matt Cutts - "0.3% of English queries noticeably affected"

Tweet from Google - "Fewer than 0.7% of queries noticeably affected"

Notice the carefully chosen phrase “X% of queries noticeably affected”. Those statements sound very specific, but once you stop to think about them, you’re inevitably left with a couple of questions: (1) percent of which queries?, and (2) exactly how much is “noticeably”?

Percent of Which Queries?

When most of us hear “X% of queries”, I think we naturally assume unique queries (different keyphrases), but there are really two interpretations: (1) unique queries, and (2) total query volume. Here’s an example, from a conversation I posted on Google+ - let’s say that we live in a world where there are only four searches anyone ever uses:

“iphone 5”
“justin bieber”
“platypus pants”
“charlie sheen”

If we ran 100 searches on our parallel universe version of Google, the volume would be distributed to our four keyphrases as follows:

That's right - suck it, Charlie Sheen. Now, let's say that query (c) is impacted by an algorithm update. By definition #1, this would affect 25% of queries. By definition #2, that same algo update would only affect 9% of queries.

I’ve put this question to Google a couple of times, and while they didn’t reply directly, a few people were kind enough to repeat it. Jacob Klein’s tweet to Matt Cutts got the following reply:

Tweet from Jacob Klein

Tweet (reply) from Matt Cutts

It’s not quite an official statement, but I’ll take it. In most cases, Google is talking about overall query volume. I wouldn’t read too much into the “typically” – Matt naturally doesn’t want to commit to “% of queries affected” only ever having one meaning when uttered by any Google employee. I’m comfortable, though, that Google’s recent statements refer to query volume. This also makes sense from the way Google views the data – overall volume is probably much easier to measure than unique queries.

I don't think Google is being deliberately cryptic in this particular case – the statement is simply ambiguous. As SEOs, we naturally think of "queries" in terms of unique keywords, because that's what we track. Google thinks in terms of overall search volume and each query is a discrete unit..

How Much is “Noticeably”?

So, now that we know we’re talking about query volume, what’s this “noticeably affected” bit all about? Does any change count, or does it have to be significant? Hat tip to Matt McGee for reporting on a follow-up conversation that sheds some light on this one:

Tweet from Rob Watts

Tweet from Matt Cutts

It’s not quite cut-and-dry, and the definition of “noticeable” may be a little complex, but basically they’re talking about Page 1 changes and probably even flux in the first few rankings. A query where the #32 ranked keyword drops to #34 isn’t going to be counted as “noticeably affected” by an algorithm update.

Knowing Is X% of the Battle

So, when Google refers to “X% of queries noticeably affected” they mean the total volume of queries and a significant change in the Top 10. Since we don’t see the entire universe of queries that they do or really know the relative volume, this still leaves a lot to interpretation, but I think any little bit of transparency helps at this point. It’s a useful gauge of relative impact as Google uses the same metric across multiple updates.

Keep in mind, though, that any given query either impacts you or doesn’t impact you. Don’t rely on the aggregate statistics – pay attention to what matters to you. Look at it this way – unemployment was just under 8% in the US in September, but you and I either have a job or we don’t. That 8% may be a useful gauge of economic prosperity or the effectiveness of our leaders, but ultimately we have to be aware of our own situation. If an update only impacts 0.3% of queries, but your money term was one of them, knowing that 99.7% of queries were untouched won’t be of much comfort to you.

For reference, I've recently added Google's impact percentages to the MozCast Events page. This page lists recent algorithm updates, along with the severity as measured both by MozCast temperature and Google's publicly stated percentage, to allow for easy comparisons.

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!