Central Perk

miercuri, 22 mai 2013

Decoding Google's Referral String (or, how I surviVED Secure Search)

Posted: 21 May 2013 07:42 PM PDT

Last week, I held a Mozinar outlining a method to extract SERP vertical -- called Universal Search by Google --- from Google referral strings. Since the Mozinar concluded, the number of people who have reached out with their own theories and ideas has been impressive. I want to post everything that I know here and then leave it up to you folks in the SEOmoz community to start hacking and sharing your insight.

For those of you who did not see the Mozinar, you can access it here (voiceover included). You can also download or view the slides without VO on Slideshare here.

Before getting into the step-by-step process and providing examples of how to use the Google referral string to interpret where in Universal Search your traffic came from, I want to lay out a problem we were having at AudienceWise. In 2011, Matthew Brown and I started an agency to help news publishers with technical SEO and audience development. In our other jobs, specifically Matthew at the New York Times, we struggled with reconciling for the lack of data around Universal Search referrals. As far as our web analytics platforms were concerned, a visit from web search, a News OneBox link, and an image result were all treated exactly the same: as organic search traffic.

Then came Google Secure Search, and referral data got even more opaque. In addition to not knowing which Universal vertical the referral came from, now in about 10% of cases we didn’t even know the keyword that referred the traffic. The question that kept going through our collective ginger minds was: how can we help our clients with content strategy if we know nothing about WHY they are receiving said search traffic? Unfortunately, Secure Search has vastly expanded and now accounts for a large percentage of all Google referral traffic. As way of an example, here is the latest percentage of keyword = (not provided) for SEOmoz:

Matthew and I knew the only way to reclaim *some* of this lost data was to start looking at other sources. Luckily, Matt speaks Spanish (sort of) and came across this blog. The author posited that the 'ved' parameter in the Google referral string held some magic in determining the vertical that result appeared in. After doing some quick searches, and looking at the “href” values for the results, it seemed like he was onto something. We immediately set up Google Analytics profile filters to extract this parameter on a client that receives 300,000 search referrals from Google per day. After a couple of hours, we were loaded with enough data to start confirming some of the authors theories and coming up with a few of our own. I will layout what we found, provide a step-by-step tutorial to setup Google Analytics filters, and provide a few examples of how to use the data.

First, let’s talk about where you can find this parameter.

Simply, the Google referral string is the “href” value assigned to each URL in a set of search results. When a user clicks on the above, she is being redirected through a google URL prior to reaching her final destination; Radiohead.com, in this case. Google most likely does this for internal data aggregation reasons -- we’re not suppose to know where our traffic comes from, but they sure make use of it -- probably for aggregating data around SERPs.

There are two parameters that I will focus on here: ‘cd’ and ‘ved.’ The ‘cd’ parameter has been written about before and tells us the position of the search result in the set. As far as I can tell, the ‘ved’ parameter is divided into three parts and tells us which Universal vertical the result is part of, the position within that vertical (relative position), and the position within the search result (absolute position). I will focus on just the Universal aspect for this post and will follow up with relative vs. absolute position in a follow-up.

Let’s have a look at a few examples.

When QFj is in the ‘ved’ parameter that the result is a standard web search result, such as:

One of the attendees of the Mozinar made this astute observation about a special variation for the web search 'ved':

When QqQIw (that’s a capital “i” not a lowercase “L”) it is a Universal result that resides within the Google News OneBox. When QpwI is present that means the result was the thumbnail image within the News OneBox.

You get the idea. Here are some other values of ‘ved.’ I suspect that there are many more and am curious to see what the community here can find and SHARE here within:

Setting up Google Analytics filters

You should have a good understanding now of potential power of this information. Did I mention that it is still available even if the keyword is “(not provided)”? We could potentially interpret the keyword by comparing ‘ved.’ Anyone up for the challenge? I go through one example below. While ‘ved’ appears to persist through Secure Search only about 50% of the search referrals within GA have this data. If anyone can shine light on this, I’m sure the rest of the community would shower you with thumbs ups!

Step 1: Set up a Google Analytics Profile filter

Go to the account’s administrative dashboard and select “New Profile.” I would recommend against setting this filter up on an existing profile as that it will overwrite some data that you otherwise want. I called mine ‘Universal Search.’

Next, you will need to set up two advanced filters; one to extract ‘ved’ and ‘cd’ from the Google referral string, and the other to display the data within Google Analytics.

Universal Extract

Here’s the text of the regex that I used

Field A (\?|&)(ved)=([^&]*)

Field B (\?|&)(cd)=([^&]*)

Universal Display

There’s many different ways to do this. I’ve decided to overwrite the campaign dimension of source since that’s where I am checking my organic search referrals.

Filters work while the data is streaming in and will not be reflected retroactively. That’s fine; you just have to wait for a day or so (or an hour or so for bigger sites) to start digging in. Here’s what it should look like:

Step 2: Set up Advanced Segments

I prefer to do this level of analysis in Excel, but Advanced Segments can be created to make it all look pretty in GA. I will walk you through the setup of one, which will inform you how to do the rest.

You will want to name your Advanced Segment something that will clue you in to which vertical you are analyzing. In this case, I have called out that it is a standard ‘blue link’ result from a News OneBox. From there, all you need to do is search on ‘Source’ for anything containing the ‘ved’ you are trying to isolate. In this case, we are looking for ‘QqQIw.’

Here’s an example of what you will see:

Wow! There is an actionable result right in front of me. It’s probably time to do some image optimization. Google apparently respects the site as a news authority, but not one that creates good images.

Another useful ‘ved’ to investigate is Sitelinks. Sitelinks are a subset of results triggered by a branded search. Google algorithmically determines which links to include, but webmasters have the ability to demote links in Webmaster Tools. The 'ved’ parameter can come in handy to measure performance of Sitelink pages and action can be taken. In order to figure out the Sitelink that sent the search referral, look at the ‘cd’ value that was passed with the referral string. We accounted for this in the filters and it is in your data here:

Here’s what the ‘cd’ values mean in relation to Sitelink results:

There are myriad of use cases for bubbling up SEO action items. Here are a few, and please add more in the comments:

Calculating ROI and resource allocation for different SEO efforts: News, image, branded, and semantic markup. As marketers, we are only as valuable as what we can quantify. A challenge with SEO is demonstrating value. This does not solve the problem, but exposes a few more variables to work with.
Optimizing branded search Sitelinks: As I outlined above, there is value in knowing which branded links send you traffic. This is also one area where you can mitigate the loss of keyword data due to Secure Search. When you see that a keyword is (not provided) AND ved = xxxxQjB, you can interpolate that keyword = YOUR BRAND.
Image optimization for Google News: The top link in the Google News OneBox is most often a different source than the image thumbnail. If ved = xxxxQqQIw ÷ ved = xxxxQpwI, or the ratio of links to images, is way off-kilter it suggests there is an image optimization issue. Publishers can then use this data to measure optimization efforts against a pre-established baseline.
Optimizing video thumbnails: Images of video that are alongside a link are always from the same source as the link. Marketers can use a similar ratio as above to analyze click-through rates and on-page analysis when ved = xxxxQuAIw.
Analyzing efficacy of semantic markup: As the occurrences of SERPS that include clickable rich-snippets and knowledge graph elements increase, being able to parse and understand the referrals using ‘ved’ is clear. I have only started looking at results that have rich-snippets, but the initial data suggests that ‘ved’ may even indicate what type event, of rich snippet was clicked. Here are a few examples: (This is one area that could use a lot more research from the community!)

Events Markup: ved = xxxBE0MGM

Music Markup: ved = xxxQ6hEw

SERP landscape analysis: If you can scrape a Google SERP, you can tell which ‘ved’ elements are on the page and know which verticals are in each. The ‘href’ lives within Java Script so the simplest way to retrieve it is by using a headless browser such PhantomJS.

That about wraps it up for my first -- of hopefully many -- posts on ‘ved.’ In the months to come, Moz will be collecting Google referral string data on a great number of SERPs for various keywords. We plan to unleash our data hound to sniff out the most useful elements. In the meantime, I would like to use this post as a place for the hacking to begin and the sharing of your thoughts in the comments.

Dig in!

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!

Dominate on Google and other major search engines

This is a SubmitStart Sponsor Update. Unsubscribe from this list.

Web CEO Software used by 950,000+ customers

THE LEADING SEO SOFTWARE

Achieve top Google rankings
with the best of breed SEO tools
from Web CEO!

Web CEO customers: PayPal, Siemens, Johnson and Johnson, Hitachi, Motorola and more

Site Optimization and Link Building Tools

Do keyword research, optimize site pages to get high search engine rankings and build natural links to your site.

Visibility Monitoring Tools

Track your search engine rankings and backlinks; analyze your visitor traffic, conversions and ROI.

Professional Features

Bring SEO data into your own apps with the Web CEO API; allow clients to logon to your own White Label SEO tools on your site.

Use FREE or Order from $49/ mo

Sent to e0nstar1.blog@gmail.com — why did I get this?

unsubscribe from this list | update subscription preferences

SubmitStart · Trade Center · Kristian IV:s väg 3 · Halmstad 302 50

Seth's Blog : Let's start with "sorry"

Let's start with "sorry"

By the time the phone rings, there's already trouble. When that manager is called or this department is reached, it's because someone is disappointed, angry or stuck. Illness, broken promises or a real urgency have led to this new conversation even taking place.

So don't start with, "[Name of company] mumble mumble" as if there's a blank slate just waiting to be written on. There's already a lot of writing on that slate. Don't demand to know the record number or begin with doubt and an edge of dismissal. Be on our team.

"It sounds like we've got a situation on our hands..." is a fine way to disarm the person you're about to talk with. He won't have to spend the first six sentences expressing his anger and urgency, because in less than ten words, you've done it for him. Or perhaps, "I'd like to help, if you'll bring me up to speed..."

It's not easy being on the receiving end of a days'-long parade of blame, but no one said it was easy. We asked you to do it because you're good and because it's important, not because it's fun.

• Email to a friend •

marți, 21 mai 2013

Mish's Global Economic Trend Analysis

What Is California Attempting to Hide?
Gold ETF Liquidates 300 Tons of Gold This Year; Reflections on Momentum Trading
Merkel Pins Cameron in Corner; Will Cameron Bury His Head in the Sand, Pretending to Not Notice?

What Is California Attempting to Hide?

Posted: 21 May 2013 12:21 PM PDT

An interesting article on a recently passed law in California came my way today regarding Obamacare secrecy in California.

Please consider California exchange granted secrecy.

A California law that created an agency to oversee national health care reforms granted it broad authority to conceal spending on the contractors that will perform most of its functions, potentially shielding the public from seeing how hundreds of millions of dollars are spent.

The degree of secrecy afforded Covered California appears unique among states attempting to establish their own health insurance exchanges under President Barack Obama's signature health law.

An Associated Press review of the 16 other states that have opted for state-run marketplaces shows the California agency was given powers that are the most restrictive in what information is required to be made public.

It's routine in government to keep bids secret until contracts are awarded, so one vendor does not get an unfair advantage over others. After a bid is awarded, contracts generally become fully public.

In setting up the California exchange, lawmakers gave it the authority to keep all contracts private for a year and the amounts paid secret indefinitely. "Except for the portion of a contract that contains the rates of payment, contracts entered into pursuant to this title shall be open to inspection one year after their effective dates," reads the code specifying what exchange records are exempt from public disclosure.

According to agency documents, Covered California plans to spend nearly $458 million on outside vendors by the end of 2014, covering lawyers, consultants, public relations advisers and other functions.

Other exchange records that are allowed to be kept secret include those that reveal recommendations, research, strategy of the board or its staff, or those that provide instructions, advice or training to employees. Minutes of the board meetings also are exempt from disclosure.

So what does California have to hide? More specifically what do the legislators (especially California Assembly Speaker John Perez, D-Los Angeles) have to hide? Contracts awarded to the non-low bidder? Contracts awarded to friends and family of legislators? Kickbacks?

With $458 million on outside vendors by the end of 2014, there are plenty of non-legitimate reasons for wanting to keep everything a secret.

With all the secrecy it's hard to say precisely who is covering up for whom, or why, but one thing is crystal clear: This secrecy is good for someone on the take and bad for taxpayers.

Mike "Mish" Shedlock
http://globaleconomicanalysis.blogspot.com

Gold ETF Liquidates 300 Tons of Gold This Year; Reflections on Momentum Trading

Posted: 21 May 2013 08:46 AM PDT

Investors in the Gold ETF - GLD liquidated 300 tons of the metal this year.

The reason? People are tired of losing money watching gold sink while the S&P soars.

Tom Lydon, the editor of ETF Trends, says the disposal of over 600,000 pounds of gold so far this year "amazing" and "incredible." Click on above link for a video interview with Lydon.

Gold vs. S&P 500

GLD data by YCharts

Reflections on Momentum Trading

Since late 2012 the S&P 500 has been on a nonstop rise, while gold has gone the other way. People have thrown in the towel on gold in favor of momentum trading in stocks.

It seems nearly everyone is a momentum trader now, one of the consequences of inept central bank bubble-blowing policy.

Louise Yamada says it's Time for Gold Bulls to Abandon Hope. See my response in Wild Swings in Gold and Silver; Time to Give Up Hope?

Mike "Mish" Shedlock
http://globaleconomicanalysis.blogspot.com

Merkel Pins Cameron in Corner; Will Cameron Bury His Head in the Sand, Pretending to Not Notice?

Posted: 20 May 2013 11:59 PM PDT

UK prime minister, David Cameron, promised to hold a referendum on whether Great Britain should remain in the EU, but only on two conditions. The first condition, that Cameron be re-elected as prime minister is iffy enough.

The second condition, that Cameron renegotiate the Lisbon Treaty, I said would never happen. And it won't.

German Chancellor Angela Merkel sealed the fate on that score as Berlin plans to streamline EU but avoid wholesale treaty change.

Berlin is drawing up plans for treaty changes to streamline decision-making in the eurozone, while stopping short of any wholesale renegotiation that would allow the UK to repatriate powers from Brussels.

Although Angela Merkel, German chancellor, has expressed her desire to keep the UK inside the EU, the move being discussed in Berlin would thwart a plan by David Cameron, UK prime minister, to piggyback on eurozone reforms to renegotiate the British relationship with Brussels.

Mr Cameron had hoped to exploit renewed interest in Berlin for wholesale EU treaty changes as a way to renegotiate the UK's membership terms. But Berlin's strategy for a new, narrowly focused treaty could force the UK premier into a repeat of the dilemma he faced in December 2011, when Mr Cameron rejected the fiscal compact treaty but most other EU countries went along without him.

Senior German officials acknowledged that they were isolated on treaty change, which is fraught with political landmines in several countries – particularly France, which would probably require a national referendum if major changes were made to EU law.

The timing of treaty changes remains a matter of debate but it could come as early as next year, after elections to the European parliament in May. The way ahead is due to be discussed at a summit next month.

Pinned in the Corner

The sooner Merkel proceeds with her strategy, the better for everyone involved, especially UK citizens. Merkel has effectively preempted Cameron's strategy in a way he cannot realistically deny.

Since there is now no possible hope of wholesale renegotiation (not that there ever really was in the first place), there is no reason for the UK to avoid a referendum now.

Will Cameron bury his head in the sand like an ostrich once again? We will find out shortly.

Mike "Mish" Shedlock
http://globaleconomicanalysis.blogspot.com

miercuri, 22 mai 2013

Decoding Google's Referral String (or, how I surviVED Secure Search)

Decoding Google's Referral String (or, how I surviVED Secure Search)

Setting up Google Analytics filters

Step 1: Set up a Google Analytics Profile filter

Step 2: Set up Advanced Segments