marți, 1 iulie 2014

One Content Metric to Rule Them All

One Content Metric to Rule Them All

One Content Metric to Rule Them All

Posted: 30 Jun 2014 05:01 PM PDT

Posted by Trevor-Klein

Let's face it: Measuring, analyzing, and reporting the success of content marketing is hard.

Not only that, but we're all busy. In its latest report on B2B trends, the Content Marketing Institute quantified some of the greatest challenges faced by today's content marketers, and a whopping 69% of companies cited a lack of time. We spend enough of our time sourcing, editing, and publishing the content, and anyone who has ever managed an editorial calendar knows that fires are constantly in need of dousing. With so little extra time on our hands, the last thing content marketers want to do is sift through a heaping pile of data that looks something like this:

Sometimes we want to dig into granular data. If a post does exceptionally well on Twitter, but just so-so everywhere else, that's noteworthy. But when we look at individual metrics, it's far too easy to read into them in all the wrong ways.

Here at Moz, it's quite easy to think that a post isn't doing well when it doesn't have a bunch of thumbs up, or to think that we've made a horrible mistake when a post gets several thumbs down. The truth is, though, that we can't simply equate metrics like thumbs to success. In fact, our most thumbed-down post in the last two years was one in which Carson Ward essentially predicted the recent demise of spammy guest blogging.

We need a solution. We need something that's easy to track at a glance, but doesn't lose the forest for the trees. We need a way to quickly sift through the noise and figure out which pieces of content were really successful, and which didn't go over nearly as well. We need something that looks more like this:

This post walks through how we combined our content metrics for the Moz Blog into a single, easy-to-digest score, and better yet, almost completely automated it.

What it is not

It is not an absolute score. Creating an absolute score, while the math would be equally easy, simply wouldn't be worthwhile. Companies that are just beginning their content marketing efforts would consistently score in the single digits, and it isn't fair to compare a multi-million dollar push from a giant corporation to a best effort from a very small company. This metric isn't meant to compare one organization's efforts with any other; it's meant to be used inside of a single organization.

What it is and what it measures

The One Metric is a single score that tells you how successful a piece of content was by comparing it to the average performance of the content that came before it. We made it by combining several other metrics, or "ingredients," that fall into three equally weighted categories:

  1. Google Analytics
  2. On-page (in-house) metrics
  3. Social metrics

It would never do to simply smash all these metrics together, as the larger numbers would inherently carry more weight. In other words, we cannot simply take the average of 10,000 visits and 200 Facebook likes, as Facebook would be weighted far more heavily—moving from 200 to 201 likes would be an increase of 0.5%, and moving from 10,000 to 10,001 visits would be an increase of 0.01%. To ensure every one of the ingredients is weighted equally, we compare them to our expectations of them individually.

Let's take a simple example using only one ingredient. If we wanted to get a sense for how well a particular post did on Twitter, we could obviously look at the number of tweets that link to it. But what does that number actually mean? How successful is a post that earns 100 tweets? 500? 2,000? In order to make sense of it, we use past performance. We take everything we've posted over the last two months, and find the average number of tweets each of those posts got. (We chose two months; you can use more or less if that works better for you.) That's our benchmark—our expectation for how many tweets our future posts will get. Then, if our next post gets more than that expected number, we can safely say that it did well by our own standards. The actual number of tweets doesn't really matter in this sense—it's about moving up and to the right, striving to continually improve our work.

Here's a more visual representation of how that looks:

Knowing a post did better or worse than expectations is quite valuable, but how much better or worse did it actually do? Did it barely miss the mark, or did it completely tank? It's time to quantify.

It's that percentage of the average (92% and 73% in the examples above) that we use to seed our One Metric. For any given ingredient, if we have 200% of the average, we have a post that did twice as well as normal. If we have 50%, we have a post that did half as well.

From there, we do the exact same thing for all the other ingredients we'd like to use, and then combine them:

This gives us a single metric that offers a quick overview of a post's performance. In the above example, our overall performance came out to 113% of what we'd expect based on our average performance. We can say it outperformed expectations by 13%.

We don't stop there, though. This percent of the average is quite useful... but we wanted this metric to be useful outside of our own minds. We wanted it to make sense to just about anyone who looked at it, so we needed a different scale. To that end, we took it one step farther and applied that percentage to a logarithmic scale, giving us a single two-digit score much like you see for Domain Authority and Page Authority.

If you're curious, we used the following equation for our scale (though you should feel free to adjust that equation to create a scale more suitable for your needs):

Where y is the One Metric score, and x is the percent of a post's expected performance it actually received. Essentially, a post that exactly meets expectations receives a score of 50.

For the above example, an overall percentage of expectations that comes out to 113% translates as follows:

Of course, you won't need to calculate the value by hand; that'll be done automatically in a spreadsheet. Which is actually a great segue...

The whole goal here is to make things easy, so what we're going for is a spreadsheet where all you have to do is "fill down" for each new piece of content as it's created. About 10-15 seconds of work for each piece. Unfortunately, I can't simply give you a ready-to-go template, as I don't have access to your Google Analytics, and have no clue how your on-page metrics might be set up. 

As a result, this might look a little daunting at first.

Once you get things working once, though, all it takes is copying the formulas into new rows for new pieces of content; the metrics will be filled automatically. It's well worth the initial effort.

Ready? Start here:

Make a copy of that document so you can make edits (File > Make a Copy), then follow the steps below to adjust that spreadsheet based on your own preferences.

  1. You'll want to add or remove columns from that sheet to match the ingredients you'll be using. Do you not have any on-page metrics like thumbs or comments? No problem—just delete them. Do you want to add Pinterest repins as an ingredient? Toss it in there. It's your metric, so make it a combination of the things that matter to you.
  2. Get some content in there. Since the performance of each new piece of content is based on the performance of what came before it, you need to add the "what came before it." If you've got access to a database for your organization (or know someone who does), that might be easiest. You can also create a new tab in that spreadsheet, then use the =IMPORTFEED function to automatically pull a list of content from your RSS feed.
  3. Populate the first row. You'll use a variety of functionality within Google Spreadsheets to pull the data you need in from various places on the web, and I go through many of them below. This is the most time-consuming part of setting this up; don't give up!
  4. Got your data successfully imported for the first row? Fill down. Make sure it's importing the right data for the rest of your initial content.
  5. Calculate the percentage of expectations. Depending on how many ingredients you're using, this equation can look mighty intimidating, but that's really just a product of the spreadsheet smooshing it all onto one line. Here's a prettier version:
    All this is doing (remember Step 2 above, where we combined the ingredients) is comparing each individual metric to past performance, and then weighting them appropriately.

    And, here's what that looks like in plain text for our metric (yours may vary):
      =((1/3)*(E48/(average(E2:E47))))+((1/3)*((F48/(average(F2:F47)))+(G48/(average(G2:G47))))/2)+((1/3)*((H48/(average(H2:H47)))+(I48/(average(I2:I47)))+(J48/(average(J2:J47)))/3))

    Note that this equation goes from row 2 through row 47 because we had 46 pieces of content that served to create our "expectation."

  6. Convert it to the One Metric score. This is a piece of cake. You can certainly use our logarithmic equation (referenced above): y = 27*ln(x) +50, where x is the percent of expectations you just finished calculating. Or, if you feel comfortable adjusting that to suit your own needs, feel free to do that as well.
  7. You're all set! Add more content, fill down, and repeat!

Update: A word of caution
After some great discussion in the comments below, I thought it prudent to include a word of caution about how this metric is used. For one thing, be smart about what the numbers actually mean, and keep the following points in mind:

  1. Make sure you have a sufficiently ample benchmark. If you only have three posts with which to set the "expected" values, then your performance against that expectation isn't going to mean much. I'd recommend having at least 10-15 posts from which you calculate that expectation before you apply the One Metric score to any subsequent posts.
  2. Be smart about averages, and know that this doesn't mean you can discard all the rest of your metrics. By smooshing all of these metrics together, we effectively soften the impact of any outliers. Thanks to Pete Wailes for this XKCD reference; while you shouldn't lose the forest for the trees, sometimes individual trees are quite important.
  3. As with any metric, knowing what to do with the One Metric takes a keen awareness of why a certain piece of content performed the way it did, and checking any intended actions against your organization's goals. There's a reason marketers haven't been replaced by algorithms: It's up to you and your brain to turn these metrics into actual insights.

</caution>

Here are more detailed instructions for pulling various types of data into the spreadsheet:

Adding new rows with IFTTT

If This Then That (IFTTT) makes it brilliantly easy to have your new posts automatically added to the spreadsheet where you track your One Metric. The one catch is that your posts need to have an RSS feed set up (more on that from FeedBurner). Sign up for a free IFTTT account if you don't already have one, and then set up a recipe that adds a row to a Google Spreadsheet for every new post in the RSS feed.

When creating that recipe, make sure you include "Entry URL" as one of the fields that's recorded in the spreadsheet; that'll be necessary for pulling in the rest of the metrics for each post.

Also, IFTTT shortens URLs by default, which you'll want to turn off, since the shortened URLs won't mean anything to the APIs we're using later. You can find that setting in your account preferences.

Pulling Google Analytics

One of the beautiful things about using a Google Spreadsheet for tracking this metric is the easy integration with Google Analytics. There's an add-on for Google Spreadsheets that makes pulling in just about any metric a simple process. The only downside is that even after setting things up correctly, you'll still need to manually refresh the data.

To get started,  install the add-on. You'll want to do so while using an account that has access to your Google Analytics.

Then, create a new report; you'll find the option under "Add-ons > Google Analytics:"

Select the GA account info that contains the metrics you want to see, and choose the metrics you'd like to track. Put "Page" in the field for "Dimensions;" that'll allow you to reference the resulting report by URL.

You can change the report's configuration later on, and if you'd like extra help figuring out how to fiddle with it, check out Google's documentation.

This will create (at least) two new tabs on your spreadsheet; one for Report Configuration, and one for each of the metrics you included when creating the report. On the Report Configuration tab, you'll want to be sure you set the date range appropriately (I'd recommend setting the end date fairly far in the future, so you don't have to go back and change it later). To make things run a bit quicker, I'd also recommend setting a filter for the section(s) of your site you'd like to evaluate. Last but not least, the default value for "Max Results" is 1,000, so if you have more pages than that, I'd change that, as well (the max value is 10,000).

Got it all set up? Run that puppy! Head to Add-ons > Google Analytics > Run Reports. Each time you return to this spreadsheet to update your info, you'll want to click "Run Reports" again, to get the most up-to-date stats.

There's one more step. Your data is now in a table on the wrong worksheet, so we need to pull it over using the VLOOKUP formula. Essentially, you're telling Excel, "See that URL over there? Find it in the table on that report tab, and tell me what the number is next to it." If you haven't used VLOOKUP before, it's well worth learning. There's a fantastic  explanation over at Search Engine Watch if you could use a primer (or a refresher).

Pulling in social metrics with scripts

This is a little trickier, as Google Spreadsheets doesn't include a way to pull in social metrics, and that info ins't included in GA. The solution? We create our own functions for the spreadsheet to use.

Relax; it's not as hard as you'd think. =)

I'll go over Facebook, Twitter, and Google Plus here, though the process would undoubtedly be similar for any other social network you'd like to measure.

We start in the script editor, which you'll find under the tools menu:

If you've been there before, you'll see a list of scripts you've already made; just click "Create a New Project." If you're new to Google Scripts, it'll plop you into a blank project—you can just dismiss the popup window that tries to get you started.

Google Scripts organizes what you create into "projects," and each project can contain multiple scripts. You'll only need one project here—just call it something like "Social Metrics Scripts"—and then create a new script within that project for each of the social networks you'd like to include as an ingredient in your One Metric.

Once you have a blank script ready for each network, go through one by one, and paste the respective code below into the large box in the script editor (make sure to replace the default "myFunction" code). 

  function fbshares(url) {  var jsondata = UrlFetchApp.fetch("http://api.facebook.com/restserver.php?method=links.getStats&format=json&urls="+url);  var object = Utilities.jsonParse(jsondata.getContentText());  return object[0].total_count;  Utilities.sleep(1000)  }  
  function tweets(url) {  var jsondata = UrlFetchApp.fetch("http://urls.api.twitter.com/1/urls/count.json?url="+url);  var object = Utilities.jsonParse(jsondata.getContentText());  Utilities.sleep(1000)  return object.count;  }  
  function plusones(url) {  var options =  {  "method" : "post",  "contentType" : "application/json",  "payload" :  '{"method":"pos.plusones.get","id":"p","params":{"nolog":true,"id":"'+url+'","source":"widget","userId":"@viewer","groupId":"@self"},"jsonrpc":"2.0","key":"p","apiVersion":"v1"}'  };  var response = UrlFetchApp.fetch("https://clients6.google.com/rpc?key=AIzaSyCKSbrvQasunBoV16zDH9R33D88CeLr9gQ", options);  var results = JSON.parse(response.getContentText());  if (results.result != undefined)  return results.result.metadata.globalCounts.count;  return "Error";  }

Make sure you save these scripts—that isn't automatic like it is with most Google applications. Done? You've now got the following functions at your disposal in Google Spreadsheets:

  • =fbshares(url)
  • =tweets(url)
  • =plusones(url)

The (url) in each of those cases is where you'll point to the URL of the post you're trying to analyze, which should be pulled in automatically by IFTTT. Voila! Social metrics.

Pulling on-page metrics

You may also have metrics built into your site that you'd like to use. For example, Moz has thumbs up on each post, and we also frequently see great discussions in our comments section, so we use both of those as success metrics for our blog. Those can usually be pulled in through one of the following two methods.

But first, obligatory note: Both of these methods involve scraping a page for information, which is obviously fine if you're scraping your own site, but it's against the ToS for many services out there (such as Google's properties and Twitter), so be careful with how you use these.

=IMPORTXML

While getting it set up correctly can be a little tricky, this is an incredibly handy function, as it allows you to scrape a piece of information from a page using an XPath. As long as your metric is displayed somewhere on the URL for your piece of content, you can use this function to pull it into your spreadsheet.

Here's how you format the function:

If you'd like a full tutorial on XPaths (they're quite useful), our friends at Distilled put together a really fantastic guide to using them for things just like this.  It's well worth a look. You can skip that for now, if you'd rather, as you can find the XPath for any given element pretty quickly with a tool built into Chrome.

Right-click on the metric you'd like to pull, and click on "Inspect element."

That'll pull up the developer tools console at the bottom of the window, and will highlight the line of code that corresponds to what you clicked. Right-click on that line of code, and you'll have the option to "Copy XPath." Have at it.

That'll copy the XPath to your clipboard, which you can then paste into the function in Google Spreadsheets.

Richard Baxter of BuiltVisible created a wonderful  guide to the IMPORTXML function a few years ago; it's worth a look if you'd like more info.

Combining =INDEX with =IMPORTHTML

If your ingredient is housed in a <table> or a list (ordered or unordered) on your pages, this method might work just as well.

=IMPORTHTML simply plucks the information from a list or table on a given URL, and =INDEX pulls the value from a cell you specify within that table. Combining them creates a function something like this:

Note that without the INDEX function, the IMPORTHTML function will pull in the entire piece of content it's given. So, if you have a 15-line table on your page and you import that using IMPORTHTML, you'll get the entire table in 15 rows in your spreadsheet. INDEX is what restricts it to a single cell in that table. For more on this function, check out this quick tutorial.

Taking it to the next level

I've got a few ideas in the works for how to make this metric even better. 

Automatically check for outlier ingredients and flag them

One of the downsides of smooshing all of these ingredients together is missing out on the insights that individual metrics can offer. If one post did fantastically well on Facebook, for example, but ended up with a non-remarkable One Metric score, you might still want to know that it did really well on Facebook.

In the next iteration of the metric, my plan is to have the spreadsheet automatically calculate not only the average performance of past content, but also the standard deviation. Then, whenever a single piece differs by more than a couple of standard deviations (in either direction), that ingredient will get called out as an outlier for further review.

Break out the categories of ingredients

In the graphic above that combines the ingredients into categories in order to calculate an overall average, it might help to monitor those individual categories, too. You might, then, have a spreadsheet that looked something like this:

Make the weight of each category adjustable based on current goals

As it stands, each of those three categories is given equal weight in coming up with our One Metric scores. If we broke the categories out, though, they could be weighted differently to reflect our company's changing goals. For example, if increased brand awareness was a goal, we could apply a heavier weight to social metrics. If retention became more important, on-page metrics from the existing community could be weighted more heavily. That weighting would adapt the metric to be a truer representation of the content's performance against current company goals.

I hope this comes in as handy for everyone else's analysis as it has for my own. If you have any questions and/or feedback, or any other interesting ways you think this metric could be used, I'd love to hear from you in the comments!


Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!
Publicat de 1 la 09:25

Niciun comentariu:

Trimiteți un comentariu

Abonați-vă la: Postare comentarii (Atom)