miercuri, 18 ianuarie 2012

10 Commandments of Affiliate Web Design Graywolf's SEO Blog

10 Commandments of Affiliate Web Design Graywolf's SEO Blog


10 Commandments of Affiliate Web Design

Posted: 18 Jan 2012 10:39 AM PST

Post image for 10 Commandments of Affiliate Web Design

Advanced SEO Book coverThis is a guest post by Gab Goldenberg, author of the advanced SEO book. Get a free chapter on link building here and check out his book affiliate program – the site boasts a 5.5% conversion rate to sale!

Affiliate web design is not only unique because it’s purpose is to generate conversions – in contrast to providing an outlet for frustrated artists – but it has to do so in a limited environment where the conversion has to happen on a third party site. Here are the 10 key web design rules for affiliates.

10. Problem: You need to send the traffic onwards, with minimal bounce rate on the merchant site.

Solution: Therefore, prepare your traffic by including visual elements of the merchant’s landing page on your own site.

Logos are a minimum, but try cropping the hero shot and using that or some stock photo face that’s on the lander… use things people will recognize after clicking through and reassure them this is what they clicked through to see. Only using text links is lazy and less likely to work.

Similarly, either repeat or paraphrase the merchant’s headline with your headline and call to action.

9. Problem: Merchant sites don’t show shipping costs [or other costs, ex.: book clubs, hidden fees] easily. This means people will abandon the cart in higher rates.

Solution: Gather the info yourself and inform your visitors exactly of all charges ahead of time.

8. Problem: Visitors care about the merchant/ the merchant’s product. Not you.

Solution: Provide a value add that encourages people to refer their friends to you. Are your reviews updated periodically? Do they get a bonus if they buy through your affiliate link? (E.g. similar to cashback affs, but you can offer bonuses like info products, tools, stickers, support etc) Do you have some unique value add like BBGeeks.com‘s reviews of network coverage organized by geography?
7. Problem: Your design is intended to yield a sale, once, and once-only.  

Solution: It’s OK to start with pushing your traffic to convert right away, you’re only ever going to get value from that customer once… so you should figure out how to get that traffic into an email (or phone, direct mail, personal contact) list you can monetize repeatedly. Generating leads for the aerospace field? Why not invite leads to your aerospace summer retreat?

6. Problem: Your claims aren’t credible.

Solution: Substantiate them with pics and video. Did you really buy that top fom the Gap? Show a picture of yourself in it with the matching bag. Weight-loss product really work? Show before/after or record yourself doing it, and prove the dates are real with newspapers in your vid.

5. Problem: You’re too enthusiastic about the affiliate product. Every reference to it on your site is linked, the “negatives” in your review are that the merchant is too helpful and generous etc… skeptical consumers don’t click.

Solution: Use one call to action above the fold, and one for every scroll of screen real estate downwards. How many pixels to a scroll? Check your analytics for visitors’ most common screen resolutions.

4. Problem: You overestimate how much persuasion people need and have too much copy.

Solution: Short copy vs long copy depends on several factors, chief amongst which is price and second degree of need. You can get names and emails with a title, picture and 2-3 bulleted benefits. (I once wrote a sales letter with 20 reasons to subscribe to my free RSS feed. Ooops.)

Note: Price includes total price, because people aren’t stupid and get that a $50 monthly subscription is a helluva lot more than a one-time $50 purchase.

3. Problem: You’re starting with visuals/templates as opposed to copy.

Solution: Form serves function. Figure out what you want to do, and create a page (or find a template) that matches that. Don’t start with visuals and then try to fit your copy/ purpose into it. For example, I’ve seen complicated SaaS software trying to sell me on a $50 subscription with hardly any copy on the landing page, because the lander featured a big spot for a hero image / video (which was meant for newbies very unfamiliar with the category of product, and thus didn’t help me) and then some horizontally aligned features with 3 lines of adwords-length copy each.

2. Problem: Your blog posts lack email subscription forms. I don’t mean in the sidebar or in a dhtml overlay-popup. I mean in the post.

Solution: Scott Brinker from IonInteractive.com (no association, though I did get a free review copy of their CRO book Honest Seduction) showed a brilliant type of landing page at Pubcon 2010. He featured a full length article featuring X ways to do Y… and then there was an opt-in form above the fold embedded in the copy, right-aligned, saying “Get 10 more ways to do Y”…

Obviously you can’t do that for every blog post, but your popular posts should have that sort of integration so you get max value out of each post. On the backend, you can need to figure out how to send a custom initial autoresponder if you want all these people on the same list, or you can create multiple duplicate lists.

Another more general way to do this is to have your forms tied to your blog post’s category or tag, in which case you have a general free report for the “seo” category” and another for the “affiliate” category.

1. Problem: Your landing page is leaky. Yes, affiliates have leaks too, not just merchants.

Solution: Get rid of all the extra junk on your landing page. Standard navigation on your product review blog post? Use a WP custom post type with a blank sidebar or comment it out for that page. Use Google Website Optimizer to test the two and watch CTR soar on the nav-free page.

p.p.s. Problem #11: WordPress is your default landing page creation/testing engine. Even using something like Thesis or Affiliate Theme, you’re strongly limiting your ability to test graphics, in terms of layout, hero shots, and more. Either use purpose built testing software that has a built-in landing page creation/editing, or get some graphics and html. I use Bharce for graphics and PSD to HTML/CSS for slicing.

photo credit: Shutterstock/SSGuy

tla starter kit

Related posts:

  1. Maximizing Profits With Website Design and Layout: Part III In Maximizing Profits With Website Design and Layout: Part I,...
  2. How to Mask Affiliate Links How to mask affiliate links is one of the most...
  3. Maximizing Profits With Website Design and Layout: Part I I was building new site this past weekend and I...
  4. Cloaking Affiliate Links, How and Why Today’s topic is one that gets asked about pretty regularly,...
  5. Maximizing Profits With Website Design and Layout: Part II In Maximizing Profits With Website Design and Layout: Part I,...

Advertisers:

  1. Text Link Ads - New customers can get $100 in free text links.
  2. BOTW.org - Get a premier listing in the internet's oldest directory.
  3. Ezilon.com Regional Directory - Check to see if your website is listed!
  4. Need an SEO Audit for your website, look at my SEO Consulting Services
  5. Directory Journal - Get permanent deep links in a search engine friendly directory
  6. LinkWheel SEO - Get Web 2.0 Backlinks
  7. TigerTech - Great Web Hosting service at a great price.
  8. Article-Writing-services.org - Article Writing Services creates quality content for websites and blogs at no cost to site owners.
  9. Link Building Services - Hire WeBuildLink.com for well-planned advanced link building campaigns. Very affordable. Contact us now for a FREE evaluation.
  10. Join 500+ top agencies & publishers who rely on The HOTH every day. US based, enterprise-grade support. High quality, contextually relevant links. 100% money back guarantee. Try it now.
  11. Krystal Glass Whiteboards - Glass writing boards for offices, boardrooms, and classrooms.

This post originally came from Michael Gray who is an SEO Consultant. Be sure not to miss the Thesis Wordpress Theme review.

10 Commandments of Affiliate Web Design

A Peek Under the Hood: How we Manage the SEOmoz Community

A Peek Under the Hood: How we Manage the SEOmoz Community


A Peek Under the Hood: How we Manage the SEOmoz Community

Posted: 17 Jan 2012 04:39 PM PST

Posted by jennita

Have you ever been a part of a community and wondered, “How does it all happen?” Well today is your lucky day! In the spirit of TAGFEE, I've decided to lift the Moz hood and show you what it takes to manage a large community. In fact, this is just the first post in a series of posts on Community Management.

Today I'll be explaining the who, what, when, where and how of how we manage the SEOmoz community. It's important to know who are the people behind the scenes, keeping the community in order and running smoothly, just as much as it is to know what exactly we consider the community to be and how we do it.

In the next posts, I'll dig deeper into subjects such as how we deal with negativity, how we gained over 100k Twitter followers and what we're planning to do for our Google+ strategy. For now, let's jump into the SEOmoz community and see how it's done.

Who Are We?

Over the last couple years, the community has grown immensely. It quickly became imperative to build a team to help take care of different aspects of the community. I simply couldn’t handle all aspects on my own anymore. So, before I jump too far into the what, why and how we manage the community, I’d like to introduce you to the “who.”

Peter Meyers (aka Dr. Pete)

Essentially Dr. Pete has been around the Moz community for about as long as Rand himself. :) No, really. He’s been an essential part of the community long before we even called it a community. Rand made the smart move long ago to bring Pete on board as an Associate.

Pete spends much of his time answering questions in Q&A (you’d be amazed at how much stuff this guy knows!) and writing on the blog. In fact he’s written some of top content on the blog for the past three years. He pretty much makes the rest of us (ok except maybe Rand) look bad at our unworthy content.

While he’s not helping manage the chaos of Q&A, writing on the blog, or being one of the funniest guys on Twitter, he runs User Effect, a successful usability and CRO company. Oh and if you’ve ever wondered if he’s a he’s a real doctor, read more here.

Casey Henry

Many of you may remember Casey from his excellent YouMoz posts that he wrote as a member of the community. The fact is, his community activity caught our eye and in 2010 we made him an Associate. At that time, he helped kill spam, answer questions in Q&A and did some dev work for us as well. It didn’t take long to realize that he was a great fit for Moz, so we hired him & moved him and his wife to Seattle.

As the resident Marketing Ninja, he manages many marketing projects as well as a number of dev responsibilities. His part within the Community Team is to help keep track of, and kill spam, spammers and scammers. Whether that’s through comments, PMs or otherwise, he’s the man on the case. He’s also quite active in Q&A and can “woot,” “whee,” and “beep” with the rest of us on the SEOmoz Twitter account.

You’ll often find him replying to Tweets to the SEOmoz account from his personal account, fixing link issues, answer questions and being an awesomely helpful guy. Aww.

Keri Morgret

Keri is well known in the industry as one of those amazing conference live-bloggers, and speaks about using negative keywords effectively for PPC. Having managed many forums and community sites in the past, including being a moderator at Sphinn, she is perfectly positioned to be a part of the Moz community team.

Currently Keri works out of her home near San Francisco as a (mostly) full-time Associate and runs her business Strike Models (go check out the site, it’s super cool) with her husband. She has quickly become an integral part of the team as well as the community in general. As the main Mozzer leading YouMoz management as well as Q&A she interacts all day long with community members. Talk about stealth, you may not realize it, but Keri pretty much knows everything that’s going on all the time. You think I’m kidding… I’m not.

Not only does she spend her time managing some of the on-site areas but she often helps out with the SEOmoz Twitter account as well. It can be a daunting task knowing that over 100k people will see your tweets (ok, in reality the number to actually see the tweets is quite less, but you get the point), but Keri jumps right in there. :) She’s also a huge help by cleaning out the Twitter “inbox” (more on that below) for me each morning.

If you’re ever curious about what’s going on at any particular time within the Moz Community, Keri is your woman.

Erica McGillivray

When we found out Erica was a founder as well as the President and Marketing Director of GeekGirlCon, we just knew she’d fit right into our community. :) With a background in SEO, Social, Email Marketing and event planning (pretty much marketing awesomeness) she easily jumped into the role of Community Attaché.

Erica can essentially do anything and everything that has to do with managing the community. A ninja in her own right. On any given day, you’ll find her managing our email marketing, answering questions in Q&A, reading through YouMoz posts, Tweeting from the Moz account, setting up webinars and organizing the upcoming MozCon.

Oh, and did I mention she’s a badass SEO? I’ve always felt strongly that you can’t manage a community unless you’re a part of the community yourself. Well Erica can talk the talk and walk the walk. Just be sure not to make her mad, she might pull these out.

Jen Lopez

Oh hi! That’s me. :) Just a quick background, I have a degree in Journalism, emphasis in Public Relations, but spent 10 years as a web developer before I turned into an SEO. Got hired as an SEO Consultant with SEOmoz in early 2009, then in January 2010 we gave up consulting. Doh! Hello Community Management. It was at that point that I created the position and over the last few years it has grown into a real job.

So what will you find me doing on an average day? I find myself Managing Twitter, Facebook & Google+, combating spam, answering questions in Q&A (usually that Keri or Erica assign me ;)), managing the blog schedule and content, responding to help tickets as needed, commenting on community posts outside of SEOmoz, and any other random thing that comes up during the day.

The truth is, my job rocks. Sure I deal with trolls sometimes, but that’s what makes the job interesting.

Mozzers

That’s you, you and YOU. Whether it’s Gianluca responding to a Tweet about SEOmoz while us West Coasters are sleeping or Ryan answering a question in Q&A about a technical PRO issue, you guys help us every day to manage the community. This is a very important aspect of the community and one that makes people want to be a part of it. It’s not just one person managing everyone else with an iron fist (OK I admit sometimes I have dreams this will happen ;), it’s all the Moz staff and community helping each other out. Holy. geeky. happiness.

What Do We Do?

Obviously there’s no way to really describe everything that we do in one blog post. When you work with a community, your day can change in an instant. Sometimes an issue comes up and you’re helping to manage an issue since you’re the public “face” of the community on the social sites. Other times you wake up to a hashtag being created and hundreds of posts being written about you. *huge grin*

Let me take a few moments to walk you through the major aspects of managing the SEOmoz community. This really is only a high-level look at we do each day. The plan is to expand on many of these areas as separate blog posts. For now, here are the what, when how and who of what it means to manage the community.

Blog

When Rand started the SEOmoz blog years ago, I’m sure he never quite imagined that it would be the base of such an expansive and amazing community. It really has become the center of everything Moz. Think about this; an average blog post gets around 40 thumbs up, 50 comments and 800+ Tweets. That’s a lot to keep up with each day!

What
Since you’re already here you probably know that we post content not only about SEO, but about Inbound Marketing in general. We focus on creating actionable takeaways and look for authors who can bring something new to the community. We like to cover hot topics in the industry but we don’t necessarily cover them as “news.” We’ll also post new updates/improvements/issues about the PRO product on the blog as well.

When
We have at least one new post each day, and sometimes publish a second one during the day (Pacific U.S. time).

How
We have a custom blog editor that we use to create the posts. When it comes to managing comments, we have a system that helps us to moderate them if they meet a certain criteria. This way we don't allow a comment to get published if we suspect that it's spam, and one of us has to approve it.

When it comes to comments and how we handle them, we take the community very seriously and will ban users if they don't "play by the rules." This is an area that I'll dig into a bit deeper on another post, but essentially, you're in our home and we request that you handle yourself as a professional.

Who
I manage the blog schedule and make sure we have a post going up each night. The idea is to set the schedule at least a couple weeks out, with openings here and there for hot topics or new authors we want to introduce. All of us watch for spam comments throughout the day and Casey set up a way to moderate and kill spam before you guys ever even see it. :)

The entire team helps manage the comments, detect spam and make sure things aren't getting out of hand anywhere.

YouMoz

Writing a post for YouMoz is a great way to get your name out in the community (remember above I mentioned Casey started out as a YouMoz author!), plus you get a nice link. ;)

What
Similar to the main blog, the community loves to read actionable posts. However, in YouMoz we do have a little more leeway than we do on the main blog. We’ll publish posts on topics that we don’t normally cover on the blog. The best part about that though is that if the post does really well in YouMoz and gets promoted to the main blog, then we see more diversity in the subjects.

When
We try to post at least one YouMoz post per day and some days we even have two. It used to take 6-8 weeks to get through the queue and now it’s only two weeks, tops! (A HUGE thanks to Keri for cleaning this up.)

How
Any member can create a blog post using our blog editor and submit it to be read by our editors. If it’s approved, it gets published to the YouMoz blog. We recently added a “Read Me First” page that has helped us to get higher quality posts submitted.

Who
Keri is the main point of contact when it comes to YouMoz. She’s done an excellent job cleaning up the queue and making it easier and quicker for authors to get their posts published. Whenever necessary, Erica and I also jump in and help by editing and approving content, declining posts, etc.

When it comes to promoting YouMoz posts to the main blog, I’m usually the one that makes that decision. There is no “golden rule” on how to get promoted and it sometimes depends on whether there’s a spot open on the main blog. When it’s good, and the community likes it, it will get promoted. :)

Q&A

When we launched the new Q&A system last year, we honestly weren’t quite sure exactly how it would be received. We were pleasantly surprised to see how much people loved to both ask and answer questions! It took us a few months to get the hang of managing it properly, but with the help of the team, we’re quite happy with this PRO feature.

What
You’ll find pretty much any and every kind of content related (sometimes barely) to internet marketing. Since it is for PRO members only, it’s fairly easy to keep spam in check since they usually don’t like to pay money. What we mainly look for here is to make sure that people are keeping in line with our Community Guidelines.

When
Twenty-four hours a day the community is asking and answering questions. Plus since many of our Associates who answer questions live in London and other “across the pond” cities, we’re able to have coverage all day long. Whee!

How
This all happens through our own proprietary Q&A system. We get asked quite often if we built this ourselves or used an out-of-box solution. This is 100% a homegrown system, which does have its own set of bugs. :) This system is only about a year old and is about 500% better than the old Q&A.

Who
While Keri manages both private and public Q&A, Dr. Pete and Erica also play a heavy role here. Keri makes sure your private questions are answered by our team of staff and expert Associates. Dr. Pete and Erica are usually in there answering questions for a few hours each day as well. Many of our Associates play a role in Q&A and you’ll see a number of them answering questions and endorsing answers every day. Even Rand goes in quite often and replies to questions personally, even ones asking how to contact him. :)

Social Community

Over the past couple years our community has grown by leaps and bounds through the help of social media sites. You may have noticed that we engage quite heavily on Twitter, Facebook, Google+ and LinkedIn. It’s not that we’ve ignored other sites; we just tend to focus our energies on these four.

Since many of our members follow us on these sites in addition to the blog, we had to figure out a way to be somewhat unique in all areas. Nothing worse than content overkill (ok, yea yea there really are lots of things that are worse, but you get my point.) Here’s a quick walk through of how we manage these four social media sites here at SEOmoz.

Twitter

Twitter is a bit of our “catch-all” and has the biggest following. With well over 100k followers, as expected this channel is used for customer service, SEO advice, content promotion and other forms of marketing.

What
We tweet about SEOmoz content, PRO membership updates, site outages, tool issues, tool upgrades/improvements, YouMoz posts, and anything related to the SEOmoz community. Additionally you’ll see a ton of replies to customer service type inquiries, issues, problems, questions, kudos, high-fives, etc.

We keep the tone of the tweets as Mozzy as possible, and speak as if we are Roger. It makes may day when someone tweets to us saying “Hey Roger, thanks for great app” or something along those lines. It’s all about Roger!

When
Our community is very international, so we can’t just tweet from 9am-5pm Pacific time. We need to be available as often as possible to respond to questions, requests and such. While we do need to sleep at some point, you’ll notice that we have people covering Twitter from about 7am until around 11pm Pacific.

Keri helps in the morning since she works from home and can jump right in. Then when I get into the office I take over for the day. She again takes over from about 5-8pm, when I jump back on.

We also schedule tweets of our content to go out during “on” hours for many in our community. While we don’t schedule tweets that ask questions and specifically ask for engagement, we do need to schedule them to promote our content. On a side-note our most retweeted tweets usually happen between 2-4am Pacific. Whee!

How
We use CoTweet to manage Twitter, which allows multiple users to manage multiple accounts. It makes it easy to assign tweets to others plus you can tag tweets and set up extensive searches. I wrote a bit more about it here on Marketing Pilgrim.

Who
While I’m the main person managing the account, at any time throughout the day you may find Keri, Erica or Casey tweeting as well. I also encourage staff to reply to tweets if the see them and simply cc: @SEOmoz so we know it’s been handled. For example, Rand will often do this. He’ll see a tweet before we do and will respond plus let us know it’s been handled. It’s a great way to give people direct interaction with Mozzers as well!

Facebook

As I mentioned before, we wanted to figure out a different approach to Facebook so it wasn’t just the same ol’ content as Twitter. So we decided to make Facebook, the “face” of SEOmoz.

What
Yes, we do post all of our content on Facebook, the same as we do on Twitter but in addition to that we’ll post fun things about the company or publish photos of events. One thing I love to do is get photos from a Meetup or conference and post them to Facebook. We find that by tagging people in photos and having them tag themselves and others, we get a boost in “likes” each time.

Additionally, I love to ask questions of the community here. People love to add their opinion and Facebook is a simple and easy way to do it. With so many people logged in all day either on their computers or through their phones or tablets, it’s very easy to get people’s attention on Facebook.

We also love to change Roger’s outfit and add some “life” to him.

When it comes to comments and wall posts, we manage them just as we would manage comments on our site. If it’s spam we remove it, if it’s obscene or someone cusses, we remove it. Essentially, as long as you’re on-topic and not a jerk, we keep your posts. :)

When
The timing of Facebook is somewhat similar to Twitter however we don’t schedule Facebook posts. We haven’t quite nailed down a science as to what time is best to post for us yet, but it’s something we’ll probably focus on this year.

How
Although I could use an outside app to manage Facebook, I choose not to. Facebook seems to not show posts created from outside apps as much in people’s feeds as they do posts directly from Facebook. I want us to show up in those feeds as often as possible. Plus I just like to know I’m seeing what users are seeing on our wall.

Who
Again, I’m the main person to manage the Facebook page, however Erica, Casey and quite a few others not a part of the official “Community team” others also have admin rights.

Google+

As soon as Google+ brand pages came out, we jumped right on it. It has taken us a couple months to shake the bones out and figure out a strategy, but I think we’re going on a nice track now.

What
In order to not seem like drones showing the exact same content over and over, on our Google+ page we not only promote our own content, but we like to promote other’s as well. This is a great place for us to introduce our readers to hot topics or content on our sites that we think they’d be interested in.

However the hottest content we’ve found, is our “Whiteboard+” video series. Essentially, it’s a Whiteboard Friday type of video, but is only posted to Google+. We’ll keep testing this and see what works best, so you might find some new content on there soon as well. J

When
Right now, since it’s so new we don’t have a specific schedule for posting to Google+. Over this past weekend we posted a special Whiteboard+ video on Friday night at 8pm Pacific and it went crazy! It currently has 353 pluses, 382 shares and 101 comments. And most of this happened over the weekend! I think we may be on to something here…

How
As far as I know there aren’t any third-party apps out there that let you set up Google+ posts, so I’m logged into Google+ all day long. I’m very happy that they started showing alerts when we get a new follower, comment, plus, etc. It makes it easier to manage than hitting refresh and scanning the page (like I did at first).

Who
Well since the account right now is connected to my personal account, I’m the only one to manage it. I sincerely hope that they allow the ability to add more users soon. A gal needs a break sometime!

LinkedIn

Ahh LinkedIn, the stepchild of our social efforts for far too long! Luckily we’re in the middle of building out our strategy, since so many people from the community are there.

What
As with the other social sites, we’ve set up our RSS feeds from both the Blog and YouMoz to show up on our group page. Additionally we’ll post updates about our tools, webinars coming up and other information the group may be interested in.

We also manage people joining the group and as with the other sites, we manage comments and posts the way we manage our own blog. There is daily management on clearing out spam and keeping it clear of sales pitches.

We’re also trying new things, like creating a book club and keeping the community connected in other ways.

When
We usually update LinkedIn during “regular business hours” on the West coast.

How
LinkedIn actually has some pretty good management features for letting you decline users and keeping out spam. Erica manages all this through LinkedIn directly.

Who
Erica manages the LinkedIn company page and group. Others on the team have admin access and can jump in at any time. But honestly Erica does such a great job, we haven’t really had to!

Whew.

Whether you're a member of the Moz community or you manage a community of your own, I hope you've found this insightful and gives you a better understanding of the community management process. Honestly, this just barely touches the surface of what happens behind the scenes. I look forward to getting more in-depth on many of these topics. If you have specific topics you'd like to see covered as it pertains to managing a community, running Social Media sites or even dealing with unruly members, please let me know in the comments. I'd love to cover topics you are interested in.

Oh and around here, our motto is "If all else fails, eat ice cream."


Do you like this post? Yes No

58 billion URLs in the Latest, Largest Linkscape Index Update Yet

Posted: 17 Jan 2012 08:01 AM PST

Posted by randfish

I've got good news. Today marks a new Linkscape index (only 14 days after our previous index rollout) which means new data in Open Site Explorer, the Mozbar, the Web App and the Moz API. It's also more than 60% larger than our previous update in early January and shows better correlations with rankings in Google.com; I'm pretty excited.

For the past couple years, SEOmoz has focused on surfacing quality links and high quality, well-correlated-with-rankings metrics to help provide a link graph that shows off a large sample of the web's link graph. However, we've heard feedback that this isn't enough and may not be exactly what many who research links are seeking (or at least, it's not fulfilling all the functions you need). We're responding by moving, starting with today's launch, to a new, consistently larger link index.

Today's data is different from how we've done Linkscape index updates in the past. Rather than take only those pages we've crawled in the past 3-4 weeks, we're using all of the pages we've found since October 2011, replacing anything that's been more recently updated/crawled with a newer version and producing an index more like what you'd see from Google or Bing (where "fresh" content gets recrawled more frequently and static content is crawled/updated less often). This new index format is something that will let us expose a much larger section of the web ongoing, and reduces the redundancies of crawling web pages that haven't been updated in months or years.

Below are two graphs showing the last year of Linkscape updates and their respective sizes in terms of individual URLs (at top) and root domains (at bottom):

Linkscape Index Size Over Time

As you can see, this latest index is considerably larger than anything we've produced recently. We had some success growing URL counts over the summer, but this actually lowered our domain diversity (and hurt some correlation numbers of metrics) so we rolled back to a previous index format until now.

This means you'll see more links pointing to your sites (on average, at least) and to those of your competitors. Our metrics' correlations are slightly increased (I hope to show off more detailed data on that in a future post with help from our data scientist, Matt), which was something we worried about with a much larger index, but we believe we've managed to retain mostly quality stuff (though I would expect there'll be more "junk" in this index than usual). The oldest crawled URLs included here were seen 82 days ago, and the newest stuff is as fresh as the New Year.

Despite this mix of old + new, the percent of "fresh" material is actually quite high. You can see a histogram below (ignore the green line) showing the distribution of URLs from various timeframes going into this new index. The most recent portion, crawled in the last 2/3rds of December, represents a solid majority.

Histogram of crawl for Index 49

Let's take a look at the raw stats for index 49:

  • 58,316,673,893 (58 billion) URLs
  • 639,806,598 (639 million) Subdomains
  • 135,392,083 (135 million) Root Domains
  • 617,554,278,005 (617 billion) Links
  • Followed vs. Nofollowed
    • 2.10% of all links found were nofollowed
    • 56.50% of nofollowed links are internal
    • 43.50% are external
  • Rel Canonical - 11.79% of all pages now employ a rel=canonical tag
  • The average page has 87.36 links on it
    • 73.06 internal links on average
    • 14.29 external links on average  

In addition to this good news, I have some potentially more hilarious and/or tragic stuff to share. I've made a deal with our Linkscape engineering group that if they release an index with 100+ billion URLs by March 30th (just 72 days away), I will shave/grow my facial hair to whatever style they collectively approve*. Thus, you may be seeing a Whiteboard Friday with a beardless or otherwise peculiar-looking presenter in the early Spring. :-)

As always, feedback is welcome and appreciated on this new index. If some of the pages or links are looking funny, please let us know.

* 20th century European dictator mustaches excluded


Do you like this post? Yes No

How To Handle Downtime During Site Maintenance

Posted: 17 Jan 2012 03:30 AM PST

Posted by Frederik Hyldig

In this post I will explain how to handle cases of planned downtime. That is, a short period of time wherein you purposely make your website inaccessible. This can be due to significant changes to the site or because of server maintenance.

It should always be the last resort to make the entire website inaccessible, but in some cases it can be necessary. Below you will find suggestions as to how to proceed with SEO in mind.

Tell both humans and robots that it's only a temporary shutdown.

In the case of a temporary shutdown, one should always inform both humans (visitors) and robots (search engines) so that they are aware that it is a planned closure, and that it is just temporary. If possible one should also state when the website is expected to be back online. This will ensure that both humans and robots will return at a later time to find what they expected to find in the first place.

There are two mistakes often seen when a website is made temporarily unavailable:

Mistake 1 - All files are removed from the server.

When both humans and robots attempt to find the website, it will result in a 404 error, which means that the requested page cannot be found. This informs neither humans nor search engines on what is actually happening. One will typically be shown a page that looks something like this:

The worst case scenario is that people will think the website no longer exists, and will therefore give up trying to find it again. Search engines handle this situation in a similar fashion. To them, a 404 error means that the page no longer exists, and it will in time be deleted from their index.

Mistake 2 - A simple page is put on the server with a short message explaining the closure.

An alternative solution to the one above is to remove all files and then put one very simple file on the server that explains why the website is closed in one or two sentences. All the old pages are then redirected to this file.

This method may tell humans what the problem is, but it still makes no sense to the search engines. The search engines can in fact become so confused by this that they believe that the temporary state of the website – the few sentences explaining the problem – is the permanent website in future.

Depending on how the redirection of the other pages has been carried out, one also risks the search engines thinking that all the other pages of the website have been (re)moved, and that only the front page is to be ranked in search results. This is a sure way to lose rankings.

Briefly on HTTP Status codes

Every time you visit a website your browser receives a message from the server that hosts the website. This message is called a HTTP Status code. As a SEO it is necessary to understand what the most important codes mean.

200 OK - The request has succeeded. This is the standard response for successful HTTP requests.

301 Moved Permanently - The requested resource has been assigned a new permanent location. This and all future requests should be directed to the given location. This status code is used for 301 redirects. In most instances, the 301 redirect is the best method for implementing redirects on a website. A 301 redirect will pass most, if not all the linkjuice from the original location.

302 Found - The requested resource resides temporarily at a different location. By using a 302 redirect instead of a 301, search engines will know that this is only a temporary state. No appreciable amount of linkjuice will be passed.

404 Not Found - The server has not found anything matching the requested location. No indication is given of whether the condition is temporary or permanent. In time, the page will be removed from the search engine's index.

503 Service Unavailable - The server is currently unavailable (this could be due to overload or maintenance). Search engines will know that this is a temporary state. This status code should be used when taking down a site for maintenance.

You can read more about HTTP status codes here. Also check out this infographic on HTTP status codes by Dr. Pete.

How to inform search engines that the downtime is temporary.

If you take down your website temporarily, you must inform search engines such as Google. As you could read above, this is done by utilizing the HTTP status code: 503 Service Unavailable, that informs the search engines that the server is temporarily unavailable. To do this one must first create a file that returns a 503 status code on the server. When the search engine sees this, it will understand the situation. This can be done by copying the four lines below into Notepad (or the like) and saving it as 503.php. You must then place this file in the root of your server.

The first two lines tell us that it is a 503 status code, and the last line is used to tell when the website is expected to be online again. Google understands this message, so it is possible to tell Google when to visit the website again. You must either provide a number (seconds) or a date. If you live in Denmark like I do and you expect to return on the 5th of January 2012, at 14:00, you must put down:

Notice that I wrote 13:00:00 in the code, even though I wrote 14:00:00 above. This is due to the fact that the time must be provided in GMT/UTC, which is, in my case, 1 hour behind local time.

But it is not enough to just put a 503 message on your server. You will receive visitors (Google included) from many different sources and to all sorts of pages of your website. They must all be redirected to the message explaining that the website is temporarily closed.

On an Apache/Linux server, this can be easily solved by using a .htaccess file to redirect all the pages towards the 503.php file. The .htaccess file is often used for 301 redirects, but that is not our purpose here. We will use a 302 redirect. You may have been previously warned about using this sort of redirect, and for good reason. It can do a great deal of damage if not used correctly. But in this case, it must be used, and in fact a 301 redirect would be detrimental in its place.

Save the 6 following lines as a .htaccess file and place it in the root of your server as well.

The 'R' in the last line indicates that this is a 302 redirect. R is 302 by default. To create a 301 redirect, it would have said [R=301, L]. The clever thing about this file, however, is that we can give ourselves access to the site and simultaneously show everyone else a 503 message. Let’s say you have the following IP address: 12.345.678.910. You then put the numbers in line 4 as shown below:

When you have placed the two files (503.php and .htaccess) on your server, you’re done. You now have peace and quiet to tinker with your website, as long as you leave those two files in the root of your server – and if Google visits, they’ll know that the site will be back later, and you’ve even let them know when to try again.

But what about passing on the message to your visitors?

How to tell your visitors that the website is only closed temporarily.

With a few additions to the 503.php file, which we made just before, we can pass on a message to visitors:

The above will result in the following message when one visits the website:

And if we look at the response the server provides Google with, with a tool such as FireBug, Web-Sniffer.net or the like, we get the following:

Now you have informed both humans and robots to come back later. This is the best way to handle server maintenance in order to prevent Google from indexing the temporary version of the website.

It should be possible to get through a temporary closure without the website’s rankings suffering serious consequences.

A Quick Note about SOPA Protests

Keri from SEOmoz here! This post is also helpful if you're wanting to protest SOPA tomorrow (January 18th) and want to minimize the effect on your rankings. Pierre Far from Google shared a a post on Google+ called Website outages and blackouts the right way that you might want to check out for some information straight from Google.


Do you like this post? Yes No