New Message Center notifications for detecting an increase in Crawl Errors

New Message Center notifications for detecting an increase in Crawl Errors: "Webmaster Level: All

When Googlebot crawls your site, it’s expected that most URLs will return a 200 response code, some a 404 response, some will be disallowed by robots.txt, etc. Whenever we’re unable to reach your content, we show this information in the Crawl errors section of Webmaster Tools (even though it might be intentional and not actually an error). Continuing with our effort to provide useful and actionable information to webmasters, we're now sending SiteNotice messages when we detect a significant increase in the number of crawl errors impacting a specific site. These notifications are meant to alert you of potential crawl-related issues and provide a sample set of URLs for diagnosing and fixing them.

A SiteNotice for a spike in the number of unreachable URLs, for example, will look like this:


We hope you find SiteNotices helpful for discovering and dealing with issues that, if left unattended, could negatively affect your crawl coverage. You’ll only receive these notifications if you’ve verified your site in Webmaster Tools and we detect significant changes to the number of crawl errors we encounter on your site. And if you don't want to miss out on any these important messages, you can use the email forwarding feature to receive these alerts in your inbox.

If you have any questions, please post them in our Webmaster Help Forum or leave your comments below.

Posted by Pooja Shah and Jonathan Simon

"

To err is human, Video Sitemap feedback is divine!

To err is human, Video Sitemap feedback is divine!: "Webmaster Level: All

You can now check your Video Sitemap for even more errors right in Webmaster Tools! It’s a new Labs feature to signal issues in your Video Sitemap such as:

• URLs disallowed by robots.txt
• Thumbnail size errors (160x120px is ideal. Anything smaller than 90x50 will be rejected.)

Video Sitemaps help us to better crawl and extract information about your videos, so we can appropriately feature them in search results.

Totally new to Video Sitemaps? Check out the Video Sitemaps center for more information. Otherwise, take a look at this new Labs feature in Webmaster Tools.

Written by Jackie Lai, Video Search Team

"

Verification time savers — Analytics included!

Verification time savers —  Analytics included!: "Webmaster Level: All

Nobody likes to duplicate effort. Unfortunately, sometimes it's a fact of life. If you want to use Google Analytics, you need to add a JavaScript tracking code to your pages. When you're ready to verify ownership of your site in other Google products (such as Webmaster Tools), you have to add a meta tag, HTML file or DNS record to your site. They're very similar tasks, but also completely independent. Until today.

You can now use a Google Analytics JavaScript snippet to verify ownership of your website. If you already have Google Analytics set up, verifying ownership is as simple as clicking a button.


This only works with the newer asynchronous Analytics JavaScript, so if you haven't migrated yet, now is a great time. If you haven't set up Google Analytics or verified yet, go ahead and set up Google Analytics first, then come verify ownership of your site. It'll save you a little time — who doesn't like that? Just as with all of our other verification methods, the Google Analytics JavaScript needs to stay in place on your site, or your verification will expire. You also need to remain an administrator on the Google Analytics account associated with the JavaScript snippet.

Don't forget that once you've verified ownership, you can add other verified owners quickly and easily through the Verification Details page. There's no need for each owner to manually verify ownership. More effort and time saved!


We've also introduced an improved interface for verification. The new verification page gives you more information about each verification method. In some cases, we can now provide detailed instructions about how to complete verification with your specific domain registrar or provider. If your provider is included, there's no need to dig through their documentation to figure out how to add a verification DNS record — we'll walk you through it.


The time you save using these new verification features might not be enough to let you take up a new hobby, but we hope it makes the verification process a little bit more pleasant. As always, please visit the Webmaster Help Forum if you have any questions.

Written by Sean Harding, Software Engineer

"

Showing more results from a domain

Showing more results from a domain: "Webmaster Level: All

Today we’ve launched a change to our ranking algorithm that will make it much easier for users to find a large number of results from a single site. For queries that indicate a strong user interest in a particular domain, like [exhibitions at amnh], we’ll now show more results from the relevant site:



Prior to today’s change, only two results from www.amnh.org would have appeared for this query. Now, we determine that the user is likely interested in the Museum of Natural History’s website, so seven results from the amnh.org domain appear. Since the user is looking for exhibitions at the museum, it’s far more likely that they’ll find what they’re looking for, faster. The last few results for this query are from other sites, preserving some diversity in the results.

We’re always reassessing our ranking and user interface, making hundreds of changes each year. We expect today’s improvement will help users find deeper results from a single site, while still providing diversity on the results page.


Written by Samarth Keshava, Software Engineer

"

Clarifying a couple points

Clarifying a couple points: "

[Just as a reminder: everything below is my personal opinion. I haven't sent it to anyone else at Google for a review, etc.]

Valleywag used a recent podcast I did as material for two points in Six Delusions of Google’s Arrogant Leaders. The two assertions that used my comments as material were “Google’s wealth means Google ‘gets it’” and “Google must sacrifice user privacy to grow.”

Valleywag has either misinterpreted what I said, or I didn’t express myself clearly, because I don’t believe either of those claims. I’ll try to explain the intent of what I said, in case I wasn’t clear during the podcast. I’ll address the latter claim first (“Google must sacrifice user privacy to grow”), because I certainly don’t believe that “Google must sacrifice user privacy to grow.” I think Google benefits the most when users understand what Google is doing and why; I also think that user trust in Google (and by extension our privacy policies) is paramount to our success.

A good example is our Google Ad Preferences page. As one blog concluded a couple days ago: “Google’s Ad Preference Manager, with its persistent opt-out plug-in, offers precisely the kind of robust opt-out that privacy advocates have always demanded.” And it’s not that we’re shy about talking about privacy; Googlers Alma Whitten and Nicole Wong recently talked privacy for an Ars Technica article that came out earlier this week. It’s a long article, but an example useful fact is that if X is the number of people who visit the Ad Preferences page and opt out, 10X people don’t opt out and 4X people actually edit their categories to improve the targeting relevance of the ads they see. Let me say that again: four times as many people change their settings to make their ads *more* relevant than opt out of interest-based targeting. I think the Ad Preferences page is a good example where users get more transparency and control regarding their privacy.

Another example where Google helps your privacy (rather than sacrificing it) is the Google Dashboard. This is a single site that gives you an overview of what information Google has from various services, and allows you to edit and to manage settings. This is another example where Google is trying to give more information to users, not less. I could point out lots of examples where we try to debunk privacy misconceptions. Where we actively fight for our users’ privacy. Or where we talk about privacy and engage in debates about user privacy. And of course there’s Google’s full privacy center (with videos!) at http://www.google.com/privacy.html . Suffice it to say, I don’t believe that Google “must sacrifice user privacy to grow.”

Okay, what about the other claim that Valleywag used me for: “Google’s wealth means Google ‘gets it’”? Ryan Tate wrote “It’s a truly bizarre moment, in which Cutts defends some horrendous management decisions based on Wall Street trades.” I don’t agree with that either, so let me try to clarify. Eric Schmidt joined the company in 2001. The first time I got to meet Eric was at the weekly TGIF meeting where he was introduced to the wider company. He answered questions for an hour, and I thought his answers were spot on. He was one of the original authors of lex, a well-known Unix utility that I had used in the past, so I knew that he was also a solid engineer and technologist. Schmidt also had experience at large companies (Sun and Novell).

All in all, I was very happy and impressed that Eric was joining Google. When I went home that day, my wife asked what had happened at work. And I replied with something like “I think the value of our stock options just went up a lot.” What I meant by that was that I thought Google had recruited the perfect person to lead the company from start-up to the next level. I still believe that. Eric has been a truly great CEO–and I’m not just saying that because for the last several years he has worked for $1 a year. Maybe I didn’t tell the anecdote well or clearly, but my intent was to explain that I think Eric Schmidt has been a great CEO right from the beginning of this decade, not to defend any decisions “based on Wall Street trades.”

If you want to listen to the full podcast, it’s available, but I hope this post helps to clarify.

"

More material for web site owners

More material for web site owners: "

Here’s some more stuff you should know about.

- I did a monster-long interview with Eric Enge. I think the interview lasted an hour or something like that, and we covered several areas in depth.

- Next, take a break and go read this post by Rhea Drysdale. Heck, maybe send her a donation by Paypal (Added: the address if you want to send a Paypal donation is rhea_drysdale [at] yahoo [dot] com). Rhea took on a big fight for the benefit of the SEO industry, saw it through to the end — and won! In the process, she earned the sort of credibility that you just can’t buy.

- We continue to answer webmaster questions over on the YouTube webmaster video channel. My recent favorite video is an eight-minute discussion of “What are some good link-building techniques?”

We now have over 200 videos live on the webmaster video channel, including topics such as “Is it worth spending time on tags and categories?”

You might want to check out the video channel; there’s a lot of good material there. You can also follow me or the Google webmaster account on Twitter; we often tweet when new webmaster videos are released.

"

New 30 day challenge: not replying to outside email

New 30 day challenge: not replying to outside email: "

I just wrapped up two 30 day challenges (no sugar for 30 days and no iPhone for 30 days). I’ll try to report back what I learned soon, but in the mean time I wanted to alert you that today I’m starting a new 30-day challenge: not replying to outside email.

When I announced my first 30 day challenge and did a poll on what challenge to do next, “no email” was near the bottom of the list. But there’s good reasons for me to try to reduce my time on email right now:

- I have a big internal project at work (nothing related to webspam) that’s going to take all my time for a couple weeks.

- I’m also planning to take a couple weeks of vacation time in the next month.

So the fact is that I probably wouldn’t have had much time to reply to outside email in the next month anyway. Also, I’ve noticed that sometimes I spend 1-2 hours a day responding to outside email. I’d like to re-assess whether that’s the best use of my time. For example, if I spent that time on more scalable ways of communicating, it might help more people. Taking a step away from responding to outside email might also help me find better ways to manage that communication. Maybe someone else can help me. Maybe I can find ways to fix what people email me about (e.g. modify or improve the spam report process so that people don’t feel the need to email me directly with spam reports). This is a chance to re-assess how I’m spending a fair chunk of my time at work and look for a better solution.

I’m not taking the attitude that “if I reply to a single external email, then I’ve failed,” because emergencies do happen. Rather, I’m going to make a deliberate effort to respond as little as possible to outside work-related emails. At any rate, I think I’ll learn something new from this challenge, so I hope folks will support me in this experiment. Wish me luck, and feel free to provide suggestions or tips in the comments about how you tackle the email monster, or ideas for new 30-day challenges.

"