Just How Smart Are Search Robots?

---

Just How Smart Are Search Robots? Posted: 29 Nov 2011 12:51 PM PST Posted by iPullRank Matt Cutts announced at Pubcon that Googlebot is “getting smarter.” He also announced that Googlebot can crawl AJAX to retrieve Facebook comments coincidentally only hours after I unveiled Joshua Giardino's research that suggested Googlebot is actually a headless browser based off the Chromium codebase at SearchLove New York. I'm going to challenge Matt Cutts' statements, Googlebot hasn't just recently gotten smarter, it actually hasn’t been a text-based crawler for some time now; nor has BingBot or Slurp for that matter. There is evidence that Search Robots are headless web browsers and the Search Engines have had this capability since 2004. Disclaimer: I do not work for any Search Engine. These ideas are speculative based on patent research done by Joshua Giardino, myself, some direction from Bill Slawski and what can be observed on Search Engine Results Pages. A headless browser is simply a full-featured web browser with no visual interface. Similar to the TSR (Terminate Stay Resident) programs that live on your system tray in Windows they run without you seeing anything on your screen but other programs may interact with them. With a headless browser you can interface with it via a command-line or scripting language and therefore load a webpage and programmatically examine the same output a user would see in Firefox, Chrome or (gasp) Internet Explorer. Vanessa Fox alluded that Google may be using these to crawl AJAX in January of 2010. However Search Engines would have us believe that their crawlers are still similar to Unix’s Lynx browser and can only see and understand text and its associated markup. Basically they have trained us to believe that Googlebot, Slurp and Bingbot are a lot like Pacman in that you point it in a direction and it gobbles up everything it can without being able to see where it’s going or what it’s looking at. Think of the dashes that Pacman eats as webpages. Every once in a while it hits a wall and is forced in another direction. Think of SEOs as the power pills. Think of ghosts as technical SEO issues that might trip up Pacman and cause him to not complete the level that is your page. When an SEO gets involved with a site it helps a search engine spider eat the ghost; when they don’t Pacman dies and starts another life on another site.  That’s what they have been selling us for years the only problem is it’s simply not true anymore and hasn’t been for some time. To be fair though Google normally only lies by omission so it’s our fault for taking so long to figure it out. I encourage you to read Josh’s paper in full but some highlights that indicate this are: A patent filed in 2004 entitled “Document Segmentation Based on Visual Gaps” discusses methods Google uses to render pages visually and traversing the Document Object Model (DOM) to better understand the content and structure of a page. A key excerpt from that patent says “Other techniques for generating appropriate weights may also be used, such as based on examination of the behavior or source code of Web browser software or using a labeled corpus of hand-segmented web pages to automatically set weights through a machine learning process.”   The wily Mr. Cutts suggested at Pubcon that GoogleBot will soon be taking into account what is happening above the fold as an indication user experience quality as though it were a new feature. That’s curious because according to the “Ranking Documents Based on User Behavior and/or Feature Data” patent from June 17, 2004 they have been able to do this for the past seven years. A key excerpt from that patent describes “Examples of features associated with a link might include the font size of the anchor text associated with the link; the position of the link (measured, for example, in a HTML list, in running text, above or below the first screenful viewed on an 800.times.600 browser display, side (top, bottom, left, right) of document, in a footer, in a sidebar, etc.); if the link is in a list, the position of the link in the list; font color and/or attributes of the link (e.g., italics, gray, same color as background, etc.);” This is evidence that Google has visually considered the fold for some time. I also would say that this is live right now as there are instant previews that show a cut-off at the point which Google is considering the fold.   It is no secret that Google has been executing JavaScript to a degree for some time now but “Searching Through Content Which is Accessible Through Web-based Forms” shows an indication that Google is using a headless browser to perform the transformations necessary to dynamically input forms. “Many web sites often use JavaScript to modify the method invocation string before form submission. This is done to prevent each crawling of their web forms. These web forms cannot be automatically invoked easily. In various embodiments, to get around this impediment, a JavaScript emulation engine is used. In one implementation, a simple browser client is invoked, which in turn invokes a JavaScript engine.” Hmmm…interesting. Google also owns a considerable amount of IBM patents as of June and August of 2011 and with that comes a lot of their awesome research into remote systems, parallel computing and headless machines for example the “Simultaneous network configuration of multiple headless machines” patent. Though Google has clearly done extensive research of their own in these areas. Not to be left out there’s a Microsoft patent entitled “High Performance Script Behavior Detection Through Browser Shimming” where there is not much room for interpretation; in so many words it says Bingbot is a browser. "A method for analyzing one or more scripts contained within a document to determine if the scripts perform one or more predefined functions, the method comprising the steps of: identifying, from the one or more scripts, one or more scripts relevant to the one or more predefined functions; interpreting the one or more relevant scripts; intercepting an external function call from the one or more relevant scripts while the one or more relevant scripts are being interpreted, the external function call directed to a document object model of the document; providing a generic response, independent of the document object model, to the external function call; requesting a browser to construct the document object model if the generic response did not enable further operation of the relevant scripts; and providing a specific response, obtained with reference to the constructed document object model, to the external function call if the browser was requested to construct the document object model."(emphasis mine) Curious, indeed.  Furthermore, Yahoo filed a patent on Feb 22, 2005 entitled  "Techniques for crawling dynamic web content" which says "The software system architecture in which embodiments of the invention are implemented may vary. FIG 1 is one example of an architecture in which plug-in modules are integrated with a conventional web crawler and a browser engine which, in one implementation, functions like a conventional web browser without a user interface (also referred to as a "headless browser")." Ladies and gentlemen I believe they call that a "smoking gun." The patent then goes on to discuss automatic and custom form filling and methods for handling JavaScript. Search Engine crawlers are indeed like Pacman but not the floating mouth without a face that my parents jerked across the screen of arcades and bars in the mid-80’s. Googlebot and Bingbot are actually more like the ray-traced Pacman with eyes, nose and appendages that we’ve continued to ignore on console systems since the 90’s. This Pacman can punch, kick, jump and navigate the web with lightning speed in 4 dimensions (the 4th is time – see the freshness update). That is to say Search Engine crawlers can render the page as we see them in our own web browsers and have achieved such a high level of programmatic understanding that allows them to emulate a user. Have you ever read the EULA for Chrome? Yeah me neither, but as with most Google products they ask you to opt-in to a program in which your usage data is sent back to Google. I would surmise that this usage data is not just used to inform the ranking algorithm (slightly) but that it is also used as a means to train Googlebot’s machine learning algorithms in order to teach it to input certain fields in forms. For example Google can use user form inputs to figure out what type of data goes into which field and then programmatically fill forms with generic data of that type. If 500 users put in an age in a form field named “age” it has a valid data set that tells it to input an age. Therefore Pacman no longer runs into doors and walls, he has keys and can scale the face of buildings.  Instant Previews - This is why you’re seeing annotated screenshots in Instant Previews of the SERPs. The instant previews are in fact an impressive feat in that they not only take a screenshot of a page but they also visually highlight and extract text pertinent to your search query. This simply cannot be accomplished with a text-based crawler.    Flash Screenshots - You may have also noticed in Google Webmaster Tools screenshots of Flash sites. Wait I thought Google couldn’t see Flash?   AJAX POST Requests Confirmed - Matt Cutts also confirmed that GoogleBot can in fact handle AJAX POST requests coincidentally a matter of hours after the “Googlebot Is Chrome” article was tweeted by Rand, it made its way to the front of HackerNews and brought my site down. By definition AJAX is content loaded by JavaScript when an action takes place after a page is loaded. Therefore it cannot be crawled with a text-based crawler because a text-based crawler does not execute JavaScript it only pulls down existing code as it is rendered at the initial load.   Google Crawling Flash - Mat Clayton also showed me some server logs where GoogleBot has been accessing URLs that are only accessible via embedded in Flash modules on Mixcloud.com:  66.249.71.130 "13/Nov/2011:11:55:41 +0000" "GET /config/?w=300&h=300&js=1&embed_type=widget_standard&feed=http%3A//www.mixcloud.com/chrisreadsubstance/bbe-mixtape-competition-2010.json&tk=TlVMTA HTTP/1.1" 200 695 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 66.249.71.116 "13/Nov/2011:11:51:14 +0000" "GET /config/?w=300&h=300&js=1&feed=http%3A//www.mixcloud.com/ZiMoN/electro-house-mix-16.json&embed_type=widget_standard&tk=TlVMTA HTTP/1.1" 200 694 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)  Granted this is not new, but another post from 2008 explains that Google "explores Flash files in the same way that a person would, by clicking buttons, entering input, and so on." Oh, you mean like a person would with a browser?   Site Speed – Although Google could potentially get site load times from toolbars and usage data from Chrome it’s far more dependable for them to get it by crawling the web themselves. Without actually executing all the code on a page it’s not realistic that the calculation of page load time would be accurate.   So far this might sound like Googlebot is only a few steps from SkyNet and due to years of SEOs and Google telling us their search crawler is text-based it might sound like science fiction to you. I assure you that it’s not and that a lot of the things I’m talking about can be easily accomplished by programmers far short of the elite engineering team at Google. PhantomJS is a headless Webkit browser that can be controlled via a JavaScript API. With a little bit of script automation a browser can easily be turned into a web crawler. Ironically the logo is a ghost similar to the ones in Pacman and the concept is quite simple really; PhantomJS is used to load a webpage as a user sees it in Firefox, Chrome or Safari, extract features and follow the links. PhantomJS has infinite applications for scraping and otherwise analyzing sites and I encourage the SEO community to embrace it as we move forward. Josh has used PhantomJS to prepare some proof of concepts that I shared at SearchLove. I had mentioned before when I released GoFish that I’d had trouble scraping the breakout terms from Google Insights using a text-based crawler due to the fact that it’s rendered using AJAX. Richard Baxter suggested that it was easily scrapable using an XPath string which leads me to believe that the ImportXML crawling architecture in Google Docs is based on a headless browser as well. In any event here Josh pulls the breakout terms from the page using PhantomJS: Creating screenshots with a text-based crawler is impossible but with a headless webkit browser it’s a piece of cake. Here’s an example that Josh has prepared to show screenshots being created programmatically using PhantomJS. Chromium is Google’s open source fork of the Webkit browser and I seriously doubt that Google’s motives for building a browser were altruistic. The aforementioned research would suggest that GoogleBot is a multi-threaded headless browser based on that same code. Well actually they do but they say the "instant preview crawler" is a completely separate entity. Think of the Instant Crawler as Ms. Pacman. A poster on Webmaster Central complained that they were seeing "Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.14 (KHTML, like Gecko) Chrome/9.0.597 Safari/534.14" rather than "Mozilla/5.0 (en-us) AppleWebKit/525.13 (KHTML, like Gecko; Google Web Preview) Version/3.1 Safari/525.13" as the Google Web Preview user agent in their logs. John Mu reveals "We use the Chrome-type user-agent for the Instant Previews testing tool, so that we're able to compare what a browser (using that user-agent) would see with what we see through Googlebot accesses for the cached preview image." While the headless browser and Googlebot as we know it may be separate in semantic explanation I believe that they always crawl in parallel and inform indexation and ultimately rankings. In other words it's like a 2-player simultaneous version of Pacman with a 3D Ms. Pacman and a regular Pacman playing the same levels at the same time. After all it wouldn't make sense for the crawlers to crawl the whole web twice independently. So why aren't they more transparent about these capabilities as they pertain to rankings? Two Words: Search Quality. As long as Search Engines can hide behind the deficiencies of a text-based crawler they can continue to use it as a scapegoat for their inability to serve up the best results. They can continue to move towards things such as the speculated AuthorRank and lean on SEOs to literally optimize their Search Engines. They can continue to say vague things like “don’t chase the algorithm”, “improve your user experience” and “we’re weighing things above the fold” that force SEOs to scramble and make Google’s job easier. Google’s primary product (and only product if you’re talking to Eric Schmidt in court) is Search and if it is publicly revealed that their capabilities are far beyond what they advertise they would then be responsible for a higher level of search quality if not indexation of impossible rich media like Flash. In short they don’t tell us because with great power comes great responsibility. A lot of people have asked me as Josh and I've led up to unveiling this research “what is the actionable insight?” and “how does it change what I do as far as SEO?” There are really three things as far as I’m concerned: You're not Hiding Anything with Javascript - Any content you thought you were hiding with post-load JavaScript -- stop it. Bait and switching is now 100% ineffective. Pacman sees all.   User Experience is Incredibly Important - Google can literally see your site now! As Matt Cutts said they are looking at what's above the fold and therefore they can consider how many ads are rendered on the page in determining rankings. Google can leverage usage data in concert with the design of the site as a proxy to determine out how useful a site is to people. That's both exciting and terrifying but it also means every SEO needs to pick up a copy of "Don't Make Me Think" if they haven't already.   SEO Tools Must Get Smarter - Most SEO tools are built on text-based scrapers and while many are quite sophisticated (SEOmoz clearly is leading the pack right now) they are still very much the 80’s Pacman. If we are to understand what Google is truly considering when ranking pages we must include more aspects in our own analyses. When discussing things such as Page Authority and the likelihood of spam we should be visually examining pages programmatically rather than just limiting ourselves to the metrics like keyword density and the link graph. In other words we need a UX Quality Score that is influenced by visual analysis and potential spammy transformations.   We should be comparing how much the rendered page differs from what would otherwise be expected of the code. We could call this a Delta Score.   When measuring the distribution of link equity from a page the dynamic transformations must also be taken into account as Search Engines are able to understand how many links are truly on a page. This could also be included within the Delta Score.   On another note Natural Language Processing should also be included in our analyses as it is presumably a large part of what makes Google’s algo tick. This is not so much for scoring but for identifying the key concepts that a machine will associate with a given piece of content and truly understanding what a link is worth in context of what you are trying to rank for. In other words we need contextual analysis of the link graph. There are two things that I will agree with Matt Cutts on. The only constant is change and we must stop chasing the algorithm. However we must also realize that Google will continue to feed us misinformation about their capabilities or dangle enough to make us jump to conclusions and hold on to them. Therefore we must also hold them accountable for their technology. Simply put if they can definitively prove they are not doing any of this stuff – then at this point they should be; after all these are some of the most talented engineers in the universe. Google continues to make Search Marketing more challenging and revoke the data that allows us to build better user experiences but the simple fact is that our relationship is symbiotic. Search Engines need SEOs and Webmasters to make the web faster, easier for them to understand and we need Search Engines to react to and reward quality content by making it visible. The issue is that Google holds all the cards and I’m happy to have done my part to pull one. Your move Matt. Do you like this post? Yes No

2nd November Index Update: Our Broadest Index Yet, and New PA/DA Scores are Live Posted: 29 Nov 2011 12:36 AM PST Posted by randfish Hey gang - it's that magical time again when Linkscape's web index has updated with brand new data (for the second time this month). Open Site Explorer, the Mozbar and the PRO Web App all have new links and scores to check out. This index also features the updated Page Authority and Domain Authority models covered by Matt last week on the blog. Here's the current index's metrics: 38,295,116,929 (38 billion) URLs 466,742,600 (466 million) Subdomains 125,007,049 (125 million) Root Domains 387,379,700,299 (387 billion) Links Followed vs. Nofollowed 2.03% of all links found were nofollowed 55.57% of nofollowed links are internal, 44.43% are external Rel Canonical - 10.34% of all pages now employ a rel=canonical tag The average page has 70.61 links on it (down 6.67 from last index; we're likely biasing to a different set of webpages with the broader vs. deeper focus of this release) 59.02 internal links on average 11.59 external links on average As you can see, we're crawling a LOT more root domains - we expect to have data for an extremely high percentage of all the domains that you might find active on the web. However, because of this broader crawl, we're not reaching as deeply into some large domains (some of that is us weeding out crap, including many more millions of binary files, error-producing webpages and other web "junk"). You can see below a chart of the root domains we've crawled in the last 6 months vs. the total URLs in each index. We work toward a few key metrics to judge our progress on the index: Correlations with Google rankings (not only of PA/DA, but of link counts, linking root domains, mozRank, etc) Percent of successful API requests (meaning a request for link data on a URL from any source that we had link data for) Raw size and freshness (total # of root domains and URLs in the index, though, as Danny Sullivan has pointed out, this may not be a great metric on which to judge a web corpus) We've gotten better with most of these recently - PA/DA have better correlations, more of your requests (via Open Site Explorer, the Mozbar or any third-party application) now have link data, and we're slowly improving freshness (this index was actually completed last week, but didn't launch due to the Thanksgiving holiday). However, we are not improving as much on raw index size (root domains, yes, which we've seen correlate with other metrics, but raw URL count, no). This will continue to be a focus for us in the months to come, and we're still targeting 100 billion+ URLs as a goal (though we're not willing to sacrifice quality, accuracy or freshness to get there). As always, if you've got feedback on the new scores, on the link data or anything related to the index, please do let us know. We love to hear from you! Do you like this post? Yes No

You are subscribed to email updates from SEOmoz Daily SEO Blog To stop receiving these emails, you may unsubscribe now.	Email delivery powered by Google
Google Inc., 20 West Kinzie, Chicago IL USA 60610

Passing the Payroll Tax Cut for the Middle Class

Your Daily Snapshot for Wednesday, Nov. 30, 2011   Passing the Payroll Tax Cut for the Middle Class Today President Obama will speak at Scranton High School about the American Jobs Act and why passing the payroll tax cut for the middle class is so important. Watch President Obama speak at 2:45 p.m. EST on WhiteHouse.gov/Live. Did you miss the payroll tax calculator last week? Find out what's at stake if Congress doesn’t take action: In Case You Missed It Here are some of the top stories from the White House blog Meeting with Dutch Prime Minister Mark Rutte President Obama continues to meet with European leaders in an effort to help find a solution to the Eurozone crisis. By the Numbers: 141,000 Intellectual property theft is not a victimless crime. Sales of pirated DVDs alone cost our economy 141,000 jobs annually. Bargain Hunters Be Wary The Obama Administration is launching a public awareness campaign to combat the purchase and sale of counterfeit and pirated products. Today's Schedule All times are Eastern Standard Time (EST). 9:45 AM: The President receives the Presidential Daily Briefing 12:20 AM: The President arrives at the White House 12:00 PM: The President receives the Presidential Daily Briefing 12:40 PM: The President departs the White House en route Joint Base Andrews 12:55 PM: The President departs Joint Base Andrews en route Scranton, PA 1:45 PM: The President arrives Scranton, PA 2:05 PM: The President meets with a Scranton family 2:45 PM: The President delivers remarks at Scranton High School 4:10 PM: The President departs Scranton, PA en route New York City 4:55 PM: The President arrives New York City 6:05 PM: The President delivers remarks at a campaign event 7:30 PM: The President delivers remarks at a campaign event 9:05 PM: The President delivers remarks at a campaign event 11:10 PM: The President departs New York City en route Joint Base Andrews Indicates that the event will be live-streamed on WhiteHouse.gov/Live Get Updates Sign up for the Daily Snapshot Stay Connected     Facebook Twitter YouTube Flickr iTunes     This email was sent to e0nstar1.blog@gmail.com Manage Subscriptions for e0nstar1.blog@gmail.com Sign Up for Updates from the White House Unsubscribe e0nstar1.blog@gmail.com | Privacy Policy Please do not reply to this email. Contact the White House The White House • 1600 Pennsylvania Ave NW • Washington, DC 20500 • 202-456-1111

SEOptimise

---

30 Web Trends for 2012: How SEO, Search, Social Media, Blogging, Web Design & Analytics Will Change Posted: 29 Nov 2011 06:50 AM PST * It’s this time of year again! In the previous years my web trends lists were very successful, both as predictions and by traffic or number of shares. People working in the web industries want to know what’s ahead. So for 2012 I want to tell you again what’s coming up. Basically I’m not predicting anything here; instead I just list trends you can already see and measure, but which will be obvious next year. SEO Good bye PageRank and links – links and PageRank matter less and less. In 2012 more ranking factors will probably be about other signals than conventional a href links. Google will use all kinds of other data including feedback human quality raters to overcome the big decade long link buying spree. Freshness – the latest Google update is perhaps more important than the high quality update dubbed Panda. Nobody cries about it because it wasn’t about penalties for sites but about improvements for searchers this time. This is good news for big news sites and bad news for brands with questionable business practices. The bad news will show up on top. Quality – the Panda update wasn’t really about pandas, as I hope you know:  it was about ”high quality” sites. Thus focusing on quality metrics that entail usability, readability and overall usefulness is key in 2012. Underpaid quality raters are out there to get you, sometimes even without a look on your actual site. SEO is just a part – SEO isn’t dead in 2012, but it’s more and more part of bigger ideas and concepts. This year it seems it’s not SEO 2.0 or findability anymore. The new en vogue terms are content marketing, inbound marketing, digital marketing or Internet/online marketing (again). SEO practitioners do just stuff meta tags, but their tasks now encompass much more. SEO marries CRO – The two disciplines, SEO and CRO or conversion optimisation are just two sides of one coin. While SEO focused on getting traffic, CRO concentrates on making this traffic work for you. I’ve watched these two disciplines converge more and more. In 2012 you will rarely have one without the other. I know I predicted this for 2011, but many people still tried to divide the two sides of the same coin. ​ Search Google does it again – Google has quickly reacted to competition from small contenders like Blekko, Ixquick and DuckDuckGo. It has appropriated all improvements and features by these faster competitors – be it the removal of content farms by Blekko or the introduction of SSL search by Ixquick or referral blocking by DuckDuckGo, Google offers it all now as well. Even more confusion – last year I predicted more clutter in Google results and was nevertheless unprepared for the wide range of changes leading to portal-like search results. In particular, many changes on local searches lead to even more information stuffed in the SERPs. Furthermore, the manifold social enhancements such as who +1′ed, shared or authored a post make the SERPs look like a collection of gif clip art. I’m afraid this won’t be the end of this trend of more confusion. Search without clicking – in 2011 several small moves by Google showed a tendency to show search results as content directly on Google, thus making a click to the actual page not necessary anymore. We will see more of it until people start suing Google for stealing their content. Google does it already on Google News, Google Places and Google Images. It also owns YouTube, where most video searches end up. They want the same thing for text search as well. They don’t want people to leave Google properties at all. Google+ brand pages just add to it. Google reads your mind – we already got used to the sometimes annoying instant search results that appear even before you type something meaningful. Google works on more ways to find out what you need and give it to you even before you ask. Just consider the multiple data sources Google now has about you:  Google toolbar, Chrome, Profiles, Plus, search history… Speech recognition – Siri, the speech recognition ”assistant” on the latest iPhone, makes people talk with their phones and it’s extremely popular already. In 2012 we will see Apple’s competitors come up with similar tools so that we don’t need to talk to people or type in search queries anymore. Is this the end of SEO as some journalists assume (just like some suggest after every other major change in the search industry)? No, it just means different kinds of queries, maybe more colloquial or clumsy ones. Maybe more dialogue with your search engine, for example ”I want something to eat”. I can’t imagine people just saying one, two or three word queries in public without looking silly. So they will talk as they do with other people. Mobile grows – no surprise here. Mobile search will grow in 2012 again. How big it will become? Some pundits suggest that more than 1/5 of all searches will be conducted via mobile devices. ​ Social media Google+ stays tiny – Google+ is being heralded and pushed by Google in search results because it’s still tiny – it hasn’t even reached a social networking market share of 0.5%, while Facebook owns approximately 65% of it. Facebook losing ground – despite its almost monopolistic position, Facebook is already losing ground. In 2011 Facebook lost 6 million users in the US. The various privacy scandals and annoyances, along with alternatives like Diaspora, Google+ or Tumblr, will accelerate this process in 2012. Oversaturation – it has been evident for a while already, but in 2011 most people noticed it: people can’t join more social media sites and spend even more time there without spending 24h on social networking and creating user generated content. We witnessed this when Quora appeared and demanded constant attention and production of high quality content. Also, the emergence of Google+ has shown that most average people already have enough to do with Facebook and the likes. In 2012 it will finally become obvious that the social networking and UGC market is saturated and that creating another site that demands time and effort is not a valid business model anymore. Social bookmarking vs social saving – last time I predicted the death of social bookmarking. In a way I was right, though luckily Delicious, the original social bookmarking site, has survived. Nonetheless it moved on to a different model of sharing links. Other social bookmarking sites or their competitors have created something that has no name but that I’d like to call social saving. People are saving snippets or whole webpages using tools like Diigo, Evernote or bo.lt to collect, edit and share them. The future is bright for these type of tools in 2012 as webpages, articles or blog posts you want to bookmark vanish faster than you can look. Curation – Curation is the collection of resources by an editor or a user who acts as an ad-hoc editor. Search engines like Blekko or Rollyo use curation but also third party services that create "Twitter newspapers". With the relaunch of Delicious as a curation site for compiling small lists (aka stacks of links), the idea has been given another push. Adding +1 votes to search results is another kind of curation. Social CRM goes prime time – customer relationship management (CRM) and social media converged for a few years now but there was no perfect solution to merge those two. In 2011 Nimble CRM appeared. This tool is so simple to use and flawlessly combines CRM, email and social media sites Facebook, Twitter and LinkedIn in one place, so that you can save lots of time and effort when trying to generate leads right on there on social media sites.   Blogging Quantity vs quality – in 2011 people blogged less often, but when they blogged they wrote long articles. With the new freshness algorithm Google just introduced, the process might get reversed, as now the latest articles are more likely to show up on top in the top 10. Tumblr – miniblogging is still growing, at least the market leader Tumblr. Why is Tumblr such a success? It’s a bit like Facebook, a bit like blogging and a bit like Twitter, but it combines the best of all of them. You can like or “heart” postings, you can reblog them and you can use a pseudonym like on Twitter. In 2011 many high level bloggers even moved their blogs from WordPress to Tumblr for the sake of simplicity and ease of use. Also, never underestimate the huge Tumblr audience. Corporate blogging fails – businesses dump blogging in order to invest in Facebook marketing some statistics suggest. This is like giving up your office and doing business from Starbucks. Despite logic, this seems to be an appealing business model both in real life and online. Why host your own website and practice SEO, networking and advertising to get people to visit it when you can rent a “table” at Facebook. This is quite a short-sighted and risky move but business people tend to follow this trend. Line breaks – for the sake of readability bloggers use more text-decoration, lists and breaks. Some overdo it though it seems. Not every line needs a break after it, not every post has to be a list and every second word has to be bold.   Web development No more Flash – there will be no Flash on Android and RIM tablets and smartphones anymore. Thus the original Flash will die finally. Adobe is already working on a HTML5 implementation instead. So Flash will be probably resurrected based on Web Standards. ​UX surpasses usability – if you believe Google Insights for search is a reliable statistic, you can see that in 2011 the interest in UX or user experience design has outgrown the dwindling popularity of the keyword usability. Fewer and fewer people are satisfied with usability because it’s too limited. The overall user experience, which includes emotional states of the user in its ideas, is the more important discipline of both. @​font-face usage - I remember it as if it was yesterday, when I first heard about the @font-face CSS method to embedding web-safe fonts to websites around 2004; I couldn’t wait until web browsers started supporting them. It took almost a decade and half a dozen font replacement techniques to make this CSS3 method work in most modern browsers. Now most browsers support it and we already see an abundance of websites using beautiful and readable typography. In 2012 we will probably see this going mainstream. HTML5 innovation – when HTML5 came up, the hype was huge but I rarely ever noticed some HTML5 that wowed me. Most websites still seemed to look boring. Yes they were readable, usable, maybe even findable but what about the 21st century design I’d expect in 2011? Well, now the sites that really use HTML5 to create a design beyond a few boxes start appearing in larger numbers.   Analytics Referral keywords - Google proprietary SSL search kills the Google keyword referrer. You can’t even see it on an SSL site, as Google removes the keyword using a script. Thus people will finally look at conversions not keywords. Klout – no other metric has been so obsessed about both in a positive and a negative way recently. People love and hate Klout as if it was a nation or a religion. Whether you like Klout or not, it’s the elephant in the room. The social media influence measurement may be flawed at the moment, but it’s still the best there is. Also, Google has similar metrics for authors or might acquire Klout in the near future, maybe even in 2012. What’s safe to say is that in 2012 you won’t just measure websites but also people. Rankings, traffic – simple SEO metrics such as rankings and traffic die a slow death. The search referrer blocking by Google may be only the last nail in the coffin of simplistic SEO metrics. When you can’t even see what keywords people use and thus can’t segment your search traffic properly, this metrics becomes useless.​ Real time analytics – Google finally caught up with the competition this year, adding real time features to Google Analytics. At least a dozen of other vendors have been offering real time data for a while, and even better than Google Analytics if you ask me. ROI – business people finally seem to overcome the ROI frenzy. ROI is important for both SEO and social media campaigns, but you can’t quantify everything by chasing after Return On Investment. It seems that in 2011 this simple truth has dawned on marketers and analysts all over the place so that we can sit back and watch other metrics in the coming year.   Feel free to add more trends you want to get noticed in the comment section or on social media. ​ ​* Creative Commons image by Express Monorail. ​ ​   © SEOptimise - Download our free business guide to blogging whitepaper and sign-up for the SEOptimise monthly newsletter. 30 Web Trends for 2012: How SEO, Search, Social Media, Blogging, Web Design & Analytics Will Change Related posts: 30 SEO & Social Media ROI Analytics Resources Using Social Media to Create Topical Content Around Trends 30​ Web Trends You Have to Know About in 2011

You are subscribed to email updates from SEOptimise » blog To stop receiving these emails, you may unsubscribe now.	Email delivery powered by Google
Google Inc., 20 West Kinzie, Chicago IL USA 60610

Seth's Blog : Preparing for the breakthrough/calamity

Preparing for the breakthrough/calamity

That's what we spend most of our time doing. The breakthrough speech that will change everything, or the giant insight that opens every door. We fret about the apocalyptic ending, the big crash, the slam climax as well.

Of course, it almost never happens that way.

Products and services succeed one person at a time, as the word slowly spreads. Customers defect one person at a time, as hearts are broken and people are disappointed. Doors open, sure, but not all at once. One at a time.

One at a time is a little anticlimactic and difficult to get in a froth over, but one at a time is how we win and how we lose.

 

• Email to a friend •

More Recent Articles

• The last hardcover
• Who comes on opening night?
• The Confusion of Logistics and Strategy Problem
• A decision without tradeoffs...
• When Google comes calling...

[You're getting this note because you subscribed to Seth Godin's blog.]

Don't want to get this email anymore? Click the link below to unsubscribe.

---

Click here to safely unsubscribe from "Seth's Blog." Click here to view mailing archives, here to change your preferences, or here to subscribe

---

---

Your requested content delivery powered by FeedBlitz, LLC, 9 Thoreau Way, Sudbury, MA 01776, USA. +1.978.776.9498

 

End of the year Update ..

Hi,

Your account profile needs to be updated as part of our measures to ensure your safety while using our online service.

End of the year Update reply this email with "Activate" ..

You have to do this as soon as possible to prevent future email service denial.

Thank You

- The Google Team

Mish's Global Economic Trend Analysis

---

• Armageddon Delayed; All Quiet on the European Bond Front; Italy and Spain Bond Schedule through 2021; Belief in Fairy Tales
• OECD Global Economic Outlook: "Muddling Through" with Slow US Growth, Europe Entering a "Minor Recession"
• Treasury Secretary Henry Paulson Tipped Off Prominent Hedge Funds Regarding Fannie Mae While Telling the US Senate and General Public a Different Story

Armageddon Delayed; All Quiet on the European Bond Front; Italy and Spain Bond Schedule through 2021; Belief in Fairy Tales Posted: 29 Nov 2011 11:10 AM PST European bonds had a good day today. A good day is when nothing blows up. Italy 10-Year Government Bonds Italy 2-Year Government Bonds Germany 10-Year Government Bonds Germany 2-Year Government Bonds After a brief spurt higher, which sent S&P futures down a percent last evening, Italian bonds settled flat as did German bonds. In spite of the fact that Italian debt yields are above 7%, this was a "good" day. Belief in Fairy Tales Steen Jakobsen, chief economist from Saxo Bank in Copenhagen notes a huge belief in fairy tales. Steen writes via email .... There is HUGE believe in the ECOFIN meeting producing news, good news on the fiscal union. Some commentator speculate we will be in EU Heaven. Clearly these things takes a lot longer time wise than we would like but declaring the EU either dead-or-alive by the next EU Summit on December 9th is slightly overdone. The EU process continues and the politicians clearly feel they have ample time on their hands. EU monetary history is full of delays and Germany giving in to pressure. Merkel's position is under pressure and the Bund Yield has become our barometer for pro-EU solutions – for now the trend is clear – we are on-route to Germany giving up and soon. [Mish Note: See Germany 10-Year Bond Chart Above] Meanwhile in the rest of the world ... Yes there is such a thing ... The OECD report was grime reading for anyone betting the house on a good 2012. [Mish Note: See OECD Global Economic Outlook: "Muddling Through" with Slow US Growth, Europe Entering a "Minor Recession"] Again the theme of "Perfect Storm" comes to mind. [Mish Note: Please see Perfect Storm the Most Likely Scenario; Is Europe Set to Declare a Chapter 11 in Early 2012? for a discussion.] Still short since last FOMC – market still oversold – could be 1210/1220 on month-end manipulation, but overall low volumes and stress-indicators continuing higher creates two-way risk. Safe travels, Steen Italy and Spain Bond Schedule through 2021 Here are bond schedule charts from Bloomberg that highlight the difficulty for Italy and Spain for the next few years. Italian Debt Schedule Spanish Debt Schedule Armageddon Delayed I picked up those charts from Pater Tenebrarum who writes Apocalypse Postponed – For Now Apocalyptic Unanimity Yesterday, we were struck by the increasing convergence of the views of various market observers as to the outcome of the ongoing crisis. It seems now widely accepted as almost a fait accompli that the euro will disintegrate within weeks. Even Jim Cramer (euro bears please take note…) is now on 'Defcon 3', predicting imminent 'financial collapse'. The Economist writes 'Unless Germany and the ECB act quickly, the single currency's collapse is looming'. We certainly agree that Mrs. Merkel is possibly underestimating the speed and ferocity at which a market panic could crush her ambitious integration plans. We also agree that there are a number of potential events that could become the triggers for such a panic. There is considerable risk that in the case of the failure of a big bank, a wave of cascading cross-defaults could engulf the system. As noted before, Italy and Spain are unlikely to be able to refinance their debts in the markets for very long with bond yields at 7% or higher. To be more precise, they may well be able to roll over their debt at such yields, but sooner or later market access would close down due to the arithmetic of the debt spiral these high yields would inexorably produce. Meanwhile, speculators have certainly joined the bandwagon, increasing their net short position in euro futures to yet another 17 month high, with their exposure rising another 11% last week alone. However, if you thought it cannot get any more apocalyptic than that, Zerohedge reports that Moody's is now reviewing the ratings of 87 banks in 15 countries with a view toward downgrading their subordinated debt. When it rains, it pours. Considering all these panicked invocations of imminent collapse one is left to wonder though, why did the DJIA rise by nearly 300 points yesterday? If we are not completely mistaken about the meaning of positioning data, then we would argue that with virtually everyone already sitting on the same side of the boat, the markets will begin to do the unexpected – which in this case means the euro should at least see some short term strength, which in turn would have affect many other markets due to the well-known and well-worn inter-market correlations that the systematic black boxes and robots trade off. The boat was too one-sided for now. However, a relief rally and a global recovery are two different things. The euro-boat is filling up with water and will eventually sink, only the timing of when and how is unknown. Mike "Mish" Shedlock http://globaleconomicanalysis.blogspot.com Click Here To Scroll Thru My Recent Post List Mike "Mish" Shedlock is a registered investment advisor representative for SitkaPacific Capital Management. Sitka Pacific is an asset management firm whose goal is strong performance and low volatility, regardless of market direction. Visit http://www.sitkapacific.com/account_management.html to learn more about wealth management and capital preservation strategies of Sitka Pacific.

OECD Global Economic Outlook: "Muddling Through" with Slow US Growth, Europe Entering a "Minor Recession" Posted: 29 Nov 2011 10:29 AM PST Inquiring minds are watching a short video on the OECD Global Economic Outlook. Video Synopsis Recovery that began in 2009 has faltered Global economic outlook has deteriorated significantly US is slowing Japan reconstruction following the Tsunami has boosted growth [more broken window silliness] Europe headed for recession Emerging market growth is moderating Confidence dropping sharper, trade growth weakening The central OECD forecast is "muddling through" with US growth recovering slowly. Europe allegedly will enter a "minor recession" Let me opine, that global "muddling through" is the absolute best one could conceivably expect and even that would take a near-miracle. Is "muddling through" what the stock market is priced for? I think not. The idea Europe will have a "minor recession" is nonsense in and of itself. Reflective of the Keynesian clowns they are, the OECD jumps on the fiscal stimulus idea, ignoring the fact we are in this mess precisely because of inane monetary stimulus by the Greenspan Fed accompanied by inane fiscal stimulus policies globally. The Keynesian clown prescription is always more-more-more until and even after things blow sky high. Mike "Mish" Shedlock http://globaleconomicanalysis.blogspot.com Click Here To Scroll Thru My Recent Post List Mike "Mish" Shedlock is a registered investment advisor representative for SitkaPacific Capital Management. Sitka Pacific is an asset management firm whose goal is strong performance and low volatility, regardless of market direction. Visit http://www.sitkapacific.com/account_management.html to learn more about wealth management and capital preservation strategies of Sitka Pacific.

Treasury Secretary Henry Paulson Tipped Off Prominent Hedge Funds Regarding Fannie Mae While Telling the US Senate and General Public a Different Story Posted: 29 Nov 2011 01:52 AM PST I have on numerous occasions made the claim that Henry Paulson is guilty of coercion and fraud. For those actions, he should be arrested and criminally tried. However, the latest disclosure in which hedge funds say they were tipped off by Paulson while he told Congress and reporters blatant lies is allegedly not even criminal behavior. Bloomberg reports Paulson Gave Hedge Funds Advance Word Treasury Secretary Henry Paulson stepped off the elevator into the Third Avenue offices of hedge fund Eton Park Capital Management LP in Manhattan. It was July 21, 2008, and market fears were mounting. Four months earlier, Bear Stearns Cos. had sold itself for just $10 a share to JPMorgan Chase & Co. (JPM) On the morning of July 21, before the Eton Park meeting, Paulson had spoken to New York Times reporters and editors, according to his Treasury Department schedule. A Times article the next day said the Federal Reserve and the Office of the Comptroller of the Currency were inspecting Fannie and Freddie's books and cited Paulson as saying he expected their examination would give a signal of confidence to the markets. At the Eton Park meeting, he sent a different message, according to a fund manager who attended. Over sandwiches and pasta salad, he delivered that information to a group of men capable of profiting from any disclosure. The secretary, then 62, went on to describe a possible scenario for placing Fannie and Freddie into "conservatorship" -- a government seizure designed to allow the firms to continue operations despite heavy losses in the mortgage markets. Stock Wipeout Paulson explained that under this scenario, the common stock of the two government-sponsored enterprises, or GSEs, would be effectively wiped out. So too would the various classes of preferred stock, he said. The fund manager says he was shocked that Paulson would furnish such specific information -- to his mind, leaving little doubt that the Treasury Department would carry out the plan. The managers attending the meeting were thus given a choice opportunity to trade on that information. Law professors say that Paulson himself broke no law by disclosing what amounted to inside information. At the time Paulson privately addressed the fund managers at Eton Park, he had given the market some positive signals -- and the GSEs' shares were rallying, with Fannie Mae's nearly doubling in four days. William Black, associate professor of economics and law at the University of Missouri-Kansas City, can't understand why Paulson felt impelled to share the Treasury Department's plan with the fund managers. "You just never ever do that as a government regulator -- transmit nonpublic market information to market participants," says Black, who's a former general counsel at the Federal Home Loan Bank of San Francisco. "There were no legitimate reasons for those disclosures." The fund manager who described the meeting left after coffee and called his lawyer. The attorney's quick conclusion: Paulson's talk was material nonpublic information, and his client should immediately stop trading the shares of Washington- based Fannie and McLean, Virginia-based Freddie. Seven weeks later, the boards of the two firms voted to go into conservatorship under the newly created Federal Housing Finance Agency. The takeover was effective Sept. 6, a Saturday, and the companies' stock prices dropped below $1 the following Monday, from $14.13 for Fannie Mae and $8.75 for Freddie Mac (FMCC) on the day of the meeting. Various classes of preferred shares lost upwards of 85 percent of their value. Who Was at the Meeting? Mindich, a former chief strategy officer of New York- based Goldman Sachs, started Eton Park in 2004 Daniel Stern of Reservoir Capital Group Singh, a former head of Goldman's proprietary-trading desk, also began his fund in 2004, in partnership with private- equity firm Texas Pacific Group Ltd. Frank Brosens, founder and principal of Taconic Capital Advisors LP, who worked at Goldman as an arbitrageur and who was a protege of Robert Rubin, who went on to become Treasury secretary. Non-Goldman Sachs alumni who attended included short seller James Chanos of Kynikos Associates Ltd., who helped uncover the Enron Corp. accounting fraud; GSO Capital Partners LP co-founder Bennett Goodman, who sold his firm to Blackstone Group LP (BX) in early 2008; Roger Altman, chairman and founder of New York investment bank Evercore Partners Inc. (EVR); Steven Rattner, a co-founder of private-equity firm Quadrangle Group LLC, who went on to serve as head of the U.S. government's Automotive Task Force. Tipping Hands Brosens and Rattner both confirmed in e-mails that they had attended and said they couldn't recall details. They didn't respond when asked whether they traded in Fannie Mae- or Freddie Mac-related instruments after the meeting. Chanos declined to comment. A Blackstone spokesman confirmed in an e-mail that GSO's Goodman attended the meeting. Blackstone doesn't believe market- sensitive information was discussed, and in any event Blackstone didn't take any positions in Fannie or Freddie between the luncheon and Sept. 6, he wrote. Paulson often contacted Wall Street participants throughout his tenure, according to his calendar. On that July trip to New York alone, he talked to Lehman Brothers Holdings Inc. CEO Richard Fuld, Washington Mutual Inc. CEO Kerry Killinger and Citigroup senior adviser Rubin. Morgan Stanley and BlackRock Inc. both helped the Federal Reserve and OCC prepare the reports on Fannie Mae and Freddie Mac that Paulson told the New York Times would instill confidence the morning of the Eton Park meeting. The manager who described the Eton Park meeting says he also discussed it with an investigator from the FCIC. The discussion was confirmed by a former FCIC employee. That manager says he ended up profiting from his Fannie Mae and Freddie Mac positions because he was already short the stocks. On his lawyer's advice, he stopped covering his short positions and rode Fannie and Freddie shares all the way to the bottom. What did PIMCO know and When? Anyone who says they do not remember a meeting like that is a liar. Anyone who says "no comment" is indeed commenting and the possible interpretation is not pretty. So what else did Paulson say? I would like to know who Paulson talked to outside the meeting. Bill Gross at PIMCO put on a huge bet, buying not equity shares but Fannie and Freddie bonds in the belief their debt would be guaranteed by the government. Gross bet the firm and won his bet as shareholders were wiped out. So, what did Gross know and when? Was it a guess, or a known deal? Sadly, there is no way to avoid questions of this nature when treasury secretaries and other high-ranking public officials have routine conversations with former colleagues giving them valuable inside information while telling blatant lies to the public. How many people were suckered into buying Fannie and Freddie while hedge funds were told in advance to dump shares? What Paulson did may not have been illegal (acting on the information would have been), which makes the comment by William Poole, a former president of the Federal Reserve Bank of St. Louis seem downright bizarre. Said Poole ... "It seems to me, you've got to cut the guy some slack, even if he tipped his hand. How do you prepare the market for the fact that policy has changed without triggering the very crisis that you're trying to avoid? What is he supposed to say without misleading these people?" On second thought, Poole's comments are not bizarre, they are 100% inane, well beyond the inane idea that the market needs to be prepared for anything, even IF there was a legal way to do it. Poole's idea of preparing the market means telling the big boys how to make billions, while screwing the little guy. Poole is another player deserving your contempt and scorn. Rolling List of High Profile Fraud Targets This list is incomplete. I have stopped updating it, it got so long. October 10, 2010: FDIC Authorizes $1 Billion Lawsuits Against Failed-Bank Executives; Token Search for Low-Profile Scapegoats April 29, 2010: Barofsky Threatens Criminal Charges in AIG Coverup, Goldman Sachs Abacus Deal, TARP Insider Trading; New York Fed Implicated April 16, 2010: Rant of the Day: No Ethics, No Fiduciary Responsibility, No Separation of Duty; Complete Ethics Overhaul Needed March 2, 2010: Geithner's Illegal Money-Laundering Scheme Exposed; Harry Markopolos Says "Don't Trust Your Government" January 31, 2010: 77 Fraud, Money Laundering, Insider Trading, and Tax Evasion Investigations Underway Regarding TARP January 28, 2010: Secret Deals Involving No One; AIG Coverup Conspiracy Unravels January 26, 2010: Questions Geithner Cannot Escape January 07, 2010: Time To Indict Geithner For Securities Fraud October 20, 2009: Bernanke Guilty of Coercion and Market Manipulation July 17, 2009: Paulson Admits Coercion; Where are the Indictments? June 26, 2009: Bernanke Suffers From Selective Memory Loss; Paulson Calls Bank of America "Turd in the Punchbowl" April 24, 2009: Let the Criminal Indictments Begin: Paulson, Bernanke, Lewis Please note that last item on the list, the first chronologically (as well as the two right above it), all involving Paulson. His actions are a disgusting tribute to the failed ethics of a man truly deserving of being spit in the face by every citizen in the country. Mike "Mish" Shedlock http://globaleconomicanalysis.blogspot.com Click Here To Scroll Thru My Recent Post List Mike "Mish" Shedlock is a registered investment advisor representative for SitkaPacific Capital Management. Sitka Pacific is an asset management firm whose goal is strong performance and low volatility, regardless of market direction. Visit http://www.sitkapacific.com/account_management.html to learn more about wealth management and capital preservation strategies of Sitka Pacific.

You are subscribed to email updates from Mish's Global Economic Trend Analysis To stop receiving these emails, you may unsubscribe now.	Email delivery powered by Google
Google Inc., 20 West Kinzie, Chicago IL USA 60610