How To Use Fetch As Googlebot Like An SEO Samurai

What Is Fetch As Googlebot

Fetch as Googlebot has two modes.

Fetch – Quick Check

In this mode, Googlebot will crawl a URL based on a path you specify, and you can then view the response your site sent to Googlebot. This can be used to check connectivity, basic errors, redirects or security issues with your website.

Fetch & Render – Deeper View

In this mode, Google crawls the URL and displays the page the way a browser would.

First, Googlebot gets all the resources referenced by your URL such as picture, CSS, and JavaScript files, running any code. to render or capture the visual layout of your page as an image. You can use the rendered image to detect differences between how Googlebot sees your page, and how your browser renders it.

How To Use Fetch As GoogleBot

Here are the basic steps:

1. On the Webmaster Tools home page, select your site.
2. In the left-hand navigation, click Crawl and then select Fetch as Google.
3. In the textbox, enter the path component of a URL on your site that you want Googlebot to retrieve.
4. From the grey drop-down, choose the type of Googlebot with which you wish to perform a fetch (or fetch and render).

The Different Types Of Googlebot Requests

There are four different Googlebot requests you can make in Fetch as Google.

If you would like to see how a desktop browser sees your site, select Desktop. To see how smartphone mobile crawlers see your site, choose Mobile Smartphone.

If you want to see how the mobile crawler for a feature phone sees the site, select Mobile cHTML or Mobile XHTML/WML. (Google notes that the Mobile cHTML is mainly for Japanese websites.)

If you do a basic Fetch, you can see the HTTP response that Google got from your page. When you do Fetch and Render, you get the response and get to see how your page looks in the specified browser.

(Note: The “Fetch And Render” option is not available for the WML and cHTML at this time, so unfortunately you won’t be able to see how your page renders for feature phones.)

Once you’ve entered your URL and fetched it, you will have the option of submitting it to Google for indexation. You’ll see a button allowing you to do so. (This button will also appear on the upper right-hand corner of the Fetch details page.)

After you submit a URL to be fetched, you can click on the path once it’s completed. This will bring you to a page with further details. At the top, you’ll see the URL that was fetched, the Googlebot type, and when it was completed. If you chose to Fetch and Render the page, you’ll see a “Fetching” tab and a “Rendering” tab.

The Fetching tab shows you the downloaded HTTP response, which can be very helpful if you are ensuring a page status, and it also shows you the code of the page. Finally, it shows you the download time of the page at the bottom.

The Rendering tab will give you an idea of how your page renders in the browser selected (desktop or smartphone). It will also list any page resources that Googlebot was not able to access.

How Many Fetches & Submissions Can You Do?

So, there actually is a quota on fetches and submissions, but most of us will never exceed it. According to Google, you are limited to 500 fetches per week. (You’ll get a notification that you’re approaching your limit once you have hit 400.)

When it comes to actual submissions to the index, they give you 500 individual URL submissions and 10 submissions where Google will crawl that URL and all of the direct links as well.
Diagnosing Your Fetch As GoogleBot Results

With this tool, you can submit a site a to get it indexed quicker, see if a page is blocked by robots.txt, see if a page is being redirected, make sure a page works on mobile, see if your server is taking too long to respond and so much more. But you can’t do any of that if you don’t know what the fetch statuses mean. So here they are.

Fetch Statuses

Fetch And Render Statuses 

Tricks Of The Trade For Fetch As Googlebot

Now that we have covered the basics, I want to give you a few ideas on how you can use this tool.

• Did you update an important page with new optimization and copy? Fetch that page and submit to the index. You will get a fresh, quick crawl and index.
• Did you launch a new section of your site? Fetch the main page and submit it, along with all the linked pages. It will get that section indexed quicker.
• Did you launch a new mobile experience (mobile site, responsive design, etc.)? Fetch as the mobile smartphone Googlebot and ensure that Google can access it.
• Did you add a new robots.txt file? Fetch a page that you might be worried about and make sure both Google and Google mobile can access it. There is also the robots.txt tester tool for this, mind you.
• Do you have rel=canonical or 301 redirects on certain URLs? Do you have multiple redirects? Want to make sure Google is interpreting them correctly? This is a great tool to test it.
• Do you have an https version of your site live, but you don’t want it in the index? Set up webmaster tools for that version and test that is its acting correctly with this tool.

As you can see, there are many applications for this tool. One thing that got me really excited about it and made me want to write this article was the indexing. I made some significant changes to a page, used this tool to submit it to the index, and within a few hours, I was visible for a competitive keyword. So, as you can see, this tool can help in a variety of areas.
www.pixotritechnologies.com