SEO Guide to Crawling Indexing and Ranking

SEO Guide to Crawling Indexing and Ranking

Published By: Admin

Share On:

Technical SEO will usually be pushed aside a little too simply in favour of things like content creation, social media and link building. But I’ve perpetually believed that there area unit several opportunities for increasing traffic by wanting inwards instead of outward. One among the largest areas of this on behalf of me is to create positive that your web site is as accessible as attainable to the search engines.


It’s quite straightforward extremely – if the search engines can’t crawl your web site with efficiency, you’re unlikely to rank. Even links and social shares won’t solve severe accessibility problems therefore the play impact is that your link building can look inaffective. Will be the very last thing you would like as a result of link building can be exhausting anyway, you don’t wish to cripple yourself before you’ve even started


The first factor we’d like to appear at is to create positive that every one of our target pages are often crawled by the search engines.


Before that, let’s check out however we tend to create our web site crawlable and the way to appear for potential issues.

Good web site design:

Good web site design isn’t solely sensible for search engines; it’s sensible for users too. Put simply, you would like to create positive that your most vital pages area unit simple to search out, ideally among some clicks of the homepage. This works well for one or two of reasons:


  • Usually, your homepage is that the most joined to and so will flow plenty of Page Rank throughout the remainder of the positioning
  • Users are going to be ready to realize your key pages quickly – increasing the chance of them finding what they need and changing into customers

Controlling the crawl :

We know that you just will build additional quality links into your web site which may facilitate together with your Page Rank , this can be one thing that we’d like to try to to anyway. However you’ll work to optimise your crawl budget by taking some steps to carefully nudge Google into the proper direction after they crawl:


  • Add the rel=”nofollow” tag to links to page you are doing not wish crawled
  • Make the links exhausting for Google to follow by creating them JavaScript or mythical being based mostly – use caution here tho’ as you would like to still create them clickable for the user and Google is recuperating at crawl these technologies all the time
  • Block bound sets of pages in your robots.txt file to prevent Google crawl them


Once you’re happy that the search engines area unit crawl your web site properly, it’s time to watch however your pages are literally being indexed and actively monitor for issues.


The easiest thanks to ensure Google is assortment a page properly is to examine the cached version and compare it to the particular version. There area unit 3 ways you’ll try this quickly.

  1. Run a Google search:
  2. Click through from Google search results
  3. Use a bookmarklet

Index Checking:

Another nice feature of Google Webmaster Tools is index standing. This provides some insight on however Google is crawl and assortment your web site moreover as providing you with an inspiration of what percentage pages Google area unit selecting to not index.


Now the ultimate a part of our work, arguably the bit that we tend to all really care concerning the most! Area unit our pages ranking moreover as they might be? We tend to area unit perpetually operating to induce our pages ranking over they already area unit, therefore I needed to specialize in a awfully specific plan of action instead.

Firstly you wish to search out what percentage pages you’re attempting to induce traffic to. This can most likely be your homepage, categories, product and content pages. There area unit some ways that you’ll try this, looking on your web site setup:


  • Look at range of URLs in your sitemap (relies on sitemaps being updated and accurate)
  • Speak to your developers World Health Organization ought to be ready to offer you a rough plan
  • You might additionally crawl your web site however this depends on all pages being accessible within the 1st place

Share On: