What the test checks
A crawler is the program a search engine sends to read your pages, such as Googlebot for Google and Bingbot for Bing. The test reads a page the way a crawler does, in four steps: the first three decide whether the page can be shown in search results, and the fourth helps search engines find it.
- The page opens: its address answers with the page itself, not with an error such as 404 (not found) or 500 (server error), the status codes a server sends. A redirect loop, addresses that keep sending visitors to each other, fails too.
- The door lets crawlers in: robots.txt, the small file of rules for crawlers on your site, does not keep them out. The Robots.txt checker shows the line that decides.
- No sign asks to leave: the page has no noindex, the tag that asks search engines to leave a page out. The Noindex checker shows it.
- The map lists it: a sitemap, the list of your pages for search engines, helps them find the page before any other page links to it. The Sitemap checker counts what yours lists.
Google explains how it finds, reads and lists pages.
What the test cannot see
Three things only Google can tell you:
- What a firewall lets Google see. A firewall is a guard in front of your site, such as Cloudflare. Some turn away readers they do not know, like this test, and still let Google in; the test then says it cannot tell whether search engines can read the page.
- What JavaScript adds to the page. JavaScript is code that adds parts of a page after it arrives. The test reads the page as it arrives; Google runs the code later.
- How often Google comes back to read the page again.
For all three, the URL Inspection tool in Google Search Console shows exactly what Google gets.
Other free checks
- Robots.txt checker: which crawlers may read a page, and the line of robots.txt that decides.
- Noindex checker: whether a page asks search engines to leave it out of their results.
- Sitemap checker: whether search engines can find and read your sitemap, and what it lists.
- Submit your website to search engines: a guide to putting your site into Google and Bing, step by step.
Questions about the crawlability test
What is crawlability?
Whether a search engine's crawler can open a page and read it.
A page it cannot read shows up in results at best as a bare address, without its text.
How do I check whether Google can crawl my page?
Use the URL Inspection tool in Google Search Console: paste the page's address and press Test live URL.
It shows whether Google could read the page and what it saw.
The test on this page checks the same basics for any site, without signing in to Search Console.
Is my site in Google?
Search Google for site: followed by your address, such as site:yoursite.com.
The Page indexing report in Google Search Console gives the full count and the reasons pages are left out.
My page passes. Why is it not in Google yet?
Passing means search engines can read the page, not that they have read it yet.
A new site that nothing links to is read last.
Search engines find it sooner when you submit the sitemap in Search Console, link to the page from your other pages (internal links) and get links from other sites.
Does the test run the page's JavaScript?
No.
It reads the page as it arrives from the server.
Google does run JavaScript, later; URL Inspection in Search Console shows the page as Google sees it after that.