Search Console vs a site crawler, and what each one cannot tell you

These get compared as if you were choosing one. They answer different questions, and the failure mode of each is being trusted for the other's job.

A search console is a report from Google about decisions it has already made. A crawler is an inspection of your site as it is served right now. One is a rear-view mirror with a delay; the other is a torch you can point wherever you like, including at sites you do not own.

The split, in one table

	Search Console	Site crawler
Needs verified ownership	Yes	No
Works on a competitor's site	No	Yes
Impressions, clicks, average position	Yes	No
The queries you actually appear for	Yes	No
Whether a URL is indexed right now	Yes	No
Which URL Google picked as canonical	Yes	No
Which canonical you declared	Only per URL	Yes, every page
Status codes and redirect chains	Partial	Yes
Titles and descriptions across the site	No	Yes
Pages that duplicate each other	No	Yes
Broken internal links	No	Yes
Data freshness	Two to three days	Live
Coverage	Only URLs Google knows	Every URL you link to

The rows that matter most are the ones only one column can fill. Nothing except Search Console will tell you which URL Google selected as canonical when it disagreed with your tag, and nothing except a crawl will give you every title on the site in one pass.

What a crawl cannot see

A crawler reads what your server returns. That means it is authoritative about markup, status codes, redirects, declared canonicals, headings, internal links, and whether two pages are substantially the same. It knows nothing about outcomes.

It cannot tell you how you rank, or for what. It cannot tell you whether a page gets traffic. It cannot tell you whether the facts on the page are correct, which no tool can. And it cannot see anything behind a login, so if the pages you care about require a session, a crawl sees the same thing an anonymous visitor sees, which is usually the point but occasionally a blind spot.

The subtler limit is that a crawl reports what should happen. A page can be perfectly crawlable, correctly canonicalised, and still not indexed, because Google chose not to. Only the console reports that choice, and its Crawled, currently not indexed state is the single most useful thing it says, because it means the technical work is done and the problem is the page itself.

What a console cannot see

It only knows URLs it has discovered. A page nobody links to and no sitemap lists is not in the report, and its absence looks identical to it being fine.

It is per-property, so it says nothing about anyone else's site. Competitive work is a crawl-only activity.

It also reports per URL rather than across the site. The URL Inspection tool is excellent and the Search Console documentation covers it properly, but checking two hundred pages one at a time is not a workflow, and the API that automates it is quota-limited per property.

And it lags. Performance data is two to three days behind, which is fine for trends and useless for "did my fix work this morning".

Using them in the right order

The pattern that works is: crawl first, console second.

Fix what is unambiguously wrong before asking Google what it thinks, because a console report on a site with broken canonicals and soft 404s mostly tells you about those. Get the technical layer clean, then read the coverage report to find what Google still refuses to index, and treat that list as a content problem rather than a technical one.

That is the order our own audits assume. The technical SEO audit covers the crawl-inspectable layer on its own, and the full SEO audit adds the content signals that sit on top of it. Neither replaces the console, and we say so on both pages.

Which crawler you use for that first pass matters less than people think, and if you already own one, it will do the technical layer better than a purpose-built approval tool.

Two specific things worth crawling for before you open the console at all: duplicate pages competing with each other, which distort every report downstream, and soft 404s, which fill a coverage report with noise that looks like an indexing problem and is not.