Screaming Frog, Ahrefs and Sitebulb for AdSense readiness

If you already run Screaming Frog or Ahrefs, the reasonable question is whether you need anything else before applying to an ad network. Partly no. These are better crawlers than any purpose-built approval checker, this one included, and for the technical layer you should use them.

The gap is not quality. It is scope.

Checked August 2026 against each vendor's own pages; limits and prices change.

What each one gives you

	Free tier	Crawl limit	Runs where	Needs verification
Screaming Frog	Yes, permanent	500 URLs per crawl free, uncapped on a licence at £199/year	Desktop	No
Ahrefs Free	Yes, permanent	5,000 crawl credits per month per project	Cloud	Yes, you must verify the site
Sitebulb	14-day trial only	10,000 URLs per audit on Lite, 500,000 on Pro	Desktop and cloud	No
Semrush	Limited free account	1 demo project, 10 reports per day	Cloud	For projects
Search Console	Yes	Only URLs Google already knows about	Cloud	Yes

Two things stand out. Screaming Frog's free tier is the most generous no-strings option for a small site: 500 URLs covers most blogs entirely, with no account and no verification. And Ahrefs giving Site Audit away for verified sites is genuinely the best free technical audit available, provided the site is yours.

Where the scope actually ends

Ahrefs publishes the list of what Site Audit checks. As of this writing it covers nineteen categories: slow pages, Core Web Vitals, heavy CSS or HTML, titles, meta descriptions, H1 tags, content, duplicates, indexability, social tags, localization, links, redirects, images, JS, CSS, robots, sitemaps and structured data.

That is a thorough technical audit. It is also, in full, a technical audit. Nothing in that list is about whether a privacy policy exists, whether a consent banner lets a visitor refuse as easily as accept, whether ads are placed so they could be mistaken for content, or whether any page carries content an ad network will not run against. The same is true of the others in the table, and it is not an oversight on their part. They are SEO tools and they are scoped like SEO tools.

Those unscoped items are a large share of first-application refusals, which is the whole reason a separate check exists. Our policy and compliance audit runs only that layer, eleven checks, and it is the half a crawler will not report on.

The overlap is real, and worth using

Where the two do overlap, use the better tool. A general crawler will find broken links, redirect chains, missing titles and duplicate pages more thoroughly than we will, and on a large site it will do it faster.

The specific things worth pulling from a crawl before an application:

• Status codes, especially pages returning 200 with an error message. Soft 404s get indexed as thin content and multiply on their own.
• Duplicate pages, which are usually one article at several addresses rather than anything written twice. The fix is normally one template change.
• Robots directives, though note that a general crawler reports whether Googlebot is blocked. The ad crawler is a separate user agent and blocking it leaves search completely unaffected, so nothing looks wrong.

That last one is the clearest example of the scope difference. A crawler tells you the file is fine. It is fine, for search.

A sensible split

Run the crawler you already pay for on the technical layer, because it is better at it. Then run a policy pass separately, because none of them do that. If you want the thresholds rather than a verdict, ours are published on each tool page: the thin content checker names its word counts, and the rest do the same.

If you own none of these and want one free thing, Screaming Frog's 500-URL tier and Google's own Search Console together cover most of what a paid crawler would tell you about a small site. The comparison between a console and a crawl is its own piece, because the two answer genuinely different questions and neither substitutes for the other.