404 page checker
Requests a path that cannot exist and checks you return a real 404, not a soft one.
What TMOD checks
- Requests a deliberately impossible path on your domain, a long random string no real page could occupy.
- Checks the HTTP status code returned. A real 404 or 410 passes; a 200 is a soft 404 and fails.
- Checks whether the response is a helpful page or a bare server default with no navigation back into the site.
- Detects redirects to the homepage, which is a common and actively harmful way of handling missing pages.
Why it matters
The status code is the part that matters and the part people get wrong. If a missing page returns 200, every mistyped URL, every dead external link, and every stale path becomes an indexable page in a crawler's eyes. Sites accumulate hundreds of these near-identical 'not found' pages, and they read exactly like thin duplicate content, because that is what they are.
Redirecting missing pages to the homepage is worse than a plain 404. It tells a crawler the content moved there, which is false, and it strands visitors on a page unrelated to what they clicked, with no explanation of what happened.
The page itself is a genuine recovery opportunity. Someone who followed a broken link is one click from leaving. A 404 page that offers search and links to the main sections often keeps them, and it preserves crawl paths that would otherwise dead-end.
How to fix it
01Return the correct status
Missing pages return 404. Permanently removed pages can return 410, which is a stronger signal to drop them from the index. Verify with `curl -I` rather than a browser, the browser shows the page, not the status.
02Never redirect missing pages to the homepage
If a specific page genuinely moved, 301 it to its specific new location. If it is gone, return 404. A blanket redirect to the homepage is a lie about what happened and helps nobody.
03Make the page useful
Say plainly that the page was not found, offer a search box, and link to the main sections and a few popular pages. This is what turns a dead end into a recovery.
04Keep it on-brand
A 404 in your normal site design with your navigation intact tells a visitor they are still on your site and the site works. A bare server error page suggests the whole thing might be broken.
404, 410 and 301, and when each is the honest answer
A 301 says this content is now at that address. Use it when there is a genuine successor: an article rewritten under a new URL, a product replaced by its next version, a category renamed. The destination has to be the specific replacement, because a redirect to something loosely related is treated as a soft 404 anyway and you have spent the redirect for nothing.
A 404 says there is nothing here, which is the correct answer far more often than people are comfortable with. It is not a failure state, it is information, and a site that answers it accurately is easier for a crawler to model than one that pretends every URL leads somewhere.
A 410 says it is gone and is not coming back. Functionally it behaves like a 404 with a stronger hint, and it is worth using deliberately when you have removed a section on purpose. Where none of these fit, because the page exists but should not be in search results, the answer is a noindex tag on a page that still returns 200, not a status code lie.
The page itself, and the log behind it
Keep it short. One sentence saying the page was not found in plain language, a search box, links to the main sections, and a couple of pages worth reading anyway. Skip the automatic redirect to the homepage after a countdown, which takes the decision away from someone who was reading the options.
Serve it in the site's normal design with the navigation intact. That is what tells a visitor the site works and one address did not, rather than the whole thing being broken. And re-probe the status code after any redesign, since a 404 template is the page most likely to be rebuilt by someone who never checked what it returns.
The more useful half is the log. The 404s worth acting on are the ones with a referrer or with repeat traffic, because those are real paths that people and crawlers are still following. Those addresses either deserve a redirect to their successor or reveal a link inside your own content pointing somewhere that no longer exists, and both are quick fixes once you know which URLs to look at.
The one-minute test you can run right now
Request a page that cannot exist, yourdomain.com/xkcd-was-here-9472, and look at the status line rather than the screen. With curl the command is curl -I followed by the URL, and the first line of the response is the whole diagnosis. HTTP/2 404 means the site is answering honestly. HTTP/2 200 means every nonexistent address on your domain is being presented to crawlers as a real page. HTTP/2 301 with a Location header pointing at the homepage means missing pages are being papered over. The browser's network panel shows the same status column if curl is not to hand; the point is to read the code, because the rendered page is identical in all three cases.
Where the 200 comes from is usually the architecture rather than a mistake anyone made. Single-page applications are the modern source: the server is configured to answer every route with the application shell so client-side routing works, the JavaScript router then displays a not-found view, and the HTTP layer has already said 200 before any of that ran. Static hosts reach the same place through a catch-all rewrite rule, and CMS themes get there when a custom error template is built as an ordinary page.
The fixes are all at the layer that speaks first. An SPA needs the server, or the framework's rendering layer, to know which routes exist and answer unknown ones with a real 404 rather than the shell. A static host needs its catch-all ordered after the real files, with an explicit 404 rule. A CMS needs the error template wired as the error handler rather than published as content. In each case, one more curl -I afterwards confirms the status actually changed, which is precisely what this check automates.
The stakes scale with how many dead URLs point at you. A site that was restructured twice and scraped once can have thousands of retired addresses in circulation, and with a soft 404 every one of them becomes an indexable page of boilerplate, a thin-content problem manufactured out of nothing. With honest 404s the same URLs simply drop out of the index. Pair the fix with a probe of your own links so you are not relying on the error page more than necessary, and let the technical audit re-test the whole path after any redesign.
Questions
What is a soft 404?
A page that says 'not found' to a human but returns HTTP 200 to a machine. Search engines specifically flag these, because they cannot distinguish them from real content without reading the page. Every nonexistent URL then becomes an indexable near-duplicate. It is the single most common misconfiguration this check finds.
Should I redirect old URLs instead of 404ing them?
When there is a genuine successor page, yes, 301 to it. When there is not, 404 is the honest answer and the right one. Redirecting an old URL to a loosely related page is treated as a soft 404 anyway, so it gains nothing and costs clarity.
Do 404s hurt my rankings?
404s on URLs that never should have existed, typos, spam probes, scraped bad links, are completely normal and harm nothing. Every site has them. What matters is 404s on URLs that should work: internal links pointing at dead pages, sitemap entries returning 404, and pages that used to rank now returning nothing.
Google indexed my 404 page itself. How?
Almost always the soft-404 route: the error page was served with a 200 status at some point, got crawled through a broken link, and stuck. Once the status is corrected to a real 404, it drops out on its own over the following weeks. If the error page has its own URL, like /404 or /not-found, that page can legitimately exist, it should just carry a noindex tag so the address people share does not become a search result.
How do I find out which missing URLs people are actually hitting?
Two sources, and they complement each other. Search Console's indexing report lists the not-found URLs Google knows about, which covers what crawlers hit. Your server or hosting logs show every 404 with its referrer, which covers what people hit and, crucially, where they came from. Sort either list by frequency: the URLs hit repeatedly are live paths that deserve a redirect or reveal a link of yours that needs fixing, and the long tail of one-off garbage can be ignored entirely.
Read more
Screaming Frog, Ahrefs and Sitebulb for AdSense readiness
The big SEO crawlers are better at technical auditing than any AdSense tool. They are also scoped to a different question. Here is where the line falls.
Search Console vs a site crawler, and what each one cannot tell you
One reports what Google decided about your site. The other inspects what your site actually serves. Neither answers the other's questions.
The robots.txt lines that quietly block ad crawlers
Blocking Mediapartners-Google costs you ad revenue while the site keeps ranking normally, so nothing looks wrong. Here is how to read the file.