This is an add-on to my post on SEO Link Juice Leakage and deals with making your 404 error page a better experience for your user.
As mentioned in my previous article you should always ensure that you protect your pages from leaking juice (I really must try to think of a better term…) by…
- Not changing the URLs at all
- If you have to change URLs then make sure you 301 redirect all of them to the new pages (or if they don’t have a defined new page then the closest match or home page)
However, there will always be some pages that slip through the net and there will always be some people who key in the wrong page into the URL bar. This is where a nice and friendly 404 page will be a great help to your visitors.
Before we get started here, please remember that this will not help your SEO efforts, but it won’t hinder them either. You need to make the effort to deal individually with all the pages that you have changed URLs on and do a 301 redirect. This is the only way that, some or all of, the power (juice!) will not be lost and instead passed on to the new page.
The dangerous part of customising your 404 page is that it will create confusion for search engines. This is because many people who create a custom 404 error page allow it to return a ‘200 OK’ header. This means that the page is actually alive and can be found and indexed by search engines.
So think of this scenario; you have a site that you have changed from a static site to a CMS based one. Your programmer either didn’t know or could be bothered to rewrite the URLs (in any case URL rewriting on this scale is a very delicate thing to do to get right), so you have a site that has completely changed its URL structure.
You think I know what I will do I’ll (or get my web guy to) create a 404 page that says something like:
- “sorry, we have changed our site and the page you want does not exist anymore, please click here to visit our home page”
- “we think you may want one of these pages” – and then list a few relevant pages that might be the best match
Even worse you may just set up a (302) redirect to the home page (from your 404 page!). There is an argument that directing the bulk of your pages with individual 301 redirects and then letting a 404 page handle all remaining redirects with a 301 redirect to the home page is an OK SEO way to do this. Well, as long as you really put the effort into redirecting your pages with this (the 301 redirect on your 404 page) element being the final bit taking up the slack, then it can be OK. However, from a user experience perspective it is better, in my view, to concentrate on doing a good job on your 301 redirects and then creating a (SEO friendly) 404 error page that will act as a guide for now and in the future.
So, what is the problem with letting your 404 page give a ‘200 OK’ header? Well it means that any page that does not really exist (whether it existed before or is just a ‘keying error’ now has a life of its own and can be indexed by search engines. Basically, the search engine spider will visit the page and index what it finds, which means that you will have lots of pages that seem to be alive and well, but really don’t exist anymore. This will mean that your power (juice) will never be passed over to your shiny new pages and you will have a load of meaningless pages that will continue to take up the majority of the search bots time. Also, your pages will cease to be relevant for the things there were before (because they will all say something like “sorry, try these pages”, etc.) and will drop like a stone down the rankings. Not a good situation.
So, after all that… here is how I have found the best way to create your customised 404 page. Please bear in mind that this is an Apache sever solution (I will look at an IIS solution at a later date, but please feel free to suggest one if you have it).
Firstly, add a line to your .htaccess file like this:
ErrorDocument 404 /404error.html
Substitute your own 404 page’s URL for the ‘/404error.html’ bit. However, please remember not to use the full CNAME e.g. http://www.yoursite.co.uk/404error.html here or this will send the request to the web and create a ‘200 OK’ header response instead of the “404 NOT FOUND” response we want.
So you get a customised 404 error page that returns a ‘404 NOT FOUND’ code, helps your visitor get to where they want to go on your site AND does not confuse the life out of search engines by giving them an incorrect header code.
Before you ask I haven’t got around to doing it on this site yet… However, try putting in a few random characters at the end of the URL on this site here, Large Format Printers Samples. You will see what you can do with this approach. Also, if you check the header code for your mistyped URL here you will see that your 404 URL actually returns a ‘404 NOT FOUND’ header code.