I’ve been getting a lot of questions about using the tag ‘rel=”nofollow”‘ on both internal and external links.

The best answer I can come up with is to look at the Google explanation here. They say that they don’t follow ‘nofollow’ links and not just not pass on PageRank (there are rumours that this is not strictly true, but I am happy to believe it is true enough).

With this in mind, the new ‘rel=”canonical”‘ header tag can help with allowing Google to spider as much of your site as it can (within reason, they won’t stay for as long as you want them to), but not leave PageRank juice all over the place (messy…).

As mentioned before, even your ‘noindex’ pages still carry PageRank and even though they will be passing a lot of this out they will still retain a bit. So the new tag really helps with large product sites with various results pages and various ways of ordering those results. Basically, you can now get the products pages spidered without dropping page rank on all of the search results pages. You still have to be a tad clever though with your dynamic URL insert into the ‘canonical’ tag.

Also, as mentioned in the Google post even though they don’t follow ‘nofollow’ you can still put these pages in the search engine sitemap if you want Google to find and index them. This answers the PageRank sculpting question with regards to ‘legals’, ‘privacy policy’, ect. pages which you don’t want to hoard PageRank but you still want Google to know you have them.

As we know there is no such thing as a duplicate content penalty. However, duplicate content is a real issue for both search engines and those who wish to do well in this area.

The big three (Google, Yahoo and Microsoft) rarely get together on general standards, but when they do it is worth taking note. All three have a problem cutting through the mess that is the world-wide web and trying to archive it into something that is usable for us all. A big issue in this respect is the amount of duplication the web throws up. This sits in two main areas…

1. The plagiarism that inherently exists on the web e.g. “ooh, that is a good article, I will use that on my site”, etc.
2. Websites that duplicate information either intentionally or unintentionally.

The later category is the one that the big three (mainly Google) have been trying for a long while to mitigate. Fortunately, the “pet insurance london”, “pet insurance cardiff”, etc. which were all basically the same page, but with different Meta and H1 tags problem has been dealt with pretty much (some are still out there though!). Also, data sorting on page (e.g. sort by price, etc.) which generates lots of very similar pages all with different URLs (i.e. usually with the “?sort=” parameter is slowly being tackled. Yahoo, also have a parameter based URL removal tool in their site explorer suite.

The recent announcement helps with issues 1 (not at all as you can’t use this tag for external domain pages) and 2 (yes) to try to help search engines and webmasters make sure that the real (and main) version of a web page is treated as the canonical (main and sort of only) one.

The addition is a tag for the head of your page which is…

link rel=”canonical” href=”The URL you want to be the main one for this web page”

As a rule it is not a bad idea to include this in all of your pages that you manually create (which includes ones that you create in WordPress, etc. – don’t worry there are some plugins available already). This means that if these pages get tagged with a differnet URLs in one way or another, at least the search engine know what you meant to be the main one. Also, if you create intentional duplicate content (landing pages, etc.) then you can use this tag to help you to not confuse the search engines.

The plagiarism one is still something that the search engines will have to deal with themselves. Also, if you are creating multiple pages (with different URLs) from a database source (feed, CMS, etc.) then this will only help you if you can incorporate a dynamic element into the ‘rel’ tag that picks the canonical URL for you. This will work as a good alternative to the ‘noindex’/'nofollow’ way which (was) my favoured method.

It is well to remember that this is not a panacea. You should still look to rationalise your URLs and ensure that they are not duplicated and indexed. Remember, you are still passing PageRank (well, the links you don’t ‘nofollow’ anyway) to any page you link to on your site (even the ‘noindex’ pages), so (still) make sure you are not giving away credit to meaningless pages.

However, overall the is really good and especially so since it creates a ’301 redirect’ environment (so the good stuff gets passed back to the original page too) and therefor better than ‘noindex’ in many ways. Remember this will only work on your domain and links between these pages (i.e. you can’t use the tag to external domains from the domain you are working on).

If used well this is an excellent addition to your SEM efforts.

Here is a really interesting post from the official Google blog.

It goes into some detail about some of the user testing and eye tracking Google are doing. The post looks at some of the differences that have occured since Google introduced universal search into its search results.

Also, if you look at the search results heatmap you can see how important the top two entries are in Google’s search results. Also, you can definately understand why Google puts more and more sponsored results above the search results as well as to the right. It must be ever so tempting for them to lessen their quality guidelines to allow more and more ads to make their way to this sweetest of spots.

Lastly, if you don’t already, this is a good time to branch out into picture and video search engine optimisation. You are missing a big win here and youe definately are if any of the search terms you would like to be listed for have an inherant visual element (just like ‘how to tie a tie’).

P.S. if you want to do some of your own (not eye but mouse pointer) heatmaps and user tracking, visit ClickTale. They have a fantastic tool that allows you to see a Flash movie of how your visitors have interacted with your site (what they click on, how long them spend on the page, where their mouse goes, etc.). The entry version is free and I would highly recommend having a look.

Here is a good post from the Google Webmaster Blog about open redirects and how redirects as a parameter in your URLs can harm your Google ranking.

As we know Google has to (and it can’t stop) indexing or at least assessing all it sees. It (as well as all the other search robots) trawls the web in search of pages, links, sites, etc. In doing this it follows all the routes it can (or rather, is ethically allowed to).

This means that if it is OK for it to do so (i.e. it can gain access and is allowed) it will seek and find many (many) different ways to find those all important web pages that keeps it ahead of the competition and up to date. This means that proxy servers and indeed server logs can be open to these information hungry creatures (OK, I hung on for as long as I could before I personalised a search robot, sorry…).

So, taking this into account if you are a victim of someone using your URL as a cloak to get people to click on a link on the web or an email link e.g. www.yoursite.com/go.php?url=www.badsite.com. Then don’t be surprised if Google finds this (and as mentioned it can be just because someone has clicked on the link from an e-mail and it is now stored on a proxy servers, etc. not just because the spammer has set up a web page with ‘click here’) and then indexes this URL. If it does and ‘www.badsite.com’ (great name isn’t it) is… well a ‘bad site’ with porn, malware, SEO black hat, whatever, etc… then unfortunately Google will probably take your site out of its index because of this. This is because you are and your site is (unwittingly) providing a bad URL to Google and this needs to be eradicated. If you are registered at Google Webmaster Central you will get a message telling you what they have done and some instruction on what to do next. If not you will probably be confused as to why your site has been dropped (best bet is to look at your server logs and see which pages on your site have been accessed or use the ‘site:www.yoursite.com’ command on Google with some date perameters added to filter for newly indexed pages).

Anyways, have a read of the article here and see if your site could be at risk.