I decided to do a bit of site admin over the last few days, by at least trying to tidy up any outstanding issues lurking in the Cipher Mysteries site, mostly from the changeover from Blogger to WordPress. While doing this, I noticed that the site had suddenly dropped about 300 places in Google’s rating for basic search terms, such as “Voynich” and “cipher” (though it has since recovered to about 180): at one stage, it wasn’t even #1 for “cipher mysteries” (Bah!) Even though I’m still getting lots of new visitors (mainly from mailing list recommendations, I think), not a lot of “love” (as SEO people like to describe PageRank, usually to sex up their dull presentations) is flowing my way from Google. Perhaps that will change soon…
Anyway, I found an excellent free tool that revealed a whole load of site problems I wasn’t previously aware of: http://www.dead-links.com/ Much as you’d expect, this is a free robot crawler that trundles happily through your website finding all your dead links, both internal and external. Links to commercial sites (such as Amazon / IMDB etc) often get reported as “405” (which basically means, “no robots allowed in here, so go away“): you’ll also get the occasional “403” (“Forbidden“, where you’ve accidentally linked to a page you had to log in to access), and perhaps a “500” (“Internal Server Error“), where Bad Stuff Is Going On But You Don’t Actually Know What It Is.
In the main, though, the bulk of the errors are likely to be “404” pages: this can encompass just about anything from miscopied or dead URLs, to “the-page-was-loading-too-slowly-so-the-robot-decided-to-give-up” (normally a false positive). Helpfully, dead-links.com lists all the errors a second time at the end of the output page, so you don’t need to cut and paste them from the huge list yourself (errrm… like I happened to do the first time).
And so, I’m going to try to sort out all my dead and malformed links over the next few days: if only 50% of the Voynich sites on the web would do the same…