Wednesday, March 19, 2008

Stop the link rot, please!

Bruce Schneier wrote recently about link rot -- web pages that are no longer available at their old URLs -- and the inconvenience it causes to internet users (in some cases, like online communities, it's much more than inconvenience). As a blogger who links to lots of online stuff, I find link rot a major pain in the neck.

I use this blog for linking to things that I find interesting; but I also use it as a place holder for things that I may need later (many links to data sources on higher ed around the world belong to this category). The hope, of course, is that these links will still take me to those pages when I need to visit them. This is just a hope.

Among Indian newspapers and magazines, I haven't had any link rot trouble with the Indian Express, Telegraph, and Outlook. While the Hindu has been good in keeping all its content open; it dishes out stuff using multiple sites, some of which don't seem to work later. And I haven't figured out which are the ones to avoid!

Some sites take their content behind a paywall after sometime. I think these sites are committing a slow suicide by ruling their content out of conversations, more and more of which are happening on the internet. Newspapers like the Business Standard belong to this category, and I wouldn't shed a tear when they all eventually die.

Some other sites go through a website redesign which creates a completely new URL structure. Examples of this sort of heightened incompetence include the Times of India and the Economic Times. All my old links to articles in these newspapers are totally useless!

Another example is the Economic and Political Weekly, whose redesign also leaves us with tons of link rot. To make things worse, their current design makes it difficult to link to their articles (pdf files) directly. What use is free content that defies direct links?

And finally, we have bloggers -- who really ought to know better! -- who simply decide that they need to start over! The most recent example is Ramnath, who retains the old blog URL, but none of the old content (A lot of it was so interesting.... Sigh!). Before him, Arunn canned his old blog and started a new one; much of the old content is there, but the URLs have changed! As a crowning example, let me cite the vanishing act of How the Other Half Lives (you can see the link -- broken, of course -- on the sidebar) to which I contributed quite a few posts.

It pains me to go back to the archives, and see that many of my posts have useless links! Grrrrr!

* * *

Some lessons:

  1. Don't get yourself a website if you cannot maintain (pay for and keep) the site's domain name for years and years.
  2. Don't go through site re-design if you don't know how to keep the old URLs intact.
  3. If you want to start a new blog on domains like WordPress or Blogspot, please start a really new blog; leave the old one alone. As someone who read and linked to your blog, I'll be grateful!

BTW, (1) and (2) should tell you why I don't have my own domain ;-)


  1. Anonymous said...

    Your advice has come to me in the right time! I moved to my paid domain yesterday and I was thinking of deleting by old blog! I may reconsider my decision, as you have linked a couple of my posts too!

  2. Anonymous said...

    Hi Sir,

    Regarding Hindu, you can use URLs like the one below

    It is permanent

    And Indian Express is a pain as the content gets behind pay after some time and is very difficult to search later

    Though you cannot call it "cute" in appearence, Hindu Website is the best newspaper site I have seen (for ease of navigation and search)

  3. Anonymous said...

    As you have told, I have redesigned my site many times, but I make sure that the user reaches the correct page in the new design even if he/she clicks the old URL

    For Example, I started blogging at

    Then moved to

    Now I am at

    But even if you click on any of the above links, you can land in the same page in the new design

  4. Yogesh K. Upadhyaya said...

    1) Thanks for bringing out this important topic. The survival of linked news items vary with different newspapers. It is few days for papers like Hindustan Times and few months for Times of India and few years for Daily Telegraph. It depends upon the server, capacity available and how it is programmed.

    2) Our monthly newsmagazine, chronicle was also facing the same problem. The links for news items could not be opened after few months.

    The problem was solved by publishing the entire article, not just the link.

    Yogesh K Upadhyaya
    New Jersey