Occurrence Of "Dead" Bioinformatics Servers/Tools

Question

[K]

Actually, more than asking a specific question, I rather wanted to share a thought, in case you want to share your impressions:

While looking for bioinformatics tools to help me in studying the proteins that are of interest to my research, I have encountered with some frequency tools or online servers which are no longer working.

In some cases, they apparently do work, but when one tries and struggles with them for a while, one encounters the bitter truth (while eternally awaiting for results that never arrive...). In other such cases, the url published in the original paper simply does not work, and no other trace of "server X" or "tool Y" exists except the original paper. This is possibly not surprising for tools ten years old or more. It is however more striking for tools which were released online only a few years ago. I guess it likely reflects the fact that, in public research, devoting a person to tool maintenance can become an overly burden from the point of view of "moving on" to get other stuff published.

Do you share this impression? Are you aware of any listing of "dead" or abandoned bioinformatics tools?

This would save time while trying to find useful, functional tools (so one would not waste time awaiting results which never come).

[Michael]

Maybe, move to Forum and make a list of the most missed abandoned bioinformatics tools and services?

Answer 1

[Pierre Lindenbaum]

see my blog posts:

• "The poor state of the java web services for Bioinformatics" http://plindenbaum.blogspot.fr/2010/05/poor-state-of-java-web-services-for.html
• "404 not found": a database of non-functional resources in the NAR database collection " http://plindenbaum.blogspot.fr/2011/04/404-not-found-database-of-non.html
• http://plindenbaum.blogspot.fr/2011/04/404-not-found-update-for.html

and

http://www.ncbi.nlm.nih.gov/pubmed/15033874 "404 not found: the stability and persistence of URLs published in MEDLINE."

[K]

Wow!

I didn't know biocatalogue.org... I will correct my previous words: maybe, YES, there has been improvement. However, I have checked a few of the tools I used or tried to used, and it seems that not all are listed there. Nevertheless, this site seems to be close to what I was imagining. I hope it keeps growing and developing. Well, thanks again for your previous reply, Pierre. Cheers!

[Neilfws]

"404 not found" is the classic paper on this issue. It's understandable: servers in academic labs are (still) frequently maintained by one person (usually a postdoc) - they move on to a new job, no-one else knows or cares about how to keep a server running - certainly no incentive for the person who left. I'm guilty of this myself, unfortunately. Academic standards are just not the same as those of professional developers or IT people.

[K]

Thanks, Pierre.

So apparently it seems that this has been happening for years already, and that the landscape has not significantly improved, right? Maybe if there were a general supervisor of all the available tools/servers...

Regards from Barcelona.

Answer 2

[Jeremy Leipzig]

The bioinformatics community views traditional web applications with increasing disdain, in part because of the abandonware issue you mention, and in part because they don't scale well for large queries. I wrote a bit about this a couple years ago here.

I actually have a lot of hope and interest in these cloud-based app solutions (BaseSpace and DNANexus) because they offer a potential for ongoing financial remuneration to the app authors as an incentive to stay current, and because they promise more in terms of scalability and "bringing the tools to the data" blah blah blah.

People need to be rewarded for their efforts to maintain tools. Because let's face it: the whole guilt trip thing just isn't cutting it here.

Answer 3

[Sudeep]

I was maintaining a tool for animal science partly during my master thesis and PhD. But the reality is that when you look at server log files and see < 10 hits per 6 months or an year (besides publishing the tool and promoting it in conferences) the "inspiration" to update it and actively maintain it is totally lost. I guess this might also be the same for the tools you mentioned here " It is however more striking for tools which were released online only a few years ago".

You were almost spot on when you said "in public research, devoting a person to tool maintenance can become an overly burden from the point of view of "moving on" to get other stuff published". Sometimes, if PhDs who are in charge of maintaing/developing tools finish and move away, there might be no one else to replace him/her and the rest will have their own priorities, so these tools will not be maintained and finally will be dead.

Answer 4

[Mary]

We toyed with this idea some time ago: Bioinformatics Tools And Resources Obituary Section.

But it seems to have died. Ha!

I will say that the new PubMedCommons comments from bioinformatics geeks have been really good and helpful for describing new/altered access details for the tools. That could be one really great use of PubMedCommons in the long run.

Answer 5

[bede.portz]

On one hand, you would assume the "best" tools are written by people who made an investment in the development of tool, and if a community of users coalesces behind a particular tool, said tool will be maintained. However, the structure of academic research means there is often little incentive in terms of career advancement and zero financial incentive to maintain a tool.

I think this is a real problem for bench scientists such as myself who lack the skills to develop my own work arounds when a tool I have come to rely on is no longer maintained (this has happened in my short data analysis "career".) Additionally, this is also negatively impacts analytical reproducibility. If you cite a publicly available tool, only to have that tool go extinct, there is little hope of anyone being able to reproduce your published analytical methods. However, given the aforementioned lack of incentives, coupled with the constant pressure to produce novel work, I don't blame the authors and curators of tools one bit when they go extinct.

The other issue is that tools going extinct is the cycle of code duplication. I think if funding agencies realized how much money they have spent funding the development of similar tools, maybe a funding vehicle could be created to provide for the continued hosting and maintenance of exisiting tools, which I suspect would cost less than the initial development itself.