Has anyone got any examples of projects in bioinformatics where untrained volunteers will evaluate or inspect molecular data and provide feedback for data analysis?
For example, I've seen fold.it and I've heard of one about DNA/protein alignments. Amazon Mturk could in theory also be used for curation.
I have had done some experiments with amazon's mturk. The quality of the work was amazing, although you should develop some curation mechanisms for spam. There are quite some so called workers out there that try to making some pennies by submitting rubbish. I ended up designing the hits in such a way, that denying work from spammers was done semi-automatically. In this specific case I used drag-and-drop functionality where the workers had to drop sections in specific drop boxes. I did some task where people were asked to parse sentences in to triples.
The best way to evaluate the value of mturk is to just do some of the available hits. I usually do some while waiting at airports/trainstations/etc/ As a non us citizen you earn amazon points. Which is a nice incentive.
That brings me to the mayor disadvantage of mturk and that is that jobs can in theory only be submitted by someone with a US-based address. I could experiment a bit with it, thanks to that account of a friend. But in general is it aimed at an US audience. I noticed though that most work was done in the US and India.
Inspired by Andra's suggestion I decided to put my NAR database web accessibility survey (Which Of The 2011 Nar Database Submissions Are Fully Accessible? ) up on mturk Which Of The 2011 Nar Database Submissions Are Fully Accessible?
https://www.mturk.com/mturk/searchbar?selectedSearchType=hitgroups&searchWords=NAR+database
"Workers" can now earn up $0.08 per NAR database surveyed!
Would be interesting to compare your results with Pierre's results: A: Is The Nar Database List Available In A Computer Readable Format?
I thought a new answer to this old question might be valuable. In this review article on Crowdsourcing for Bioinformatics, we described several papers that used MTurk for text annotation. Including:
And also this one recently came out:
The one about sequence alignment is called Phylo.
Are you specifically looking for molecular biology projects because otherwise I can suggest Open Dinosaur Project which asks that the crowd aggregates published measurements of dinosaur limb bones for many different taxa from the literature to study the evolutionary transitions from bipedality to quadrupedality. Then there are the very impressive astrophysic projects like Galaxy Zoo and Stardust@Home. Zooniverse have other projects like gathering information about past climates from hand written nautical records (OldWeather).
UPDATE: FoldIT is another good example of crowd sourcing bioinformatics projects.
What about the crowdsourcing analysis of the German outbreak E. coli genome?
https://github.com/ehec-outbreak-crowdsourced/BGI-data-analysis/wiki
Hi,
I recently heard about eteRNA which is similar to Fold IT but focused on RNA.
Also, the university of Marseille used some undergraduate students to annotate/validate some metagenomes. It is a bit different since the students where neither volunteer or "untrained" but still of interest Metagenome Annotation Using a Distributed Grid of Undergraduate Students The project has gone worldwide under the name of Annotathon
Another example, non related at all to biology, is GalaxyZoo which aim at helping astrologists to classify galaxies.
I hope this helped you.
The yrGATE module at PlantGDB allow volunteers to evaluate de novo gene predictions, splice-aligned cDNAs/ESTs, and even re-run annotation tools with custom parameters to build their own custom gene models for a given genomic sequence. All volunteer-submitted gene models are saved in a pending state, and expert curators subsequently approve, modify, or delete the models.
I just found out about the idea of a Gene Ontology annotation game:
Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
version 2.3.6
We are working on one... http://afra.sbcs.qmul.ac.uk https://github.com/yeban/afra