Software for cleaning FASTA files

Question

[l.souza]

What software can I use to remove sequences:

• with a lot of unkown nucleotides (N)
• duplicated
• too short

?

[WouterDeCoster]

Hi Lucas Souza,

This question contains insufficient information to get answered. Please elaborate, for example on the technology used for generating the data. We are really bad at reading your mind (that's a bug in biostars which will get fixed in a next release). So for now, put some effort in your question and don't make it hard for people trying to help you.

Cheers,
Wouter

[l.souza]

Edited. Is it better?

[WouterDeCoster]

Obviously, all information you just added is vital for answering the question. By now we have lost 9 hours since you asked the question. Remember this for further questions that it's important to be as informative as possible.

[GenoMax]

Must you use entire genomes? Are there particular genes that are perhaps informative enough?

[l.souza]

The entire genome produces a single polyprotein that is cleaved in post-translational processes. For this reason I think it would be better to use the entire genome. But I may be wrong with this thinking

[GenoMax]

Lucas Souza : Please do not change the entire contents of an original question. That makes the chain of responses/comments here meaningless. Consider this a fair warning. I will tag @Istvan to see if he can restore the original question.

[GenoMax]

Tagging: Istvan Albert

Answer 1

[Brian Bushnell]

Hi Lucas,

You can accomplish most of what you want using the BBMap package.

Filtering Ns and short sequences:

bbduk.sh in=file.fasta out=filtered.fasta maxns=50 minlen=1000

Removing exact duplicates and containments:

dedupe.sh in=file.fasta out=deduped.fasta

Unfortunately BBDuk does not have a mechanism for removing sequences based on the % of Ns, just the absolute number, but I may add that.

[finswimmer]

Wait, there is a task that none of the BB-Tools can do? :)

[Brian Bushnell]

Geez, like I said, I may add it :) I already wrote myself a note!