Extracting specific sequences from a big fasta file using ids of the sequences to be excluded

Question

[hasche89]

I have a huge fasta file of around 20 GB size. I also have some sequence IDS from the same fasta file in text format. Now, I want to retrieve those sequences which don't have those particular ids in the text file.

How shall I proceed? I use Ubuntu 12. I am a novice and have very little knowledge of bash, shell or perl. Any Linux or Samtools or Bioperl command will be helpful.

Thanks.

Answer 1

[thackl]

This would work:

git clone https://github.com/BioInf-Wuerzburg/SeqFilter.git
cd SeqFilter
make  # just fetches some libraries, no root or anything required

bin/SeqFilter big.fasta --ids idx.txt --ids-exclude --out big-filtered.fasta

Answer 2

[GouthamAtla]

Simple way is to get a list of IDs that you would like to fetch from fasta. This could be done with 'grep' .

grep "^>" input.fasta | sed 's/>//' | grep -v - -f Ids.txt > retreive_IDs.txt

Then you could use something like pyfaidx or samtools

samtools faidx input.fasta `cat retreive_IDs.txt` 

[venu]

and also faSomeRecords

./faSomeRecords input.fa retreive_IDs.txt output.fa

[hasche89]

Thanks for the commands.

I am a beginner in this field. Can you please tell me what does each component of your command does?

Thanks

[venu]

Execute each command on your own, then you will understand very easily what each command is doing.

Answer 3

[Brian Bushnell]

Boy, this really comes up a lot. Using the BBMap package:

filterbyname.sh in=file.fasta out=filtered.fasta names=names.txt include=f

[thackl]

Always important to keep busy ;)