filter out fasta File by pattern

Question

[Assa Yeroslaviz]

Hi,

I would like to filter out some sequences from a fasta file by using a specific pattern.

For example I have this file:

>input1
UGAGGUAGUAGG
>input2
CUAUGCUUACC
>out1
UCCCUGAGACCGUGA
>out2
CUCCGGGUACC
>desc1
ACUUCCUUACAUGCCC

I know already how I can extract all the fasta sequences with a specific pattern into a new file by using awk.

But what I would like to do is to remove all entries of a specific pattern from the original fasta file and save the newly made file into a new one. In my file above, I would like for example to remove all sequences with the header pattern out. and save only the other to a new file.

Is there a tool somewhere for doing that, or is it possible in awk/sed or even grep

Thanks

Assa

Answer 1

[Pierre Lindenbaum]

I would like for example to remove all sequences with the header pattern out. and save only the other to a new file

awk '/^>/ {P=index($0,"out")==0} {if(P) print} ' in.fasta > out.fasta

[Assa Yeroslaviz]

Thanks. That was fast :-)

[xiachongjing]

Hi Pierre! Thanks for your command.

I have a question on this issue. Instead of one header pattern "out" (in your case), I am looking for many patterns that are stored in a file. So what should I do? Your help is appreciated in advance.

[Matt Shirley]

You could modify my answer below:

$ pip install pyfaidx
$ xargs faidx in.fasta -g > out.fasta < patterns.txt

[tlorin]

See also this post: How to remove some fasta sequences by header information from a large fasta file, any command and script please?

[bioyas]

Thank you Pierre for your answer. I used your command and it seems it is removing the header with a particular pattern from fasta file . I was wondering if this line of command removes the whole sequence associated to that header as my fasta file is not linearized and sequences are stored in multiple lines.

Also do you have any Idea how can I store the target sequences for deletion in another file?

[cpad0112]

Please

1. Do not revive years old post as the answers are years old and new tools have come in.
2. Post your query as a new post with example input and output

Answer 2

[Matt Shirley]

I like Pierre's answer to this since it's simple. However, I had been thinking about adding regular expression filtering to my pyfaidx project, and this morning I finished up adding this functionality:

$ pip install pyfaidx
$ faidx in.fasta -g "out" > out.fasta

The (small) advantage here is that faidx will perform filtering on an indexed file, preventing you from reading the entire file through your filter.

Answer 3

[hans]

Using samtools

$ samtools faidx in.fasta
$ awk '/^>/ {print substr($1,2,400) }'  in.fasta | grep -v "out" list >selected_list
$ samtools faidx in.fasta -r selected_list > out.fasta

This will give you all the functionality of "grep" to select the desired sequences.

[Pierre Lindenbaum]

Why print substr($1,2,400)?

Why piping in grep -v "out" while the same grep reads a file list?

[hans]

print substr($1,2,400) 

Keeps only the first field of the pasta header while deleting the ">" from the header. The file name "list" in the grep command is a mistake, it should be deleted:

awk '/^>/ {print substr($1,2,400) }'  in.fasta | grep -v "out"  >selected_list