Search a read by its name in a big fastq.gz file

Question

[xiaoleiusc]

Dear All,

How can I search for a read by part of its name in a big fastq.gz file (size around 13GB)? For example, I would like to search for a read name containing the VH01677:31:AACCMFHHV:1:1101:6586:25290 string in a fastq.gz file. For a small-size fastq.gz file, I just use the gunzip command in the Mac OS terminal to unzip the file, open it with the text editor, and use the "Ctrl + F" key to search the read. But for a big-size fastq.gz file, I do not want to do it this way as it is very inefficient.

Thanks,
Xiao

Answer 1

[rpolicastro]

Plain grep should work.

gzip -dc input.fastq.gz | grep -A3 'VH01677:31:AACCMFHHV:1:1101:6586:25290' | gzip > match.fastq.gz

You could try seqkit grep also if you want to use a more formal fastq parser.

seqkit grep -rp 'VH01677:31:AACCMFHHV:1:1101:6586:25290' -o match.fastq.gz input.fastq.gz

[xiaoleiusc]

Hi, rpolicastro,

Thank you very much! I tried seqkit grep and it works! It seems to me you changed the arguments from -Irp to just -rp.

Xiao

[rpolicastro]

I'm glad it worked! Whether or not you include -I as an argument you'll get the same results, so I decided to edit my post and remove it just to simplify the answer.

[xiaoleiusc]

I want to print the matched read on the screen, so I use the -Irp argument without the -o argument, and it works. Thanks again!

Xiao

Answer 2

[GenoMax]

You can also use filterbyname.sh from BBMap suite.

$ filterbyname.sh -Xmx2g in=file_R1_001.fastq.gz names=MXXXXX:469:000000000-XXXX7:1:1101:16837:2353 out=stdout.fq include=t

names= A list of strings or files.  The files can have one name per line, or be a standard read file 

You can also do substring matches, case-sensitive matches etc.Note that the read name being searched for should not have the beginning @.

Answer 3

[Ram]

Another option is seqtk subseq: https://github.com/lh3/seqtk

With a list of names in a file named name.lst, you can use the following command to extract the corresponding read(s):

seqtk subseq in.fq name.lst > out.fq