HOw to modify the description line 1 of a fastaq file to match with third line?

Question

[majeedaasim]

I am using paired end SRA data, it looks like this

@HWI-ST915_0064:2:1101:1420:2104/1
GTCTCTTCGCACGCTTTCACTGTGAACGGTTCGGCATCGAGAAGGACGCAGTTCCTCTCCGGCTTGGACCAGTTTCTGGTGGCCACGGCTGCCCCCATCC
+SRR1188607.1 HWI-ST915_0064:2:1101:1420:2104 length=100
HDHHHHHHHHHHHHHHHHHHHHHHGHHHHHHHHHHHHHHFHFFHHHHHHHHEHEHHHHHHHGHHHED?EE=A@BACDDECCE@DB74?############

@HWI-ST915_0064:2:1101:1498:2108/1
AAAGATTGCAATGGAGGAGAAAGGGAAGACCCTGCCTGAAGAAATGCAATTGATAAATAAGTTGTTGTCTGAGGAAAAGGGTTCGGAGAGGATGAGAATG
+SRR1188607.2 HWI-ST915_0064:2:1101:1498:2108 length=100
GFHHHHHHHHHHHHHHHHHHHHHGHGHHHHHHHHHHGHHHHGHHHHHHDHGGHHFHHHHHGEGGFGGFGFHFHHDHHGHHGFGFGHGFHHHHHHHHHHHH

Othe read file is similar except with /2 at the end

For further analysis I need to match the description in line one with line three after +, but in line three there is extra information as SRR1188607. 1, SRR1188607.2 etc how to get rid of this so that it matches with line one description. Also how to delete everything after +

Thanks

[Pierre Lindenbaum]

I wonder why you need to do this... most programs ignore the 3rd line...

[WouterDeCoster]

For example, the biopython parser throws an error if the third line is present and not equal to the first line. AFAIK per convention the third line has to be equal to the first, or absent.

Answer 1

[Pierre Lindenbaum]

awk '(NR%4==3) {i=index($0," ");printf("+%s\n",substr($0,i+1));next;}{print}' input.fastq

Answer 2

[lakhujanivijay]

Pattern to exclude is

SRR[some_digits][a dot][another digit][a space]

---

sed combined with regex

sed 's/SRR[0-9]*.[0-9] //g'

Input

[Desktop]$ cat test.fq 
@HWI-ST915_0064:2:1101:1420:2104/1
GTCTCTTCGCACGCTTTCACTGTGAACGGTTCGGCATCGAGAAGGACGCAGTTCCTCTCCGGCTTGGACCAGTTTCTGGTGGCCACGGCTGCCCCCATCC
+SRR1188607.1 HWI-ST915_0064:2:1101:1420:2104 length=100
HDHHHHHHHHHHHHHHHHHHHHHHGHHHHHHHHHHHHHHFHFFHHHHHHHHEHEHHHHHHHGHHHED?EE=A@BACDDECCE@DB74?############

Output

[Desktop]$  sed 's/SRR[0-9]*.[0-9] //g' test.fq
@HWI-ST915_0064:2:1101:1420:2104/1
GTCTCTTCGCACGCTTTCACTGTGAACGGTTCGGCATCGAGAAGGACGCAGTTCCTCTCCGGCTTGGACCAGTTTCTGGTGGCCACGGCTGCCCCCATCC
+HWI-ST915_0064:2:1101:1420:2104 length=100
HDHHHHHHHHHHHHHHHHHHHHHHGHHHHHHHHHHHHHHFHFFHHHHHHHHEHEHHHHHHHGHHHED?EE=A@BACDDECCE@DB74?############

[GenoMax]

This would still lead to line 1 header not matching line 3. So as @Wouter said above will cause problems with some programs.

[majeedaasim]

I agree and indeed the "length=100" causes the problem so doesn't match.

[lakhujanivijay]

ah! not a big task to remove that as well! sed once again my friend; adding sed 's/length=[0-9]*//' :)

sed 's/SRR[0-9]*.[0-9] //g' test.fq | sed 's/length=[0-9]*//'

---

[Desktop]$ sed 's/SRR[0-9]*.[0-9] //g' test.fq | sed 's/length=[0-9]*//'
@HWI-ST915_0064:2:1101:1420:2104/1
GTCTCTTCGCACGCTTTCACTGTGAACGGTTCGGCATCGAGAAGGACGCAGTTCCTCTCCGGCTTGGACCAGTTTCTGGTGGCCACGGCTGCCCCCATCC
+HWI-ST915_0064:2:1101:1420:2104 
HDHHHHHHHHHHHHHHHHHHHHHHGHHHHHHHHHHHHHHFHFFHHHHHHHHEHEHHHHHHHGHHHED?EE=A@BACDDECCE@DB74?############

Is that /1 an issue? If yes, let me know.

[GenoMax]

Yes it will be. Line 1 and 3 need to match or line 3 need to be blank (past +) for many tools.

Answer 3

[Tm]

To remove anything after '+' from 3rd line, you can use simple sed one liner:

sed "s/+SRR1188607.*/+/g" in_file.fastq >out_file.fastq

[GenoMax]

To properly format code you should select the text you want to format first and then click on the '101` button you see in the editor window.

[majeedaasim]

thanks toralmanvar it works But what if the SRR... is present in the the beginning of the first line as

SRR1188607.1@HWI-ST915_0064:2:1101:1498:2108/1

And I want to get rid of these SRR IDs to begin the description line with @HWI- onwards

[GenoMax]

Dump the reads with -F option.

fastq-dump -F --split-files SRR1188607

and then add 1: and 2: to the read headers by using reformat.sh from BBMap suite.

 reformat.sh in1=SRR1188607_1.fastq in2=SRR1188607_2.fastq out1=R1.fq out2=R2.fq addcolon=t

[Tm]

To replace SRR1188607.1 from the begining, you can use:

sed "s/SRR1188607.1@HWI-ST915/@HWI-ST915/g" in_file.fastq >out_file.fastq

Here SRR1188607.1@HWI-ST915 part will be substituted by @HWI-ST915 in your first header line.

[majeedaasim]

It tried it, but the description was same and there was no impact.

Also one point I shall mention is that the change shall take place in all reads and not in a single read as the code you mentioned reflects.

Thanks

[GenoMax]

Use the method I had posted above.

Answer 4

[cpad0112]

$ sed 's/^\(\+\).*\(HWI.*\)/\1\2/g' test.fq

[GenoMax]

This also leads to line 1 header not matching line 3. So as @Wouter said above will cause problems with some programs.

[cpad0112]

output (for other read file replace /1 with /2):

$ sed 's/^\(\+\).*\(HWI.*\)\s.*/\1\2\/1/g' test.fq

or

 $ sed 's/\bSRR.\{9\} \b//;s/\b len.*\b/\/1/' test.fq 
@HWI-ST915_0064:2:1101:1420:2104/1
GTCTCTTCGCACGCTTTCACTGTGAACGGTTCGGCATCGAGAAGGACGCAGTTCCTCTCCGGCTTGGACCAGTTTCTGGTGGCCACGGCTGCCCCCATCC
+HWI-ST915_0064:2:1101:1420:2104/1
HDHHHHHHHHHHHHHHHHHHHHHHGHHHHHHHHHHHHHHFHFFHHHHHHHHEHEHHHHHHHGHHHED?EE=A@BACDDECCE@DB74?############

@HWI-ST915_0064:2:1101:1498:2108/1
AAAGATTGCAATGGAGGAGAAAGGGAAGACCCTGCCTGAAGAAATGCAATTGATAAATAAGTTGTTGTCTGAGGAAAAGGGTTCGGAGAGGATGAGAATG
+HWI-ST915_0064:2:1101:1498:2108/1
GFHHHHHHHHHHHHHHHHHHHHHGHGHHHHHHHHHHGHHHHGHHHHHHDHGGHHFHHHHHGEGGFGGFGFHFHHDHHGHHGFGFGHGFHHHHHHHHHHHH

input:

$ cat test.fq 
@HWI-ST915_0064:2:1101:1420:2104/1
GTCTCTTCGCACGCTTTCACTGTGAACGGTTCGGCATCGAGAAGGACGCAGTTCCTCTCCGGCTTGGACCAGTTTCTGGTGGCCACGGCTGCCCCCATCC
+SRR1188607.1 HWI-ST915_0064:2:1101:1420:2104 length=100
HDHHHHHHHHHHHHHHHHHHHHHHGHHHHHHHHHHHHHHFHFFHHHHHHHHEHEHHHHHHHGHHHED?EE=A@BACDDECCE@DB74?############

@HWI-ST915_0064:2:1101:1498:2108/1
AAAGATTGCAATGGAGGAGAAAGGGAAGACCCTGCCTGAAGAAATGCAATTGATAAATAAGTTGTTGTCTGAGGAAAAGGGTTCGGAGAGGATGAGAATG
+SRR1188607.2 HWI-ST915_0064:2:1101:1498:2108 length=100
GFHHHHHHHHHHHHHHHHHHHHHGHGHHHHHHHHHHGHHHHGHHHHHHDHGGHHFHHHHHGEGGFGGFGFHFHHDHHGHHGFGFGHGFHHHHHHHHHHHH