Print based on character strings

Question

[selplat21]

I have a series of fastq files in different folders like so:

Historic_WestFork/80_S74_L002_R1_001.fastq.gz
Historic_WestFork/80_S74_L002_R2_001.fastq.gz
Historic_WestFork/82_S75_L002_R1_001.fastq.gz
Historic_WestFork/82_S75_L002_R2_001.fastq.gz
Historic_WestFork/83_S76_L002_R1_001.fastq.gz
Historic_WestFork/83_S76_L002_R2_001.fastq.gz
Historic_WestFork/84_S77_L002_R1_001.fastq.gz
Historic_WestFork/84_S77_L002_R2_001.fastq.gz
Historic_WestFork/85_S78_L002_R1_001.fastq.gz
Historic_WestFork/85_S78_L002_R2_001.fastq.gz
Historic_WestFork/86_S79_L002_R1_001.fastq.gz
Historic_WestFork/86_S79_L002_R2_001.fastq.gz
Historic_WestFork/88_S80_L002_R1_001.fastq.gz
Historic_WestFork/88_S80_L002_R2_001.fastq.gz
Historic_WestFork/90_S81_L002_R1_001.fastq.gz
Historic_WestFork/90_S81_L002_R2_001.fastq.gz
Historic_WestFork/91_S82_L002_R1_001.fastq.gz
Historic_WestFork/91_S82_L002_R2_001.fastq.gz
Historic_WestFork2/80_S70_L002_R1_001.fastq.gz
Historic_WestFork2/80_S70_L002_R2_001.fastq.gz
Historic_WestFork2/81_S71_L002_R1_001.fastq.gz
Historic_WestFork2/81_S71_L002_R2_001.fastq.gz
Historic_WestFork2/82_S72_L002_R1_001.fastq.gz
Historic_WestFork2/82_S72_L002_R2_001.fastq.gz
Historic_WestFork2/83_S73_L002_R1_001.fastq.gz
Historic_WestFork2/83_S73_L002_R2_001.fastq.gz
Historic_WestFork2/84_S74_L002_R1_001.fastq.gz
Historic_WestFork2/84_S74_L002_R2_001.fastq.gz
Historic_WestFork2/88_S75_L002_R1_001.fastq.gz
Historic_WestFork2/88_S75_L002_R2_001.fastq.gz

Each unique sample is denoted by the number (e.g 88). Only 5 of the 10 samples have two R1 files and two R2 files. I need to merge the R1 files and the R2 files for each sample and want to write a loop to echo each command to merge files.

For each sample, I want to echo for example "cat Historic_WestFork/88_S80_L002_R1_001.fastq.gz Historic_WestFork2/88_S75_L002_R1_001.fastq.gz > 88_merged_R1.fastq.gz" (used 88 for example here). And then the same for the R2 files.

Any help would be appreciated for R or command line.

Thank you.

[Ram]

Your example doesn't match your data. Do you wish to replace the _S*_L00* part with _merged_? Please show a proper example.

[selplat21]

Edited, Yes!

[Ram]

This is a fun exercise but if you only have to do this 10 times, you're better off doing it manually. Figuring out a loop is going to take longer than doing it manually.

[selplat21]

Thank you so much!!

[GenoMax]

If an answer was helpful, you should upvote it; if the answer resolved your question, you should mark it as accepted. You can accept more than one if they work.

Answer 1

[Ram]

If the above is all you have, this should work:

for r1fq in $(grep "Historic_WestFork/.+_R1_" list_of_fastq_files.txt)
do
    smpl=$(basename $r1fq) #remove dir name
    smpl=${smpl%%_*} #trim everything from end up to first underscore
    r2f1=${r1fq/_R1_/_R2_} #this is R2 fastq

    echo "cat $r1fq Historic_WestFork2/$smpl*_R1_*.fastq.gz > ${smpl}_merged_R1.fastq.gz"
    echo "cat $r1fq Historic_WestFork2/$smpl*_R2_*.fastq.gz > ${smpl}_merged_R2.fastq.gz"
    ## Remove the `echo` to do the actual operation
done

This won't work if you have multiple folders or even a difference in the folder names. You might need to use a complicated find operation in those cases. It would just be easier to create soft-links to all files manually in a separate folder, then use grobs within that folder to get things done.