I'm confused as to why the two bowtie lines are running at the same time? I was under the impression that everything in unix is run top to bottom and the second bowtie line wouldn't start running until the first is completed. The reason I ask is these commands are running simultaneously and I'm having memory issues of sorting two large bam files at the same time.. I'm assuming it has something to do with the piping but I am confused. Script below
(I have changed the sample names per privacy)
!/bin/bash
echo BOWTIE LOG > ../BAM/bowtie_log.txt
mkdir ../BAM/
bowtie2 --very-sensitive -x /PATH/TO/genome -X 2000 2>> ../BAM/bowtie_log.txt -p 1 -1 trimmed/Sample1_R1_001_val_1.fq.gz -2 trimmed/Sample1_R2_001_val_2.fq.gz | samtools view -Sb - | samtools sort - -o ../BAM/Sample1.bam
bowtie2 --very-sensitive -x /PATH/TO/genome -X 2000 2>> ../BAM/bowtie_log.txt -p 1 -1 trimmed/Sample2_R1_001_val_1.fq.gz -2 trimmed/Sample2_R2_001_val_2.fq.gz | samtools view -Sb - | samtools sort - -o ../BAM/Sample2.bam
(Edited my original comment based on @Jean-Karim's comment and some other searches): It may be the pipes in the command that are backgrounding something and allowing both jobs to run. There are no logical/control operators.
Adding
{ }
around the entire command (as suggested by @steve below) may prevent that backgrounding from happening (if it works, my assumption).(see this, specifically A.2 logical operators).
Wow learn something new about bash everyday. Thanks for the input
I don't follow this. This has to be more subtle than that. The newline is a terminator and the shell should wait for the command to complete before continuing on the next one.
I think the OP does not have a new line at all and the text wraps around and shows as if there were a newline.
I added a comment to Jean's answer? Does this bg have this effect?
does the same thing happen if you write it as
{ bowtie2 .... | samtools ... ; } && { bowtie2 .... | samtools ... ; }
?What is making you think that both are running concurrently?
Reason I thought that was because Sample2 tmp merge files were being made before Sample1 was merged and got a memory error