delete IDs from a list file

Question

[Cindy]

I am using gsnap for alignment. I have several hundreds of paired fastq files, So I use ls to put them in a sampleID then align one by one.

sampleID=$(ls *fastq | awk 'BEGIN{FS="."}{count[$1]++}END{for(j in count) print j}')
for i in $sampleID; do...

However, after one week running, my computer crushed. So I need to removed all IDs that have been aligned from the sampleID. I tried to output sampleID to sampleID.ls and manually removed finished ID, and use "for i in $sampleID.ls; do.."., but it does not work.

Anyone could help? Appreciate ahead.

Answer 1

[Pierre Lindenbaum]

use a Makefile to only compute the remaining things. Something like

sampleIDs=$(shell ls *.fastq | cut -d '.' -f 1 | sort | uniq)
define run
$(1).txt: $(1).R1.fq  $(1).R2.fq 
    echo $$^ > $$@
endef

all: $(addsuffix .txt,${sampleIDs})
$(eval $(foreach S,${sampleIDs},$(call run,$S)))

and

make