Error running multiple blast on multiple fasta queries

Question

[er.doug.ragnar]

I'm using the following code to run blasts on a server, of several fasta queries against a database; I already used blasttools to create the database. This is the code:

#!/bin/bash


#Script to run entire blast analysis at once

display_usage() {
echo '
1st argument is the list of query sequences (full path) in a file
2nd argument is the directory containing the database files to use (full path)
3rd argument is the number of threads available to use. example "15"
4th argument is the directory were results must be saved (full path)
'
}


#check if required arguments are there and display usage message
if [ -z "$1" ] || [ -z "$2" ] || [ -z "$3" ] || [ -z "$4" ] ; then
    printf "Please provide the arguments required for the script.\n\n"
    display_usage
    exit 1
fi

echo ""
date
echo ""


#variables 
query_list="$1"
db_DIR="$2"
threads="$3"
output_dir="$4"

user=$USER
data=$(date +"%d-%m-%y-%Hh-%Mmin")

DIR_name="$user-$data"

WD=/dev/shm/"$DIR_name"

mkdir $WD
mkdir $WD'/queries'
mkdir $WD'/db'
mkdir $WD'/tmp_results'

cat $query_list | while read q; do cp $q $WD'/queries'; done
cp $db_DIR $WD'/db'

cd $WD'/queries'

ls | while read query; do for db in `ls $WD'/db/'*.faa`; do query_size=$(ls -l $query | cut -d " " -f5); db_name=$(echo $db | rev | cut -d "/" -f1 | rev | cut -d "." -f1); echo -e "A processar a amostra \e[1m\e[33m$query \e[0mcontra a base de dados \e[1m\e[35m$db_name \e[0m" ; cat $query | parallel --will-cite --pipe --block $[$query_size/($threads*5)] -j $threads --recstart '>' blastp -query - -db $WD/db/$db_name.faa -outfmt '6' -evalue .0000000000000001 >> $WD'/tmp_results/'$query.$db_name.eval.tab.out; done; cp $WD'/tmp_results/'$query* $output_dir ;done

echo ""
echo "A eliminar ficheiros temporários na RAM"
echo ""

rm -r $WD

echo "Blast finalizado"

echo ""
date
echo ""

But it keeps giving me the following error for each query file:

ls: cannot access '/dev/shm/hdtr-11-02-20-08h-54min/db/*.faa': No such file or directory
cp: cannot stat '/dev/shm/hdtr-11-02-20-08h-54min/tmp_results/Aeromonas_sp__ASNIH4_GCF_2934505_1_ASM293450v1protein.faa*': No such file or directory
ls: cannot access '/dev/shm/hdarmancier-11-02-20-08h-54min/db/*.faa': No such file or directory

[lieven.sterck]

A small educational note: I added (code) markup to your post for increased readability. You can do this by selecting the text and clicking the 101010 button. When you compose or edit a post that button is in your toolbar, see image below:

[Joe]

Are all those file paths definitely correct?

[er.doug.ragnar]

I've checked the paths 3 times and didn't see any error.

[lieven.sterck]

are you sure (and thus can you check) if you have your file paths correct?

as it says, it can't find any of the files you are trying to manipulate

[er.doug.ragnar]

I've checked the paths 3 times and didn't see any error.

Answer 1

[GenoMax]

/dev/shm (tempfs) refers to shared memory.

If your files are on disk then the part that is moving the files to /dev/shm is not working. Perhaps you are running out of memory on your system.

I suggest you change

WD=/dev/shm/"$DIR_name"

and set that to a real storage path.

[er.doug.ragnar]

It was the memory. Thank you, that solved it.

[Ram]

If an answer was helpful, you should upvote it; if the answer resolved your question, you should mark it as accepted. You can accept more than one answer if they all work.