How to execute curl with dynamically created URL using Bash

Question

[nadarajan_v]

In bash, I have created a function. In this function, I am constructing an URL that will be used to download a selected chromosome, unzip it and save it as a fasta file using curl. The following is my code.

 #!/bin/bash
function getSequence {
#echo $1
URL="http://hgdownload.cse.ucsc.edu/goldenPath/hg38/chromosomes/chr" 
UNZIP=".fa.gz | gunzip -c > chr"
FE=".fa"
SPACE=" "
FINAL="$SPACE$URL$1$UNZIP$1$FE"

curl "$FINAL"

}

echo "Enter a number between 1-22 or the letters X or Y"
read chromosome
regex='^[0-9]+$'
if [[ $chromosome =~ $regex ]]
then
if [ $chromosome -gt 0 -a $chromosome -lt 23 ]
    then
        #echo "$chromosome is numeric"
        getSequence $chromosome
    else
        echo "Your input is invalid!"
fi
else
if [ $chromosome = "y" ] || [ $chromosome = "Y" ]
then 
    echo "You entered $chromosome"
elif [ $chromosome = "x" ] || [ $chromosome = "X" ]
then 
    echo "You entered $chromosome"
else
    echo "Your input is invalid!"
fi
fi

echo "$FINAL", gives http://hgdownload.cse.ucsc.edu/goldenPath/hg38/chromosomes/chr20.fa.gz | gunzip -c > chr20.fa If I copy and paste on the terminal it works fine. However, the line curl "$FINAL" in the function causes this error: curl: (3) URL using bad/illegal format or missing URL

Thanks

Answer 1

[Pierre Lindenbaum]

you are inserting a useless space in front of the url.

SPACE=" "
FINAL="$SPACE$URL$1$UNZIP$1$FE"

that said, Shell programs requiring such user input :

echo "Enter a number between 1-22 or the letters X or Y"
read chromosome

are evil. Please don't. Use a command line parser (getpopt).

[nadarajan_v]

Yes, I will be using getopt later. If I remove the $SPACE, I get the following warning and the .fa file is not created.

Warning: Binary output can mess up your terminal. Use "--output -" to tell
Warning: curl to output it to your terminal anyway, or consider "--output
Warning: <FILE>" to save to a file.

Thanks

[Pierre Lindenbaum]

curl --output -  "${URL}"  | gunzip -c > chr

[nadarajan_v]

This worked great. Thank you!

Answer 2

[zorbax]

You can simplify your function as Pierre suggests:

getSequence() {
    local nth=$1
    local URL="http://hgdownload.cse.ucsc.edu/goldenPath/hg38/chromosomes/chr${nth}.fa.gz"
    curl -sS -o - "${URL}" | gunzip -c > chr"${nth}".fa
}

Otherwise, you will get an error something like: curl: (6) Could not resolve host: "http because of split the URL.

The option -S when is used with -s, --silent, it makes curl show an error message if it fails.

[nadarajan_v]

Thank you so much. It worked perfectly!