Question about CIBERSORTx Docker to Singularity

Question

[jinghuang800]

Hello,

I'm trying to run gene expression imputation analysis with CIBERSORTx docker. The university shell only support singularity. Therefore I need to run CIBEROSRTx Docker with Singularity.

Here is what I did:

# pull docker image into singularity    
singularity pull docker://cibersortx/hires

# get "hires_latest.sif" image file.    

# run singularity 
singularity run -B /my_path/hires_input:/src/data -B /my_path/hires_output:/src/outdir \
hires_latest.sif --username email \
--token my_token \
--mixture /src/data/Fig4a-arrays-SimulatedMixtures.MAS5.txt \
--sigmatrix /src/data/Fig4a-LM4.txt \
--window 20 \
--QN TRUE

When I try to run the analysis with the data in tutorial, I get the error:

FATAL:   stat /my_path/CIBERSORTxHiRes: no such file or directory

Does anyone know how to resolve this? Thanks for your time.

The commend for docker would be this:

docker run -v absolute/path/to/input/dir:/src/data -v absolute/path/to/output/dir:/src/outdir cibersortx/hires --username email_address_registered_on_CIBERSORTx_website --token token_obtained_from_CIBERSORTx_website --mixture Fig4a-arrays-SimulatedMixtures.MAS5.txt --sigmatrix Fig4a-LM4.txt --classes Fig4a-LM4-mergedclasses.txt --window 20 --QN TRUE

[Ram]

Did you change the bind paths while pasting the command here? The error message does not match your command.

[jinghuang800]

I stored the "hires_latest.sif" image file in /xdisks/PI_name/my_name/cibersortx/

I sbatch the job to Slurm to run it, and the commend in my job script is this:

cd /xdisk/PI_name/my_name/cibersortx/

singularity run -B /xdisk/PI_name/my_name/cibersortx/hires_input:/src/data -B /xdisk/PI_name/my_name/cibersortx/hires_output:/src/outdir   hires_latest.sif
 --username email --token My_token --mixture /src/data/Fig4a-arrays-SimulatedMixtures.MAS5.txt --sigmatrix /src/data/Fig4a-LM4.txt --window 20 --QN TRUE

The slurm.out I got just contains "FATAL stat /xdisk/PI_name/my_name/cibersortx/CIBERSORTxHiRes: no such file or directory"

Which path should I change? I'm new for both containers.

[GenoMax]

Is that location accessible on nodes where this job is running?

[Raphael]

Hello,

I'm getting the same error as you trying to run a fraction analysis.

My command is

$ singularity run -B /home/rmoreira:/src/data -B /home/rmoreira/Results:/src/outdir /cibersortx/fractions.sif --username // --token // --single_cell TRUE --refsample ReferenceFile2.txt --mixture dataRNAseq.txt --fraction 0 --rmbatchSmode TRUE 

The error

FATAL: While checking container encryption: could not open image /cibersortx/fractions.sif: failed to retrieve path for /cibersortx/fractions.sif: lstat /cibersortx: no such file or directory

Could you help me?

Answer 1

[jinghuang800]

I got the result. I should add /src/CIBERSORTxHiRes after hires_latest.sif.

[Ram]

Can you explain why that worked?

[Jose]

You point singularity to the container but then also the executable (if you check the hires image it contains both hires and fractions within it as it needs the executables to provide adjusted fractions if you use batch correction).

Without it you've said singularity run <hires_latest.sif> but given it nothing to execute as there's a few options inside: CIBERSORTxFractions, CIBERSORTxHiRes, install_R_dependencies.R, R_modules

[GS_BioInfo]

Hello, I'm running into the same error and your fix is not working for me. To clarify your command was as follow ?

# run singularity
singularity run -B /my_path/hires_input:/src/data -B /my_path/hires_output:/src/outdir \ hires_latest.sif /src/CIBERSORTxHiRes \ --username email \ --token my_token \ --mixture /src/data/Fig4a-arrays-SimulatedMixtures.MAS5.txt \ --sigmatrix /src/data/Fig4a-LM4.txt \ --window 20 \ --QN TRUE

I am also wondering about run vs exec, because with singularity run I get this error but with singularity exec no (I get another error down the line) does anyone know why ?

[Jose]

run is for when the container has a runscript (default workflow or tool). I don't think the HiRes container has a runscript in its metadata so run can't work. exec on the other hand allows you to execute something inside the container which is why you need to provide

the container

hires_latest.sif 

and the executable

/src/CIBERSORTxHiRes

Answer 2

[Jose]

I figured out how to get the previous suggestion of adding /src/CIBERSORTxHiRes after the .sif (.simg) to work. It runs pretty quickly if you use --threads

Running this successfully pulls documentation

singularity exec CIBERSORTxHiRes.simg /src/CIBERSORTxHiRes

If you run

singularity shell CIBERSORTxHiRes.simg

And then within the container you run this to find the executables

ls -lh /src/

So just provide the same executable path within the command (you can add full paths also if needed)

singularity exec \
-B /scratch/.../cibersortx:/src/data \               
-B /scratch/.../cibersortx/output/2kHVG:/src/outdir \ 
/scratch/.../CIBERSORTxHiRes.simg \   #container image
/src/CIBERSORTxHiRes \                #executable within the container
--username <email>  \                     
--token <token> \           
--mixture tpm_results_filtered.txt \                 
--sigmatrix signature_matrix_file.txt \              
--rmbatchSmode TRUE \                                
--threads 12 \                                       
--subsetgenes tpm_variable_genes_2k.txt \            
--refsample signature_matrix_file.txt \