bad alloc with admixture32

Question

[Loïs]

Hello everyone,

I'm having a problem calculating delta K with the admixture32 pipeline on my RADseq data generated by STACKS.

Here is the command I use:

#!/bin/bash
#SBATCH -J mpi_job
#SBATCH --nodes=1
#SBATCH --time=80:00:00
#SBATCH --cpus-per-task=10
#SBATCH --mem=20G

for K in 4 5; \
do /usr/local/bioinfo/src/ADMIXTURE/admixture_linux-1.3.0/admixture32 --cv /home/groux/work/populations_48.bed $K -j1 | tee log${K}.out; done

and here is the log obtained :

****                   ADMIXTURE Version 1.3.0                  ****
****                    Copyright 2008-2015                     ****
****           David Alexander, Suyash Shringarpure,            ****
****                John  Novembre, Ken Lange                   ****
****                                                            ****
****                 Please cite our paper!                     ****
****   Information at www.genetics.ucla.edu/software/admixture  ****

Cross-validation will be performed.  Folds=5.
Parallel execution requested.  Will use 2 threads.
Random seed: 43
Point estimation method: Block relaxation algorithm
Convergence acceleration algorithm: QuasiNewton, 3 secant conditions
Point estimation will terminate when objective function delta < 0.0001
Estimation of standard errors disabled; will compute point estimates only.
Size of G: 133x5009874
Performing five EM steps to prime main algorithm
1 (EM)  Elapsed: 15.909 Loglikelihood: -7.33575e+07 (delta): 5.53733e+07
2 (EM)  Elapsed: 15.246 Loglikelihood: -6.91772e+07 (delta): 4.18036e+06
3 (EM)  Elapsed: 15.131 Loglikelihood: -6.74197e+07 (delta): 1.75748e+06
4 (EM)  Elapsed: 15.221 Loglikelihood: -6.62251e+07 (delta): 1.19455e+06
5 (EM)  Elapsed: 15.182 Loglikelihood: -6.53358e+07 (delta): 889323
Initial loglikelihood: -6.53358e+07
Starting main algorithm
1 (QN/Block)    Elapsed: 45.736 Loglikelihood: -6.24718e+07 (delta): 2.86401e+06
terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
****                   ADMIXTURE Version 1.3.0                  ****
****                    Copyright 2008-2015                     ****
****           David Alexander, Suyash Shringarpure,            ****
****                John  Novembre, Ken Lange                   ****
****                                                            ****
****                 Please cite our paper!                     ****
****   Information at www.genetics.ucla.edu/software/admixture  ****

Cross-validation will be performed.  Folds=5.
Parallel execution requested.  Will use 2 threads.
Random seed: 43
Point estimation method: Block relaxation algorithm
Convergence acceleration algorithm: QuasiNewton, 3 secant conditions
Point estimation will terminate when objective function delta < 0.0001
Estimation of standard errors disabled; will compute point estimates only.
Size of G: 133x5009874
Performing five EM steps to prime main algorithm
1 (EM)  Elapsed: 18.231 Loglikelihood: -7.30901e+07 (delta): 5.31587e+07
2 (EM)  Elapsed: 17.638 Loglikelihood: -6.8697e+07  (delta): 4.39313e+06
3 (EM)  Elapsed: 17.708 Loglikelihood: -6.68036e+07 (delta): 1.89339e+06
4 (EM)  Elapsed: 17.788 Loglikelihood: -6.54971e+07 (delta): 1.30655e+06
5 (EM)  Elapsed: 17.843 Loglikelihood: -6.4523e+07  (delta): 974028
Initial loglikelihood: -6.4523e+07
Starting main algorithm
terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc

Do you have any idea what the problem is and how to solve it?

Thank you and have a nice day

Loïs

Answer 1

[Vic]

This could be a memory problem, try giving the job more memory (this post or this post and this post ). Can you check your memory usage for the failed job? Looks like you're using the slurm workflow manager, try checking the job using this command:

sacct -j <job number here> --units=G --format JobID,MaxVMSize,MaxRSS,NodeList,AllocCPUS,TotalCPU,State,Start,End

[Loïs]

Hello,

Thank you for your feedback!

The recommended command gives me the following result:

JobID         MaxVMSize     MaxRSS        NodeList  AllocCPUS   TotalCPU      State               Start                 End 
------------ ---------- ---------- --------------- ---------- ---------- ---------- ------------------- ------------------- 
49320451                                   node041         32  08:13.157  COMPLETED 2023-06-29T20:39:23 2023-06-29T20:47:36 
49320451.ba+      0.14G      2.36G         node041         32  08:13.156  COMPLETED 2023-06-29T20:39:23 2023-06-29T20:47:36 
49320451.ex+      0.10G          0         node041         32  00:00.001  COMPLETED 2023-06-29T20:39:23 2023-06-29T20:47:36 

I've made several attempts by increasing the number of cpus but I always get the same error.

Best,

[GenoMax]

It is not the CPU that is the problem. It is memory.

#SBATCH --cpus-per-task=10
#SBATCH --mem=20G

If you are using 10 cores then you probably need more memory, try

#SBATCH --cpus-per-task=10
#SBATCH --mem=60G

see if that helps.

If 20G is all you can assign then reduce the number of CPU's to 2.

[Loïs]

Hello,

I really don't understand, I've tried the recommended settings and I get the same error. I even tried with mem=300G and Cpu=5, I still get the same error...

[Vic]

Thats ok, the memory problem is the first issue to check, which you seem to have ruled out. The other issue could be to do with how you input file was generated? You said STACKS, but could you be more specific, how did you produce the .bed file? From a vcf? Additionallly, programs like ADMIXTURE need to have unlinked sites, so did you LD prune your sites? Have you tried regenerating the file (for example, just the last step or so)?