need help with bwa

Question

[Cony Lab]

I successfully used bwa to index chr19 using bwa index chr19.fa , but when I do index of hg38.fa, it dose not work. I have plenty of storage on my ubuntu cluster and I don't know why. Can anyone help me please? I'm new to this place.

gunzip hg38.fa.gz
ubuntu@VM-0-17-ubuntu:~/wes_cancer/data$ nohup bwa index hg38.fa &
~/wes_cancer/data$ bwa index hg38.fa
[bwa_index] Pack FASTA... Killed

I used Homo_sapiens_assembly38.fasta, but it still not working. I did successfully download the files from websites and bwa is fine, but don't know why it is not working?

bwa index Homo_sapiens_assembly38.fasta
[bwa_index] Pack FASTA... Killed

time bwa index -a bwtsw -p gatk_hg38 ~/wes_cancer/data/Homo_sapiens_assembly38.fasta
[bwa_index] Pack FASTA... Killed

real    0m12.877s
user    0m4.771s
sys 0m1.196s

[JC]

how much memory do you have? Seems like it is being killed because mem

[Cony Lab]

I have 177 G available, don't know why it is not working for the whole genome

df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            445M     0  445M   0% /dev
tmpfs            99M  948K   98M   1% /run
/dev/vda2       515G  318G  177G  65% /
tmpfs           491M   24K  491M   1% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           491M     0  491M   0% /sys/fs/cgroup

[GenoMax]

memory =/= disk storage space

How much RAM do you have?

[Cony Lab]

what do you mean, could you please tell me how to allocate more RAM space to do bwa?

[Michael]

type cat /proc/meminfo into your terminal and post the first lines of output, like so:

(base) michaeld@kjempefuru ~ $ cat /proc/meminfo
MemTotal:       2113425932 kB
MemFree:        502596012 kB
MemAvailable:   2043964440 kB

[Cony Lab]

[GenoMax]

Unfortunately no. 1004904 kb is just 1Gb of RAM. You should ask for at least 16G to be able to work comfortably.

[Cony Lab]

Thank you very much! Now I get it!

[Pierre Lindenbaum]

on my ubuntu cluster

are you working on the main/login node do you use the cluster ?

[Cony Lab]

I'm using my personal HPC account I purchased and I guess yes I'm using the main login node? Could you please help me? I'm confused how to allocate more space so that I can do bwa job?

Answer 1

[Istvan Albert]

What system is that? Are you sure you have "purchased and HPC"? That does not sound likely, people don't just go around "purchasing HPCs".

What all advice here tries to uncover is whether you have sufficient resources to run the tool. When you "purchased the HPC" what kinds of resources were you given? How many CPUs and how much memory. If it is indeed a purchase, these values are usually the first listed as the cost is directly proportional to resources.

Or link to the site where you "purchased the HPC"

[Cony Lab]

Thank you so much for replying me back. I'm really new to this area and used the term inappropriately. So I bought an account that is link to a ubuntu ip and it has 256 G memory I think. I have download files, and I run CHR 19 bwa sucessfully, but when it comes to the whole genome, it get killed. This really prevent me from doing the following steps.

[Istvan Albert]

I see above that you only have 1 GB of memory on that system. That is insufficient for mapping against Human sized genomes.

Get a system with more RAM, like GenoMax says, 16GB should work.

[Cony Lab]

Thank you very much! Now I get it!