Abnormalties regarding the use of HPC02 (a CPU server from the school) server for R programming

Question

[applepie]

Recently I have been using the HPC02 for the R programming, but it turns out that I could not successfully log in the R console. The webpage kept poppint out the message " is taking longer to start than usual". Even if I tried to press the "terminate" button to abort the R session. What's happening? Can this problem be resolved? Can I use linux to terminate the R session that is currently running? Thank you so much.

[ATpoint]

Please consult with the IT support or the person maintaining this resource locally. This is not something one can solve from remote. Maybe the Docker (or whatever container or RStudio server instance that is) needs a reboot.

[LauferVA]

unlikely to be the solution to OPs problem

Answer 1

[LauferVA]

What's happening?

The most likely scenario here is that you are trying to load a huge .Rdata file and the interactive session is timing out before the load is complete.

Can this problem be resolved?

No problem is insoluble in all conceivable circumstances. Here, your best bet here is to check the size / amount of the data you are loading in the background (i.e., what .Rdata image file is being loaded at the outset of your instance being created?). In particular, if you kill your current instance, and make a new instance, and this still fails to load, then this becomes by far the most likely option. Additionally, how large is that file in relation to the total amount of RAM you reserved for your instance of Rstudio?

At any rate, the simplest solution is to find the .Rdata image file on the HPC that is being loaded when you Rstudio interactive session kicks off. If it is huge, move the file (don't delete it unless you don't need it) and leave the rest of the folder structure intact. Then you can restart your Rstudio session.

Can I use linux to terminate the R session that is currently running?

Yes, following the syntax of the scheduler used by the HPC to schedule the jobs. For instance, if it is SLURM, you'd write scancel ${job_id}; if sungrid, qdel ${JobId}, etc.

---- you didn't ask about this, but arguably most important comment ----

Finally, I think this stratification scheme will help you:

1. Use your HPCs Rstudio interactive functionality for:

   • visualization
   • getting your initial scripts working, then kill the command and run it via your scheduler on the command line of your HPC
   • Tasks that are part of work flows, but finish in seconds to minutes and won't require you to store huge amounts of info. in RAM.

2. By contrast, run time/memory intensive tasks on the scheduler. Doing this will cut the failure rate by > 1 OoM (or more depending on specs), and will eliminate the load problems you are having because you won't be trying to load a huge .Rdata image into your interactive session.

VAL