.slurm vs .sh files

Question

[Sammy]

Hello. I am writing my first ever script.

Is there any difference between somefile.slurm and somefile.sh? Are both terminologies human convention? I know that you can use them both to submit jobs but I don't understand the practical differences between them (if there is any).

Thanks.

[GenoMax]

Using somefile_slurm will make it explicit that it is a SLURM script rather than a pure shell script, without having to open the file to see what is in it.

Note: I have not tested to see if SLURM needs the .sh extension when doing sbatch somefile_SLURM.sh or if it will happy with sbatch somefile_SLURM, if you want to follow @Ram's suggestion.

[Ram]

Would sbatch <(head somefile.txt) work?

[GenoMax]

Update: sbatch is happy with just a file name. .sh is not required.

[Sammy]

Thanks. So I have different types of shells: C shell (#!/bin/csh); BASH (#!/bin/bash); Korn shell (#!/bin/ksh) etc... But BASH does not need a suffix like .sh or .csh. (Should it be .bash?)

You use SLURM to interact with HPC to run your script (written in BASH or Bourne Shell (.sh)). You can mark that is a SLURM script also (human convention) if you want to. Is this right?

Maybe is a naive question but how do I choose what kind of shell to use? On my personal computer I have bash by default, but also sh, rbash and a few others are installed.

For my script to work I have to make sure that the shell type is installed?

[Ram]

Please do not add an answer unless you're answering the top-level question. Use Add Comment and Add Reply instead.

Suffixes are not needed for any type of file. It's for human convenience and other conventions (such as Windows and macOS automatic program associations). Imagine having to remember what type each file is and choosing a program to open it with every time - that is the pain avoided by having automatic program associations based on filename extensions/suffixes.

My statement is not to be interpreted as "each shell gets it own file extension". It is to be interpreted as "there is a chance for confusion if sh is both a shell and a file extension that is not specific to the sh shell." This only matters because sh only has a subset of bash-like features.

You use SLURM to interact with HPC to run your script (written in BASH or Bourne Shell (.sh)). You can mark that is a SLURM script also (human convention) if you want to. Is this right?

That is correct.

Maybe is a naive question but how do I choose what kind of shell to use? On my personal computer I have bash by default, but also sh, rbash and a few others are installed.

This is a matter of personal preference. Most of us use bash. I used bash for years before switching to zsh a few years ago. A lot of smaller HPC systems don't have zsh, so bash might be the safer bet.

For my script to work I have to make sure that the shell type is installed?

Not if you stick to bash - that is almost universally available. You're just starting out, so keep an eye out for when you stumble into shell-specific features.

Answer 1

[Ram]

Broadly speaking, file names and extensions are more for human understanding than anything to do with software. .sh is a common extension for shell executables, and .slurm seems to indicate a slurm-specific script.

Personal opinion: I'd recommend not using .sh for shell executables, since sh is a specific type of shell. .sh does not unambiguously convey the shell that should execute the script, so it's better to name executables without extensions and specify the shell on the first line.

#!/bin/bash
diff <(cut -f2 file1 | sort -u) <(cut -f3 file2 | sort) #sample command

I cannot run the above script using sh scriptName, but I can run it using bash scriptName. Hence, naming the file scriptName.sh might lead to some confusion.

[Mensur Dlakic]

I cannot run the above script using sh scriptName, but I can run it using bash scriptName. Hence, naming the file scriptName.sh might lead to some confusion.

You are way more pedantic than I am, which is certainly not a bad thing. All my .sh files have the shell defined one the first line, like in your example. As long as they are executable, one can run them with scriptName without knowing the shell, so to me there is nothing confusing. I don't use bash anyway :)

By the way, slurm doesn't care at all about file extension, and neither do Unix/Linux shells. All my slurm jobs end in .job.

[Ram]

As long as the shell is mentioned in the first line, the script can be run as long as it's not run as $1 to a shell executable. That is the confusion I wish to avoid. Being pedantic saves time while cleaning up scripts and managing data :-)