Screen processes continue to run but can't find the original screen
1
1
Entering edit mode
3.0 years ago

I'm new to setting processes running via ssh on a server. To allow me to step away and return to them I'm trying to run them in a screen session.

# ssh onto server
ssh myid@server.co.uk

# Start screen session
screen -S myscreen

# Set the process running.
gatk CombineGVCFs \
   -R "/refs/hg38/gatk download/Homo_sapiens_assembly38.fasta" \
   --variant "/data/MSH3repeat_S1_L001_g.vcf.gz" \
   --variant "/data/20201028_CCDG_14151_B01_GRM_WGS_2020-08-05_chr5.recalibrated_variants.vcf.gz" \
   -O "/data/cohort.g.vcf.gz"

When I check the screens that are running immediately with screen -ls then I can see myscreen. However, when I switch off the computer and then return the next day I run screen -ls and it says no sockets.

I think it might be something to do with the fact when I set up a new screen with e.g. screen -S myscreen I always get this error:

  File "/usr/bin/pixstor-notify", line 64
    print f.read().strip()
          ^
SyntaxError: invalid syntax
(base) skgtmdf@rds-gw-007:~ $ 

But I don't know what it means!

However, the screen is clearly still there as the output files continue to accumulate in the output destination!!

screen linux • 2.7k views
ADD COMMENT
0
Entering edit mode

If you added the ampersand symbol (&) at the end of your command, you have set it to run in the background. That means it will run even after you log out, and will continue so until it is done, encounters the error, runs out of disk space, or the actual server is turned off. If you logged out from your original screen, however, your command most likely will not produce any more screen output.

What you are referring to as screen still being there is actually a process that is still active. You can check the running processes using top.

ADD REPLY
2
Entering edit mode

I don't see any mention of the & command. Also, commands run in the bg end on logout unless disowned.

ADD REPLY
0
Entering edit mode

partially correct indeed. Though absolutely not advised to use if you plan on logging out from your session. In that case screen is your best/only option.

Moreover the process will be killed as soon as it tries to print something to either STDERR or even STDOUT as those stream are not available anymore after log-out.

ADD REPLY
0
Entering edit mode

No '&' in the command I'm afraid.

I'm still experiencing the same problem. I set three screens going, detached from them. An hour later my vpn disconnected. When I reconnected and ssh back in, the screens have disappeared and screen -ls shows no sockets. However, the output files continue to accumulate in the output directory.

ADD REPLY
0
Entering edit mode

top shows processes running, but I don't think it's possible to see what the actual process is, and I don't think it helps me re-enter the screen??

enter image description here

ADD REPLY
0
Entering edit mode

did you detach your screen session before you logged out? (ctrl-a + d ) If not it will become a ghost process (which you will indeed see in the process list, but you likely can't access it anymore).

EDIT: usually you run screen session on a server (== some computer system that does not shut down) or your desktop in the lab (if you keep that one running)

ADD REPLY
0
Entering edit mode

Think Lieven is right, sounds like you arent detaching the screen session correctly. Should be no need for ampersands or disowning unless you want to continue to use the screen session and avoid having your CLI locked up.

ADD REPLY
0
Entering edit mode

Yes, I detached. Last night I set a long process going in a screen on my server, detached, then during the night my vpn dropped out. This morning I reconnected, and screen -ls shows no sockets. However, the output files continue to accumulate in the output directory.

ADD REPLY
0
Entering edit mode

I don't know what I did, but the ghost screen just reappeared when I ran screen -ls. I've done quite a lot in the meantime, but it includes creating other screen windows. Any idea what the problem is anyone?

ADD REPLY
0
Entering edit mode

Strange - long shot, but maybe you need to update or reinstall screen itself?

What happens if you do the following immediately after one another?:

  1. Start a screen
  2. Set a dummy process running in the screen (or even just leave the screen idle)
  3. Detach
  4. Exit the remote machine (quit SSH)
  5. Re-connect
  6. List your screen sessions?
ADD REPLY
4
Entering edit mode
3.0 years ago
Carambakaracho ★ 3.3k

Maybe a weird idea, but are you logging into the same node each time? A friend worked on a HPC-cluster with login-nodes being assigned to according to availability, so he was never guaranteed to login to the exact same machine.

Related to detaching "properly", I frequently get detached due to connection drops, my sessions stay attached, and get listed. Killing the terminal without detaching never affected my screen sessions, afaict.

ADD COMMENT
3
Entering edit mode

It turns out there are indeed 8 nodes on our server, and on ssh you join a random one. By ssh to a consistent node my screen sessions appeared. Thanks all

ADD REPLY

Login before adding your answer.

Traffic: 1644 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6