Darius Knowledge Hub

Running python script jupyter even when close

2 min read

Running python script / jupyter even when closed ssh (nohup)

Created: 21 Nov 2022, 05:27 PM | Modified: =dateformat(this.file.mtime,"dd MMM yyyy, hh:mm a") Tags: knowledge, unix

nohup python3 cifar10_classifier.py &

https://stackoverflow.com/a/55462300

Write shell script as per https://stackoverflow.com/a/17389526

nohup my_command > my.log 2>&1 &

echo $! > save_pid.txt

kill -9 `cat save_pid.txt`

rm save_pid.txt

Use the following to view the logging in real time

tail -f my.log

nohup python3 -u cifar10_classifier.py > ../results/nohup.out 2>&1 &

for python, add -u to prevent output buffering python - Nohup is not writing log to output file - Stack Overflow

Check running processes

top -U darius

https://unix.stackexchange.com/questions/85466/how-to-see-process-created-by-specific-user-in-unix-linux

Running jupyter notebook with nohup

https://gist.github.com/33eyes/e1da2d78979dc059433849c466ff5996

Start jupyter notebook on remote server

In the remote server terminal, run:

nohup jupyter notebook &

(the & sends the process to run in the background, so that the terminal window can be used)

Tunnel into the remote jupyter notebook from local

In the local terminal, run:

ssh awsgpu -NL 8157:localhost:8888 ubuntu@11.111.111.11

ssh darius@heat-AI-bigdata-3090-aftershock -NL 8157:localhost:8888

replacing awsgpu with remote server ssh login, 8157 with a free local port, 8888 with the port used by jupyter notebook on the remote server (usually it’s 8888), and 11.111.111.11 with remote server IP.

Stop the remote jupyter notebook

Since we now have a jupyter notebook running in the background, we can’t stop the notebook by closing the terminal window or Ctrl C. Here are the steps to stop it.

check what jupyter notebooks are running:

jupyter notebook list

(note which ports are being used)

find which process IDs (PIDs) are being run on the port used by the jupyter notebook(s):

netstat -tulpn

Look for processes that are running on port 8888 (or whichever port was listed in the step above), and note the PID.

now kill the process(es):

kill 2759

replace 2759 with the PID from the step above.

Or can use Tmux

Graph View