Running Python on ManeFrame II

Custom Python Evironments and Package Installations

Python environments allow users to use specific versions of Python with the packages of their choice. The packages can be installed via conda or pip, depending on the type of environment being used.

module purge
module load python/3
conda create -y -n jupyter_37 -c conda-forge jupyterlab python=3.7
source activate ~/.conda/envs/jupyter_37
deactivate

module purge
module load spack gcc-9.2
source <(spack module tcl loads --dependencies python@3.7%gcc@9.2)
python3 -m venv ~/.venv/jupyter_37
source ~/.venv/jupyter_37/bin/activate
pip3 install --upgrade pip
pip3 install --upgrade jupyterlab
deactivate

Submitting a Python Job to a Queue Using Wrap

A Python script can be executed non-interactively in batch mode directly using sbatch’s wrapping function.

1. Log into the cluster using SSH and run the following commands at the command prompt.

2. module load python to enable access to Python.

3. cd to the directory with Python script.

4. sbatch -p <partition and options> --wrap "python <Python script file name>" where <partition and options> is the partition and associated Slurm flags for each partition outlined in the table above. and <Python script file name> is the Python script to be run.

5. squeue -u $USER to verify that the job has been submitted to the queue.

Example:

module load python
sbatch -p standard-mem-s --exclusive --mem=250G --wrap "python example.py"

Submitting a Python Job to a Queue Using an sbatch Script

A Python script can be executed non-interactively in batch mode by creating an sbatch script. The sbatch script gives the Slurm resource scheduler information about what compute resources your calculation requires to run and also how to run the Python script when the job is executed by Slurm.

1. Log into the cluster using SSH and run the following commands at the command prompt.

2. cd to the directory with the Python script.

3. cp /hpc/examples/python/python_example.sbatch <descriptive file name> where <descriptive file name> is meaningful for the calculation being done. It is suggested to not use spaces in the file name and that it end with .sbatch for clarity.

4. Edit the sbatch file using using preferred text editor. Change the partition and flags and Python script file name as required for your specific calculation.

#!/bin/bash
#SBATCH -J python_example              # Job name
#SBATCH -o example.txt                 # Output file name
#SBATCH -p standard-mem-s              # Partition (queue)
#SBATCH --exclusive                    # Exclusivity
#SBATCH --mem=250G                     # Total memory required per node

module purge                           # Unload all modules
module load python                     # Load R, change version as needed

python example.py                      # Edit Python script name as needed

5. sbatch <descriptive file name> where <descriptive file name> is the sbatch script name chosen previously.

6. squeue -u $USER to verify that the job has been submitted to the queue.

Submitting Multiple Python Jobs to a Queue Using a Single sbatch Script

Multiple Python scripts can be executed non-interactively in batch mode by creating a single sbatch script. The sbatch script gives the Slurm resource scheduler information about what compute resources your calculations requires to run and also how to run the Python script for each job when the job is executed by Slurm.

1. Log into the cluster using SSH and run the following commands at the command prompt.

2. cd to the directory with the Python script or scripts.

3. cp -R /hpc/examples/python/python_array_example ~/.

4. cd python_array_example.

5. Edit the sbatch file using using preferred text editor. Change the partition and flags, Python script file name, and number of jobs that will be executed as required for your specific calculation.

#!/bin/bash
#SBATCH -J python_example              # Job name
#SBATCH -p standard-mem-s              # Partition (queue)
#SBATCH --exclusive                    # Exclusivity
#SBATCH --mem=250G                     # Total memory required per node
#SBATCH -o python_example_%A-%a.out    # Job output; %A is job ID and %a is array index
#SBATCH --array=1-2                    # Range of indices to be executed

module purge                           # Unload all modules
module load python                     # Load R, change version as needed

python array_example_${SLURM_ARRAY_TASK_ID}.py
# Edit Python script name as needed; ${SLURM_ARRAY_TASK_ID} is array index

6. sbatch python_array.sbatch.

7. squeue -u $USER to verify that the job has been submitted to the queue.

Submitting Jupyter Notebooks to a Queue Using a sbatch Script

Jupyter Notebooks can be executed non-interactively using sbatch.

1. Log into the cluster using SSH and run the following commands at the command prompt. Alternatively, you can access a terminal from the web portal [hpc.smu.edu](https://hpc.smu.edu) and selecting _MainFrame II Shell Access from the Clusters menu at the top.

2. cd to the directory with the Jupyter notebook file(s).

3. Edit or create an sbatch file using using preferred text editor. For this example, we’ll call this file notebook_job.sbatch Change the partition and flags, Notebook file name, and Conda environment as required for your specific application.

#!/bin/bash

#SBATCH -J run_notebook              # Job name to display in the queue
#SBATCH -o run_notebook_%j.txt       # terminal output from the job, %j expands to the job id
#SBATCH -e run_notebook_%j.err       # error log from the job, %j expands to the job id
#SBATCH --partition=standard-mem-s   # partition to run on
#SBATCH --nodes=1                    # number of nodes to run on
#SBATCH --ntasks-per-node=1          # number of tasks per node
#SBATCH --cpus-per-task=1            # number of cores per task
#SBATCH --time=0-00:30:00            # job runtime, format is D-HH:MM:SS
#SBATCH --mem=10G                    # memory for the job, in GB
#SBATCH --gres=gpu:0                 # number of GPUs gpu:1 = 1 gpu, gpu2:2 = 2 gpu

# make sure there are no modules loaded from the environment
module purge

# load the modules we want. We just need the python module
# for jupyterlabs and/or Anacaonda. If you are using other
# modules add them here
module load python/3

# make sure Conda commands are available in the script
# and load our Conda environment. If you are not using
# a Conda environment, you can omit these.
eval "$(conda shell.bash hook)"
conda activate my_conda_environment

# run the notebook
# This command is usually used to convert a notebook to a python script,
# but we can also use it to run the notebook and write the output into
# the same notebook, so when you open it the output areas are populated
jupyter nbconvert --to notebook --inplace --execute my_jupyter_notebook.ipynb

4. Submit the job by running sbatch notebook_job.sbatch.

5. squeue -u $USER to verify that the job has been submitted to the queue.