How to Run Tensorflow in a Jupyter Notebook (LCC)

This document shows how to run TensorFlow on LCC through Jupyter notebooks.

Log in to LCC and install TensorFlow and the jupyter kernel. What you need to change below is line 6, where you define a directory for the Tensorflow software environment inside the TF_ENV variable. On like 7, remember the name stored in the variable JUPYTER_KERNEL. Below, the variable is set as “python-tensorflow”. This is the name of the Jupyter kernel that you will select in a later step:

# Load Miniforge
module load ccs/Miniforge3
source activate

# Create conda Environment
TF_ENV=/project/yourprojectname/tf_env
JUPYTER_KERNEL=python-tensorflow
JUPYTER_KERNEL_DESC=Python-TensorFlow
mkdir $TF_ENV
conda create --prefix ${TF_ENV}
conda activate $TF_ENV
conda config --env --add pkgs_dirs ${TF_ENV}/packages
cat $TF_ENV/.condarc   # validate that the cache dir is there as priority.
conda install tensorflow jupyter

python -m ipykernel install --user --name $JUPYTER_KERNEL --display-name "${JUPYTER_KERNEL_DESC}"

Check in the python interpreter to check for TensorFlow installation success by seeing if the version number is printed:

>>> import tensorflow
print(tensorflow.__version__)
>>> print(tensorflow.__version__)
2.6.2

Go to LCC's OpenOndemand (OOD) at https://ood.ccs.uky.edu and go to the page that lets you request a Jupyter session.

In the OOD jupyter app job request, there's a textbox titled "Custom Conda Environment Path". Set it to the path to the Conva environment you just set above (the value of $TF_ENV). Enter your project account name and other job attributes. Also, put in your project id in the “Account” box (it should look something like “col_somename_uksr”. Below, the Debug queue is used, where jobs can only go for 1 hour before being terminated. This queue is to check if your TensorFlow setup works. Once you know that it works, you can select a different queue that lets you run longer jobs:

Once you are in a Jupyter session, load the newly created Python Kernel by going to Kernel > Change Kernel
Select the Kernel you’ve created (it will depend on what you’ve set in JUPYTER_KERNEL and JUPYTER_KERNEL_DESC in step 1.

References

How to create custom conda Environment
Creating a Custom Conda Environment on HPC

How to run Jupyter notebook:

JupyterHub