Creating a custom kernel for Jupyter Notebooks

Jupyter Notebooks are widely recognized as a popular interactive platform for coding in Python, used in data science, research, and education. Open OnDemand simplifies the process of running Jupyter Notebooks by offering a user-friendly, web-based interface. This guide provides step-by-step instructions to create a custom Python kernel for Jupyter Notebooks, allowing you to leverage specific conda environments or Singularity containers for your projects.

1 1) Test if ipykernel is installed.
- 1.1 Singularity Container
- 1.2 Conda Environment
2 2) Create custom-kernel directory.
3 3) Create custom-kernel helper script.
- 3.1 Singularity Container
- 3.2 Conda Environment
4 4) Create custom-kernel JSON file.
5 5) Check Jupyter Notebook.
6 Acknowledgements

1) Test if ipykernel is installed.

Singularity Container

To test whether a container has ipykernel installed, you can execute a command similar to the following for your specific container:

singularity run --app feems100 /share/singularity/images/ccs/conda/lcc-conda-11-rocky9.sinf python -m pip list | grep ipykernel

This command will run python -m pip list | grep ipykernel within the container /share/singularity/images/ccs/conda/lcc-conda-11-rocky9.sinf, specifically in the feems100 app environment.

If there is a response from this command, you can continue to build the custom-kernel. For example:

[link_blue_id@dtn ~]$ singularity run --app feems100 /share/singularity/images/ccs/conda/lcc-conda-11-rocky9.sinf python -m pip list | grep ipykernel
ipykernel                6.29.5

Conda Environment

To test whether a conda environment has ipykernel installed, you can execute a command similar to the following for your specific environment:

module load ccs/Miniforge3
source activate /path/to/conda/environment
python -m pip list | grep ipykernel

This command will run python -m pip list | grep ipykernel within the conda environment at /path/to/conda/environment.

If there is a response from this command, you can continue to build the custom-kernel. For example:

2) Create custom-kernel directory.

Jupyter will look for custom-kernels in the user directory: ~/.local/share/jupyter/kernels

We will need to create a new subdirectory for our new kernel:

3) Create custom-kernel helper script.

Singularity Container

The following is an example helper script that should be placed in the custom-kernel directory for a Singularity container.

This can be modified by including additional modules, for example, if you needed to load CUDA you would include module load ccs/cuda; into the string after module load ccs/singularity;.

Remember to change the singularity container path and app to reflect the container you wish to load.

Conda Environment

The following is an example helper script that should be placed in the custom-kernel directory for a conda environment.

This can be modified by including additional modules, for example, if you needed to load CUDA you would include module load ccs/cuda; into the string after module load ccs/Miniforge3;.

Remember to change the conda environment path to reflect the environment you wish to load.

4) Create custom-kernel JSON file.

Jupyter looks for a file in the directory named kernel.json. We must create this file and populate it to provide a name for the kernel and direct it to our helper scripts. Two examples are provided below one for conda (.../xlstm/kernel.json) and one for singularity container (.../feems/kernel.json).

5) Check Jupyter Notebook.

See Running Interactive JupyterNotebook Session Using Open OnDemand - RCDDocs - docs to initiate a JupyterNotebook session using Open OnDemand.

To see the available kernels, click “Kernel” > “Change Kernel” in the menu bar.

Successfully installed kernels appear as the following:

Acknowledgements

This document was produced with aid from https://docs.nersc.gov/services/jupyter/how-to-guides/