Creating a custom kernel for Jupyter Notebooks

Custom Jupyter kernels allow you to run notebooks using specific conda environments or Singularity containers.

This guide explains how to:

verify required Python packages
create a kernel directory
configure a helper script
register the kernel

1 Step 1 — Verify ipykernel is Installed
- 1.1 For Singularity Containers
- 1.2 For Conda Environments
2 Step 2 — Create a Kernel Directory
3 Step 3 — Create a Kernel Helper Script
- 3.1 Example — Singularity Kernel Helper
- 3.2 Example — Conda Kernel Helper
4 Step 4 — Create the kernel.json File
5 Step 5 — Verify the Kernel in Jupyter
- 5.1 Acknowledgements

Step 1 — Verify `ipykernel` is Installed

A custom kernel requires the ipykernel package.

For Singularity Containers

Run:

singularity run --app <app_name> <container_path> python -m pip list | grep ipykernel

If ipykernel appears in the output, continue.

For Conda Environments

Run:

module load ccs/Miniforge3
source activate /path/to/conda/environment
python -m pip list | grep ipykernel

If ipykernel appears, continue.

Step 2 — Create a Kernel Directory

Jupyter searches for kernels in:

~/.local/share/jupyter/kernels

Create a directory for your kernel:

cd ~/.local/share/jupyter/kernels
mkdir my_kernel
cd my_kernel

Step 3 — Create a Kernel Helper Script

This script launches Python inside your container or environment.

Example — Singularity Kernel Helper

#!/bin/bash
ssh $USER@$HOSTNAME -T "
module load ccs/singularity;
singularity run --app <app_name> <container_path> \$@
"

Example — Conda Kernel Helper

#!/bin/bash
ssh $USER@$HOSTNAME -T "
module load ccs/Miniforge3;
source activate /path/to/environment;
\$@
"

Make the script executable:

chmod +x kernel-helper.sh

Step 4 — Create the `kernel.json` File

Create a file named kernel.json in the kernel directory:

{
  "argv": [
    "/home/$USER/.local/share/jupyter/kernels/my_kernel/kernel-helper.sh",
    "python",
    "-m",
    "ipykernel_launcher",
    "-f",
    "{connection_file}"
  ],
  "display_name": "My Custom Kernel",
  "language": "python"
}

Step 5 — Verify the Kernel in Jupyter

Jupyter Notebook interface showing the “Change Kernel” menu with multiple kernel options including a custom kernel.

Start a Jupyter session using Open OnDemand.

Then:

Open a notebook
Click Kernel → Change Kernel
Select your new kernel

Acknowledgements

Adapted from guidance provided by:

https://docs.nersc.gov/services/jupyter/how-to-guides/