/
Starting Your First Nextflow Job on LCC and MCC

Starting Your First Nextflow Job on LCC and MCC

 

This guide walks you through running your first Nextflow job on the LCC or MCC clusters using the nf-core/fastqrepair test pipeline. It includes running an initial interactive test on the DTN (Data Transfer Node) to download required assets, then submitting jobs to SLURM for production use.

📋 Prerequisites

✅ You have an active LCC or MCC account.
✅ You know your PI’s project allocation account name.
✅ You have access to the DTN node on LCC or MCC.

🔷 Why use the DTN?

The first run of an nf-core pipeline downloads Singularity containers and pipeline assets from the internet. To avoid heavy I/O or network load on compute nodes, we recommend launching this initial test run on the DTN. After this one-time setup, all subsequent runs should be submitted as SLURM jobs.

🚀 Steps

1️⃣ Connect to the DTN node

For LCC:

SSH command to connect to LCC DTN node
ssh <your-username>@dtn.ccs.uky.edu

For MCC:

SSH command to connect to MCC DTN node
ssh <your-username>@mcc-dtn.ccs.uky.edu

2️⃣ Load the Nextflow module

On LCC:

Loading Nextflow and Singularity modules on LCC
module load ccs/conda/nextflow-25.04.4 module load ccs/singularity-3.8.2

On MCC:

Loading Nextflow module on MCC
module load ccs/conda/nextflow/24.10.4

3️⃣ Set Singularity cache to scratch

Setting Singularity cache directory to scratch space
export NXF_SINGULARITY_CACHEDIR=/scratch/$USER/singularity_cache

Info: On LCC, scratch directories could be /mnt/gpfs2_4m/scratch/$USER; /mnt/gpfs2_16m/scratch/$USER; or /scratch/$USER.

You can execute echo $SCRATCH to see your scratch location.

4️⃣ Edit the nextflow.config file

Below are example configurations you can use:

LCC nextflow.config:

LCC Nextflow configuration file example
process { executor = 'slurm' queue = 'SKY32M192_L' // Note: Only one queue can be specified here; you cannot list multiple queues in Nextflow. Choose the most appropriate CPU queue for your job, such as SKY32M192_L, CAL48M192_L, or CAC48M192_L clusterOptions = '-A col_PILinkBlue_uksr' cpus = 1 memory = '4 GB' time = '1h' } singularity { enabled = true autoMounts = true }

MCC nextflow.config:

MCC Nextflow configuration file example
process { executor = 'slurm' queue = 'normal' clusterOptions = '-A coa_PILinkBlue_uksr' cpus = 1 memory = '4 GB' time = '1h' } singularity { enabled = true autoMounts = true }

5️⃣ Run the test pipeline on the DTN

Once your nextflow.config file is in place, run the test pipeline on the DTN:

Running the fastqrepair test pipeline on DTN
nextflow run nf-core/fastqrepair -profile test,singularity -r 1.0.0 --outdir /scratch/$USER/fastqrepair_test_results -c nextflow.config

This step downloads the required assets and containers.

Info: On LCC, scratch directories could be /mnt/gpfs2_4m/scratch/$USER; /mnt/gpfs2_16m/scratch/$USER; or /scratch/$USER.

You can execute echo $SCRATCH to see your scratch location.

6️⃣ Submit the pipeline as a SLURM job

Below are example SLURM job scripts you can use to create a submit.sh submission script:

LCC Job Script

LCC SLURM submission script for Nextflow
#!/bin/bash #SBATCH --job-name=nextflow_test #SBATCH --output nextflow_test-%j.out #SBATCH --mail-type ALL #SBATCH --mail-user email_address@uky.edu #SBATCH --ntasks=1 #SBATCH --cpus-per-task 1 #SBATCH --mem-per-cpu 4gb #SBATCH --account=col_vgazu2_uksr #SBATCH --partition=SKY32M192_L #SBATCH --time=1:00:00 module load ccs/conda/nextflow-25.04.4 module load ccs/singularity-3.8.2 export NXF_SINGULARITY_CACHEDIR=/scratch/$USER/singularity_cache nextflow run nf-core/fastqrepair \ -profile test,singularity \ -r 1.0.0 \ --outdir /scratch/$USER/fastqrepair_test_results \ -c nextflow.config

Submit with:

Submitting the SLURM job script
sbatch submit.sh

MCC Job Script

MCC SLURM submission script for Nextflow
#!/bin/bash #SBATCH --job-name=nextflow_test #SBATCH --output nextflow_test-%j.out #SBATCH --mail-type ALL #SBATCH --mail-user email_address@uky.edu #SBATCH --ntasks=1 #SBATCH --cpus-per-task 1 #SBATCH --mem-per-cpu 4gb #SBATCH --account=coa_PILinkBlue_uksr #SBATCH --partition=normal #SBATCH --time=1:00:00 module load ccs/conda/nextflow/24.10.4 export NXF_SINGULARITY_CACHEDIR=/scratch/$USER/singularity_cache nextflow run nf-core/fastqrepair \ -profile test,singularity \ -r 1.0.0 \ --outdir /scratch/$USER/fastqrepair_test_results \ -c nextflow.config

Submit with:

Submitting the SLURM job script
sbatch submit.sh

7️⃣ Proceeding with Production Data

Once you have successfully run the test pipeline and downloaded the required assets, you are ready to process your own data. Before submitting your SLURM job for production, transfer your input FASTQ files to your scratch or project directory using Globus. Then modify your SLURM job script to remove the -profile test argument and replace it with appropriate --input and --outdir parameters pointing to your data locations. Consult the nf-core/fastqrepair Usage page for details about supported input parameters.

🌟 Notes

✅ Always use --outdir to specify a scratch or project directory for results.
✅ Always specify -r when running an nf-core pipeline.
✅ Use the DTN interactively only for the initial container download; submit all subsequent runs through SLURM.
✅ You can also create additional profiles in your nextflow.config – for example, if you would like to submit to different queues on LCC.
✅ For production runs, remove the -profile test option and instead specify your real --input and --outdir parameters. Use Globus to transfer your FASTQ files to the scratch or project space before running.

For more details, see:

If you have any questions, please contact us at CCS Support .

Center for Computational Sciences