SLURM Queues

SLURM Queues for Open Access Only Resources

Summary

This page explains Simple Linux Utility for Resource Management (SLURM) queue options for Open Access and Condo (owned-share) resources, usage limits, and naming conventions. Tables list queue parameters and definitions.

The roughly 400 nodes comprising the Lipscomb Compute Cluster (LCC) and Morgan Compute Cluster (MCC) supercomputer continue to be supported, offering approximately 260 million Service Units (SUs) per year to all users for Open Access allocations use only.

As was the case with the previous systems, these resources are scheduled using a fair share scheduling algorithm. (Unlike the Condo resources, there are no privileged SLURM accounts; all users have the same priority). The only allocations that can be used on this system are the Open Access allocations — specifically for collaborative (col) and general (gol) purposes. Like the previous DLX 2/3 systems, there are Long, Medium, Short, and Debug SLURM queues that limit the number of days a job can run based on the number of cores it needs. The more cores required, the shorter the maximum run time.

Moreover, SLURM is configured to give higher priority to jobs that run for short periods of time (see Note 2). In addition, no Principal Investigator (PI) group may use more than 1536 (on LCC) and 2048 (on MCC) cores at any given time. The maximum runtime for the long queues has been set at 14 days based on historical data indicating that few jobs need more than 14 days. If you need to run a job longer than 14 days, please submit a support request: Submit a support request.

The Queue Names shown below are based on the Naming Conventions described at the bottom of this page.

SLURM Queues for Condo Resources

Table 1. Condo queues, maximum runtimes, billing rates, and node details.

Condo jobs are "billed" for the resources they use. Because each type of resource has a different billing rate, there are different queues for each type of Condo resource. At present, the Condo resource types are SKY1 (Skylake 6130 CPUs), P100 (NVIDIA P100 GPUs), and V100 (NVIDIA V100 GPUs). In the future there may be additional resource types (e.g., a new Skylake processor, say SKY2). Within each type of resource, there are short, medium, long, and flex queues to prevent users from monopolizing any particular resource (see below). In addition, there are two debug queues used to debug CPU or GPU jobs, respectively.

No PI may use more than 1024 cores at any given time.

No PI may use more than 32 GPU cards at any given time.

If you need to run a job longer than 14 days, please submit a support request: Submit a support request.

 

---

Naming Conventions (used in the above names)

Table 2. CPU naming convention components.

CPU queues names have the following format:

Table 3. GPU naming convention components.

and GPU queues names have the following format: (Note: GPU cards are housed in a server with CPUs)

where

Notes

²: For both Lipscomb Compute Cluster (LCC) and Morgan Compute Cluster (MCC), and for Condo resources, SLURM is configured to give slighly higher priority to "wide jobs" requiring many cores to ensure they are not starved by "narrow jobs". So-called "back filling" is also used to allow narrow jobs to make effective use of available cores.