Combating 'Disk quota exceeded'

Owned by UKY CCS RCD
Aug 08, 2024

LCC enforces storage quotas, meaning certain directories have an upper limit on how much data you can store in them. If you exceed this limit, you'll receive a 'Disk quota exceeded' error, and your job will crash. You can review the established quotas in our /wiki/spaces/UPR/pages/7669196.

Some applications will invisibly create directories under /home and store large files there. Because /home/$USER has the most strict quotas, this is the most frequent culprit for errors. For example, when running Singularity containers, Singularity will create a /home/$USER/.singularity directory and download large image files which will quickly hit your home quota. But there is a simple solution for this problem.

The solution is to move the directory in question to your /scratch/$USER space, and provide a symlink from home to scratch. This will allow applications to look under /home/$USER but actually be directed to /scratch/$USER where the quotas are much more forgiving. Following is a two-command example of this, using .singularity as a case study.

mv /home/$USER/.singularity $SCRATCH/.singularity
ln -s $SCRATCH/.singularity /home/$USER/.singularity

-----------
For users installing conda packages:

Please do the same as above since conda installs all the packages in your home directory in the folder $HOME/.conda. Since you need conda packages for long term use put them in your shared project space instead of your scratch folder.

Eg: If your shared group project space is /project/xxxx_uksr  then

      mkdir -p /project/xxxx_uksr/$USER/my_conda
      ln -s /project/xxxx_uksr/$USER/my_conda /home/$USER/.conda

     

-----------
For users installing R packages:

Please do the same as above since R installs all the packages in your home directory in the folder $HOME/R. Since you need R packages for long term use put them in your shared project space instead of your scratch folder.

Eg: If your shared group project space is /project/xxxx_uksr  then

      mkdir -p /project/xxxx_uksr/$USER/my_R
      ln -s /project/xxxx_uksr/$USER/my_R /home/$USER/R

Also make sure you load the same R module (eg: module load ccs/conda/r-base-4.0.0) each time if you want to use the R packages that were installed with that version of R.

--------------

For users installing Julia packages:

Please do the same as above since Julia installs all the packages and the registries in your home directory in the folder $HOME/.julia. Since you need Julia packages for long term use put them in your shared project space instead of your scratch folder.

Eg: If your shared group project space is /project/xxxx_uksr  then

      mkdir -p /project/xxxx_uksr/$USER/my_julia
      ln -s /project/xxxx_uksr/$USER/my_julia /home/$USER/.julia

Also make sure you load the same Julia module (eg: module load ccs/julia/1.6.1) each time if you want to use the Julia packages that were installed with that version of Julia.

     

Here are the storage quotas on LCC https://docs.ccs.uky.edu/display/HPC/LCC+storage+Quotas


Center for Computational Sciences