    \fBcgroup.conf\fP is an ASCII file which defines parameters used by
    Slurm's Linux cgroup related plugins.
    The file location can be modified at system build time using the
    DEFAULT_SLURM_CONF parameter or at execution time by setting the SLURM_CONF
    environment variable. The file will always be located in the
    same directory as the \fBslurm.conf\fP file.
    .LP
    Parameter names are case insensitive.
    Any text following a "#" in the configuration file is treated
    as a comment through the end of that line.
    Changes to the configuration file take effect upon restart of
    Slurm daemons, daemon receipt of the SIGHUP signal, or execution
    of the command "scontrol reconfigure" unless otherwise noted.

    .LP
    For general Slurm cgroups information, see the Cgroups Guide at
    <https://slurm.schedmd.com/cgroups.html>.

    .LP
    The following cgroup.conf parameters are defined to control the general behavior
    of Slurm cgroup plugins.

    .TP
    \fBCgroupAutomount\fR=<yes|no>
    Slurm cgroup plugins require valid and functional cgroup subsystem to be mounted
    under /sys/fs/cgroup/<subsystem_name>.
    When launched, plugins check their subsystem availability. If not available,
    the plugin launch fails unless CgroupAutomount is set to yes. In that case, the
    plugin will first try to mount the required subsystems.

    .TP
    \fBCgroupMountpoint\fR=\fIPATH\fR
    Specify the \fIPATH\fR under which cgroups should be mounted. This
    should be a writable directory which will contain cgroups mounted
    one per subsystem. The default \fIPATH\fR is /sys/fs/cgroup.

    .SH "TASK/CGROUP PLUGIN"

    .LP
    The following cgroup.conf parameters are defined to control the behavior
    of this particular plugin:

    .TP
    \fBAllowedKmemSpace\fR=<number>
    Constrain the job cgroup kernel memory to this amount of the allocated memory,
    specified in bytes. The \fBAllowedKmemSpace\fR must be between the upper and
    lower memory limits, specified by \fBMaxKmemPercent\fR and \fBMinKmemSpace\fR,
    respectively. If \fBAllowedKmemSpace\fR goes beyond the upper or lower limit,
    it will be reset to that upper or lower limit, whichever has been exceeded.

    .TP
    \fBAllowedRAMSpace\fR=<number>

    .TP
    \fBAllowedSwapSpace\fR=<number>
    Constrain the job cgroup swap space to this percentage of the allocated
    memory.  The default value is 0, which means that RAM+Swap will be limited
    to \fBAllowedRAMSpace\fR. The supplied percentage may be expressed as a
    floating point number, e.g. 50.5.  If the limit is exceeded, the job steps
    will be killed and a warning message will be written to standard error.
    Also see \fBConstrainSwapSpace\fR.
    NOTE: Setting AllowedSwapSpace to 0 does not restrict the Linux kernel from
    using swap space. To control how the kernel uses swap space, see
    \fBMemorySwappiness\fR.

    .TP
    \fBConstrainCores\fR=<yes|no>
    If configured to "yes" then constrain allowed cores to the subset of
    allocated resources. This functionality makes use of the cpuset subsystem.
    Due to a bug fixed in version 1.11.5 of HWLOC, the task/affinity plugin may be
    required in addition to task/cgroup for this to function properly.
    The default value is "no".

    .TP
    \fBConstrainDevices\fR=<yes|no>
    If configured to "yes" then constrain the job's allowed devices based on GRES
    allocated resources. It uses the devices subsystem for that.
    The default value is "no".

    .TP
    \fBConstrainKmemSpace\fR=<yes|no>
    If configured to "yes" then constrain the job's Kmem RAM usage in addition to
    RAM usage. Only takes effect if \fBConstrainRAMSpace\fR is set to "yes". If
    enabled, the job's Kmem limit will be assigned the value of
    \fBAllowedKmemSpace\fR or the value coming from \fBMaxKmemPercent\fR.
    The default value is "no" which will leave Kmem setting untouched by Slurm.
    Also see \fBAllowedKmemSpace\fR, \fBMaxKmemPercent\fR.

    .TP
    \fBConstrainRAMSpace\fR=<yes|no>
    If configured to "yes" then constrain the job's RAM usage by setting
    the memory soft limit to the allocated memory and the hard limit to
    the allocated memory * \fBAllowedRAMSpace\fR.  The default value is "no", in
    which case the job's RAM limit will be set to its swap space limit if
    \fBConstrainSwapSpace\fR is set to "yes".
    Also see \fBAllowedSwapSpace\fR, \fBAllowedRAMSpace\fR and
    \fBConstrainSwapSpace\fR.

    \fBNOTE\fR: When using \fBConstrainRAMSpace\fR, if a process tries to consume
    more memory than is available, the step that process is running in will be
    killed. This differs from the behavior when using \fBOverMemoryKill\fR,
    where just the offending process will be killed.

    \fBNOTE\fR: When enabled, ConstrainRAMSpace can lead to a noticeable decline in
    used to constrain swap space.
    Also see \fBAllowedSwapSpace\fR.

    .TP
    \fBMaxRAMPercent\fR=\fIPERCENT\fR
    Set an upper bound in percent of total RAM on the RAM constraint for a job.
    This will be the memory constraint applied to jobs that are not explicitly
    allocated memory by Slurm (i.e. Slurm's select plugin is not configured to manage
    memory allocations). The \fIPERCENT\fR may be an arbitrary floating
    point number. The default value is 100.

    .TP
    \fBMaxSwapPercent\fR=\fIPERCENT\fR
    Set an upper bound (in percent of total RAM) on the amount of RAM+Swap
    that may be used for a job. This will be the swap limit applied to jobs
    on systems where memory is not being explicitly allocated to job. The
    \fIPERCENT\fR may be an arbitrary floating point number between 0 and 100.
    The default value is 100.

    .TP
    \fBMaxKmemPercent\fR=\fIPERCENT\fR
    Set an upper bound in percent of total RAM as the maximum Kmem for a job. The
    \fIPERCENT\fR may be an arbitrary floating point number, however, the product
    of \fBMaxKmemPercent\fR and job requested memory has to fall between
    \fBMinKmemSpace\fR and job requested memory, otherwise the boundary value is
    used. The default value is 100.

    .TP
    \fBMemorySwappiness\fR=<number>
    Configure the kernel's priority for swapping out anonymous pages (such as
    program data) verses file cache pages for the job cgroup. Valid values are
    between 0 and 100, inclusive. A value of 0 prevents the kernel from swapping
    out program data. A value of 100 gives equal priority to swapping out file
    cache or anonymous pages. If not set, then the kernel's default swappiness
    value will be used. \fBConstrainSwapSpace\fR
    must be set to \fByes\fR in order for this parameter to be applied.

    .TP
    \fBMinKmemSpace\fR=<number>
    Set a lower bound (in MB) on the memory limits defined by
    \fBAllowedKmemSpace\fR. The default limit is 30M.

    .TP
    \fBMinRAMSpace\fR=<number>
    Set a lower bound (in MB) on the memory limits defined by
    \fBAllowedRAMSpace\fR and \fBAllowedSwapSpace\fR. This prevents
    accidentally creating a memory cgroup with such a low limit that slurmstepd
    is immediately killed due to lack of RAM. The default limit is 30M.

    .TP
    \fBTaskAffinity\fR=<yes|no>
    If configured to "yes" then set a default task affinity to bind each step

    .LP
    Debian and derivatives (e.g. Ubuntu) usually exclude the memory and memsw (swap)
    cgroups by default. To include them, add the following parameters to the kernel
    command line: \fBcgroup_enable=memory swapaccount=1\fR
    .LP
    This can usually be placed in /etc/default/grub inside the
    \fBGRUB_CMDLINE_LINUX\fR variable. A command such as update\-grub must be run
    after updating the file.

    .SH "EXAMPLE"
    .TP
    \fB/etc/slurm/cgroup.conf\fR:
    This example cgroup.conf file shows a configuration that enables the more
    commonly used cgroup enforcement mechanisms.

    .nf
    ###
    # Slurm cgroup support configuration file.
    ###
    CgroupAutomount=yes
    CgroupMountpoint=/sys/fs/cgroup
    ConstrainCores=yes
    ConstrainDevices=yes
    ConstrainKmemSpace=no        #avoid known Kernel issues
    ConstrainRAMSpace=yes
    ConstrainSwapSpace=yes
    TaskAffinity=no              #use task/affinity plugin instead
    .fi
    .TP
    \fB/etc/slurm/slurm.conf\fR:
    These are the entries required in \fBslurm.conf\fR to activate the cgroup
    enforcement mechanisms. Make sure that the node definitions in your
    \fBslurm.conf\fR closely match the configuration as shown by "\fBslurmd -C\fR".
    Either MemSpecLimit should be set or RealMemory should be defined with less
    than the actual amount of memory for a node to ensure that all system/non-job
    processes will have sufficient memory at all times. Sites should also configure
    \fBpam_slurm_adopt\fR to ensure users can not escape the cgroups via \fBssh\fR.

    .nf
    ###
    # Slurm configuration entries for cgroups
    ###
    ProctrackType=proctrack/cgroup
    TaskPlugin=task/cgroup,task/affinity
    JobAcctGatherType=jobacct_gather/cgroup #optional for gathering metrics
    PrologFlags=Contain                     #X11 flag is also suggested
    .fi

    .SH "COPYING"
    Copyright (C) 2010\-2012 Lawrence Livermore National Security.
    Produced at Lawrence Livermore National Laboratory (cf, DISCLAIMER).
    details.

    .SH "SEE ALSO"
    .LP
    \fBslurm.conf\<B>fR(5)