Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 1 | CGROUPS |
| 2 | ------- |
| 3 | |
Li Zefan | 45ce80f | 2009-01-15 13:50:59 -0800 | [diff] [blame] | 4 | Written by Paul Menage <menage@google.com> based on |
| 5 | Documentation/cgroups/cpusets.txt |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 6 | |
| 7 | Original copyright statements from cpusets.txt: |
| 8 | Portions Copyright (C) 2004 BULL SA. |
| 9 | Portions Copyright (c) 2004-2006 Silicon Graphics, Inc. |
| 10 | Modified by Paul Jackson <pj@sgi.com> |
| 11 | Modified by Christoph Lameter <clameter@sgi.com> |
| 12 | |
| 13 | CONTENTS: |
| 14 | ========= |
| 15 | |
| 16 | 1. Control Groups |
| 17 | 1.1 What are cgroups ? |
| 18 | 1.2 Why are cgroups needed ? |
| 19 | 1.3 How are cgroups implemented ? |
| 20 | 1.4 What does notify_on_release do ? |
Daniel Lezcano | 97978e6 | 2010-10-27 15:33:35 -0700 | [diff] [blame] | 21 | 1.5 What does clone_children do ? |
| 22 | 1.6 How do I use cgroups ? |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 23 | 2. Usage Examples and Syntax |
| 24 | 2.1 Basic Usage |
| 25 | 2.2 Attaching processes |
Kirill A. Shutemov | 8ca712e | 2010-03-10 15:22:10 -0800 | [diff] [blame] | 26 | 2.3 Mounting hierarchies by name |
Kirill A. Shutemov | 0dea116 | 2010-03-10 15:22:20 -0800 | [diff] [blame] | 27 | 2.4 Notification API |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 28 | 3. Kernel API |
| 29 | 3.1 Overview |
| 30 | 3.2 Synchronization |
| 31 | 3.3 Subsystem API |
Aristeu Rozanski | 19ec256 | 2012-09-11 16:28:10 -0400 | [diff] [blame] | 32 | 4. Extended attributes usage |
| 33 | 5. Questions |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 34 | |
| 35 | 1. Control Groups |
Li Zefan | d19e058 | 2008-02-23 15:24:08 -0800 | [diff] [blame] | 36 | ================= |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 37 | |
| 38 | 1.1 What are cgroups ? |
| 39 | ---------------------- |
| 40 | |
| 41 | Control Groups provide a mechanism for aggregating/partitioning sets of |
| 42 | tasks, and all their future children, into hierarchical groups with |
| 43 | specialized behaviour. |
| 44 | |
| 45 | Definitions: |
| 46 | |
| 47 | A *cgroup* associates a set of tasks with a set of parameters for one |
| 48 | or more subsystems. |
| 49 | |
| 50 | A *subsystem* is a module that makes use of the task grouping |
| 51 | facilities provided by cgroups to treat groups of tasks in |
| 52 | particular ways. A subsystem is typically a "resource controller" that |
| 53 | schedules a resource or applies per-cgroup limits, but it may be |
| 54 | anything that wants to act on a group of processes, e.g. a |
| 55 | virtualization subsystem. |
| 56 | |
| 57 | A *hierarchy* is a set of cgroups arranged in a tree, such that |
| 58 | every task in the system is in exactly one of the cgroups in the |
| 59 | hierarchy, and a set of subsystems; each subsystem has system-specific |
| 60 | state attached to each cgroup in the hierarchy. Each hierarchy has |
| 61 | an instance of the cgroup virtual filesystem associated with it. |
| 62 | |
Chris Samuel | caa790b | 2009-01-17 00:01:18 +1100 | [diff] [blame] | 63 | At any one time there may be multiple active hierarchies of task |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 64 | cgroups. Each hierarchy is a partition of all tasks in the system. |
| 65 | |
Michael Kerrisk | 83b061f | 2012-09-11 13:20:20 +0200 | [diff] [blame] | 66 | User-level code may create and destroy cgroups by name in an |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 67 | instance of the cgroup virtual file system, specify and query to |
Michael Kerrisk | 83b061f | 2012-09-11 13:20:20 +0200 | [diff] [blame] | 68 | which cgroup a task is assigned, and list the task PIDs assigned to |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 69 | a cgroup. Those creations and assignments only affect the hierarchy |
| 70 | associated with that instance of the cgroup file system. |
| 71 | |
| 72 | On their own, the only use for cgroups is for simple job |
| 73 | tracking. The intention is that other subsystems hook into the generic |
| 74 | cgroup support to provide new attributes for cgroups, such as |
| 75 | accounting/limiting the resources which processes in a cgroup can |
Michael Kerrisk | 83b061f | 2012-09-11 13:20:20 +0200 | [diff] [blame] | 76 | access. For example, cpusets (see Documentation/cgroups/cpusets.txt) allow |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 77 | you to associate a set of CPUs and a set of memory nodes with the |
| 78 | tasks in each cgroup. |
| 79 | |
| 80 | 1.2 Why are cgroups needed ? |
| 81 | ---------------------------- |
| 82 | |
| 83 | There are multiple efforts to provide process aggregations in the |
Michael Kerrisk | 83b061f | 2012-09-11 13:20:20 +0200 | [diff] [blame] | 84 | Linux kernel, mainly for resource-tracking purposes. Such efforts |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 85 | include cpusets, CKRM/ResGroups, UserBeanCounters, and virtual server |
| 86 | namespaces. These all require the basic notion of a |
| 87 | grouping/partitioning of processes, with newly forked processes ending |
Michael Kerrisk | 83b061f | 2012-09-11 13:20:20 +0200 | [diff] [blame] | 88 | up in the same group (cgroup) as their parent process. |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 89 | |
| 90 | The kernel cgroup patch provides the minimum essential kernel |
| 91 | mechanisms required to efficiently implement such groups. It has |
| 92 | minimal impact on the system fast paths, and provides hooks for |
| 93 | specific subsystems such as cpusets to provide additional behaviour as |
| 94 | desired. |
| 95 | |
| 96 | Multiple hierarchy support is provided to allow for situations where |
| 97 | the division of tasks into cgroups is distinctly different for |
| 98 | different subsystems - having parallel hierarchies allows each |
| 99 | hierarchy to be a natural division of tasks, without having to handle |
| 100 | complex combinations of tasks that would be present if several |
| 101 | unrelated subsystems needed to be forced into the same tree of |
| 102 | cgroups. |
| 103 | |
| 104 | At one extreme, each resource controller or subsystem could be in a |
| 105 | separate hierarchy; at the other extreme, all subsystems |
| 106 | would be attached to the same hierarchy. |
| 107 | |
| 108 | As an example of a scenario (originally proposed by vatsa@in.ibm.com) |
| 109 | that can benefit from multiple hierarchies, consider a large |
| 110 | university server with various users - students, professors, system |
| 111 | tasks etc. The resource planning for this server could be along the |
| 112 | following lines: |
| 113 | |
Geunsik Lim | 6ad8523 | 2011-04-04 15:10:45 -0700 | [diff] [blame] | 114 | CPU : "Top cpuset" |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 115 | / \ |
| 116 | CPUSet1 CPUSet2 |
Geunsik Lim | 6ad8523 | 2011-04-04 15:10:45 -0700 | [diff] [blame] | 117 | | | |
| 118 | (Professors) (Students) |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 119 | |
| 120 | In addition (system tasks) are attached to topcpuset (so |
| 121 | that they can run anywhere) with a limit of 20% |
| 122 | |
Geunsik Lim | 6ad8523 | 2011-04-04 15:10:45 -0700 | [diff] [blame] | 123 | Memory : Professors (50%), Students (30%), system (20%) |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 124 | |
Geunsik Lim | 6ad8523 | 2011-04-04 15:10:45 -0700 | [diff] [blame] | 125 | Disk : Professors (50%), Students (30%), system (20%) |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 126 | |
| 127 | Network : WWW browsing (20%), Network File System (60%), others (20%) |
| 128 | / \ |
Geunsik Lim | 6ad8523 | 2011-04-04 15:10:45 -0700 | [diff] [blame] | 129 | Professors (15%) students (5%) |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 130 | |
Michael Kerrisk | 83b061f | 2012-09-11 13:20:20 +0200 | [diff] [blame] | 131 | Browsers like Firefox/Lynx go into the WWW network class, while (k)nfsd goes |
| 132 | into the NFS network class. |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 133 | |
Chris Samuel | caa790b | 2009-01-17 00:01:18 +1100 | [diff] [blame] | 134 | At the same time Firefox/Lynx will share an appropriate CPU/Memory class |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 135 | depending on who launched it (prof/student). |
| 136 | |
| 137 | With the ability to classify tasks differently for different resources |
Michael Kerrisk | 83b061f | 2012-09-11 13:20:20 +0200 | [diff] [blame] | 138 | (by putting those resource subsystems in different hierarchies), |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 139 | the admin can easily set up a script which receives exec notifications |
| 140 | and depending on who is launching the browser he can |
| 141 | |
Jörg Sommer | f6e07d3 | 2011-06-15 12:59:45 -0700 | [diff] [blame] | 142 | # echo browser_pid > /sys/fs/cgroup/<restype>/<userclass>/tasks |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 143 | |
| 144 | With only a single hierarchy, he now would potentially have to create |
| 145 | a separate cgroup for every browser launched and associate it with |
Jörg Sommer | 67de016 | 2011-06-15 13:00:47 -0700 | [diff] [blame] | 146 | appropriate network and other resource class. This may lead to |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 147 | proliferation of such cgroups. |
| 148 | |
Michael Kerrisk | 83b061f | 2012-09-11 13:20:20 +0200 | [diff] [blame] | 149 | Also let's say that the administrator would like to give enhanced network |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 150 | access temporarily to a student's browser (since it is night and the user |
Michael Kerrisk | 83b061f | 2012-09-11 13:20:20 +0200 | [diff] [blame] | 151 | wants to do online gaming :)) OR give one of the student's simulation |
| 152 | apps enhanced CPU power. |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 153 | |
Michael Kerrisk | 83b061f | 2012-09-11 13:20:20 +0200 | [diff] [blame] | 154 | With ability to write PIDs directly to resource classes, it's just a |
| 155 | matter of: |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 156 | |
Jörg Sommer | f6e07d3 | 2011-06-15 12:59:45 -0700 | [diff] [blame] | 157 | # echo pid > /sys/fs/cgroup/network/<new_class>/tasks |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 158 | (after some time) |
Jörg Sommer | f6e07d3 | 2011-06-15 12:59:45 -0700 | [diff] [blame] | 159 | # echo pid > /sys/fs/cgroup/network/<orig_class>/tasks |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 160 | |
Michael Kerrisk | 83b061f | 2012-09-11 13:20:20 +0200 | [diff] [blame] | 161 | Without this ability, the administrator would have to split the cgroup into |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 162 | multiple separate ones and then associate the new cgroups with the |
| 163 | new resource classes. |
| 164 | |
| 165 | |
| 166 | |
| 167 | 1.3 How are cgroups implemented ? |
| 168 | --------------------------------- |
| 169 | |
| 170 | Control Groups extends the kernel as follows: |
| 171 | |
| 172 | - Each task in the system has a reference-counted pointer to a |
| 173 | css_set. |
| 174 | |
| 175 | - A css_set contains a set of reference-counted pointers to |
| 176 | cgroup_subsys_state objects, one for each cgroup subsystem |
| 177 | registered in the system. There is no direct link from a task to |
| 178 | the cgroup of which it's a member in each hierarchy, but this |
| 179 | can be determined by following pointers through the |
| 180 | cgroup_subsys_state objects. This is because accessing the |
| 181 | subsystem state is something that's expected to happen frequently |
| 182 | and in performance-critical code, whereas operations that require a |
| 183 | task's actual cgroup assignments (in particular, moving between |
Paul Menage | 817929e | 2007-10-18 23:39:36 -0700 | [diff] [blame] | 184 | cgroups) are less common. A linked list runs through the cg_list |
| 185 | field of each task_struct using the css_set, anchored at |
| 186 | css_set->tasks. |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 187 | |
Michael Kerrisk | 83b061f | 2012-09-11 13:20:20 +0200 | [diff] [blame] | 188 | - A cgroup hierarchy filesystem can be mounted for browsing and |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 189 | manipulation from user space. |
| 190 | |
Michael Kerrisk | 83b061f | 2012-09-11 13:20:20 +0200 | [diff] [blame] | 191 | - You can list all the tasks (by PID) attached to any cgroup. |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 192 | |
| 193 | The implementation of cgroups requires a few, simple hooks |
Michael Kerrisk | 83b061f | 2012-09-11 13:20:20 +0200 | [diff] [blame] | 194 | into the rest of the kernel, none in performance-critical paths: |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 195 | |
| 196 | - in init/main.c, to initialize the root cgroups and initial |
| 197 | css_set at system boot. |
| 198 | |
| 199 | - in fork and exit, to attach and detach a task from its css_set. |
| 200 | |
Michael Kerrisk | 83b061f | 2012-09-11 13:20:20 +0200 | [diff] [blame] | 201 | In addition, a new file system of type "cgroup" may be mounted, to |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 202 | enable browsing and modifying the cgroups presently known to the |
| 203 | kernel. When mounting a cgroup hierarchy, you may specify a |
| 204 | comma-separated list of subsystems to mount as the filesystem mount |
| 205 | options. By default, mounting the cgroup filesystem attempts to |
| 206 | mount a hierarchy containing all registered subsystems. |
| 207 | |
| 208 | If an active hierarchy with exactly the same set of subsystems already |
| 209 | exists, it will be reused for the new mount. If no existing hierarchy |
| 210 | matches, and any of the requested subsystems are in use in an existing |
| 211 | hierarchy, the mount will fail with -EBUSY. Otherwise, a new hierarchy |
| 212 | is activated, associated with the requested subsystems. |
| 213 | |
Tejun Heo | 26d5bbe | 2013-04-12 10:29:04 -0700 | [diff] [blame] | 214 | It's not currently possible to bind a new subsystem to an active |
| 215 | cgroup hierarchy, or to unbind a subsystem from an active cgroup |
| 216 | hierarchy. This may be possible in future, but is fraught with nasty |
| 217 | error-recovery issues. |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 218 | |
| 219 | When a cgroup filesystem is unmounted, if there are any |
| 220 | child cgroups created below the top-level cgroup, that hierarchy |
| 221 | will remain active even though unmounted; if there are no |
| 222 | child cgroups then the hierarchy will be deactivated. |
| 223 | |
| 224 | No new system calls are added for cgroups - all support for |
| 225 | querying and modifying cgroups is via this cgroup file system. |
| 226 | |
| 227 | Each task under /proc has an added file named 'cgroup' displaying, |
| 228 | for each active hierarchy, the subsystem names and the cgroup name |
| 229 | as the path relative to the root of the cgroup file system. |
| 230 | |
| 231 | Each cgroup is represented by a directory in the cgroup file system |
| 232 | containing the following files describing that cgroup: |
| 233 | |
Michael Kerrisk | 83b061f | 2012-09-11 13:20:20 +0200 | [diff] [blame] | 234 | - tasks: list of tasks (by PID) attached to that cgroup. This list |
| 235 | is not guaranteed to be sorted. Writing a thread ID into this file |
Paul Menage | 7823da3 | 2009-10-07 16:32:26 -0700 | [diff] [blame] | 236 | moves the thread into this cgroup. |
Michael Kerrisk | 83b061f | 2012-09-11 13:20:20 +0200 | [diff] [blame] | 237 | - cgroup.procs: list of thread group IDs in the cgroup. This list is |
| 238 | not guaranteed to be sorted or free of duplicate TGIDs, and userspace |
Paul Menage | 7823da3 | 2009-10-07 16:32:26 -0700 | [diff] [blame] | 239 | should sort/uniquify the list if this property is required. |
Michael Kerrisk | 83b061f | 2012-09-11 13:20:20 +0200 | [diff] [blame] | 240 | Writing a thread group ID into this file moves all threads in that |
Ben Blum | 74a1166 | 2011-05-26 16:25:20 -0700 | [diff] [blame] | 241 | group into this cgroup. |
Li Zefan | d19e058 | 2008-02-23 15:24:08 -0800 | [diff] [blame] | 242 | - notify_on_release flag: run the release agent on exit? |
| 243 | - release_agent: the path to use for release notifications (this file |
| 244 | exists in the top cgroup only) |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 245 | |
| 246 | Other subsystems such as cpusets may add additional files in each |
Li Zefan | d19e058 | 2008-02-23 15:24:08 -0800 | [diff] [blame] | 247 | cgroup dir. |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 248 | |
| 249 | New cgroups are created using the mkdir system call or shell |
| 250 | command. The properties of a cgroup, such as its flags, are |
| 251 | modified by writing to the appropriate file in that cgroups |
| 252 | directory, as listed above. |
| 253 | |
| 254 | The named hierarchical structure of nested cgroups allows partitioning |
| 255 | a large system into nested, dynamically changeable, "soft-partitions". |
| 256 | |
| 257 | The attachment of each task, automatically inherited at fork by any |
| 258 | children of that task, to a cgroup allows organizing the work load |
| 259 | on a system into related sets of tasks. A task may be re-attached to |
| 260 | any other cgroup, if allowed by the permissions on the necessary |
| 261 | cgroup file system directories. |
| 262 | |
| 263 | When a task is moved from one cgroup to another, it gets a new |
| 264 | css_set pointer - if there's an already existing css_set with the |
Michael Kerrisk | 83b061f | 2012-09-11 13:20:20 +0200 | [diff] [blame] | 265 | desired collection of cgroups then that group is reused, otherwise a new |
Li Zefan | b851ee7 | 2009-02-18 14:48:14 -0800 | [diff] [blame] | 266 | css_set is allocated. The appropriate existing css_set is located by |
| 267 | looking into a hash table. |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 268 | |
Paul Menage | 817929e | 2007-10-18 23:39:36 -0700 | [diff] [blame] | 269 | To allow access from a cgroup to the css_sets (and hence tasks) |
| 270 | that comprise it, a set of cg_cgroup_link objects form a lattice; |
| 271 | each cg_cgroup_link is linked into a list of cg_cgroup_links for |
Li Zefan | d19e058 | 2008-02-23 15:24:08 -0800 | [diff] [blame] | 272 | a single cgroup on its cgrp_link_list field, and a list of |
Paul Menage | 817929e | 2007-10-18 23:39:36 -0700 | [diff] [blame] | 273 | cg_cgroup_links for a single css_set on its cg_link_list. |
| 274 | |
| 275 | Thus the set of tasks in a cgroup can be listed by iterating over |
| 276 | each css_set that references the cgroup, and sub-iterating over |
| 277 | each css_set's task set. |
| 278 | |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 279 | The use of a Linux virtual file system (vfs) to represent the |
| 280 | cgroup hierarchy provides for a familiar permission and name space |
| 281 | for cgroups, with a minimum of additional kernel code. |
| 282 | |
| 283 | 1.4 What does notify_on_release do ? |
| 284 | ------------------------------------ |
| 285 | |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 286 | If the notify_on_release flag is enabled (1) in a cgroup, then |
| 287 | whenever the last task in the cgroup leaves (exits or attaches to |
| 288 | some other cgroup) and the last child cgroup of that cgroup |
| 289 | is removed, then the kernel runs the command specified by the contents |
| 290 | of the "release_agent" file in that hierarchy's root directory, |
| 291 | supplying the pathname (relative to the mount point of the cgroup |
| 292 | file system) of the abandoned cgroup. This enables automatic |
| 293 | removal of abandoned cgroups. The default value of |
| 294 | notify_on_release in the root cgroup at system boot is disabled |
| 295 | (0). The default value of other cgroups at creation is the current |
Michael Kerrisk | 83b061f | 2012-09-11 13:20:20 +0200 | [diff] [blame] | 296 | value of their parents' notify_on_release settings. The default value of |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 297 | a cgroup hierarchy's release_agent path is empty. |
| 298 | |
Daniel Lezcano | 97978e6 | 2010-10-27 15:33:35 -0700 | [diff] [blame] | 299 | 1.5 What does clone_children do ? |
| 300 | --------------------------------- |
| 301 | |
Tejun Heo | 2260e7f | 2012-11-19 08:13:38 -0800 | [diff] [blame] | 302 | This flag only affects the cpuset controller. If the clone_children |
| 303 | flag is enabled (1) in a cgroup, a new cpuset cgroup will copy its |
| 304 | configuration from the parent during initialization. |
Daniel Lezcano | 97978e6 | 2010-10-27 15:33:35 -0700 | [diff] [blame] | 305 | |
| 306 | 1.6 How do I use cgroups ? |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 307 | -------------------------- |
| 308 | |
| 309 | To start a new job that is to be contained within a cgroup, using |
| 310 | the "cpuset" cgroup subsystem, the steps are something like: |
| 311 | |
Jörg Sommer | f6e07d3 | 2011-06-15 12:59:45 -0700 | [diff] [blame] | 312 | 1) mount -t tmpfs cgroup_root /sys/fs/cgroup |
| 313 | 2) mkdir /sys/fs/cgroup/cpuset |
| 314 | 3) mount -t cgroup -ocpuset cpuset /sys/fs/cgroup/cpuset |
| 315 | 4) Create the new cgroup by doing mkdir's and write's (or echo's) in |
| 316 | the /sys/fs/cgroup virtual file system. |
| 317 | 5) Start a task that will be the "founding father" of the new job. |
Michael Kerrisk | 83b061f | 2012-09-11 13:20:20 +0200 | [diff] [blame] | 318 | 6) Attach that task to the new cgroup by writing its PID to the |
Jörg Sommer | f6e07d3 | 2011-06-15 12:59:45 -0700 | [diff] [blame] | 319 | /sys/fs/cgroup/cpuset/tasks file for that cgroup. |
| 320 | 7) fork, exec or clone the job tasks from this founding father task. |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 321 | |
| 322 | For example, the following sequence of commands will setup a cgroup |
| 323 | named "Charlie", containing just CPUs 2 and 3, and Memory Node 1, |
| 324 | and then start a subshell 'sh' in that cgroup: |
| 325 | |
Jörg Sommer | f6e07d3 | 2011-06-15 12:59:45 -0700 | [diff] [blame] | 326 | mount -t tmpfs cgroup_root /sys/fs/cgroup |
| 327 | mkdir /sys/fs/cgroup/cpuset |
| 328 | mount -t cgroup cpuset -ocpuset /sys/fs/cgroup/cpuset |
| 329 | cd /sys/fs/cgroup/cpuset |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 330 | mkdir Charlie |
| 331 | cd Charlie |
Dhaval Giani | 0f146a7 | 2008-05-12 14:02:31 -0700 | [diff] [blame] | 332 | /bin/echo 2-3 > cpuset.cpus |
| 333 | /bin/echo 1 > cpuset.mems |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 334 | /bin/echo $$ > tasks |
| 335 | sh |
| 336 | # The subshell 'sh' is now running in cgroup Charlie |
| 337 | # The next line should display '/Charlie' |
| 338 | cat /proc/self/cgroup |
| 339 | |
| 340 | 2. Usage Examples and Syntax |
| 341 | ============================ |
| 342 | |
| 343 | 2.1 Basic Usage |
| 344 | --------------- |
| 345 | |
Michael Kerrisk | 83b061f | 2012-09-11 13:20:20 +0200 | [diff] [blame] | 346 | Creating, modifying, using cgroups can be done through the cgroup |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 347 | virtual filesystem. |
| 348 | |
Chris Samuel | caa790b | 2009-01-17 00:01:18 +1100 | [diff] [blame] | 349 | To mount a cgroup hierarchy with all available subsystems, type: |
Jörg Sommer | f6e07d3 | 2011-06-15 12:59:45 -0700 | [diff] [blame] | 350 | # mount -t cgroup xxx /sys/fs/cgroup |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 351 | |
| 352 | The "xxx" is not interpreted by the cgroup code, but will appear in |
| 353 | /proc/mounts so may be any useful identifying string that you like. |
| 354 | |
Eric B Munson | bb6405e | 2011-03-15 16:12:18 -0700 | [diff] [blame] | 355 | Note: Some subsystems do not work without some user input first. For instance, |
| 356 | if cpusets are enabled the user will have to populate the cpus and mems files |
| 357 | for each new cgroup created before that group can be used. |
| 358 | |
Jörg Sommer | f6e07d3 | 2011-06-15 12:59:45 -0700 | [diff] [blame] | 359 | As explained in section `1.2 Why are cgroups needed?' you should create |
| 360 | different hierarchies of cgroups for each single resource or group of |
| 361 | resources you want to control. Therefore, you should mount a tmpfs on |
| 362 | /sys/fs/cgroup and create directories for each cgroup resource or resource |
| 363 | group. |
| 364 | |
| 365 | # mount -t tmpfs cgroup_root /sys/fs/cgroup |
| 366 | # mkdir /sys/fs/cgroup/rg1 |
| 367 | |
Trevor Woerner | 595f4b6 | 2010-05-26 14:42:35 -0700 | [diff] [blame] | 368 | To mount a cgroup hierarchy with just the cpuset and memory |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 369 | subsystems, type: |
Jörg Sommer | f6e07d3 | 2011-06-15 12:59:45 -0700 | [diff] [blame] | 370 | # mount -t cgroup -o cpuset,memory hier1 /sys/fs/cgroup/rg1 |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 371 | |
Daniel Wagner | 9a8054a | 2012-07-10 10:49:18 +0200 | [diff] [blame] | 372 | While remounting cgroups is currently supported, it is not recommend |
| 373 | to use it. Remounting allows changing bound subsystems and |
| 374 | release_agent. Rebinding is hardly useful as it only works when the |
| 375 | hierarchy is empty and release_agent itself should be replaced with |
| 376 | conventional fsnotify. The support for remounting will be removed in |
| 377 | the future. |
Li Zefan | b6719ec | 2009-04-02 16:57:28 -0700 | [diff] [blame] | 378 | |
| 379 | To Specify a hierarchy's release_agent: |
| 380 | # mount -t cgroup -o cpuset,release_agent="/sbin/cpuset_release_agent" \ |
Jörg Sommer | f6e07d3 | 2011-06-15 12:59:45 -0700 | [diff] [blame] | 381 | xxx /sys/fs/cgroup/rg1 |
Li Zefan | b6719ec | 2009-04-02 16:57:28 -0700 | [diff] [blame] | 382 | |
| 383 | Note that specifying 'release_agent' more than once will return failure. |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 384 | |
Tejun Heo | 26d5bbe | 2013-04-12 10:29:04 -0700 | [diff] [blame] | 385 | Note that changing the set of subsystems is currently only supported |
| 386 | when the hierarchy consists of a single (root) cgroup. Supporting |
| 387 | the ability to arbitrarily bind/unbind subsystems from an existing |
| 388 | cgroup hierarchy is intended to be implemented in the future. |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 389 | |
Jörg Sommer | f6e07d3 | 2011-06-15 12:59:45 -0700 | [diff] [blame] | 390 | Then under /sys/fs/cgroup/rg1 you can find a tree that corresponds to the |
| 391 | tree of the cgroups in the system. For instance, /sys/fs/cgroup/rg1 |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 392 | is the cgroup that holds the whole system. |
| 393 | |
Li Zefan | b6719ec | 2009-04-02 16:57:28 -0700 | [diff] [blame] | 394 | If you want to change the value of release_agent: |
Jörg Sommer | f6e07d3 | 2011-06-15 12:59:45 -0700 | [diff] [blame] | 395 | # echo "/sbin/new_release_agent" > /sys/fs/cgroup/rg1/release_agent |
Li Zefan | b6719ec | 2009-04-02 16:57:28 -0700 | [diff] [blame] | 396 | |
| 397 | It can also be changed via remount. |
| 398 | |
Jörg Sommer | f6e07d3 | 2011-06-15 12:59:45 -0700 | [diff] [blame] | 399 | If you want to create a new cgroup under /sys/fs/cgroup/rg1: |
| 400 | # cd /sys/fs/cgroup/rg1 |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 401 | # mkdir my_cgroup |
| 402 | |
| 403 | Now you want to do something with this cgroup. |
| 404 | # cd my_cgroup |
| 405 | |
| 406 | In this directory you can find several files: |
| 407 | # ls |
Paul Menage | 7823da3 | 2009-10-07 16:32:26 -0700 | [diff] [blame] | 408 | cgroup.procs notify_on_release tasks |
Li Zefan | d19e058 | 2008-02-23 15:24:08 -0800 | [diff] [blame] | 409 | (plus whatever files added by the attached subsystems) |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 410 | |
| 411 | Now attach your shell to this cgroup: |
| 412 | # /bin/echo $$ > tasks |
| 413 | |
| 414 | You can also create cgroups inside your cgroup by using mkdir in this |
| 415 | directory. |
| 416 | # mkdir my_sub_cs |
| 417 | |
| 418 | To remove a cgroup, just use rmdir: |
| 419 | # rmdir my_sub_cs |
| 420 | |
| 421 | This will fail if the cgroup is in use (has cgroups inside, or |
| 422 | has processes attached, or is held alive by other subsystem-specific |
| 423 | reference). |
| 424 | |
| 425 | 2.2 Attaching processes |
| 426 | ----------------------- |
| 427 | |
| 428 | # /bin/echo PID > tasks |
| 429 | |
| 430 | Note that it is PID, not PIDs. You can only attach ONE task at a time. |
| 431 | If you have several tasks to attach, you have to do it one after another: |
| 432 | |
| 433 | # /bin/echo PID1 > tasks |
| 434 | # /bin/echo PID2 > tasks |
| 435 | ... |
| 436 | # /bin/echo PIDn > tasks |
| 437 | |
Li Zefan | bef67c5 | 2008-07-04 09:59:55 -0700 | [diff] [blame] | 438 | You can attach the current shell task by echoing 0: |
| 439 | |
| 440 | # echo 0 > tasks |
| 441 | |
Ben Blum | 74a1166 | 2011-05-26 16:25:20 -0700 | [diff] [blame] | 442 | You can use the cgroup.procs file instead of the tasks file to move all |
Michael Kerrisk | 83b061f | 2012-09-11 13:20:20 +0200 | [diff] [blame] | 443 | threads in a threadgroup at once. Echoing the PID of any task in a |
Ben Blum | 74a1166 | 2011-05-26 16:25:20 -0700 | [diff] [blame] | 444 | threadgroup to cgroup.procs causes all tasks in that threadgroup to be |
Rami Rosen | 1ae65ae | 2013-04-03 20:32:04 +0300 | [diff] [blame] | 445 | attached to the cgroup. Writing 0 to cgroup.procs moves all tasks |
Ben Blum | 74a1166 | 2011-05-26 16:25:20 -0700 | [diff] [blame] | 446 | in the writing task's threadgroup. |
| 447 | |
Eric B Munson | bb6405e | 2011-03-15 16:12:18 -0700 | [diff] [blame] | 448 | Note: Since every task is always a member of exactly one cgroup in each |
| 449 | mounted hierarchy, to remove a task from its current cgroup you must |
| 450 | move it into a new cgroup (possibly the root cgroup) by writing to the |
| 451 | new cgroup's tasks file. |
| 452 | |
Li Zefan | 5fe69d7 | 2011-11-04 11:22:05 -0700 | [diff] [blame] | 453 | Note: Due to some restrictions enforced by some cgroup subsystems, moving |
| 454 | a process to another cgroup can fail. |
Eric B Munson | bb6405e | 2011-03-15 16:12:18 -0700 | [diff] [blame] | 455 | |
Paul Menage | c6d57f3 | 2009-09-23 15:56:19 -0700 | [diff] [blame] | 456 | 2.3 Mounting hierarchies by name |
| 457 | -------------------------------- |
| 458 | |
| 459 | Passing the name=<x> option when mounting a cgroups hierarchy |
| 460 | associates the given name with the hierarchy. This can be used when |
| 461 | mounting a pre-existing hierarchy, in order to refer to it by name |
| 462 | rather than by its set of active subsystems. Each hierarchy is either |
| 463 | nameless, or has a unique name. |
| 464 | |
| 465 | The name should match [\w.-]+ |
| 466 | |
| 467 | When passing a name=<x> option for a new hierarchy, you need to |
| 468 | specify subsystems manually; the legacy behaviour of mounting all |
| 469 | subsystems when none are explicitly specified is not supported when |
| 470 | you give a subsystem a name. |
| 471 | |
| 472 | The name of the subsystem appears as part of the hierarchy description |
| 473 | in /proc/mounts and /proc/<pid>/cgroups. |
| 474 | |
Kirill A. Shutemov | 0dea116 | 2010-03-10 15:22:20 -0800 | [diff] [blame] | 475 | 2.4 Notification API |
| 476 | -------------------- |
| 477 | |
| 478 | There is mechanism which allows to get notifications about changing |
| 479 | status of a cgroup. |
| 480 | |
Michael Kerrisk | 83b061f | 2012-09-11 13:20:20 +0200 | [diff] [blame] | 481 | To register a new notification handler you need to: |
Kirill A. Shutemov | 0dea116 | 2010-03-10 15:22:20 -0800 | [diff] [blame] | 482 | - create a file descriptor for event notification using eventfd(2); |
| 483 | - open a control file to be monitored (e.g. memory.usage_in_bytes); |
| 484 | - write "<event_fd> <control_fd> <args>" to cgroup.event_control. |
| 485 | Interpretation of args is defined by control file implementation; |
| 486 | |
| 487 | eventfd will be woken up by control file implementation or when the |
| 488 | cgroup is removed. |
| 489 | |
Michael Kerrisk | 83b061f | 2012-09-11 13:20:20 +0200 | [diff] [blame] | 490 | To unregister a notification handler just close eventfd. |
Kirill A. Shutemov | 0dea116 | 2010-03-10 15:22:20 -0800 | [diff] [blame] | 491 | |
| 492 | NOTE: Support of notifications should be implemented for the control |
| 493 | file. See documentation for the subsystem. |
Paul Menage | c6d57f3 | 2009-09-23 15:56:19 -0700 | [diff] [blame] | 494 | |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 495 | 3. Kernel API |
| 496 | ============= |
| 497 | |
| 498 | 3.1 Overview |
| 499 | ------------ |
| 500 | |
| 501 | Each kernel subsystem that wants to hook into the generic cgroup |
| 502 | system needs to create a cgroup_subsys object. This contains |
| 503 | various methods, which are callbacks from the cgroup system, along |
Michael Kerrisk | 83b061f | 2012-09-11 13:20:20 +0200 | [diff] [blame] | 504 | with a subsystem ID which will be assigned by the cgroup system. |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 505 | |
| 506 | Other fields in the cgroup_subsys object include: |
| 507 | |
| 508 | - subsys_id: a unique array index for the subsystem, indicating which |
Li Zefan | d19e058 | 2008-02-23 15:24:08 -0800 | [diff] [blame] | 509 | entry in cgroup->subsys[] this subsystem should be managing. |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 510 | |
Li Zefan | d19e058 | 2008-02-23 15:24:08 -0800 | [diff] [blame] | 511 | - name: should be initialized to a unique subsystem name. Should be |
| 512 | no longer than MAX_CGROUP_TYPE_NAMELEN. |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 513 | |
Li Zefan | d19e058 | 2008-02-23 15:24:08 -0800 | [diff] [blame] | 514 | - early_init: indicate if the subsystem needs early initialization |
| 515 | at system boot. |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 516 | |
| 517 | Each cgroup object created by the system has an array of pointers, |
Michael Kerrisk | 83b061f | 2012-09-11 13:20:20 +0200 | [diff] [blame] | 518 | indexed by subsystem ID; this pointer is entirely managed by the |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 519 | subsystem; the generic cgroup code will never touch this pointer. |
| 520 | |
| 521 | 3.2 Synchronization |
| 522 | ------------------- |
| 523 | |
| 524 | There is a global mutex, cgroup_mutex, used by the cgroup |
| 525 | system. This should be taken by anything that wants to modify a |
| 526 | cgroup. It may also be taken to prevent cgroups from being |
| 527 | modified, but more specific locks may be more appropriate in that |
| 528 | situation. |
| 529 | |
| 530 | See kernel/cgroup.c for more details. |
| 531 | |
| 532 | Subsystems can take/release the cgroup_mutex via the functions |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 533 | cgroup_lock()/cgroup_unlock(). |
| 534 | |
| 535 | Accessing a task's cgroup pointer may be done in the following ways: |
| 536 | - while holding cgroup_mutex |
| 537 | - while holding the task's alloc_lock (via task_lock()) |
| 538 | - inside an rcu_read_lock() section via rcu_dereference() |
| 539 | |
| 540 | 3.3 Subsystem API |
Li Zefan | d19e058 | 2008-02-23 15:24:08 -0800 | [diff] [blame] | 541 | ----------------- |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 542 | |
| 543 | Each subsystem should: |
| 544 | |
| 545 | - add an entry in linux/cgroup_subsys.h |
| 546 | - define a cgroup_subsys object called <name>_subsys |
| 547 | |
Ben Blum | e6a1105 | 2010-03-10 15:22:09 -0800 | [diff] [blame] | 548 | If a subsystem can be compiled as a module, it should also have in its |
Ben Blum | cf5d594 | 2010-03-10 15:22:09 -0800 | [diff] [blame] | 549 | module initcall a call to cgroup_load_subsys(), and in its exitcall a |
| 550 | call to cgroup_unload_subsys(). It should also set its_subsys.module = |
| 551 | THIS_MODULE in its .c file. |
Ben Blum | e6a1105 | 2010-03-10 15:22:09 -0800 | [diff] [blame] | 552 | |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 553 | Each subsystem may export the following methods. The only mandatory |
Tejun Heo | 92fb974 | 2012-11-19 08:13:38 -0800 | [diff] [blame] | 554 | methods are css_alloc/free. Any others that are null are presumed to |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 555 | be successful no-ops. |
| 556 | |
Tejun Heo | 92fb974 | 2012-11-19 08:13:38 -0800 | [diff] [blame] | 557 | struct cgroup_subsys_state *css_alloc(struct cgroup *cgrp) |
Paul Menage | 8dc4f3e | 2008-02-07 00:13:45 -0800 | [diff] [blame] | 558 | (cgroup_mutex held by caller) |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 559 | |
Tejun Heo | 92fb974 | 2012-11-19 08:13:38 -0800 | [diff] [blame] | 560 | Called to allocate a subsystem state object for a cgroup. The |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 561 | subsystem should allocate its subsystem state object for the passed |
| 562 | cgroup, returning a pointer to the new object on success or a |
Tejun Heo | 92fb974 | 2012-11-19 08:13:38 -0800 | [diff] [blame] | 563 | ERR_PTR() value. On success, the subsystem pointer should point to |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 564 | a structure of type cgroup_subsys_state (typically embedded in a |
| 565 | larger subsystem-specific object), which will be initialized by the |
| 566 | cgroup system. Note that this will be called at initialization to |
| 567 | create the root subsystem state for this subsystem; this case can be |
| 568 | identified by the passed cgroup object having a NULL parent (since |
| 569 | it's the root of the hierarchy) and may be an appropriate place for |
| 570 | initialization code. |
| 571 | |
Tejun Heo | 92fb974 | 2012-11-19 08:13:38 -0800 | [diff] [blame] | 572 | int css_online(struct cgroup *cgrp) |
Paul Menage | 8dc4f3e | 2008-02-07 00:13:45 -0800 | [diff] [blame] | 573 | (cgroup_mutex held by caller) |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 574 | |
Tejun Heo | 92fb974 | 2012-11-19 08:13:38 -0800 | [diff] [blame] | 575 | Called after @cgrp successfully completed all allocations and made |
| 576 | visible to cgroup_for_each_child/descendant_*() iterators. The |
| 577 | subsystem may choose to fail creation by returning -errno. This |
| 578 | callback can be used to implement reliable state sharing and |
| 579 | propagation along the hierarchy. See the comment on |
| 580 | cgroup_for_each_descendant_pre() for details. |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 581 | |
Tejun Heo | 92fb974 | 2012-11-19 08:13:38 -0800 | [diff] [blame] | 582 | void css_offline(struct cgroup *cgrp); |
Li Zefan | d7eeac1 | 2013-03-12 15:35:59 -0700 | [diff] [blame] | 583 | (cgroup_mutex held by caller) |
Li Zefan | d19e058 | 2008-02-23 15:24:08 -0800 | [diff] [blame] | 584 | |
Tejun Heo | 92fb974 | 2012-11-19 08:13:38 -0800 | [diff] [blame] | 585 | This is the counterpart of css_online() and called iff css_online() |
| 586 | has succeeded on @cgrp. This signifies the beginning of the end of |
| 587 | @cgrp. @cgrp is being removed and the subsystem should start dropping |
| 588 | all references it's holding on @cgrp. When all references are dropped, |
| 589 | cgroup removal will proceed to the next step - css_free(). After this |
| 590 | callback, @cgrp should be considered dead to the subsystem. |
| 591 | |
| 592 | void css_free(struct cgroup *cgrp) |
| 593 | (cgroup_mutex held by caller) |
| 594 | |
| 595 | The cgroup system is about to free @cgrp; the subsystem should free |
| 596 | its subsystem state object. By the time this method is called, @cgrp |
| 597 | is completely unused; @cgrp->parent is still valid. (Note - can also |
| 598 | be called for a newly-created cgroup if an error occurs after this |
| 599 | subsystem's create() method has been called for the new cgroup). |
Li Zefan | d19e058 | 2008-02-23 15:24:08 -0800 | [diff] [blame] | 600 | |
Li Zefan | 761b3ef | 2012-01-31 13:47:36 +0800 | [diff] [blame] | 601 | int can_attach(struct cgroup *cgrp, struct cgroup_taskset *tset) |
Paul Menage | 8dc4f3e | 2008-02-07 00:13:45 -0800 | [diff] [blame] | 602 | (cgroup_mutex held by caller) |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 603 | |
Tejun Heo | 2f7ee56 | 2011-12-12 18:12:21 -0800 | [diff] [blame] | 604 | Called prior to moving one or more tasks into a cgroup; if the |
| 605 | subsystem returns an error, this will abort the attach operation. |
| 606 | @tset contains the tasks to be attached and is guaranteed to have at |
| 607 | least one task in it. |
| 608 | |
| 609 | If there are multiple tasks in the taskset, then: |
| 610 | - it's guaranteed that all are from the same thread group |
| 611 | - @tset contains all tasks from the thread group whether or not |
| 612 | they're switching cgroups |
| 613 | - the first task is the leader |
| 614 | |
| 615 | Each @tset entry also contains the task's old cgroup and tasks which |
| 616 | aren't switching cgroup can be skipped easily using the |
| 617 | cgroup_taskset_for_each() iterator. Note that this isn't called on a |
| 618 | fork. If this method returns 0 (success) then this should remain valid |
| 619 | while the caller holds cgroup_mutex and it is ensured that either |
Ben Blum | f780bdb | 2011-05-26 16:25:19 -0700 | [diff] [blame] | 620 | attach() or cancel_attach() will be called in future. |
| 621 | |
Li Zefan | 761b3ef | 2012-01-31 13:47:36 +0800 | [diff] [blame] | 622 | void cancel_attach(struct cgroup *cgrp, struct cgroup_taskset *tset) |
Daisuke Nishimura | 2468c72 | 2010-03-10 15:22:03 -0800 | [diff] [blame] | 623 | (cgroup_mutex held by caller) |
| 624 | |
| 625 | Called when a task attach operation has failed after can_attach() has succeeded. |
| 626 | A subsystem whose can_attach() has some side-effects should provide this |
Thomas Weber | 8839316 | 2010-03-16 11:47:56 +0100 | [diff] [blame] | 627 | function, so that the subsystem can implement a rollback. If not, not necessary. |
Daisuke Nishimura | 2468c72 | 2010-03-10 15:22:03 -0800 | [diff] [blame] | 628 | This will be called only about subsystems whose can_attach() operation have |
Tejun Heo | 2f7ee56 | 2011-12-12 18:12:21 -0800 | [diff] [blame] | 629 | succeeded. The parameters are identical to can_attach(). |
Daisuke Nishimura | 2468c72 | 2010-03-10 15:22:03 -0800 | [diff] [blame] | 630 | |
Li Zefan | 761b3ef | 2012-01-31 13:47:36 +0800 | [diff] [blame] | 631 | void attach(struct cgroup *cgrp, struct cgroup_taskset *tset) |
Li Zefan | 18e7f1f | 2009-01-07 18:07:32 -0800 | [diff] [blame] | 632 | (cgroup_mutex held by caller) |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 633 | |
| 634 | Called after the task has been attached to the cgroup, to allow any |
| 635 | post-attachment activity that requires memory allocations or blocking. |
Tejun Heo | 2f7ee56 | 2011-12-12 18:12:21 -0800 | [diff] [blame] | 636 | The parameters are identical to can_attach(). |
Ben Blum | f780bdb | 2011-05-26 16:25:19 -0700 | [diff] [blame] | 637 | |
Li Zefan | 761b3ef | 2012-01-31 13:47:36 +0800 | [diff] [blame] | 638 | void fork(struct task_struct *task) |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 639 | |
Li Zefan | e8d55fd | 2008-04-29 01:00:13 -0700 | [diff] [blame] | 640 | Called when a task is forked into a cgroup. |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 641 | |
Li Zefan | 761b3ef | 2012-01-31 13:47:36 +0800 | [diff] [blame] | 642 | void exit(struct task_struct *task) |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 643 | |
Li Zefan | d19e058 | 2008-02-23 15:24:08 -0800 | [diff] [blame] | 644 | Called during task exit. |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 645 | |
Tejun Heo | 26d5bbe | 2013-04-12 10:29:04 -0700 | [diff] [blame] | 646 | void bind(struct cgroup *root) |
| 647 | (cgroup_mutex held by caller) |
| 648 | |
| 649 | Called when a cgroup subsystem is rebound to a different hierarchy |
| 650 | and root cgroup. Currently this will only involve movement between |
| 651 | the default hierarchy (which never has sub-cgroups) and a hierarchy |
| 652 | that is being created/destroyed (and hence has no sub-cgroups). |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 653 | |
Aristeu Rozanski | 19ec256 | 2012-09-11 16:28:10 -0400 | [diff] [blame] | 654 | 4. Extended attribute usage |
| 655 | =========================== |
| 656 | |
| 657 | cgroup filesystem supports certain types of extended attributes in its |
| 658 | directories and files. The current supported types are: |
| 659 | - Trusted (XATTR_TRUSTED) |
| 660 | - Security (XATTR_SECURITY) |
| 661 | |
| 662 | Both require CAP_SYS_ADMIN capability to set. |
| 663 | |
| 664 | Like in tmpfs, the extended attributes in cgroup filesystem are stored |
| 665 | using kernel memory and it's advised to keep the usage at minimum. This |
| 666 | is the reason why user defined extended attributes are not supported, since |
| 667 | any user can do it and there's no limit in the value size. |
| 668 | |
| 669 | The current known users for this feature are SELinux to limit cgroup usage |
| 670 | in containers and systemd for assorted meta data like main PID in a cgroup |
| 671 | (systemd creates a cgroup per service). |
| 672 | |
| 673 | 5. Questions |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 674 | ============ |
| 675 | |
| 676 | Q: what's up with this '/bin/echo' ? |
| 677 | A: bash's builtin 'echo' command does not check calls to write() against |
| 678 | errors. If you use it in the cgroup file system, you won't be |
| 679 | able to tell whether a command succeeded or failed. |
| 680 | |
| 681 | Q: When I attach processes, only the first of the line gets really attached ! |
| 682 | A: We can only return one error code per call to write(). So you should also |
Michael Kerrisk | 83b061f | 2012-09-11 13:20:20 +0200 | [diff] [blame] | 683 | put only ONE PID. |
Paul Menage | ddbcc7e | 2007-10-18 23:39:30 -0700 | [diff] [blame] | 684 | |