blob: f12fda31dcdc488c62963779a4e3360acd30a5d4 [file] [log] [blame]
David Howells8f0aa2f2009-04-03 16:42:35 +01001 ====================================
2 SLOW WORK ITEM EXECUTION THREAD POOL
3 ====================================
4
5By: David Howells <dhowells@redhat.com>
6
7The slow work item execution thread pool is a pool of threads for performing
8things that take a relatively long time, such as making mkdir calls.
9Typically, when processing something, these items will spend a lot of time
10blocking a thread on I/O, thus making that thread unavailable for doing other
11work.
12
13The standard workqueue model is unsuitable for this class of work item as that
14limits the owner to a single thread or a single thread per CPU. For some
15tasks, however, more threads - or fewer - are required.
16
17There is just one pool per system. It contains no threads unless something
18wants to use it - and that something must register its interest first. When
19the pool is active, the number of threads it contains is dynamic, varying
20between a maximum and minimum setting, depending on the load.
21
22
23====================
24CLASSES OF WORK ITEM
25====================
26
27This pool support two classes of work items:
28
29 (*) Slow work items.
30
31 (*) Very slow work items.
32
33The former are expected to finish much quicker than the latter.
34
35An operation of the very slow class may do a batch combination of several
36lookups, mkdirs, and a create for instance.
37
38An operation of the ordinarily slow class may, for example, write stuff or
39expand files, provided the time taken to do so isn't too long.
40
41Operations of both types may sleep during execution, thus tying up the thread
42loaned to it.
43
44
45THREAD-TO-CLASS ALLOCATION
46--------------------------
47
48Not all the threads in the pool are available to work on very slow work items.
49The number will be between one and one fewer than the number of active threads.
50This is configurable (see the "Pool Configuration" section).
51
52All the threads are available to work on ordinarily slow work items, but a
53percentage of the threads will prefer to work on very slow work items.
54
55The configuration ensures that at least one thread will be available to work on
56very slow work items, and at least one thread will be available that won't work
57on very slow work items at all.
58
59
60=====================
61USING SLOW WORK ITEMS
62=====================
63
64Firstly, a module or subsystem wanting to make use of slow work items must
65register its interest:
66
David Howells3d7a6412009-11-19 18:10:23 +000067 int ret = slow_work_register_user(struct module *module);
David Howells8f0aa2f2009-04-03 16:42:35 +010068
David Howells3d7a6412009-11-19 18:10:23 +000069This will return 0 if successful, or a -ve error upon failure. The module
70pointer should be the module interested in using this facility (almost
71certainly THIS_MODULE).
David Howells8f0aa2f2009-04-03 16:42:35 +010072
73
74Slow work items may then be set up by:
75
76 (1) Declaring a slow_work struct type variable:
77
78 #include <linux/slow-work.h>
79
80 struct slow_work myitem;
81
82 (2) Declaring the operations to be used for this item:
83
84 struct slow_work_ops myitem_ops = {
85 .get_ref = myitem_get_ref,
86 .put_ref = myitem_put_ref,
87 .execute = myitem_execute,
88 };
89
90 [*] For a description of the ops, see section "Item Operations".
91
92 (3) Initialising the item:
93
94 slow_work_init(&myitem, &myitem_ops);
95
96 or:
97
98 vslow_work_init(&myitem, &myitem_ops);
99
100 depending on its class.
101
102A suitably set up work item can then be enqueued for processing:
103
104 int ret = slow_work_enqueue(&myitem);
105
106This will return a -ve error if the thread pool is unable to gain a reference
107on the item, 0 otherwise.
108
109
110The items are reference counted, so there ought to be no need for a flush
111operation. When all a module's slow work items have been processed, and the
112module has no further interest in the facility, it should unregister its
113interest:
114
David Howells3d7a6412009-11-19 18:10:23 +0000115 slow_work_unregister_user(struct module *module);
116
117The module pointer is used to wait for all outstanding work items for that
118module before completing the unregistration. This prevents the put_ref() code
119from being taken away before it completes. module should almost certainly be
120THIS_MODULE.
David Howells8f0aa2f2009-04-03 16:42:35 +0100121
122
123===============
124ITEM OPERATIONS
125===============
126
127Each work item requires a table of operations of type struct slow_work_ops.
128All members are required:
129
130 (*) Get a reference on an item:
131
132 int (*get_ref)(struct slow_work *work);
133
134 This allows the thread pool to attempt to pin an item by getting a
135 reference on it. This function should return 0 if the reference was
136 granted, or a -ve error otherwise. If an error is returned,
137 slow_work_enqueue() will fail.
138
139 The reference is held whilst the item is queued and whilst it is being
140 executed. The item may then be requeued with the same reference held, or
141 the reference will be released.
142
143 (*) Release a reference on an item:
144
145 void (*put_ref)(struct slow_work *work);
146
147 This allows the thread pool to unpin an item by releasing the reference on
148 it. The thread pool will not touch the item again once this has been
149 called.
150
151 (*) Execute an item:
152
153 void (*execute)(struct slow_work *work);
154
155 This should perform the work required of the item. It may sleep, it may
156 perform disk I/O and it may wait for locks.
157
158
159==================
160POOL CONFIGURATION
161==================
162
163The slow-work thread pool has a number of configurables:
164
165 (*) /proc/sys/kernel/slow-work/min-threads
166
167 The minimum number of threads that should be in the pool whilst it is in
168 use. This may be anywhere between 2 and max-threads.
169
170 (*) /proc/sys/kernel/slow-work/max-threads
171
172 The maximum number of threads that should in the pool. This may be
173 anywhere between min-threads and 255 or NR_CPUS * 2, whichever is greater.
174
175 (*) /proc/sys/kernel/slow-work/vslow-percentage
176
177 The percentage of active threads in the pool that may be used to execute
178 very slow work items. This may be between 1 and 99. The resultant number
179 is bounded to between 1 and one fewer than the number of active threads.
180 This ensures there is always at least one thread that can process very
181 slow work items, and always at least one thread that won't.