blob: 2dcaf9adb7a75d3e80ced6dd7edab52a0c181915 [file] [log] [blame]
Paul E. McKenney0c87f9b2013-03-14 16:27:31 -07001 NO_HZ: Reducing Scheduling-Clock Ticks
2
3
4This document describes Kconfig options and boot parameters that can
5reduce the number of scheduling-clock interrupts, thereby improving energy
6efficiency and reducing OS jitter. Reducing OS jitter is important for
7some types of computationally intensive high-performance computing (HPC)
8applications and for real-time applications.
9
Paul E. McKenney295fde82013-04-29 10:09:41 -070010There are three main ways of managing scheduling-clock interrupts
11(also known as "scheduling-clock ticks" or simply "ticks"):
Paul E. McKenney0c87f9b2013-03-14 16:27:31 -070012
Paul E. McKenney295fde82013-04-29 10:09:41 -0700131. Never omit scheduling-clock ticks (CONFIG_HZ_PERIODIC=y or
14 CONFIG_NO_HZ=n for older kernels). You normally will -not-
15 want to choose this option.
Paul E. McKenney0c87f9b2013-03-14 16:27:31 -070016
Paul E. McKenney295fde82013-04-29 10:09:41 -0700172. Omit scheduling-clock ticks on idle CPUs (CONFIG_NO_HZ_IDLE=y or
18 CONFIG_NO_HZ=y for older kernels). This is the most common
19 approach, and should be the default.
Paul E. McKenney0c87f9b2013-03-14 16:27:31 -070020
Paul E. McKenney295fde82013-04-29 10:09:41 -0700213. Omit scheduling-clock ticks on CPUs that are either idle or that
22 have only one runnable task (CONFIG_NO_HZ_FULL=y). Unless you
23 are running realtime applications or certain types of HPC
24 workloads, you will normally -not- want this option.
25
26These three cases are described in the following three sections, followed
Paul E. McKenney8bdf7a22013-06-18 11:15:21 -070027by a third section on RCU-specific considerations, a fourth section
28discussing testing, and a fifth and final section listing known issues.
Paul E. McKenney0c87f9b2013-03-14 16:27:31 -070029
30
Paul E. McKenney295fde82013-04-29 10:09:41 -070031NEVER OMIT SCHEDULING-CLOCK TICKS
32
33Very old versions of Linux from the 1990s and the very early 2000s
34are incapable of omitting scheduling-clock ticks. It turns out that
35there are some situations where this old-school approach is still the
36right approach, for example, in heavy workloads with lots of tasks
37that use short bursts of CPU, where there are very frequent idle
38periods, but where these idle periods are also quite short (tens or
39hundreds of microseconds). For these types of workloads, scheduling
40clock interrupts will normally be delivered any way because there
41will frequently be multiple runnable tasks per CPU. In these cases,
42attempting to turn off the scheduling clock interrupt will have no effect
43other than increasing the overhead of switching to and from idle and
44transitioning between user and kernel execution.
45
46This mode of operation can be selected using CONFIG_HZ_PERIODIC=y (or
47CONFIG_NO_HZ=n for older kernels).
48
49However, if you are instead running a light workload with long idle
50periods, failing to omit scheduling-clock interrupts will result in
51excessive power consumption. This is especially bad on battery-powered
52devices, where it results in extremely short battery lifetimes. If you
53are running light workloads, you should therefore read the following
54section.
55
56In addition, if you are running either a real-time workload or an HPC
57workload with short iterations, the scheduling-clock interrupts can
58degrade your applications performance. If this describes your workload,
59you should read the following two sections.
60
61
62OMIT SCHEDULING-CLOCK TICKS FOR IDLE CPUs
Paul E. McKenney0c87f9b2013-03-14 16:27:31 -070063
64If a CPU is idle, there is little point in sending it a scheduling-clock
65interrupt. After all, the primary purpose of a scheduling-clock interrupt
66is to force a busy CPU to shift its attention among multiple duties,
67and an idle CPU has no duties to shift its attention among.
68
69The CONFIG_NO_HZ_IDLE=y Kconfig option causes the kernel to avoid sending
70scheduling-clock interrupts to idle CPUs, which is critically important
71both to battery-powered devices and to highly virtualized mainframes.
72A battery-powered device running a CONFIG_HZ_PERIODIC=y kernel would
73drain its battery very quickly, easily 2-3 times as fast as would the
74same device running a CONFIG_NO_HZ_IDLE=y kernel. A mainframe running
751,500 OS instances might find that half of its CPU time was consumed by
76unnecessary scheduling-clock interrupts. In these situations, there
77is strong motivation to avoid sending scheduling-clock interrupts to
78idle CPUs. That said, dyntick-idle mode is not free:
79
801. It increases the number of instructions executed on the path
81 to and from the idle loop.
82
832. On many architectures, dyntick-idle mode also increases the
84 number of expensive clock-reprogramming operations.
85
86Therefore, systems with aggressive real-time response constraints often
87run CONFIG_HZ_PERIODIC=y kernels (or CONFIG_NO_HZ=n for older kernels)
88in order to avoid degrading from-idle transition latencies.
89
90An idle CPU that is not receiving scheduling-clock interrupts is said to
91be "dyntick-idle", "in dyntick-idle mode", "in nohz mode", or "running
92tickless". The remainder of this document will use "dyntick-idle mode".
93
94There is also a boot parameter "nohz=" that can be used to disable
95dyntick-idle mode in CONFIG_NO_HZ_IDLE=y kernels by specifying "nohz=off".
96By default, CONFIG_NO_HZ_IDLE=y kernels boot with "nohz=on", enabling
97dyntick-idle mode.
98
99
Paul E. McKenney295fde82013-04-29 10:09:41 -0700100OMIT SCHEDULING-CLOCK TICKS FOR CPUs WITH ONLY ONE RUNNABLE TASK
Paul E. McKenney0c87f9b2013-03-14 16:27:31 -0700101
102If a CPU has only one runnable task, there is little point in sending it
103a scheduling-clock interrupt because there is no other task to switch to.
Paul E. McKenney295fde82013-04-29 10:09:41 -0700104Note that omitting scheduling-clock ticks for CPUs with only one runnable
105task implies also omitting them for idle CPUs.
Paul E. McKenney0c87f9b2013-03-14 16:27:31 -0700106
107The CONFIG_NO_HZ_FULL=y Kconfig option causes the kernel to avoid
108sending scheduling-clock interrupts to CPUs with a single runnable task,
109and such CPUs are said to be "adaptive-ticks CPUs". This is important
110for applications with aggressive real-time response constraints because
111it allows them to improve their worst-case response times by the maximum
112duration of a scheduling-clock interrupt. It is also important for
113computationally intensive short-iteration workloads: If any CPU is
114delayed during a given iteration, all the other CPUs will be forced to
115wait idle while the delayed CPU finishes. Thus, the delay is multiplied
116by one less than the number of CPUs. In these situations, there is
117again strong motivation to avoid sending scheduling-clock interrupts.
118
119By default, no CPU will be an adaptive-ticks CPU. The "nohz_full="
120boot parameter specifies the adaptive-ticks CPUs. For example,
121"nohz_full=1,6-8" says that CPUs 1, 6, 7, and 8 are to be adaptive-ticks
122CPUs. Note that you are prohibited from marking all of the CPUs as
123adaptive-tick CPUs: At least one non-adaptive-tick CPU must remain
Paul E. McKenney8bdf7a22013-06-18 11:15:21 -0700124online to handle timekeeping tasks in order to ensure that system
125calls like gettimeofday() returns accurate values on adaptive-tick CPUs.
126(This is not an issue for CONFIG_NO_HZ_IDLE=y because there are no running
127user processes to observe slight drifts in clock rate.) Therefore, the
128boot CPU is prohibited from entering adaptive-ticks mode. Specifying a
129"nohz_full=" mask that includes the boot CPU will result in a boot-time
130error message, and the boot CPU will be removed from the mask. Note that
131this means that your system must have at least two CPUs in order for
132CONFIG_NO_HZ_FULL=y to do anything for you.
Paul E. McKenney0c87f9b2013-03-14 16:27:31 -0700133
134Alternatively, the CONFIG_NO_HZ_FULL_ALL=y Kconfig parameter specifies
135that all CPUs other than the boot CPU are adaptive-ticks CPUs. This
136Kconfig parameter will be overridden by the "nohz_full=" boot parameter,
137so that if both the CONFIG_NO_HZ_FULL_ALL=y Kconfig parameter and
138the "nohz_full=1" boot parameter is specified, the boot parameter will
139prevail so that only CPU 1 will be an adaptive-ticks CPU.
140
141Finally, adaptive-ticks CPUs must have their RCU callbacks offloaded.
142This is covered in the "RCU IMPLICATIONS" section below.
143
144Normally, a CPU remains in adaptive-ticks mode as long as possible.
145In particular, transitioning to kernel mode does not automatically change
146the mode. Instead, the CPU will exit adaptive-ticks mode only if needed,
147for example, if that CPU enqueues an RCU callback.
148
149Just as with dyntick-idle mode, the benefits of adaptive-tick mode do
150not come for free:
151
1521. CONFIG_NO_HZ_FULL selects CONFIG_NO_HZ_COMMON, so you cannot run
153 adaptive ticks without also running dyntick idle. This dependency
154 extends down into the implementation, so that all of the costs
155 of CONFIG_NO_HZ_IDLE are also incurred by CONFIG_NO_HZ_FULL.
156
1572. The user/kernel transitions are slightly more expensive due
158 to the need to inform kernel subsystems (such as RCU) about
159 the change in mode.
160
Paul E. McKenneyc2519782015-01-25 11:28:28 -08001613. POSIX CPU timers prevent CPUs from entering adaptive-tick mode.
162 Real-time applications needing to take actions based on CPU time
163 consumption need to use other means of doing so.
Paul E. McKenney0c87f9b2013-03-14 16:27:31 -0700164
1654. If there are more perf events pending than the hardware can
166 accommodate, they are normally round-robined so as to collect
167 all of them over time. Adaptive-tick mode may prevent this
168 round-robining from happening. This will likely be fixed by
169 preventing CPUs with large numbers of perf events pending from
170 entering adaptive-tick mode.
171
1725. Scheduler statistics for adaptive-tick CPUs may be computed
173 slightly differently than those for non-adaptive-tick CPUs.
174 This might in turn perturb load-balancing of real-time tasks.
175
1766. The LB_BIAS scheduler feature is disabled by adaptive ticks.
177
178Although improvements are expected over time, adaptive ticks is quite
179useful for many types of real-time and compute-intensive applications.
180However, the drawbacks listed above mean that adaptive ticks should not
181(yet) be enabled by default.
182
183
184RCU IMPLICATIONS
185
186There are situations in which idle CPUs cannot be permitted to
187enter either dyntick-idle mode or adaptive-tick mode, the most
188common being when that CPU has RCU callbacks pending.
189
190The CONFIG_RCU_FAST_NO_HZ=y Kconfig option may be used to cause such CPUs
191to enter dyntick-idle mode or adaptive-tick mode anyway. In this case,
192a timer will awaken these CPUs every four jiffies in order to ensure
193that the RCU callbacks are processed in a timely fashion.
194
195Another approach is to offload RCU callback processing to "rcuo" kthreads
196using the CONFIG_RCU_NOCB_CPU=y Kconfig option. The specific CPUs to
Paul E. McKenney44c65ff2017-05-15 16:26:34 -0700197offload may be selected using The "rcu_nocbs=" kernel boot parameter,
198which takes a comma-separated list of CPUs and CPU ranges, for example,
199"1,3-5" selects CPUs 1, 3, 4, and 5.
Paul E. McKenney0c87f9b2013-03-14 16:27:31 -0700200
201The offloaded CPUs will never queue RCU callbacks, and therefore RCU
202never prevents offloaded CPUs from entering either dyntick-idle mode
203or adaptive-tick mode. That said, note that it is up to userspace to
204pin the "rcuo" kthreads to specific CPUs if desired. Otherwise, the
205scheduler will decide where to run them, which might or might not be
206where you want them to run.
207
208
Paul E. McKenney8bdf7a22013-06-18 11:15:21 -0700209TESTING
210
211So you enable all the OS-jitter features described in this document,
212but do not see any change in your workload's behavior. Is this because
213your workload isn't affected that much by OS jitter, or is it because
214something else is in the way? This section helps answer this question
215by providing a simple OS-jitter test suite, which is available on branch
216master of the following git archive:
217
218git://git.kernel.org/pub/scm/linux/kernel/git/frederic/dynticks-testing.git
219
220Clone this archive and follow the instructions in the README file.
221This test procedure will produce a trace that will allow you to evaluate
222whether or not you have succeeded in removing OS jitter from your system.
223If this trace shows that you have removed OS jitter as much as is
224possible, then you can conclude that your workload is not all that
225sensitive to OS jitter.
226
227Note: this test requires that your system have at least two CPUs.
228We do not currently have a good way to remove OS jitter from single-CPU
229systems.
230
231
Paul E. McKenney0c87f9b2013-03-14 16:27:31 -0700232KNOWN ISSUES
233
234o Dyntick-idle slows transitions to and from idle slightly.
235 In practice, this has not been a problem except for the most
236 aggressive real-time workloads, which have the option of disabling
237 dyntick-idle mode, an option that most of them take. However,
238 some workloads will no doubt want to use adaptive ticks to
239 eliminate scheduling-clock interrupt latencies. Here are some
240 options for these workloads:
241
242 a. Use PMQOS from userspace to inform the kernel of your
243 latency requirements (preferred).
244
245 b. On x86 systems, use the "idle=mwait" boot parameter.
246
247 c. On x86 systems, use the "intel_idle.max_cstate=" to limit
248 ` the maximum C-state depth.
249
250 d. On x86 systems, use the "idle=poll" boot parameter.
251 However, please note that use of this parameter can cause
252 your CPU to overheat, which may cause thermal throttling
253 to degrade your latencies -- and that this degradation can
254 be even worse than that of dyntick-idle. Furthermore,
255 this parameter effectively disables Turbo Mode on Intel
256 CPUs, which can significantly reduce maximum performance.
257
258o Adaptive-ticks slows user/kernel transitions slightly.
259 This is not expected to be a problem for computationally intensive
260 workloads, which have few such transitions. Careful benchmarking
261 will be required to determine whether or not other workloads
262 are significantly affected by this effect.
263
264o Adaptive-ticks does not do anything unless there is only one
265 runnable task for a given CPU, even though there are a number
266 of other situations where the scheduling-clock tick is not
267 needed. To give but one example, consider a CPU that has one
268 runnable high-priority SCHED_FIFO task and an arbitrary number
269 of low-priority SCHED_OTHER tasks. In this case, the CPU is
270 required to run the SCHED_FIFO task until it either blocks or
271 some other higher-priority task awakens on (or is assigned to)
272 this CPU, so there is no point in sending a scheduling-clock
273 interrupt to this CPU. However, the current implementation
274 nevertheless sends scheduling-clock interrupts to CPUs having a
275 single runnable SCHED_FIFO task and multiple runnable SCHED_OTHER
276 tasks, even though these interrupts are unnecessary.
277
Paul E. McKenneyce5f4fc2013-05-13 10:32:10 -0700278 And even when there are multiple runnable tasks on a given CPU,
279 there is little point in interrupting that CPU until the current
280 running task's timeslice expires, which is almost always way
281 longer than the time of the next scheduling-clock interrupt.
282
Paul E. McKenney0c87f9b2013-03-14 16:27:31 -0700283 Better handling of these sorts of situations is future work.
284
285o A reboot is required to reconfigure both adaptive idle and RCU
286 callback offloading. Runtime reconfiguration could be provided
287 if needed, however, due to the complexity of reconfiguring RCU at
288 runtime, there would need to be an earthshakingly good reason.
289 Especially given that you have the straightforward option of
290 simply offloading RCU callbacks from all CPUs and pinning them
291 where you want them whenever you want them pinned.
292
293o Additional configuration is required to deal with other sources
294 of OS jitter, including interrupts and system-utility tasks
295 and processes. This configuration normally involves binding
296 interrupts and tasks to particular CPUs.
297
298o Some sources of OS jitter can currently be eliminated only by
299 constraining the workload. For example, the only way to eliminate
300 OS jitter due to global TLB shootdowns is to avoid the unmapping
301 operations (such as kernel module unload operations) that
302 result in these shootdowns. For another example, page faults
303 and TLB misses can be reduced (and in some cases eliminated) by
304 using huge pages and by constraining the amount of memory used
305 by the application. Pre-faulting the working set can also be
306 helpful, especially when combined with the mlock() and mlockall()
307 system calls.
308
309o Unless all CPUs are idle, at least one CPU must keep the
310 scheduling-clock interrupt going in order to support accurate
311 timekeeping.
312
Paul E. McKenneyce5f4fc2013-05-13 10:32:10 -0700313o If there might potentially be some adaptive-ticks CPUs, there
314 will be at least one CPU keeping the scheduling-clock interrupt
315 going, even if all CPUs are otherwise idle.
316
317 Better handling of this situation is ongoing work.
318
319o Some process-handling operations still require the occasional
320 scheduling-clock tick. These operations include calculating CPU
321 load, maintaining sched average, computing CFS entity vruntime,
322 computing avenrun, and carrying out load balancing. They are
323 currently accommodated by scheduling-clock tick every second
324 or so. On-going work will eliminate the need even for these
325 infrequent scheduling-clock ticks.