blob: c776968f446300db6ff7d43695c0828e222d6694 [file] [log] [blame]
Paul E. McKenney64db4cf2008-12-18 21:55:32 +01001CONFIG_RCU_TRACE debugfs Files and Formats
2
3
Paul E. McKenney8e79e1f2010-11-04 14:31:19 -07004The rcutree and rcutiny implementations of RCU provide debugfs trace
5output that summarizes counters and state. This information is useful for
6debugging RCU itself, and can sometimes also help to debug abuses of RCU.
7The following sections describe the debugfs files and formats, first
8for rcutree and next for rcutiny.
Paul E. McKenney64db4cf2008-12-18 21:55:32 +01009
10
Paul E. McKenney8e79e1f2010-11-04 14:31:19 -070011CONFIG_TREE_RCU and CONFIG_TREE_PREEMPT_RCU debugfs Files and Formats
Paul E. McKenney64db4cf2008-12-18 21:55:32 +010012
Paul E. McKenney40e80c42012-10-31 13:00:15 -070013These implementations of RCU provide several debugfs directories under the
Paul E. McKenney90e6ac32011-04-06 15:20:47 -070014top-level directory "rcu":
15
Paul E. McKenney40e80c42012-10-31 13:00:15 -070016rcu/rcu_bh
17rcu/rcu_preempt
18rcu/rcu_sched
19
20Each directory contains files for the corresponding flavor of RCU.
21Note that rcu/rcu_preempt is only present for CONFIG_TREE_PREEMPT_RCU.
22For CONFIG_TREE_RCU, the RCU flavor maps onto the RCU-sched flavor,
23so that activity for both appears in rcu/rcu_sched.
24
25In addition, the following file appears in the top-level directory:
26rcu/rcutorture. This file displays rcutorture test progress. The output
27of "cat rcu/rcutorture" looks as follows:
28
29rcutorture test sequence: 0 (test in progress)
30rcutorture update version number: 615
31
32The first line shows the number of rcutorture tests that have completed
33since boot. If a test is currently running, the "(test in progress)"
34string will appear as shown above. The second line shows the number of
35update cycles that the current test has started, or zero if there is
36no test in progress.
37
38
39Within each flavor directory (rcu/rcu_bh, rcu/rcu_sched, and possibly
40also rcu/rcu_preempt) the following files will be present:
41
42rcudata:
Paul E. McKenney90e6ac32011-04-06 15:20:47 -070043 Displays fields in struct rcu_data.
Paul E. McKenneyd484a212012-10-31 13:22:08 -070044rcuexp:
45 Displays statistics for expedited grace periods.
Paul E. McKenney40e80c42012-10-31 13:00:15 -070046rcugp:
Paul E. McKenney90e6ac32011-04-06 15:20:47 -070047 Displays grace-period counters.
Paul E. McKenney40e80c42012-10-31 13:00:15 -070048rcuhier:
Paul E. McKenney90e6ac32011-04-06 15:20:47 -070049 Displays the struct rcu_node hierarchy.
Paul E. McKenney40e80c42012-10-31 13:00:15 -070050rcu_pending:
Paul E. McKenney90e6ac32011-04-06 15:20:47 -070051 Displays counts of the reasons rcu_pending() decided that RCU had
52 work to do.
Paul E. McKenney40e80c42012-10-31 13:00:15 -070053rcuboost:
Paul E. McKenney90e6ac32011-04-06 15:20:47 -070054 Displays RCU boosting statistics. Only present if
55 CONFIG_RCU_BOOST=y.
Paul E. McKenney64db4cf2008-12-18 21:55:32 +010056
Paul E. McKenney40e80c42012-10-31 13:00:15 -070057The output of "cat rcu/rcu_preempt/rcudata" looks as follows:
Paul E. McKenney64db4cf2008-12-18 21:55:32 +010058
Paul E. McKenney40e80c42012-10-31 13:00:15 -070059 0!c=30455 g=30456 pq=1 qp=1 dt=126535/140000000000000/0 df=2002 of=4 ql=0/0 qs=N... b=10 ci=74572 nci=0 co=1131 ca=716
60 1!c=30719 g=30720 pq=1 qp=0 dt=132007/140000000000000/0 df=1874 of=10 ql=0/0 qs=N... b=10 ci=123209 nci=0 co=685 ca=982
61 2!c=30150 g=30151 pq=1 qp=1 dt=138537/140000000000000/0 df=1707 of=8 ql=0/0 qs=N... b=10 ci=80132 nci=0 co=1328 ca=1458
62 3 c=31249 g=31250 pq=1 qp=0 dt=107255/140000000000000/0 df=1749 of=6 ql=0/450 qs=NRW. b=10 ci=151700 nci=0 co=509 ca=622
63 4!c=29502 g=29503 pq=1 qp=1 dt=83647/140000000000000/0 df=965 of=5 ql=0/0 qs=N... b=10 ci=65643 nci=0 co=1373 ca=1521
64 5 c=31201 g=31202 pq=1 qp=1 dt=70422/0/0 df=535 of=7 ql=0/0 qs=.... b=10 ci=58500 nci=0 co=764 ca=698
65 6!c=30253 g=30254 pq=1 qp=1 dt=95363/140000000000000/0 df=780 of=5 ql=0/0 qs=N... b=10 ci=100607 nci=0 co=1414 ca=1353
66 7 c=31178 g=31178 pq=1 qp=0 dt=91536/0/0 df=547 of=4 ql=0/0 qs=.... b=10 ci=109819 nci=0 co=1115 ca=969
Paul E. McKenney64db4cf2008-12-18 21:55:32 +010067
Paul E. McKenney40e80c42012-10-31 13:00:15 -070068This file has one line per CPU, or eight for this 8-CPU system.
69The fields are as follows:
Paul E. McKenney64db4cf2008-12-18 21:55:32 +010070
71o The number at the beginning of each line is the CPU number.
72 CPUs numbers followed by an exclamation mark are offline,
73 but have been online at least once since boot. There will be
74 no output for CPUs that have never been online, which can be
75 a good thing in the surprisingly common case where NR_CPUS is
76 substantially larger than the number of actual CPUs.
77
78o "c" is the count of grace periods that this CPU believes have
Paul E. McKenney40e80c42012-10-31 13:00:15 -070079 completed. Offlined CPUs and CPUs in dynticks idle mode may lag
80 quite a ways behind, for example, CPU 4 under "rcu_sched" above,
81 which has been offline through 16 RCU grace periods. It is not
82 unusual to see offline CPUs lagging by thousands of grace periods.
83 Note that although the grace-period number is an unsigned long,
84 it is printed out as a signed long to allow more human-friendly
85 representation near boot time.
Paul E. McKenney64db4cf2008-12-18 21:55:32 +010086
87o "g" is the count of grace periods that this CPU believes have
Paul E. McKenney2fa218d2011-03-27 21:37:58 -070088 started. Again, offlined CPUs and CPUs in dynticks idle mode
89 may lag behind. If the "c" and "g" values are equal, this CPU
90 has already reported a quiescent state for the last RCU grace
91 period that it is aware of, otherwise, the CPU believes that it
92 owes RCU a quiescent state.
Paul E. McKenney64db4cf2008-12-18 21:55:32 +010093
94o "pq" indicates that this CPU has passed through a quiescent state
95 for the current grace period. It is possible for "pq" to be
96 "1" and "c" different than "g", which indicates that although
97 the CPU has passed through a quiescent state, either (1) this
98 CPU has not yet reported that fact, (2) some other CPU has not
99 yet reported for this grace period, or (3) both.
100
Paul E. McKenney64db4cf2008-12-18 21:55:32 +0100101o "qp" indicates that RCU still expects a quiescent state from
Paul E. McKenney2fa218d2011-03-27 21:37:58 -0700102 this CPU. Offlined CPUs and CPUs in dyntick idle mode might
103 well have qp=1, which is OK: RCU is still ignoring them.
Paul E. McKenney64db4cf2008-12-18 21:55:32 +0100104
Paul E. McKenney64db4cf2008-12-18 21:55:32 +0100105o "dt" is the current value of the dyntick counter that is incremented
Paul E. McKenney40e80c42012-10-31 13:00:15 -0700106 when entering or leaving idle, either due to a context switch or
107 due to an interrupt. This number is even if the CPU is in idle
108 from RCU's viewpoint and odd otherwise. The number after the
109 first "/" is the interrupt nesting depth when in idle state,
110 or a large number added to the interrupt-nesting depth when
111 running a non-idle task. Some architectures do not accurately
112 count interrupt nesting when running in non-idle kernel context,
113 which can result in interesting anomalies such as negative
114 interrupt-nesting levels. The number after the second "/"
115 is the NMI nesting depth.
Paul E. McKenney64db4cf2008-12-18 21:55:32 +0100116
Paul E. McKenney64db4cf2008-12-18 21:55:32 +0100117o "df" is the number of times that some other CPU has forced a
118 quiescent state on behalf of this CPU due to this CPU being in
Paul E. McKenney40e80c42012-10-31 13:00:15 -0700119 idle state.
Paul E. McKenney64db4cf2008-12-18 21:55:32 +0100120
Paul E. McKenney64db4cf2008-12-18 21:55:32 +0100121o "of" is the number of times that some other CPU has forced a
122 quiescent state on behalf of this CPU due to this CPU being
Paul E. McKenney2fa218d2011-03-27 21:37:58 -0700123 offline. In a perfect world, this might never happen, but it
Paul E. McKenney64db4cf2008-12-18 21:55:32 +0100124 turns out that offlining and onlining a CPU can take several grace
125 periods, and so there is likely to be an extended period of time
126 when RCU believes that the CPU is online when it really is not.
127 Please note that erring in the other direction (RCU believing a
128 CPU is offline when it is really alive and kicking) is a fatal
129 error, so it makes sense to err conservatively.
130
Paul E. McKenney64db4cf2008-12-18 21:55:32 +0100131o "ql" is the number of RCU callbacks currently residing on
Paul E. McKenney40e80c42012-10-31 13:00:15 -0700132 this CPU. The first number is the number of "lazy" callbacks
133 that are known to RCU to only be freeing memory, and the number
134 after the "/" is the total number of callbacks, lazy or not.
135 These counters count callbacks regardless of what phase of
136 grace-period processing that they are in (new, waiting for
137 grace period to start, waiting for grace period to end, ready
138 to invoke).
Paul E. McKenney64db4cf2008-12-18 21:55:32 +0100139
Paul E. McKenney0ac3d132011-03-28 15:47:07 -0700140o "qs" gives an indication of the state of the callback queue
141 with four characters:
142
143 "N" Indicates that there are callbacks queued that are not
144 ready to be handled by the next grace period, and thus
145 will be handled by the grace period following the next
146 one.
147
148 "R" Indicates that there are callbacks queued that are
149 ready to be handled by the next grace period.
150
151 "W" Indicates that there are callbacks queued that are
152 waiting on the current grace period.
153
154 "D" Indicates that there are callbacks queued that have
155 already been handled by a prior grace period, and are
156 thus waiting to be invoked. Note that callbacks in
157 the process of being invoked are not counted here.
158 Callbacks in the process of being invoked are those
159 that have been removed from the rcu_data structures
160 queues by rcu_do_batch(), but which have not yet been
161 invoked.
162
163 If there are no callbacks in a given one of the above states,
164 the corresponding character is replaced by ".".
165
Paul E. McKenney40e80c42012-10-31 13:00:15 -0700166o "b" is the batch limit for this CPU. If more than this number
167 of RCU callbacks is ready to invoke, then the remainder will
168 be deferred.
169
170o "ci" is the number of RCU callbacks that have been invoked for
171 this CPU. Note that ci+nci+ql is the number of callbacks that have
172 been registered in absence of CPU-hotplug activity.
173
174o "nci" is the number of RCU callbacks that have been offloaded from
175 this CPU. This will always be zero unless the kernel was built
176 with CONFIG_RCU_NOCB_CPU=y and the "rcu_nocbs=" kernel boot
177 parameter was specified.
178
179o "co" is the number of RCU callbacks that have been orphaned due to
180 this CPU going offline. These orphaned callbacks have been moved
181 to an arbitrarily chosen online CPU.
182
183o "ca" is the number of RCU callbacks that have been adopted by this
184 CPU due to other CPUs going offline. Note that ci+co-ca+ql is
185 the number of RCU callbacks registered on this CPU.
186
187
188Kernels compiled with CONFIG_RCU_BOOST=y display the following from
189/debug/rcu/rcu_preempt/rcudata:
190
191 0!c=12865 g=12866 pq=1 qp=1 dt=83113/140000000000000/0 df=288 of=11 ql=0/0 qs=N... kt=0/O ktl=944 b=10 ci=60709 nci=0 co=748 ca=871
192 1 c=14407 g=14408 pq=1 qp=0 dt=100679/140000000000000/0 df=378 of=7 ql=0/119 qs=NRW. kt=0/W ktl=9b6 b=10 ci=109740 nci=0 co=589 ca=485
193 2 c=14407 g=14408 pq=1 qp=0 dt=105486/0/0 df=90 of=9 ql=0/89 qs=NRW. kt=0/W ktl=c0c b=10 ci=83113 nci=0 co=533 ca=490
194 3 c=14407 g=14408 pq=1 qp=0 dt=107138/0/0 df=142 of=8 ql=0/188 qs=NRW. kt=0/W ktl=b96 b=10 ci=121114 nci=0 co=426 ca=290
195 4 c=14405 g=14406 pq=1 qp=1 dt=50238/0/0 df=706 of=7 ql=0/0 qs=.... kt=0/W ktl=812 b=10 ci=34929 nci=0 co=643 ca=114
196 5!c=14168 g=14169 pq=1 qp=0 dt=45465/140000000000000/0 df=161 of=11 ql=0/0 qs=N... kt=0/O ktl=b4d b=10 ci=47712 nci=0 co=677 ca=722
197 6 c=14404 g=14405 pq=1 qp=0 dt=59454/0/0 df=94 of=6 ql=0/0 qs=.... kt=0/W ktl=e57 b=10 ci=55597 nci=0 co=701 ca=811
198 7 c=14407 g=14408 pq=1 qp=1 dt=68850/0/0 df=31 of=8 ql=0/0 qs=.... kt=0/W ktl=14bd b=10 ci=77475 nci=0 co=508 ca=1042
199
200This is similar to the output discussed above, but contains the following
201additional fields:
202
Paul E. McKenney90e6ac32011-04-06 15:20:47 -0700203o "kt" is the per-CPU kernel-thread state. The digit preceding
Paul E. McKenney15ba0ba2011-04-06 16:01:16 -0700204 the first slash is zero if there is no work pending and 1
Paul E. McKenney5ece5ba2011-04-22 18:08:51 -0700205 otherwise. The character between the first pair of slashes is
206 as follows:
Paul E. McKenney90e6ac32011-04-06 15:20:47 -0700207
208 "S" The kernel thread is stopped, in other words, all
209 CPUs corresponding to this rcu_node structure are
210 offline.
211
212 "R" The kernel thread is running.
213
214 "W" The kernel thread is waiting because there is no work
215 for it to do.
216
Paul E. McKenney15ba0ba2011-04-06 16:01:16 -0700217 "O" The kernel thread is waiting because it has been
218 forced off of its designated CPU or because its
219 ->cpus_allowed mask permits it to run on other than
220 its designated CPU.
221
Paul E. McKenney90e6ac32011-04-06 15:20:47 -0700222 "Y" The kernel thread is yielding to avoid hogging CPU.
223
224 "?" Unknown value, indicates a bug.
225
Paul E. McKenney15ba0ba2011-04-06 16:01:16 -0700226 The number after the final slash is the CPU that the kthread
227 is actually running on.
228
Paul E. McKenneyd5988af2011-06-15 23:07:55 -0700229 This field is displayed only for CONFIG_RCU_BOOST kernels.
230
Paul E. McKenney5ece5ba2011-04-22 18:08:51 -0700231o "ktl" is the low-order 16 bits (in hexadecimal) of the count of
232 the number of times that this CPU's per-CPU kthread has gone
233 through its loop servicing invoke_rcu_cpu_kthread() requests.
234
Paul E. McKenneyd5988af2011-06-15 23:07:55 -0700235 This field is displayed only for CONFIG_RCU_BOOST kernels.
236
Paul E. McKenney64db4cf2008-12-18 21:55:32 +0100237
Paul E. McKenneyd484a212012-10-31 13:22:08 -0700238The output of "cat rcu/rcu_preempt/rcuexp" looks as follows:
239
240s=21872 d=21872 w=0 tf=0 wd1=0 wd2=0 n=0 sc=21872 dt=21872 dl=0 dx=21872
241
242These fields are as follows:
243
244o "s" is the starting sequence number.
245
246o "d" is the ending sequence number. When the starting and ending
247 numbers differ, there is an expedited grace period in progress.
248
249o "w" is the number of times that the sequence numbers have been
250 in danger of wrapping.
251
252o "tf" is the number of times that contention has resulted in a
253 failure to begin an expedited grace period.
254
255o "wd1" and "wd2" are the number of times that an attempt to
256 start an expedited grace period found that someone else had
257 completed an expedited grace period that satisfies the
258 attempted request. "Our work is done."
259
260o "n" is number of times that contention was so great that
261 the request was demoted from an expedited grace period to
262 a normal grace period.
263
264o "sc" is the number of times that the attempt to start a
265 new expedited grace period succeeded.
266
267o "dt" is the number of times that we attempted to update
268 the "d" counter.
269
270o "dl" is the number of times that we failed to update the "d"
271 counter.
272
273o "dx" is the number of times that we succeeded in updating
274 the "d" counter.
275
276
Paul E. McKenney40e80c42012-10-31 13:00:15 -0700277The output of "cat rcu/rcu_preempt/rcugp" looks as follows:
Paul E. McKenney269dcc12010-09-07 14:23:09 -0700278
Paul E. McKenney40e80c42012-10-31 13:00:15 -0700279completed=31249 gpnum=31250 age=1 max=18
Paul E. McKenney269dcc12010-09-07 14:23:09 -0700280
Paul E. McKenney40e80c42012-10-31 13:00:15 -0700281These fields are taken from the rcu_state structure, and are as follows:
Paul E. McKenney64db4cf2008-12-18 21:55:32 +0100282
283o "completed" is the number of grace periods that have completed.
284 It is comparable to the "c" field from rcu/rcudata in that a
285 CPU whose "c" field matches the value of "completed" is aware
286 that the corresponding RCU grace period has completed.
287
288o "gpnum" is the number of grace periods that have started. It is
Paul E. McKenney40e80c42012-10-31 13:00:15 -0700289 similarly comparable to the "g" field from rcu/rcudata in that
290 a CPU whose "g" field matches the value of "gpnum" is aware that
291 the corresponding RCU grace period has started.
Paul E. McKenney64db4cf2008-12-18 21:55:32 +0100292
Paul E. McKenney40e80c42012-10-31 13:00:15 -0700293 If these two fields are equal, then there is no grace period
294 in progress, in other words, RCU is idle. On the other hand,
295 if the two fields differ (as they are above), then an RCU grace
296 period is in progress.
Paul E. McKenney64db4cf2008-12-18 21:55:32 +0100297
Paul E. McKenney40e80c42012-10-31 13:00:15 -0700298o "age" is the number of jiffies that the current grace period
299 has extended for, or zero if there is no grace period currently
300 in effect.
Paul E. McKenney64db4cf2008-12-18 21:55:32 +0100301
Paul E. McKenney40e80c42012-10-31 13:00:15 -0700302o "max" is the age in jiffies of the longest-duration grace period
303 thus far.
Paul E. McKenney64db4cf2008-12-18 21:55:32 +0100304
Paul E. McKenney40e80c42012-10-31 13:00:15 -0700305The output of "cat rcu/rcu_preempt/rcuhier" looks as follows:
Paul E. McKenney64db4cf2008-12-18 21:55:32 +0100306
Paul E. McKenney40e80c42012-10-31 13:00:15 -0700307c=14407 g=14408 s=0 jfq=2 j=c863 nfqs=12040/nfqsng=0(12040) fqlh=1051 oqlen=0/0
3083/3 ..>. 0:7 ^0
309e/e ..>. 0:3 ^0 d/d ..>. 4:7 ^1
Paul E. McKenney64db4cf2008-12-18 21:55:32 +0100310
Paul E. McKenney40e80c42012-10-31 13:00:15 -0700311The fields are as follows:
Paul E. McKenney64db4cf2008-12-18 21:55:32 +0100312
Paul E. McKenney40e80c42012-10-31 13:00:15 -0700313o "c" is exactly the same as "completed" under rcu/rcu_preempt/rcugp.
Paul E. McKenney64db4cf2008-12-18 21:55:32 +0100314
Paul E. McKenney40e80c42012-10-31 13:00:15 -0700315o "g" is exactly the same as "gpnum" under rcu/rcu_preempt/rcugp.
316
317o "s" is the current state of the force_quiescent_state()
Paul E. McKenney64db4cf2008-12-18 21:55:32 +0100318 state machine.
319
320o "jfq" is the number of jiffies remaining for this grace period
321 before force_quiescent_state() is invoked to help push things
Paul E. McKenney40e80c42012-10-31 13:00:15 -0700322 along. Note that CPUs in idle mode throughout the grace period
323 will not report on their own, but rather must be check by some
324 other CPU via force_quiescent_state().
Paul E. McKenney64db4cf2008-12-18 21:55:32 +0100325
326o "j" is the low-order four hex digits of the jiffies counter.
327 Yes, Paul did run into a number of problems that turned out to
328 be due to the jiffies counter no longer counting. Why do you ask?
329
330o "nfqs" is the number of calls to force_quiescent_state() since
331 boot.
332
333o "nfqsng" is the number of useless calls to force_quiescent_state(),
334 where there wasn't actually a grace period active. This can
Paul E. McKenney40e80c42012-10-31 13:00:15 -0700335 no longer happen due to grace-period processing being pushed
336 into a kthread. The number in parentheses is the difference
Paul E. McKenney64db4cf2008-12-18 21:55:32 +0100337 between "nfqs" and "nfqsng", or the number of times that
338 force_quiescent_state() actually did some real work.
339
340o "fqlh" is the number of calls to force_quiescent_state() that
341 exited immediately (without even being counted in nfqs above)
342 due to contention on ->fqslock.
343
Paul E. McKenney40e80c42012-10-31 13:00:15 -0700344o Each element of the form "3/3 ..>. 0:7 ^0" represents one rcu_node
345 structure. Each line represents one level of the hierarchy,
346 from root to leaves. It is best to think of the rcu_data
347 structures as forming yet another level after the leaves.
348 Note that there might be either one, two, three, or even four
349 levels of rcu_node structures, depending on the relationship
350 between CONFIG_RCU_FANOUT, CONFIG_RCU_FANOUT_LEAF (possibly
351 adjusted using the rcu_fanout_leaf kernel boot parameter), and
352 CONFIG_NR_CPUS (possibly adjusted using the nr_cpu_ids count of
353 possible CPUs for the booting hardware).
Paul E. McKenney0edf1a62009-10-14 10:15:59 -0700354
Paul E. McKenney64db4cf2008-12-18 21:55:32 +0100355 o The numbers separated by the "/" are the qsmask followed
356 by the qsmaskinit. The qsmask will have one bit
Paul E. McKenney40e80c42012-10-31 13:00:15 -0700357 set for each entity in the next lower level that has
358 not yet checked in for the current grace period ("e"
359 indicating CPUs 5, 6, and 7 in the example above).
Paul E. McKenney64db4cf2008-12-18 21:55:32 +0100360 The qsmaskinit will have one bit for each entity that is
361 currently expected to check in during each grace period.
362 The value of qsmaskinit is assigned to that of qsmask
363 at the beginning of each grace period.
364
Paul E. McKenney0edf1a62009-10-14 10:15:59 -0700365 o The characters separated by the ">" indicate the state
Paul E. McKenney12f5f522010-11-29 21:56:39 -0800366 of the blocked-tasks lists. A "G" preceding the ">"
Paul E. McKenney0edf1a62009-10-14 10:15:59 -0700367 indicates that at least one task blocked in an RCU
368 read-side critical section blocks the current grace
Paul E. McKenney12f5f522010-11-29 21:56:39 -0800369 period, while a "E" preceding the ">" indicates that
370 at least one task blocked in an RCU read-side critical
371 section blocks the current expedited grace period.
372 A "T" character following the ">" indicates that at
373 least one task is blocked within an RCU read-side
374 critical section, regardless of whether any current
375 grace period (expedited or normal) is inconvenienced.
376 A "." character appears if the corresponding condition
377 does not hold, so that "..>." indicates that no tasks
378 are blocked. In contrast, "GE>T" indicates maximal
Paul E. McKenney40e80c42012-10-31 13:00:15 -0700379 inconvenience from blocked tasks. CONFIG_TREE_RCU
380 builds of the kernel will always show "..>.".
Paul E. McKenney0edf1a62009-10-14 10:15:59 -0700381
Paul E. McKenney64db4cf2008-12-18 21:55:32 +0100382 o The numbers separated by the ":" are the range of CPUs
383 served by this struct rcu_node. This can be helpful
384 in working out how the hierarchy is wired together.
385
Paul E. McKenney40e80c42012-10-31 13:00:15 -0700386 For example, the example rcu_node structure shown above
387 has "0:7", indicating that it covers CPUs 0 through 7.
Paul E. McKenney64db4cf2008-12-18 21:55:32 +0100388
389 o The number after the "^" indicates the bit in the
Paul E. McKenney40e80c42012-10-31 13:00:15 -0700390 next higher level rcu_node structure that this rcu_node
391 structure corresponds to. For example, the "d/d ..>. 4:7
392 ^1" has a "1" in this position, indicating that it
393 corresponds to the "1" bit in the "3" shown in the
394 "3/3 ..>. 0:7 ^0" entry on the next level up.
Paul E. McKenney6fd9b3a2009-04-13 21:31:18 -0700395
396
Paul E. McKenney40e80c42012-10-31 13:00:15 -0700397The output of "cat rcu/rcu_sched/rcu_pending" looks as follows:
Paul E. McKenney6fd9b3a2009-04-13 21:31:18 -0700398
Paul E. McKenney40e80c42012-10-31 13:00:15 -0700399 0!np=26111 qsp=29 rpq=5386 cbr=1 cng=570 gpc=3674 gps=577 nn=15903
400 1!np=28913 qsp=35 rpq=6097 cbr=1 cng=448 gpc=3700 gps=554 nn=18113
401 2!np=32740 qsp=37 rpq=6202 cbr=0 cng=476 gpc=4627 gps=546 nn=20889
402 3 np=23679 qsp=22 rpq=5044 cbr=1 cng=415 gpc=3403 gps=347 nn=14469
403 4!np=30714 qsp=4 rpq=5574 cbr=0 cng=528 gpc=3931 gps=639 nn=20042
404 5 np=28910 qsp=2 rpq=5246 cbr=0 cng=428 gpc=4105 gps=709 nn=18422
405 6!np=38648 qsp=5 rpq=7076 cbr=0 cng=840 gpc=4072 gps=961 nn=25699
406 7 np=37275 qsp=2 rpq=6873 cbr=0 cng=868 gpc=3416 gps=971 nn=25147
Paul E. McKenney6fd9b3a2009-04-13 21:31:18 -0700407
Paul E. McKenney40e80c42012-10-31 13:00:15 -0700408The fields are as follows:
409
410o The leading number is the CPU number, with "!" indicating
411 an offline CPU.
Paul E. McKenney6fd9b3a2009-04-13 21:31:18 -0700412
413o "np" is the number of times that __rcu_pending() has been invoked
414 for the corresponding flavor of RCU.
415
416o "qsp" is the number of times that the RCU was waiting for a
417 quiescent state from this CPU.
418
Paul E. McKenneyd21670a2010-04-14 17:39:26 -0700419o "rpq" is the number of times that the CPU had passed through
420 a quiescent state, but not yet reported it to RCU.
421
Paul E. McKenney6fd9b3a2009-04-13 21:31:18 -0700422o "cbr" is the number of times that this CPU had RCU callbacks
423 that had passed through a grace period, and were thus ready
424 to be invoked.
425
426o "cng" is the number of times that this CPU needed another
427 grace period while RCU was idle.
428
429o "gpc" is the number of times that an old grace period had
430 completed, but this CPU was not yet aware of it.
431
432o "gps" is the number of times that a new grace period had started,
433 but this CPU was not yet aware of it.
434
Paul E. McKenney40e80c42012-10-31 13:00:15 -0700435o "nn" is the number of times that this CPU needed nothing.
Paul E. McKenney90e6ac32011-04-06 15:20:47 -0700436
437
438The output of "cat rcu/rcuboost" looks as follows:
439
Paul E. McKenney40e80c42012-10-31 13:00:15 -07004400:3 tasks=.... kt=W ntb=0 neb=0 nnb=0 j=c864 bt=c894
441 balk: nt=0 egt=4695 bt=0 nb=0 ny=56 nos=0
4424:7 tasks=.... kt=W ntb=0 neb=0 nnb=0 j=c864 bt=c894
443 balk: nt=0 egt=6541 bt=0 nb=0 ny=126 nos=0
Paul E. McKenney90e6ac32011-04-06 15:20:47 -0700444
445This information is output only for rcu_preempt. Each two-line entry
446corresponds to a leaf rcu_node strcuture. The fields are as follows:
447
448o "n:m" is the CPU-number range for the corresponding two-line
449 entry. In the sample output above, the first entry covers
Paul E. McKenney40e80c42012-10-31 13:00:15 -0700450 CPUs zero through three and the second entry covers CPUs four
451 through seven.
Paul E. McKenney90e6ac32011-04-06 15:20:47 -0700452
453o "tasks=TNEB" gives the state of the various segments of the
454 rnp->blocked_tasks list:
455
456 "T" This indicates that there are some tasks that blocked
457 while running on one of the corresponding CPUs while
458 in an RCU read-side critical section.
459
460 "N" This indicates that some of the blocked tasks are preventing
461 the current normal (non-expedited) grace period from
462 completing.
463
464 "E" This indicates that some of the blocked tasks are preventing
465 the current expedited grace period from completing.
466
467 "B" This indicates that some of the blocked tasks are in
468 need of RCU priority boosting.
469
470 Each character is replaced with "." if the corresponding
471 condition does not hold.
472
473o "kt" is the state of the RCU priority-boosting kernel
474 thread associated with the corresponding rcu_node structure.
475 The state can be one of the following:
476
477 "S" The kernel thread is stopped, in other words, all
478 CPUs corresponding to this rcu_node structure are
479 offline.
480
481 "R" The kernel thread is running.
482
483 "W" The kernel thread is waiting because there is no work
484 for it to do.
485
486 "Y" The kernel thread is yielding to avoid hogging CPU.
487
488 "?" Unknown value, indicates a bug.
489
490o "ntb" is the number of tasks boosted.
491
492o "neb" is the number of tasks boosted in order to complete an
493 expedited grace period.
494
495o "nnb" is the number of tasks boosted in order to complete a
496 normal (non-expedited) grace period. When boosting a task
497 that was blocking both an expedited and a normal grace period,
498 it is counted against the expedited total above.
499
500o "j" is the low-order 16 bits of the jiffies counter in
501 hexadecimal.
502
503o "bt" is the low-order 16 bits of the value that the jiffies
504 counter will have when we next start boosting, assuming that
505 the current grace period does not end beforehand. This is
506 also in hexadecimal.
507
508o "balk: nt" counts the number of times we didn't boost (in
509 other words, we balked) even though it was time to boost because
510 there were no blocked tasks to boost. This situation occurs
511 when there is one blocked task on one rcu_node structure and
512 none on some other rcu_node structure.
513
514o "egt" counts the number of times we balked because although
515 there were blocked tasks, none of them were blocking the
516 current grace period, whether expedited or otherwise.
517
518o "bt" counts the number of times we balked because boosting
519 had already been initiated for the current grace period.
520
521o "nb" counts the number of times we balked because there
522 was at least one task blocking the current non-expedited grace
523 period that never had blocked. If it is already running, it
524 just won't help to boost its priority!
525
526o "ny" counts the number of times we balked because it was
527 not yet time to start boosting.
528
529o "nos" counts the number of times we balked for other
530 reasons, e.g., the grace period ended first.
531
532
Paul E. McKenney8e79e1f2010-11-04 14:31:19 -0700533CONFIG_TINY_RCU and CONFIG_TINY_PREEMPT_RCU debugfs Files and Formats
534
535These implementations of RCU provides a single debugfs file under the
536top-level directory RCU, namely rcu/rcudata, which displays fields in
537rcu_bh_ctrlblk, rcu_sched_ctrlblk and, for CONFIG_TINY_PREEMPT_RCU,
538rcu_preempt_ctrlblk.
539
540The output of "cat rcu/rcudata" is as follows:
541
542rcu_preempt: qlen=24 gp=1097669 g197/p197/c197 tasks=...
543 ttb=. btg=no ntb=184 neb=0 nnb=183 j=01f7 bt=0274
544 normal balk: nt=1097669 gt=0 bt=371 b=0 ny=25073378 nos=0
545 exp balk: bt=0 nos=0
546rcu_sched: qlen: 0
547rcu_bh: qlen: 0
548
549This is split into rcu_preempt, rcu_sched, and rcu_bh sections, with the
550rcu_preempt section appearing only in CONFIG_TINY_PREEMPT_RCU builds.
551The last three lines of the rcu_preempt section appear only in
552CONFIG_RCU_BOOST kernel builds. The fields are as follows:
553
554o "qlen" is the number of RCU callbacks currently waiting either
555 for an RCU grace period or waiting to be invoked. This is the
556 only field present for rcu_sched and rcu_bh, due to the
557 short-circuiting of grace period in those two cases.
558
559o "gp" is the number of grace periods that have completed.
560
561o "g197/p197/c197" displays the grace-period state, with the
562 "g" number being the number of grace periods that have started
563 (mod 256), the "p" number being the number of grace periods
564 that the CPU has responded to (also mod 256), and the "c"
565 number being the number of grace periods that have completed
566 (once again mode 256).
567
568 Why have both "gp" and "g"? Because the data flowing into
569 "gp" is only present in a CONFIG_RCU_TRACE kernel.
570
571o "tasks" is a set of bits. The first bit is "T" if there are
572 currently tasks that have recently blocked within an RCU
573 read-side critical section, the second bit is "N" if any of the
574 aforementioned tasks are blocking the current RCU grace period,
575 and the third bit is "E" if any of the aforementioned tasks are
576 blocking the current expedited grace period. Each bit is "."
577 if the corresponding condition does not hold.
578
579o "ttb" is a single bit. It is "B" if any of the blocked tasks
580 need to be priority boosted and "." otherwise.
581
582o "btg" indicates whether boosting has been carried out during
583 the current grace period, with "exp" indicating that boosting
584 is in progress for an expedited grace period, "no" indicating
585 that boosting has not yet started for a normal grace period,
586 "begun" indicating that boosting has bebug for a normal grace
587 period, and "done" indicating that boosting has completed for
588 a normal grace period.
589
590o "ntb" is the total number of tasks subjected to RCU priority boosting
591 periods since boot.
592
593o "neb" is the number of expedited grace periods that have had
594 to resort to RCU priority boosting since boot.
595
596o "nnb" is the number of normal grace periods that have had
597 to resort to RCU priority boosting since boot.
598
Paul E. McKenney90e6ac32011-04-06 15:20:47 -0700599o "j" is the low-order 16 bits of the jiffies counter in hexadecimal.
Paul E. McKenney8e79e1f2010-11-04 14:31:19 -0700600
Paul E. McKenney90e6ac32011-04-06 15:20:47 -0700601o "bt" is the low-order 16 bits of the value that the jiffies counter
Paul E. McKenney8e79e1f2010-11-04 14:31:19 -0700602 will have at the next time that boosting is scheduled to begin.
603
604o In the line beginning with "normal balk", the fields are as follows:
605
606 o "nt" is the number of times that the system balked from
607 boosting because there were no blocked tasks to boost.
608 Note that the system will balk from boosting even if the
609 grace period is overdue when the currently running task
610 is looping within an RCU read-side critical section.
611 There is no point in boosting in this case, because
612 boosting a running task won't make it run any faster.
613
614 o "gt" is the number of times that the system balked
615 from boosting because, although there were blocked tasks,
616 none of them were preventing the current grace period
617 from completing.
618
619 o "bt" is the number of times that the system balked
620 from boosting because boosting was already in progress.
621
622 o "b" is the number of times that the system balked from
623 boosting because boosting had already completed for
624 the grace period in question.
625
626 o "ny" is the number of times that the system balked from
627 boosting because it was not yet time to start boosting
628 the grace period in question.
629
630 o "nos" is the number of times that the system balked from
631 boosting for inexplicable ("not otherwise specified")
632 reasons. This can actually happen due to races involving
633 increments of the jiffies counter.
634
635o In the line beginning with "exp balk", the fields are as follows:
636
637 o "bt" is the number of times that the system balked from
638 boosting because there were no blocked tasks to boost.
639
640 o "nos" is the number of times that the system balked from
641 boosting for inexplicable ("not otherwise specified")
642 reasons.