| CONFIG_RCU_TRACE debugfs Files and Formats |
| |
| |
| The rcutree and rcutiny implementations of RCU provide debugfs trace |
| output that summarizes counters and state. This information is useful for |
| debugging RCU itself, and can sometimes also help to debug abuses of RCU. |
| The following sections describe the debugfs files and formats, first |
| for rcutree and next for rcutiny. |
| |
| |
| CONFIG_TREE_RCU and CONFIG_TREE_PREEMPT_RCU debugfs Files and Formats |
| |
| These implementations of RCU provides several debugfs files under the |
| top-level directory "rcu": |
| |
| rcu/rcudata: |
| Displays fields in struct rcu_data. |
| rcu/rcudata.csv: |
| Comma-separated values spreadsheet version of rcudata. |
| rcu/rcugp: |
| Displays grace-period counters. |
| rcu/rcuhier: |
| Displays the struct rcu_node hierarchy. |
| rcu/rcu_pending: |
| Displays counts of the reasons rcu_pending() decided that RCU had |
| work to do. |
| rcu/rcutorture: |
| Displays rcutorture test progress. |
| rcu/rcuboost: |
| Displays RCU boosting statistics. Only present if |
| CONFIG_RCU_BOOST=y. |
| |
| The output of "cat rcu/rcudata" looks as follows: |
| |
| rcu_sched: |
| 0 c=20972 g=20973 pq=1 pqc=20972 qp=0 dt=545/1/0 df=50 of=0 ri=0 ql=163 qs=NRW. kt=0/W/0 ktl=ebc3 b=10 ci=153737 co=0 ca=0 |
| 1 c=20972 g=20973 pq=1 pqc=20972 qp=0 dt=967/1/0 df=58 of=0 ri=0 ql=634 qs=NRW. kt=0/W/1 ktl=58c b=10 ci=191037 co=0 ca=0 |
| 2 c=20972 g=20973 pq=1 pqc=20972 qp=0 dt=1081/1/0 df=175 of=0 ri=0 ql=74 qs=N.W. kt=0/W/2 ktl=da94 b=10 ci=75991 co=0 ca=0 |
| 3 c=20942 g=20943 pq=1 pqc=20942 qp=1 dt=1846/0/0 df=404 of=0 ri=0 ql=0 qs=.... kt=0/W/3 ktl=d1cd b=10 ci=72261 co=0 ca=0 |
| 4 c=20972 g=20973 pq=1 pqc=20972 qp=0 dt=369/1/0 df=83 of=0 ri=0 ql=48 qs=N.W. kt=0/W/4 ktl=e0e7 b=10 ci=128365 co=0 ca=0 |
| 5 c=20972 g=20973 pq=1 pqc=20972 qp=0 dt=381/1/0 df=64 of=0 ri=0 ql=169 qs=NRW. kt=0/W/5 ktl=fb2f b=10 ci=164360 co=0 ca=0 |
| 6 c=20972 g=20973 pq=1 pqc=20972 qp=0 dt=1037/1/0 df=183 of=0 ri=0 ql=62 qs=N.W. kt=0/W/6 ktl=d2ad b=10 ci=65663 co=0 ca=0 |
| 7 c=20897 g=20897 pq=1 pqc=20896 qp=0 dt=1572/0/0 df=382 of=0 ri=0 ql=0 qs=.... kt=0/W/7 ktl=cf15 b=10 ci=75006 co=0 ca=0 |
| rcu_bh: |
| 0 c=1480 g=1480 pq=1 pqc=1479 qp=0 dt=545/1/0 df=6 of=0 ri=1 ql=0 qs=.... kt=0/W/0 ktl=ebc3 b=10 ci=0 co=0 ca=0 |
| 1 c=1480 g=1480 pq=1 pqc=1479 qp=0 dt=967/1/0 df=3 of=0 ri=1 ql=0 qs=.... kt=0/W/1 ktl=58c b=10 ci=151 co=0 ca=0 |
| 2 c=1480 g=1480 pq=1 pqc=1479 qp=0 dt=1081/1/0 df=6 of=0 ri=1 ql=0 qs=.... kt=0/W/2 ktl=da94 b=10 ci=0 co=0 ca=0 |
| 3 c=1480 g=1480 pq=1 pqc=1479 qp=0 dt=1846/0/0 df=8 of=0 ri=1 ql=0 qs=.... kt=0/W/3 ktl=d1cd b=10 ci=0 co=0 ca=0 |
| 4 c=1480 g=1480 pq=1 pqc=1479 qp=0 dt=369/1/0 df=6 of=0 ri=1 ql=0 qs=.... kt=0/W/4 ktl=e0e7 b=10 ci=0 co=0 ca=0 |
| 5 c=1480 g=1480 pq=1 pqc=1479 qp=0 dt=381/1/0 df=4 of=0 ri=1 ql=0 qs=.... kt=0/W/5 ktl=fb2f b=10 ci=0 co=0 ca=0 |
| 6 c=1480 g=1480 pq=1 pqc=1479 qp=0 dt=1037/1/0 df=6 of=0 ri=1 ql=0 qs=.... kt=0/W/6 ktl=d2ad b=10 ci=0 co=0 ca=0 |
| 7 c=1474 g=1474 pq=1 pqc=1473 qp=0 dt=1572/0/0 df=8 of=0 ri=1 ql=0 qs=.... kt=0/W/7 ktl=cf15 b=10 ci=0 co=0 ca=0 |
| |
| The first section lists the rcu_data structures for rcu_sched, the second |
| for rcu_bh. Note that CONFIG_TREE_PREEMPT_RCU kernels will have an |
| additional section for rcu_preempt. Each section has one line per CPU, |
| or eight for this 8-CPU system. The fields are as follows: |
| |
| o The number at the beginning of each line is the CPU number. |
| CPUs numbers followed by an exclamation mark are offline, |
| but have been online at least once since boot. There will be |
| no output for CPUs that have never been online, which can be |
| a good thing in the surprisingly common case where NR_CPUS is |
| substantially larger than the number of actual CPUs. |
| |
| o "c" is the count of grace periods that this CPU believes have |
| completed. Offlined CPUs and CPUs in dynticks idle mode may |
| lag quite a ways behind, for example, CPU 6 under "rcu_sched" |
| above, which has been offline through not quite 40,000 RCU grace |
| periods. It is not unusual to see CPUs lagging by thousands of |
| grace periods. |
| |
| o "g" is the count of grace periods that this CPU believes have |
| started. Again, offlined CPUs and CPUs in dynticks idle mode |
| may lag behind. If the "c" and "g" values are equal, this CPU |
| has already reported a quiescent state for the last RCU grace |
| period that it is aware of, otherwise, the CPU believes that it |
| owes RCU a quiescent state. |
| |
| o "pq" indicates that this CPU has passed through a quiescent state |
| for the current grace period. It is possible for "pq" to be |
| "1" and "c" different than "g", which indicates that although |
| the CPU has passed through a quiescent state, either (1) this |
| CPU has not yet reported that fact, (2) some other CPU has not |
| yet reported for this grace period, or (3) both. |
| |
| o "pqc" indicates which grace period the last-observed quiescent |
| state for this CPU corresponds to. This is important for handling |
| the race between CPU 0 reporting an extended dynticks-idle |
| quiescent state for CPU 1 and CPU 1 suddenly waking up and |
| reporting its own quiescent state. If CPU 1 was the last CPU |
| for the current grace period, then the CPU that loses this race |
| will attempt to incorrectly mark CPU 1 as having checked in for |
| the next grace period! |
| |
| o "qp" indicates that RCU still expects a quiescent state from |
| this CPU. Offlined CPUs and CPUs in dyntick idle mode might |
| well have qp=1, which is OK: RCU is still ignoring them. |
| |
| o "dt" is the current value of the dyntick counter that is incremented |
| when entering or leaving dynticks idle state, either by the |
| scheduler or by irq. The number after the "/" is the interrupt |
| nesting depth when in dyntick-idle state, or one greater than |
| the interrupt-nesting depth otherwise. |
| |
| This field is displayed only for CONFIG_NO_HZ kernels. |
| |
| o "dn" is the current value of the dyntick counter that is incremented |
| when entering or leaving dynticks idle state via NMI. If both |
| the "dt" and "dn" values are even, then this CPU is in dynticks |
| idle mode and may be ignored by RCU. If either of these two |
| counters is odd, then RCU must be alert to the possibility of |
| an RCU read-side critical section running on this CPU. |
| |
| This field is displayed only for CONFIG_NO_HZ kernels. |
| |
| o "df" is the number of times that some other CPU has forced a |
| quiescent state on behalf of this CPU due to this CPU being in |
| dynticks-idle state. |
| |
| This field is displayed only for CONFIG_NO_HZ kernels. |
| |
| o "of" is the number of times that some other CPU has forced a |
| quiescent state on behalf of this CPU due to this CPU being |
| offline. In a perfect world, this might never happen, but it |
| turns out that offlining and onlining a CPU can take several grace |
| periods, and so there is likely to be an extended period of time |
| when RCU believes that the CPU is online when it really is not. |
| Please note that erring in the other direction (RCU believing a |
| CPU is offline when it is really alive and kicking) is a fatal |
| error, so it makes sense to err conservatively. |
| |
| o "ri" is the number of times that RCU has seen fit to send a |
| reschedule IPI to this CPU in order to get it to report a |
| quiescent state. |
| |
| o "ql" is the number of RCU callbacks currently residing on |
| this CPU. This is the total number of callbacks, regardless |
| of what state they are in (new, waiting for grace period to |
| start, waiting for grace period to end, ready to invoke). |
| |
| o "qs" gives an indication of the state of the callback queue |
| with four characters: |
| |
| "N" Indicates that there are callbacks queued that are not |
| ready to be handled by the next grace period, and thus |
| will be handled by the grace period following the next |
| one. |
| |
| "R" Indicates that there are callbacks queued that are |
| ready to be handled by the next grace period. |
| |
| "W" Indicates that there are callbacks queued that are |
| waiting on the current grace period. |
| |
| "D" Indicates that there are callbacks queued that have |
| already been handled by a prior grace period, and are |
| thus waiting to be invoked. Note that callbacks in |
| the process of being invoked are not counted here. |
| Callbacks in the process of being invoked are those |
| that have been removed from the rcu_data structures |
| queues by rcu_do_batch(), but which have not yet been |
| invoked. |
| |
| If there are no callbacks in a given one of the above states, |
| the corresponding character is replaced by ".". |
| |
| o "kt" is the per-CPU kernel-thread state. The digit preceding |
| the first slash is zero if there is no work pending and 1 |
| otherwise. The character between the first pair of slashes is |
| as follows: |
| |
| "S" The kernel thread is stopped, in other words, all |
| CPUs corresponding to this rcu_node structure are |
| offline. |
| |
| "R" The kernel thread is running. |
| |
| "W" The kernel thread is waiting because there is no work |
| for it to do. |
| |
| "O" The kernel thread is waiting because it has been |
| forced off of its designated CPU or because its |
| ->cpus_allowed mask permits it to run on other than |
| its designated CPU. |
| |
| "Y" The kernel thread is yielding to avoid hogging CPU. |
| |
| "?" Unknown value, indicates a bug. |
| |
| The number after the final slash is the CPU that the kthread |
| is actually running on. |
| |
| o "ktl" is the low-order 16 bits (in hexadecimal) of the count of |
| the number of times that this CPU's per-CPU kthread has gone |
| through its loop servicing invoke_rcu_cpu_kthread() requests. |
| |
| o "b" is the batch limit for this CPU. If more than this number |
| of RCU callbacks is ready to invoke, then the remainder will |
| be deferred. |
| |
| o "ci" is the number of RCU callbacks that have been invoked for |
| this CPU. Note that ci+ql is the number of callbacks that have |
| been registered in absence of CPU-hotplug activity. |
| |
| o "co" is the number of RCU callbacks that have been orphaned due to |
| this CPU going offline. These orphaned callbacks have been moved |
| to an arbitrarily chosen online CPU. |
| |
| o "ca" is the number of RCU callbacks that have been adopted due to |
| other CPUs going offline. Note that ci+co-ca+ql is the number of |
| RCU callbacks registered on this CPU. |
| |
| There is also an rcu/rcudata.csv file with the same information in |
| comma-separated-variable spreadsheet format. |
| |
| |
| The output of "cat rcu/rcugp" looks as follows: |
| |
| rcu_sched: completed=33062 gpnum=33063 |
| rcu_bh: completed=464 gpnum=464 |
| |
| Again, this output is for both "rcu_sched" and "rcu_bh". Note that |
| kernels built with CONFIG_TREE_PREEMPT_RCU will have an additional |
| "rcu_preempt" line. The fields are taken from the rcu_state structure, |
| and are as follows: |
| |
| o "completed" is the number of grace periods that have completed. |
| It is comparable to the "c" field from rcu/rcudata in that a |
| CPU whose "c" field matches the value of "completed" is aware |
| that the corresponding RCU grace period has completed. |
| |
| o "gpnum" is the number of grace periods that have started. It is |
| comparable to the "g" field from rcu/rcudata in that a CPU |
| whose "g" field matches the value of "gpnum" is aware that the |
| corresponding RCU grace period has started. |
| |
| If these two fields are equal (as they are for "rcu_bh" above), |
| then there is no grace period in progress, in other words, RCU |
| is idle. On the other hand, if the two fields differ (as they |
| do for "rcu_sched" above), then an RCU grace period is in progress. |
| |
| |
| The output of "cat rcu/rcuhier" looks as follows, with very long lines: |
| |
| c=6902 g=6903 s=2 jfq=3 j=72c7 nfqs=13142/nfqsng=0(13142) fqlh=6 |
| 1/1 ..>. 0:127 ^0 |
| 3/3 ..>. 0:35 ^0 0/0 ..>. 36:71 ^1 0/0 ..>. 72:107 ^2 0/0 ..>. 108:127 ^3 |
| 3/3f ..>. 0:5 ^0 2/3 ..>. 6:11 ^1 0/0 ..>. 12:17 ^2 0/0 ..>. 18:23 ^3 0/0 ..>. 24:29 ^4 0/0 ..>. 30:35 ^5 0/0 ..>. 36:41 ^0 0/0 ..>. 42:47 ^1 0/0 ..>. 48:53 ^2 0/0 ..>. 54:59 ^3 0/0 ..>. 60:65 ^4 0/0 ..>. 66:71 ^5 0/0 ..>. 72:77 ^0 0/0 ..>. 78:83 ^1 0/0 ..>. 84:89 ^2 0/0 ..>. 90:95 ^3 0/0 ..>. 96:101 ^4 0/0 ..>. 102:107 ^5 0/0 ..>. 108:113 ^0 0/0 ..>. 114:119 ^1 0/0 ..>. 120:125 ^2 0/0 ..>. 126:127 ^3 |
| rcu_bh: |
| c=-226 g=-226 s=1 jfq=-5701 j=72c7 nfqs=88/nfqsng=0(88) fqlh=0 |
| 0/1 ..>. 0:127 ^0 |
| 0/3 ..>. 0:35 ^0 0/0 ..>. 36:71 ^1 0/0 ..>. 72:107 ^2 0/0 ..>. 108:127 ^3 |
| 0/3f ..>. 0:5 ^0 0/3 ..>. 6:11 ^1 0/0 ..>. 12:17 ^2 0/0 ..>. 18:23 ^3 0/0 ..>. 24:29 ^4 0/0 ..>. 30:35 ^5 0/0 ..>. 36:41 ^0 0/0 ..>. 42:47 ^1 0/0 ..>. 48:53 ^2 0/0 ..>. 54:59 ^3 0/0 ..>. 60:65 ^4 0/0 ..>. 66:71 ^5 0/0 ..>. 72:77 ^0 0/0 ..>. 78:83 ^1 0/0 ..>. 84:89 ^2 0/0 ..>. 90:95 ^3 0/0 ..>. 96:101 ^4 0/0 ..>. 102:107 ^5 0/0 ..>. 108:113 ^0 0/0 ..>. 114:119 ^1 0/0 ..>. 120:125 ^2 0/0 ..>. 126:127 ^3 |
| |
| This is once again split into "rcu_sched" and "rcu_bh" portions, |
| and CONFIG_TREE_PREEMPT_RCU kernels will again have an additional |
| "rcu_preempt" section. The fields are as follows: |
| |
| o "c" is exactly the same as "completed" under rcu/rcugp. |
| |
| o "g" is exactly the same as "gpnum" under rcu/rcugp. |
| |
| o "s" is the "signaled" state that drives force_quiescent_state()'s |
| state machine. |
| |
| o "jfq" is the number of jiffies remaining for this grace period |
| before force_quiescent_state() is invoked to help push things |
| along. Note that CPUs in dyntick-idle mode throughout the grace |
| period will not report on their own, but rather must be check by |
| some other CPU via force_quiescent_state(). |
| |
| o "j" is the low-order four hex digits of the jiffies counter. |
| Yes, Paul did run into a number of problems that turned out to |
| be due to the jiffies counter no longer counting. Why do you ask? |
| |
| o "nfqs" is the number of calls to force_quiescent_state() since |
| boot. |
| |
| o "nfqsng" is the number of useless calls to force_quiescent_state(), |
| where there wasn't actually a grace period active. This can |
| happen due to races. The number in parentheses is the difference |
| between "nfqs" and "nfqsng", or the number of times that |
| force_quiescent_state() actually did some real work. |
| |
| o "fqlh" is the number of calls to force_quiescent_state() that |
| exited immediately (without even being counted in nfqs above) |
| due to contention on ->fqslock. |
| |
| o Each element of the form "1/1 0:127 ^0" represents one struct |
| rcu_node. Each line represents one level of the hierarchy, from |
| root to leaves. It is best to think of the rcu_data structures |
| as forming yet another level after the leaves. Note that there |
| might be either one, two, or three levels of rcu_node structures, |
| depending on the relationship between CONFIG_RCU_FANOUT and |
| CONFIG_NR_CPUS. |
| |
| o The numbers separated by the "/" are the qsmask followed |
| by the qsmaskinit. The qsmask will have one bit |
| set for each entity in the next lower level that |
| has not yet checked in for the current grace period. |
| The qsmaskinit will have one bit for each entity that is |
| currently expected to check in during each grace period. |
| The value of qsmaskinit is assigned to that of qsmask |
| at the beginning of each grace period. |
| |
| For example, for "rcu_sched", the qsmask of the first |
| entry of the lowest level is 0x14, meaning that we |
| are still waiting for CPUs 2 and 4 to check in for the |
| current grace period. |
| |
| o The characters separated by the ">" indicate the state |
| of the blocked-tasks lists. A "G" preceding the ">" |
| indicates that at least one task blocked in an RCU |
| read-side critical section blocks the current grace |
| period, while a "E" preceding the ">" indicates that |
| at least one task blocked in an RCU read-side critical |
| section blocks the current expedited grace period. |
| A "T" character following the ">" indicates that at |
| least one task is blocked within an RCU read-side |
| critical section, regardless of whether any current |
| grace period (expedited or normal) is inconvenienced. |
| A "." character appears if the corresponding condition |
| does not hold, so that "..>." indicates that no tasks |
| are blocked. In contrast, "GE>T" indicates maximal |
| inconvenience from blocked tasks. |
| |
| o The numbers separated by the ":" are the range of CPUs |
| served by this struct rcu_node. This can be helpful |
| in working out how the hierarchy is wired together. |
| |
| For example, the first entry at the lowest level shows |
| "0:5", indicating that it covers CPUs 0 through 5. |
| |
| o The number after the "^" indicates the bit in the |
| next higher level rcu_node structure that this |
| rcu_node structure corresponds to. |
| |
| For example, the first entry at the lowest level shows |
| "^0", indicating that it corresponds to bit zero in |
| the first entry at the middle level. |
| |
| |
| The output of "cat rcu/rcu_pending" looks as follows: |
| |
| rcu_sched: |
| 0 np=255892 qsp=53936 rpq=85 cbr=0 cng=14417 gpc=10033 gps=24320 nf=6445 nn=146741 |
| 1 np=261224 qsp=54638 rpq=33 cbr=0 cng=25723 gpc=16310 gps=2849 nf=5912 nn=155792 |
| 2 np=237496 qsp=49664 rpq=23 cbr=0 cng=2762 gpc=45478 gps=1762 nf=1201 nn=136629 |
| 3 np=236249 qsp=48766 rpq=98 cbr=0 cng=286 gpc=48049 gps=1218 nf=207 nn=137723 |
| 4 np=221310 qsp=46850 rpq=7 cbr=0 cng=26 gpc=43161 gps=4634 nf=3529 nn=123110 |
| 5 np=237332 qsp=48449 rpq=9 cbr=0 cng=54 gpc=47920 gps=3252 nf=201 nn=137456 |
| 6 np=219995 qsp=46718 rpq=12 cbr=0 cng=50 gpc=42098 gps=6093 nf=4202 nn=120834 |
| 7 np=249893 qsp=49390 rpq=42 cbr=0 cng=72 gpc=38400 gps=17102 nf=41 nn=144888 |
| rcu_bh: |
| 0 np=146741 qsp=1419 rpq=6 cbr=0 cng=6 gpc=0 gps=0 nf=2 nn=145314 |
| 1 np=155792 qsp=12597 rpq=3 cbr=0 cng=0 gpc=4 gps=8 nf=3 nn=143180 |
| 2 np=136629 qsp=18680 rpq=1 cbr=0 cng=0 gpc=7 gps=6 nf=0 nn=117936 |
| 3 np=137723 qsp=2843 rpq=0 cbr=0 cng=0 gpc=10 gps=7 nf=0 nn=134863 |
| 4 np=123110 qsp=12433 rpq=0 cbr=0 cng=0 gpc=4 gps=2 nf=0 nn=110671 |
| 5 np=137456 qsp=4210 rpq=1 cbr=0 cng=0 gpc=6 gps=5 nf=0 nn=133235 |
| 6 np=120834 qsp=9902 rpq=2 cbr=0 cng=0 gpc=6 gps=3 nf=2 nn=110921 |
| 7 np=144888 qsp=26336 rpq=0 cbr=0 cng=0 gpc=8 gps=2 nf=0 nn=118542 |
| |
| As always, this is once again split into "rcu_sched" and "rcu_bh" |
| portions, with CONFIG_TREE_PREEMPT_RCU kernels having an additional |
| "rcu_preempt" section. The fields are as follows: |
| |
| o "np" is the number of times that __rcu_pending() has been invoked |
| for the corresponding flavor of RCU. |
| |
| o "qsp" is the number of times that the RCU was waiting for a |
| quiescent state from this CPU. |
| |
| o "rpq" is the number of times that the CPU had passed through |
| a quiescent state, but not yet reported it to RCU. |
| |
| o "cbr" is the number of times that this CPU had RCU callbacks |
| that had passed through a grace period, and were thus ready |
| to be invoked. |
| |
| o "cng" is the number of times that this CPU needed another |
| grace period while RCU was idle. |
| |
| o "gpc" is the number of times that an old grace period had |
| completed, but this CPU was not yet aware of it. |
| |
| o "gps" is the number of times that a new grace period had started, |
| but this CPU was not yet aware of it. |
| |
| o "nf" is the number of times that this CPU suspected that the |
| current grace period had run for too long, and thus needed to |
| be forced. |
| |
| Please note that "forcing" consists of sending resched IPIs |
| to holdout CPUs. If that CPU really still is in an old RCU |
| read-side critical section, then we really do have to wait for it. |
| The assumption behing "forcing" is that the CPU is not still in |
| an old RCU read-side critical section, but has not yet responded |
| for some other reason. |
| |
| o "nn" is the number of times that this CPU needed nothing. Alert |
| readers will note that the rcu "nn" number for a given CPU very |
| closely matches the rcu_bh "np" number for that same CPU. This |
| is due to short-circuit evaluation in rcu_pending(). |
| |
| |
| The output of "cat rcu/rcutorture" looks as follows: |
| |
| rcutorture test sequence: 0 (test in progress) |
| rcutorture update version number: 615 |
| |
| The first line shows the number of rcutorture tests that have completed |
| since boot. If a test is currently running, the "(test in progress)" |
| string will appear as shown above. The second line shows the number of |
| update cycles that the current test has started, or zero if there is |
| no test in progress. |
| |
| |
| The output of "cat rcu/rcuboost" looks as follows: |
| |
| 0:5 tasks=.... kt=W ntb=0 neb=0 nnb=0 j=2f95 bt=300f |
| balk: nt=0 egt=989 bt=0 nb=0 ny=0 nos=16 |
| 6:7 tasks=.... kt=W ntb=0 neb=0 nnb=0 j=2f95 bt=300f |
| balk: nt=0 egt=225 bt=0 nb=0 ny=0 nos=6 |
| |
| This information is output only for rcu_preempt. Each two-line entry |
| corresponds to a leaf rcu_node strcuture. The fields are as follows: |
| |
| o "n:m" is the CPU-number range for the corresponding two-line |
| entry. In the sample output above, the first entry covers |
| CPUs zero through five and the second entry covers CPUs 6 |
| and 7. |
| |
| o "tasks=TNEB" gives the state of the various segments of the |
| rnp->blocked_tasks list: |
| |
| "T" This indicates that there are some tasks that blocked |
| while running on one of the corresponding CPUs while |
| in an RCU read-side critical section. |
| |
| "N" This indicates that some of the blocked tasks are preventing |
| the current normal (non-expedited) grace period from |
| completing. |
| |
| "E" This indicates that some of the blocked tasks are preventing |
| the current expedited grace period from completing. |
| |
| "B" This indicates that some of the blocked tasks are in |
| need of RCU priority boosting. |
| |
| Each character is replaced with "." if the corresponding |
| condition does not hold. |
| |
| o "kt" is the state of the RCU priority-boosting kernel |
| thread associated with the corresponding rcu_node structure. |
| The state can be one of the following: |
| |
| "S" The kernel thread is stopped, in other words, all |
| CPUs corresponding to this rcu_node structure are |
| offline. |
| |
| "R" The kernel thread is running. |
| |
| "W" The kernel thread is waiting because there is no work |
| for it to do. |
| |
| "Y" The kernel thread is yielding to avoid hogging CPU. |
| |
| "?" Unknown value, indicates a bug. |
| |
| o "ntb" is the number of tasks boosted. |
| |
| o "neb" is the number of tasks boosted in order to complete an |
| expedited grace period. |
| |
| o "nnb" is the number of tasks boosted in order to complete a |
| normal (non-expedited) grace period. When boosting a task |
| that was blocking both an expedited and a normal grace period, |
| it is counted against the expedited total above. |
| |
| o "j" is the low-order 16 bits of the jiffies counter in |
| hexadecimal. |
| |
| o "bt" is the low-order 16 bits of the value that the jiffies |
| counter will have when we next start boosting, assuming that |
| the current grace period does not end beforehand. This is |
| also in hexadecimal. |
| |
| o "balk: nt" counts the number of times we didn't boost (in |
| other words, we balked) even though it was time to boost because |
| there were no blocked tasks to boost. This situation occurs |
| when there is one blocked task on one rcu_node structure and |
| none on some other rcu_node structure. |
| |
| o "egt" counts the number of times we balked because although |
| there were blocked tasks, none of them were blocking the |
| current grace period, whether expedited or otherwise. |
| |
| o "bt" counts the number of times we balked because boosting |
| had already been initiated for the current grace period. |
| |
| o "nb" counts the number of times we balked because there |
| was at least one task blocking the current non-expedited grace |
| period that never had blocked. If it is already running, it |
| just won't help to boost its priority! |
| |
| o "ny" counts the number of times we balked because it was |
| not yet time to start boosting. |
| |
| o "nos" counts the number of times we balked for other |
| reasons, e.g., the grace period ended first. |
| |
| |
| CONFIG_TINY_RCU and CONFIG_TINY_PREEMPT_RCU debugfs Files and Formats |
| |
| These implementations of RCU provides a single debugfs file under the |
| top-level directory RCU, namely rcu/rcudata, which displays fields in |
| rcu_bh_ctrlblk, rcu_sched_ctrlblk and, for CONFIG_TINY_PREEMPT_RCU, |
| rcu_preempt_ctrlblk. |
| |
| The output of "cat rcu/rcudata" is as follows: |
| |
| rcu_preempt: qlen=24 gp=1097669 g197/p197/c197 tasks=... |
| ttb=. btg=no ntb=184 neb=0 nnb=183 j=01f7 bt=0274 |
| normal balk: nt=1097669 gt=0 bt=371 b=0 ny=25073378 nos=0 |
| exp balk: bt=0 nos=0 |
| rcu_sched: qlen: 0 |
| rcu_bh: qlen: 0 |
| |
| This is split into rcu_preempt, rcu_sched, and rcu_bh sections, with the |
| rcu_preempt section appearing only in CONFIG_TINY_PREEMPT_RCU builds. |
| The last three lines of the rcu_preempt section appear only in |
| CONFIG_RCU_BOOST kernel builds. The fields are as follows: |
| |
| o "qlen" is the number of RCU callbacks currently waiting either |
| for an RCU grace period or waiting to be invoked. This is the |
| only field present for rcu_sched and rcu_bh, due to the |
| short-circuiting of grace period in those two cases. |
| |
| o "gp" is the number of grace periods that have completed. |
| |
| o "g197/p197/c197" displays the grace-period state, with the |
| "g" number being the number of grace periods that have started |
| (mod 256), the "p" number being the number of grace periods |
| that the CPU has responded to (also mod 256), and the "c" |
| number being the number of grace periods that have completed |
| (once again mode 256). |
| |
| Why have both "gp" and "g"? Because the data flowing into |
| "gp" is only present in a CONFIG_RCU_TRACE kernel. |
| |
| o "tasks" is a set of bits. The first bit is "T" if there are |
| currently tasks that have recently blocked within an RCU |
| read-side critical section, the second bit is "N" if any of the |
| aforementioned tasks are blocking the current RCU grace period, |
| and the third bit is "E" if any of the aforementioned tasks are |
| blocking the current expedited grace period. Each bit is "." |
| if the corresponding condition does not hold. |
| |
| o "ttb" is a single bit. It is "B" if any of the blocked tasks |
| need to be priority boosted and "." otherwise. |
| |
| o "btg" indicates whether boosting has been carried out during |
| the current grace period, with "exp" indicating that boosting |
| is in progress for an expedited grace period, "no" indicating |
| that boosting has not yet started for a normal grace period, |
| "begun" indicating that boosting has bebug for a normal grace |
| period, and "done" indicating that boosting has completed for |
| a normal grace period. |
| |
| o "ntb" is the total number of tasks subjected to RCU priority boosting |
| periods since boot. |
| |
| o "neb" is the number of expedited grace periods that have had |
| to resort to RCU priority boosting since boot. |
| |
| o "nnb" is the number of normal grace periods that have had |
| to resort to RCU priority boosting since boot. |
| |
| o "j" is the low-order 16 bits of the jiffies counter in hexadecimal. |
| |
| o "bt" is the low-order 16 bits of the value that the jiffies counter |
| will have at the next time that boosting is scheduled to begin. |
| |
| o In the line beginning with "normal balk", the fields are as follows: |
| |
| o "nt" is the number of times that the system balked from |
| boosting because there were no blocked tasks to boost. |
| Note that the system will balk from boosting even if the |
| grace period is overdue when the currently running task |
| is looping within an RCU read-side critical section. |
| There is no point in boosting in this case, because |
| boosting a running task won't make it run any faster. |
| |
| o "gt" is the number of times that the system balked |
| from boosting because, although there were blocked tasks, |
| none of them were preventing the current grace period |
| from completing. |
| |
| o "bt" is the number of times that the system balked |
| from boosting because boosting was already in progress. |
| |
| o "b" is the number of times that the system balked from |
| boosting because boosting had already completed for |
| the grace period in question. |
| |
| o "ny" is the number of times that the system balked from |
| boosting because it was not yet time to start boosting |
| the grace period in question. |
| |
| o "nos" is the number of times that the system balked from |
| boosting for inexplicable ("not otherwise specified") |
| reasons. This can actually happen due to races involving |
| increments of the jiffies counter. |
| |
| o In the line beginning with "exp balk", the fields are as follows: |
| |
| o "bt" is the number of times that the system balked from |
| boosting because there were no blocked tasks to boost. |
| |
| o "nos" is the number of times that the system balked from |
| boosting for inexplicable ("not otherwise specified") |
| reasons. |