blob: 5001280e9d824d360cfb5589eb40751968b841d3 [file] [log] [blame]
Ingo Molnarf3e97da2006-07-03 00:24:52 -07001Runtime locking correctness validator
2=====================================
3
4started by Ingo Molnar <mingo@redhat.com>
5additions by Arjan van de Ven <arjan@linux.intel.com>
6
7Lock-class
8----------
9
10The basic object the validator operates upon is a 'class' of locks.
11
12A class of locks is a group of locks that are logically the same with
13respect to locking rules, even if the locks may have multiple (possibly
14tens of thousands of) instantiations. For example a lock in the inode
15struct is one class, while each inode has its own instantiation of that
16lock class.
17
18The validator tracks the 'state' of lock-classes, and it tracks
19dependencies between different lock-classes. The validator maintains a
20rolling proof that the state and the dependencies are correct.
21
22Unlike an lock instantiation, the lock-class itself never goes away: when
23a lock-class is used for the first time after bootup it gets registered,
24and all subsequent uses of that lock-class will be attached to this
25lock-class.
26
27State
28-----
29
Peter Zijlstraf510b232009-01-22 17:53:47 +010030The validator tracks lock-class usage history into 4n + 1 separate state bits:
Ingo Molnarf3e97da2006-07-03 00:24:52 -070031
Peter Zijlstraf510b232009-01-22 17:53:47 +010032- 'ever held in STATE context'
Li Zefan0e692a92009-08-07 15:10:54 +080033- 'ever held as readlock in STATE context'
34- 'ever held with STATE enabled'
35- 'ever held as readlock with STATE enabled'
Peter Zijlstraf510b232009-01-22 17:53:47 +010036
SeongJae Parka97af332015-01-14 04:22:01 +090037Where STATE can be either one of (kernel/locking/lockdep_states.h)
Peter Zijlstraf510b232009-01-22 17:53:47 +010038 - hardirq
39 - softirq
40 - reclaim_fs
Ingo Molnarf3e97da2006-07-03 00:24:52 -070041
42- 'ever used' [ == !unused ]
43
Peter Zijlstraf510b232009-01-22 17:53:47 +010044When locking rules are violated, these state bits are presented in the
45locking error messages, inside curlies. A contrived example:
Jim Cromiefd7bcea2006-09-30 23:27:40 -070046
47 modprobe/2287 is trying to acquire lock:
Peter Zijlstraf510b232009-01-22 17:53:47 +010048 (&sio_locks[i].lock){-.-...}, at: [<c02867fd>] mutex_lock+0x21/0x24
Jim Cromiefd7bcea2006-09-30 23:27:40 -070049
50 but task is already holding lock:
Peter Zijlstraf510b232009-01-22 17:53:47 +010051 (&sio_locks[i].lock){-.-...}, at: [<c02867fd>] mutex_lock+0x21/0x24
Jim Cromiefd7bcea2006-09-30 23:27:40 -070052
53
Peter Zijlstraf510b232009-01-22 17:53:47 +010054The bit position indicates STATE, STATE-read, for each of the states listed
55above, and the character displayed in each indicates:
Jim Cromiefd7bcea2006-09-30 23:27:40 -070056
Ming Lei992d7ce2009-04-24 23:10:06 +080057 '.' acquired while irqs disabled and not in irq context
58 '-' acquired in irq context
59 '+' acquired with irqs enabled
Peter Zijlstraf510b232009-01-22 17:53:47 +010060 '?' acquired in irq context with irqs enabled.
Jim Cromiefd7bcea2006-09-30 23:27:40 -070061
62Unused mutexes cannot be part of the cause of an error.
63
64
Ingo Molnarf3e97da2006-07-03 00:24:52 -070065Single-lock state rules:
66------------------------
67
68A softirq-unsafe lock-class is automatically hardirq-unsafe as well. The
69following states are exclusive, and only one of them is allowed to be
70set for any lock-class:
71
72 <hardirq-safe> and <hardirq-unsafe>
73 <softirq-safe> and <softirq-unsafe>
74
75The validator detects and reports lock usage that violate these
76single-lock state rules.
77
78Multi-lock dependency rules:
79----------------------------
80
81The same lock-class must not be acquired twice, because this could lead
82to lock recursion deadlocks.
83
84Furthermore, two locks may not be taken in different order:
85
86 <L1> -> <L2>
87 <L2> -> <L1>
88
89because this could lead to lock inversion deadlocks. (The validator
90finds such dependencies in arbitrary complexity, i.e. there can be any
91other locking sequence between the acquire-lock operations, the
92validator will still track all dependencies between locks.)
93
94Furthermore, the following usage based lock dependencies are not allowed
95between any two lock-classes:
96
97 <hardirq-safe> -> <hardirq-unsafe>
98 <softirq-safe> -> <softirq-unsafe>
99
100The first rule comes from the fact the a hardirq-safe lock could be
101taken by a hardirq context, interrupting a hardirq-unsafe lock - and
102thus could result in a lock inversion deadlock. Likewise, a softirq-safe
103lock could be taken by an softirq context, interrupting a softirq-unsafe
104lock.
105
106The above rules are enforced for any locking sequence that occurs in the
107kernel: when acquiring a new lock, the validator checks whether there is
108any rule violation between the new lock and any of the held locks.
109
110When a lock-class changes its state, the following aspects of the above
111dependency rules are enforced:
112
113- if a new hardirq-safe lock is discovered, we check whether it
114 took any hardirq-unsafe lock in the past.
115
116- if a new softirq-safe lock is discovered, we check whether it took
117 any softirq-unsafe lock in the past.
118
119- if a new hardirq-unsafe lock is discovered, we check whether any
120 hardirq-safe lock took it in the past.
121
122- if a new softirq-unsafe lock is discovered, we check whether any
123 softirq-safe lock took it in the past.
124
125(Again, we do these checks too on the basis that an interrupt context
126could interrupt _any_ of the irq-unsafe or hardirq-unsafe locks, which
127could lead to a lock inversion deadlock - even if that lock scenario did
128not trigger in practice yet.)
129
130Exception: Nested data dependencies leading to nested locking
131-------------------------------------------------------------
132
133There are a few cases where the Linux kernel acquires more than one
134instance of the same lock-class. Such cases typically happen when there
135is some sort of hierarchy within objects of the same type. In these
136cases there is an inherent "natural" ordering between the two objects
137(defined by the properties of the hierarchy), and the kernel grabs the
138locks in this fixed order on each of the objects.
139
Matt LaPlante2fe0ae72006-10-03 22:50:39 +0200140An example of such an object hierarchy that results in "nested locking"
Ingo Molnarf3e97da2006-07-03 00:24:52 -0700141is that of a "whole disk" block-dev object and a "partition" block-dev
142object; the partition is "part of" the whole device and as long as one
143always takes the whole disk lock as a higher lock than the partition
144lock, the lock ordering is fully correct. The validator does not
145automatically detect this natural ordering, as the locking rule behind
146the ordering is not static.
147
148In order to teach the validator about this correct usage model, new
149versions of the various locking primitives were added that allow you to
150specify a "nesting level". An example call, for the block device mutex,
151looks like this:
152
153enum bdev_bd_mutex_lock_class
154{
155 BD_MUTEX_NORMAL,
156 BD_MUTEX_WHOLE,
157 BD_MUTEX_PARTITION
158};
159
160 mutex_lock_nested(&bdev->bd_contains->bd_mutex, BD_MUTEX_PARTITION);
161
162In this case the locking is done on a bdev object that is known to be a
163partition.
164
Matt LaPlantea2ffd272006-10-03 22:49:15 +0200165The validator treats a lock that is taken in such a nested fashion as a
Ingo Molnarf3e97da2006-07-03 00:24:52 -0700166separate (sub)class for the purposes of validation.
167
168Note: When changing code to use the _nested() primitives, be careful and
Matt LaPlante2fe0ae72006-10-03 22:50:39 +0200169check really thoroughly that the hierarchy is correctly mapped; otherwise
Ingo Molnarf3e97da2006-07-03 00:24:52 -0700170you can get false positives or false negatives.
171
172Proof of 100% correctness:
173--------------------------
174
175The validator achieves perfect, mathematical 'closure' (proof of locking
176correctness) in the sense that for every simple, standalone single-task
Matt LaPlante992caac2006-10-03 22:52:05 +0200177locking sequence that occurred at least once during the lifetime of the
Ingo Molnarf3e97da2006-07-03 00:24:52 -0700178kernel, the validator proves it with a 100% certainty that no
179combination and timing of these locking sequences can cause any class of
180lock related deadlock. [*]
181
182I.e. complex multi-CPU and multi-task locking scenarios do not have to
183occur in practice to prove a deadlock: only the simple 'component'
184locking chains have to occur at least once (anytime, in any
185task/context) for the validator to be able to prove correctness. (For
186example, complex deadlocks that would normally need more than 3 CPUs and
187a very unlikely constellation of tasks, irq-contexts and timings to
188occur, can be detected on a plain, lightly loaded single-CPU system as
189well!)
190
191This radically decreases the complexity of locking related QA of the
192kernel: what has to be done during QA is to trigger as many "simple"
193single-task locking dependencies in the kernel as possible, at least
194once, to prove locking correctness - instead of having to trigger every
195possible combination of locking interaction between CPUs, combined with
196every possible hardirq and softirq nesting scenario (which is impossible
197to do in practice).
198
199[*] assuming that the validator itself is 100% correct, and no other
200 part of the system corrupts the state of the validator in any way.
201 We also assume that all NMI/SMM paths [which could interrupt
202 even hardirq-disabled codepaths] are correct and do not interfere
203 with the validator. We also assume that the 64-bit 'chain hash'
204 value is unique for every lock-chain in the system. Also, lock
205 recursion must not be higher than 20.
206
207Performance:
208------------
209
210The above rules require _massive_ amounts of runtime checking. If we did
211that for every lock taken and for every irqs-enable event, it would
212render the system practically unusably slow. The complexity of checking
213is O(N^2), so even with just a few hundred lock-classes we'd have to do
214tens of thousands of checks for every event.
215
216This problem is solved by checking any given 'locking scenario' (unique
217sequence of locks taken after each other) only once. A simple stack of
218held locks is maintained, and a lightweight 64-bit hash value is
219calculated, which hash is unique for every lock chain. The hash value,
220when the chain is validated for the first time, is then put into a hash
221table, which hash-table can be checked in a lockfree manner. If the
222locking chain occurs again later on, the hash table tells us that we
223dont have to validate the chain again.
Paul E. McKenneyb804cb92011-09-28 10:23:39 -0700224
225Troubleshooting:
226----------------
227
228The validator tracks a maximum of MAX_LOCKDEP_KEYS number of lock classes.
229Exceeding this number will trigger the following lockdep warning:
230
231 (DEBUG_LOCKS_WARN_ON(id >= MAX_LOCKDEP_KEYS))
232
233By default, MAX_LOCKDEP_KEYS is currently set to 8191, and typical
234desktop systems have less than 1,000 lock classes, so this warning
235normally results from lock-class leakage or failure to properly
236initialize locks. These two problems are illustrated below:
237
2381. Repeated module loading and unloading while running the validator
239 will result in lock-class leakage. The issue here is that each
240 load of the module will create a new set of lock classes for
241 that module's locks, but module unloading does not remove old
242 classes (see below discussion of reuse of lock classes for why).
243 Therefore, if that module is loaded and unloaded repeatedly,
244 the number of lock classes will eventually reach the maximum.
245
2462. Using structures such as arrays that have large numbers of
247 locks that are not explicitly initialized. For example,
248 a hash table with 8192 buckets where each bucket has its own
249 spinlock_t will consume 8192 lock classes -unless- each spinlock
250 is explicitly initialized at runtime, for example, using the
251 run-time spin_lock_init() as opposed to compile-time initializers
252 such as __SPIN_LOCK_UNLOCKED(). Failure to properly initialize
253 the per-bucket spinlocks would guarantee lock-class overflow.
254 In contrast, a loop that called spin_lock_init() on each lock
255 would place all 8192 locks into a single lock class.
256
257 The moral of this story is that you should always explicitly
258 initialize your locks.
259
260One might argue that the validator should be modified to allow
261lock classes to be reused. However, if you are tempted to make this
262argument, first review the code and think through the changes that would
263be required, keeping in mind that the lock classes to be removed are
264likely to be linked into the lock-dependency graph. This turns out to
265be harder to do than to say.
266
267Of course, if you do run out of lock classes, the next thing to do is
268to find the offending lock classes. First, the following command gives
269you the number of lock classes currently in use along with the maximum:
270
271 grep "lock-classes" /proc/lockdep_stats
272
273This command produces the following output on a modest system:
274
275 lock-classes: 748 [max: 8191]
276
277If the number allocated (748 above) increases continually over time,
278then there is likely a leak. The following command can be used to
279identify the leaking lock classes:
280
281 grep "BD" /proc/lockdep
282
283Run the command and save the output, then compare against the output from
284a later run of this command to identify the leakers. This same output
285can also help you find situations where runtime lock initialization has
286been omitted.