blob: 6ae89a9edf2aac8d3ed6a9cf00d8cbad34fbaca7 [file] [log] [blame]
Andi Kleenf58ee002009-10-04 02:28:42 +02001What is hwpoison?
2
3Upcoming Intel CPUs have support for recovering from some memory errors
4(``MCA recovery''). This requires the OS to declare a page "poisoned",
5kill the processes associated with it and avoid using it in the future.
6
7This patchkit implements the necessary infrastructure in the VM.
8
9To quote the overview comment:
10
11 * High level machine check handler. Handles pages reported by the
12 * hardware as being corrupted usually due to a 2bit ECC memory or cache
13 * failure.
14 *
15 * This focusses on pages detected as corrupted in the background.
16 * When the current CPU tries to consume corruption the currently
17 * running process can just be killed directly instead. This implies
18 * that if the error cannot be handled for some reason it's safe to
19 * just ignore it because no corruption has been consumed yet. Instead
20 * when that happens another machine check will happen.
21 *
22 * Handles page cache pages in various states. The tricky part
23 * here is that we can access any page asynchronous to other VM
24 * users, because memory failures could happen anytime and anywhere,
25 * possibly violating some of their assumptions. This is why this code
26 * has to be extremely careful. Generally it tries to use normal locking
27 * rules, as in get the standard locks, even if that means the
28 * error handling takes potentially a long time.
29 *
30 * Some of the operations here are somewhat inefficient and have non
31 * linear algorithmic complexity, because the data structures have not
32 * been optimized for this case. This is in particular the case
33 * for the mapping from a vma to a process. Since this case is expected
34 * to be rare we hope we can get away with this.
35
36The code consists of a the high level handler in mm/memory-failure.c,
37a new page poison bit and various checks in the VM to handle poisoned
38pages.
39
40The main target right now is KVM guests, but it works for all kinds
41of applications. KVM support requires a recent qemu-kvm release.
42
43For the KVM use there was need for a new signal type so that
44KVM can inject the machine check into the guest with the proper
45address. This in theory allows other applications to handle
46memory failures too. The expection is that near all applications
47won't do that, but some very specialized ones might.
48
49---
50
51There are two (actually three) modi memory failure recovery can be in:
52
53vm.memory_failure_recovery sysctl set to zero:
54 All memory failures cause a panic. Do not attempt recovery.
55 (on x86 this can be also affected by the tolerant level of the
56 MCE subsystem)
57
58early kill
59 (can be controlled globally and per process)
60 Send SIGBUS to the application as soon as the error is detected
61 This allows applications who can process memory errors in a gentle
62 way (e.g. drop affected object)
63 This is the mode used by KVM qemu.
64
65late kill
66 Send SIGBUS when the application runs into the corrupted page.
67 This is best for memory error unaware applications and default
68 Note some pages are always handled as late kill.
69
70---
71
72User control:
73
74vm.memory_failure_recovery
75 See sysctl.txt
76
77vm.memory_failure_early_kill
78 Enable early kill mode globally
79
80PR_MCE_KILL
81 Set early/late kill mode/revert to system default
82 arg1: PR_MCE_KILL_CLEAR: Revert to system default
83 arg1: PR_MCE_KILL_SET: arg2 defines thread specific mode
84 PR_MCE_KILL_EARLY: Early kill
85 PR_MCE_KILL_LATE: Late kill
86 PR_MCE_KILL_DEFAULT: Use system global default
Naoya Horiguchi3ba08122014-06-04 16:11:02 -070087 Note that if you want to have a dedicated thread which handles
88 the SIGBUS(BUS_MCEERR_AO) on behalf of the process, you should
89 call prctl(PR_MCE_KILL_EARLY) on the designated thread. Otherwise,
90 the SIGBUS is sent to the main thread.
91
Andi Kleenf58ee002009-10-04 02:28:42 +020092PR_MCE_KILL_GET
93 return current mode
94
95
96---
97
98Testing:
99
Andi Kleenfe194d32009-12-16 12:20:00 +0100100madvise(MADV_HWPOISON, ....)
Andi Kleenf58ee002009-10-04 02:28:42 +0200101 (as root)
102 Poison a page in the process for testing
103
104
105hwpoison-inject module through debugfs
Andi Kleenf58ee002009-10-04 02:28:42 +0200106
Wu Fengguang847ce402009-12-16 12:19:58 +0100107/sys/debug/hwpoison/
Andi Kleenf58ee002009-10-04 02:28:42 +0200108
Wu Fengguang847ce402009-12-16 12:19:58 +0100109corrupt-pfn
110
Wu Fengguang31d3d342009-12-16 12:19:59 +0100111Inject hwpoison fault at PFN echoed into this file. This does
112some early filtering to avoid corrupted unintended pages in test suites.
Wu Fengguang847ce402009-12-16 12:19:58 +0100113
114unpoison-pfn
115
116Software-unpoison page at PFN echoed into this file. This
117way a page can be reused again.
118This only works for Linux injected failures, not for real
119memory failures.
120
121Note these injection interfaces are not stable and might change between
122kernel versions
Andi Kleenf58ee002009-10-04 02:28:42 +0200123
Wu Fengguang7c116f22009-12-16 12:19:59 +0100124corrupt-filter-dev-major
125corrupt-filter-dev-minor
126
127Only handle memory failures to pages associated with the file system defined
128by block device major/minor. -1U is the wildcard value.
129This should be only used for testing with artificial injection.
130
Andi Kleen4fd466e2009-12-16 12:19:59 +0100131corrupt-filter-memcg
132
133Limit injection to pages owned by memgroup. Specified by inode number
134of the memcg.
135
136Example:
Jörg Sommerf6e07d32011-06-15 12:59:45 -0700137 mkdir /sys/fs/cgroup/mem/hwpoison
Andi Kleen4fd466e2009-12-16 12:19:59 +0100138
139 usemem -m 100 -s 1000 &
Jörg Sommerf6e07d32011-06-15 12:59:45 -0700140 echo `jobs -p` > /sys/fs/cgroup/mem/hwpoison/tasks
Andi Kleen4fd466e2009-12-16 12:19:59 +0100141
Jörg Sommerf6e07d32011-06-15 12:59:45 -0700142 memcg_ino=$(ls -id /sys/fs/cgroup/mem/hwpoison | cut -f1 -d' ')
Andi Kleen4fd466e2009-12-16 12:19:59 +0100143 echo $memcg_ino > /debug/hwpoison/corrupt-filter-memcg
144
145 page-types -p `pidof init` --hwpoison # shall do nothing
146 page-types -p `pidof usemem` --hwpoison # poison its pages
Wu Fengguang478c5ff2009-12-16 12:19:59 +0100147
148corrupt-filter-flags-mask
149corrupt-filter-flags-value
150
151When specified, only poison pages if ((page_flags & mask) == value).
152This allows stress testing of many kinds of pages. The page_flags
153are the same as in /proc/kpageflags. The flag bits are defined in
154include/linux/kernel-page-flags.h and documented in
155Documentation/vm/pagemap.txt
156
Andi Kleenf58ee002009-10-04 02:28:42 +0200157Architecture specific MCE injector
158
159x86 has mce-inject, mce-test
160
161Some portable hwpoison test programs in mce-test, see blow.
162
163---
164
165References:
166
167http://halobates.de/mce-lc09-2.pdf
168 Overview presentation from LinuxCon 09
169
170git://git.kernel.org/pub/scm/utils/cpu/mce/mce-test.git
171 Test suite (hwpoison specific portable tests in tsrc)
172
173git://git.kernel.org/pub/scm/utils/cpu/mce/mce-inject.git
174 x86 specific injector
175
176
177---
178
179Limitations:
180
181- Not all page types are supported and never will. Most kernel internal
182objects cannot be recovered, only LRU pages for now.
183- Right now hugepage support is missing.
184
185---
186Andi Kleen, Oct 2009
187