blob: 19533f93b7a2ac4175142947142569199dacc51f [file] [log] [blame]
KAMEZAWA Hiroyuki9836d892009-01-07 18:08:27 -08001Memory Resource Controller(Memcg) Implementation Memo.
KAMEZAWA Hiroyuki03f3c432009-01-07 18:08:31 -08002Last Updated: 2008/12/15
3Base Kernel Version: based on 2.6.28-rc8-mm.
KAMEZAWA Hiroyuki9836d892009-01-07 18:08:27 -08004
5Because VM is getting complex (one of reasons is memcg...), memcg's behavior
6is complex. This is a document for memcg's internal behavior.
7Please note that implementation details can be changed.
8
Li Zefan45ce80f2009-01-15 13:50:59 -08009(*) Topics on API should be in Documentation/cgroups/memory.txt)
KAMEZAWA Hiroyuki9836d892009-01-07 18:08:27 -080010
110. How to record usage ?
12 2 objects are used.
13
14 page_cgroup ....an object per page.
15 Allocated at boot or memory hotplug. Freed at memory hot removal.
16
17 swap_cgroup ... an entry per swp_entry.
18 Allocated at swapon(). Freed at swapoff().
19
20 The page_cgroup has USED bit and double count against a page_cgroup never
21 occurs. swap_cgroup is used only when a charged page is swapped-out.
22
231. Charge
24
25 a page/swp_entry may be charged (usage += PAGE_SIZE) at
26
27 mem_cgroup_newpage_charge()
28 Called at new page fault and Copy-On-Write.
29
30 mem_cgroup_try_charge_swapin()
31 Called at do_swap_page() (page fault on swap entry) and swapoff.
32 Followed by charge-commit-cancel protocol. (With swap accounting)
33 At commit, a charge recorded in swap_cgroup is removed.
34
35 mem_cgroup_cache_charge()
36 Called at add_to_page_cache()
37
38 mem_cgroup_cache_charge_swapin()
39 Called at shmem's swapin.
40
41 mem_cgroup_prepare_migration()
42 Called before migration. "extra" charge is done and followed by
43 charge-commit-cancel protocol.
44 At commit, charge against oldpage or newpage will be committed.
45
462. Uncharge
47 a page/swp_entry may be uncharged (usage -= PAGE_SIZE) by
48
49 mem_cgroup_uncharge_page()
50 Called when an anonymous page is fully unmapped. I.e., mapcount goes
51 to 0. If the page is SwapCache, uncharge is delayed until
52 mem_cgroup_uncharge_swapcache().
53
54 mem_cgroup_uncharge_cache_page()
55 Called when a page-cache is deleted from radix-tree. If the page is
56 SwapCache, uncharge is delayed until mem_cgroup_uncharge_swapcache().
57
58 mem_cgroup_uncharge_swapcache()
59 Called when SwapCache is removed from radix-tree. The charge itself
60 is moved to swap_cgroup. (If mem+swap controller is disabled, no
61 charge to swap occurs.)
62
63 mem_cgroup_uncharge_swap()
64 Called when swp_entry's refcnt goes down to 0. A charge against swap
65 disappears.
66
67 mem_cgroup_end_migration(old, new)
68 At success of migration old is uncharged (if necessary), a charge
69 to new page is committed. At failure, charge to old page is committed.
70
713. charge-commit-cancel
72 In some case, we can't know this "charge" is valid or not at charging
73 (because of races).
74 To handle such case, there are charge-commit-cancel functions.
75 mem_cgroup_try_charge_XXX
76 mem_cgroup_commit_charge_XXX
77 mem_cgroup_cancel_charge_XXX
78 these are used in swap-in and migration.
79
80 At try_charge(), there are no flags to say "this page is charged".
81 at this point, usage += PAGE_SIZE.
82
83 At commit(), the function checks the page should be charged or not
84 and set flags or avoid charging.(usage -= PAGE_SIZE)
85
86 At cancel(), simply usage -= PAGE_SIZE.
87
88Under below explanation, we assume CONFIG_MEM_RES_CTRL_SWAP=y.
89
904. Anonymous
91 Anonymous page is newly allocated at
92 - page fault into MAP_ANONYMOUS mapping.
93 - Copy-On-Write.
94 It is charged right after it's allocated before doing any page table
95 related operations. Of course, it's uncharged when another page is used
96 for the fault address.
97
98 At freeing anonymous page (by exit() or munmap()), zap_pte() is called
99 and pages for ptes are freed one by one.(see mm/memory.c). Uncharges
100 are done at page_remove_rmap() when page_mapcount() goes down to 0.
101
102 Another page freeing is by page-reclaim (vmscan.c) and anonymous
103 pages are swapped out. In this case, the page is marked as
104 PageSwapCache(). uncharge() routine doesn't uncharge the page marked
105 as SwapCache(). It's delayed until __delete_from_swap_cache().
106
107 4.1 Swap-in.
108 At swap-in, the page is taken from swap-cache. There are 2 cases.
109
110 (a) If the SwapCache is newly allocated and read, it has no charges.
111 (b) If the SwapCache has been mapped by processes, it has been
112 charged already.
113
KAMEZAWA Hiroyuki03f3c432009-01-07 18:08:31 -0800114 This swap-in is one of the most complicated work. In do_swap_page(),
115 following events occur when pte is unchanged.
116
117 (1) the page (SwapCache) is looked up.
118 (2) lock_page()
119 (3) try_charge_swapin()
120 (4) reuse_swap_page() (may call delete_swap_cache())
121 (5) commit_charge_swapin()
122 (6) swap_free().
123
124 Considering following situation for example.
125
126 (A) The page has not been charged before (2) and reuse_swap_page()
127 doesn't call delete_from_swap_cache().
128 (B) The page has not been charged before (2) and reuse_swap_page()
129 calls delete_from_swap_cache().
130 (C) The page has been charged before (2) and reuse_swap_page() doesn't
131 call delete_from_swap_cache().
132 (D) The page has been charged before (2) and reuse_swap_page() calls
133 delete_from_swap_cache().
134
135 memory.usage/memsw.usage changes to this page/swp_entry will be
136 Case (A) (B) (C) (D)
137 Event
138 Before (2) 0/ 1 0/ 1 1/ 1 1/ 1
139 ===========================================
140 (3) +1/+1 +1/+1 +1/+1 +1/+1
141 (4) - 0/ 0 - -1/ 0
142 (5) 0/-1 0/ 0 -1/-1 0/ 0
143 (6) - 0/-1 - 0/-1
144 ===========================================
145 Result 1/ 1 1/ 1 1/ 1 1/ 1
146
147 In any cases, charges to this page should be 1/ 1.
KAMEZAWA Hiroyuki9836d892009-01-07 18:08:27 -0800148
149 4.2 Swap-out.
150 At swap-out, typical state transition is below.
151
152 (a) add to swap cache. (marked as SwapCache)
153 swp_entry's refcnt += 1.
154 (b) fully unmapped.
155 swp_entry's refcnt += # of ptes.
156 (c) write back to swap.
157 (d) delete from swap cache. (remove from SwapCache)
158 swp_entry's refcnt -= 1.
159
160
161 At (b), the page is marked as SwapCache and not uncharged.
162 At (d), the page is removed from SwapCache and a charge in page_cgroup
163 is moved to swap_cgroup.
164
165 Finally, at task exit,
166 (e) zap_pte() is called and swp_entry's refcnt -=1 -> 0.
167 Here, a charge in swap_cgroup disappears.
168
1695. Page Cache
170 Page Cache is charged at
171 - add_to_page_cache_locked().
172
173 uncharged at
174 - __remove_from_page_cache().
175
176 The logic is very clear. (About migration, see below)
177 Note: __remove_from_page_cache() is called by remove_from_page_cache()
178 and __remove_mapping().
179
1806. Shmem(tmpfs) Page Cache
181 Memcg's charge/uncharge have special handlers of shmem. The best way
182 to understand shmem's page state transition is to read mm/shmem.c.
183 But brief explanation of the behavior of memcg around shmem will be
184 helpful to understand the logic.
185
186 Shmem's page (just leaf page, not direct/indirect block) can be on
187 - radix-tree of shmem's inode.
188 - SwapCache.
189 - Both on radix-tree and SwapCache. This happens at swap-in
190 and swap-out,
191
192 It's charged when...
193 - A new page is added to shmem's radix-tree.
194 - A swp page is read. (move a charge from swap_cgroup to page_cgroup)
195 It's uncharged when
196 - A page is removed from radix-tree and not SwapCache.
197 - When SwapCache is removed, a charge is moved to swap_cgroup.
198 - When swp_entry's refcnt goes down to 0, a charge in swap_cgroup
199 disappears.
200
2017. Page Migration
202 One of the most complicated functions is page-migration-handler.
203 Memcg has 2 routines. Assume that we are migrating a page's contents
204 from OLDPAGE to NEWPAGE.
205
206 Usual migration logic is..
207 (a) remove the page from LRU.
208 (b) allocate NEWPAGE (migration target)
209 (c) lock by lock_page().
210 (d) unmap all mappings.
211 (e-1) If necessary, replace entry in radix-tree.
212 (e-2) move contents of a page.
213 (f) map all mappings again.
214 (g) pushback the page to LRU.
215 (-) OLDPAGE will be freed.
216
217 Before (g), memcg should complete all necessary charge/uncharge to
218 NEWPAGE/OLDPAGE.
219
220 The point is....
221 - If OLDPAGE is anonymous, all charges will be dropped at (d) because
222 try_to_unmap() drops all mapcount and the page will not be
223 SwapCache.
224
225 - If OLDPAGE is SwapCache, charges will be kept at (g) because
226 __delete_from_swap_cache() isn't called at (e-1)
227
228 - If OLDPAGE is page-cache, charges will be kept at (g) because
229 remove_from_swap_cache() isn't called at (e-1)
230
231 memcg provides following hooks.
232
233 - mem_cgroup_prepare_migration(OLDPAGE)
234 Called after (b) to account a charge (usage += PAGE_SIZE) against
235 memcg which OLDPAGE belongs to.
236
237 - mem_cgroup_end_migration(OLDPAGE, NEWPAGE)
238 Called after (f) before (g).
239 If OLDPAGE is used, commit OLDPAGE again. If OLDPAGE is already
240 charged, a charge by prepare_migration() is automatically canceled.
241 If NEWPAGE is used, commit NEWPAGE and uncharge OLDPAGE.
242
243 But zap_pte() (by exit or munmap) can be called while migration,
244 we have to check if OLDPAGE/NEWPAGE is a valid page after commit().
245
2468. LRU
247 Each memcg has its own private LRU. Now, it's handling is under global
248 VM's control (means that it's handled under global zone->lru_lock).
249 Almost all routines around memcg's LRU is called by global LRU's
250 list management functions under zone->lru_lock().
251
252 A special function is mem_cgroup_isolate_pages(). This scans
253 memcg's private LRU and call __isolate_lru_page() to extract a page
254 from LRU.
255 (By __isolate_lru_page(), the page is removed from both of global and
256 private LRU.)
257
258
2599. Typical Tests.
260
261 Tests for racy cases.
262
263 9.1 Small limit to memcg.
264 When you do test to do racy case, it's good test to set memcg's limit
265 to be very small rather than GB. Many races found in the test under
266 xKB or xxMB limits.
267 (Memory behavior under GB and Memory behavior under MB shows very
268 different situation.)
269
270 9.2 Shmem
271 Historically, memcg's shmem handling was poor and we saw some amount
272 of troubles here. This is because shmem is page-cache but can be
273 SwapCache. Test with shmem/tmpfs is always good test.
274
275 9.3 Migration
276 For NUMA, migration is an another special case. To do easy test, cpuset
277 is useful. Following is a sample script to do migration.
278
279 mount -t cgroup -o cpuset none /opt/cpuset
280
281 mkdir /opt/cpuset/01
282 echo 1 > /opt/cpuset/01/cpuset.cpus
283 echo 0 > /opt/cpuset/01/cpuset.mems
284 echo 1 > /opt/cpuset/01/cpuset.memory_migrate
285 mkdir /opt/cpuset/02
286 echo 1 > /opt/cpuset/02/cpuset.cpus
287 echo 1 > /opt/cpuset/02/cpuset.mems
288 echo 1 > /opt/cpuset/02/cpuset.memory_migrate
289
290 In above set, when you moves a task from 01 to 02, page migration to
291 node 0 to node 1 will occur. Following is a script to migrate all
292 under cpuset.
293 --
294 move_task()
295 {
296 for pid in $1
297 do
298 /bin/echo $pid >$2/tasks 2>/dev/null
299 echo -n $pid
300 echo -n " "
301 done
302 echo END
303 }
304
305 G1_TASK=`cat ${G1}/tasks`
306 G2_TASK=`cat ${G2}/tasks`
307 move_task "${G1_TASK}" ${G2} &
308 --
309 9.4 Memory hotplug.
310 memory hotplug test is one of good test.
311 to offline memory, do following.
312 # echo offline > /sys/devices/system/memory/memoryXXX/state
313 (XXX is the place of memory)
314 This is an easy way to test page migration, too.
315
316 9.5 mkdir/rmdir
317 When using hierarchy, mkdir/rmdir test should be done.
318 Use tests like the following.
319
320 echo 1 >/opt/cgroup/01/memory/use_hierarchy
321 mkdir /opt/cgroup/01/child_a
322 mkdir /opt/cgroup/01/child_b
323
324 set limit to 01.
325 add limit to 01/child_b
326 run jobs under child_a and child_b
327
328 create/delete following groups at random while jobs are running.
329 /opt/cgroup/01/child_a/child_aa
330 /opt/cgroup/01/child_b/child_bb
331 /opt/cgroup/01/child_c
332
333 running new jobs in new group is also good.
334
335 9.6 Mount with other subsystems.
336 Mounting with other subsystems is a good test because there is a
337 race and lock dependency with other cgroup subsystems.
338
339 example)
340 # mount -t cgroup none /cgroup -t cpuset,memory,cpu,devices
341
342 and do task move, mkdir, rmdir etc...under this.