blob: f0465064c13dee194fe36a9e3157fbc818dc7403 [file] [log] [blame]
Sasha Goldshteindda47692016-02-08 03:10:13 -08001Demonstrations of memleak.
2
3
4memleak traces and matches memory allocation and deallocation requests, and
5collects call stacks for each allocation. memleak can then print a summary
6of which call stacks performed allocations that weren't subsequently freed.
7For example:
8
9# ./memleak.py -p $(pidof allocs)
10Attaching to malloc and free in pid 5193, Ctrl+C to quit.
11*** Outstanding allocations:
12 80 bytes in 5 allocations from stack
13 main+0x6d [/home/vagrant/allocs] (400862)
14 __libc_start_main+0xf0 [/usr/lib64/libc-2.21.so] (7fd460ac2790)
15
16*** Outstanding allocations:
17 160 bytes in 10 allocations from stack
18 main+0x6d [/home/vagrant/allocs] (400862)
19 __libc_start_main+0xf0 [/usr/lib64/libc-2.21.so] (7fd460ac2790)
20
21
Sasha Goldshteinb9b9ad32016-02-08 03:41:43 -080022Each entry printed is a set of allocations that originate from the same call
23stack, and that weren't freed yet. The number of bytes and number of allocs
24are followed by the call stack, top to bottom, of the allocation site.
25
Sasha Goldshtein33522d72016-02-08 03:39:44 -080026As time goes on, it becomes apparent that the main function in the allocs
27process is leaking memory, 16 bytes at a time. Fortunately, you don't have to
28inspect each allocation individually -- you get a nice summary of which stack
29is responsible for a large leak.
30
31Occasionally, you do want the individual allocation details. Perhaps the same
32stack is allocating various sizes and you want to confirm which sizes are
33prevalent. Use the -a switch:
34
35# ./memleak.py -p $(pidof allocs) -a
36Attaching to malloc and free in pid 5193, Ctrl+C to quit.
37*** Outstanding allocations:
38 addr = 948cd0 size = 16
39 addr = 948d10 size = 16
40 addr = 948d30 size = 16
41 addr = 948cf0 size = 16
42 64 bytes in 4 allocations from stack
43 main+0x6d [/home/vagrant/allocs] (400862)
44 __libc_start_main+0xf0 [/usr/lib64/libc-2.21.so] (7fd460ac2790)
45
46*** Outstanding allocations:
47 addr = 948d50 size = 16
48 addr = 948cd0 size = 16
49 addr = 948d10 size = 16
50 addr = 948d30 size = 16
51 addr = 948cf0 size = 16
52 addr = 948dd0 size = 16
53 addr = 948d90 size = 16
54 addr = 948db0 size = 16
55 addr = 948d70 size = 16
56 addr = 948df0 size = 16
57 160 bytes in 10 allocations from stack
58 main+0x6d [/home/vagrant/allocs] (400862)
59 __libc_start_main+0xf0 [/usr/lib64/libc-2.21.so] (7fd460ac2790)
60
61
Sasha Goldshteindda47692016-02-08 03:10:13 -080062When using the -p switch, memleak traces the allocations of a particular
63process. Without this switch, kernel allocations (kmalloc) are traced instead.
64For example:
65
66# ./memleak.py
67Attaching to kmalloc and kfree, Ctrl+C to quit.
68...
69 248 bytes in 4 allocations from stack
70 bpf_prog_load [kernel] (ffffffff8118c471)
71 sys_bpf [kernel] (ffffffff8118c8b5)
72
73 328 bytes in 1 allocations from stack
74 perf_mmap [kernel] (ffffffff811990fd)
75 mmap_region [kernel] (ffffffff811df5d4)
76 do_mmap [kernel] (ffffffff811dfb83)
77 vm_mmap_pgoff [kernel] (ffffffff811c494f)
78 sys_mmap_pgoff [kernel] (ffffffff811ddf02)
79 sys_mmap [kernel] (ffffffff8101b0ab)
80
81 464 bytes in 1 allocations from stack
82 traceprobe_command [kernel] (ffffffff81187cf2)
83 traceprobe_probes_write [kernel] (ffffffff81187d86)
84 probes_write [kernel] (ffffffff81181580)
85 __vfs_write [kernel] (ffffffff812237b7)
86 vfs_write [kernel] (ffffffff81223ec6)
87 sys_write [kernel] (ffffffff81224b85)
88 entry_SYSCALL_64_fastpath [kernel] (ffffffff8178182e)
89
90 8192 bytes in 1 allocations from stack
91 alloc_and_copy_ftrace_hash.constprop.59 [kernel] (ffffffff8115d17e)
92 ftrace_set_hash [kernel] (ffffffff8115e767)
93 ftrace_set_filter_ip [kernel] (ffffffff8115e9a8)
94 arm_kprobe [kernel] (ffffffff81148600)
95 enable_kprobe [kernel] (ffffffff811486f6)
96 kprobe_register [kernel] (ffffffff81182399)
97 perf_trace_init [kernel] (ffffffff8117c4e0)
98 perf_tp_event_init [kernel] (ffffffff81192479)
99
100
Sasha Goldshtein33522d72016-02-08 03:39:44 -0800101Here you can see that arming the kprobe to which our eBPF program is attached
102consumed 8KB of memory. Loading the BPF program also consumed a couple hundred
103bytes (in bpf_prog_load).
104
Sasha Goldshteindda47692016-02-08 03:10:13 -0800105memleak stores each allocated block along with its size, timestamp, and the
106stack that allocated it. When the block is deleted, this information is freed
107to reduce the memory overhead.
108
109To avoid false positives, allocations younger than a certain age (500ms by
110default) are not printed. To change this threshold, use the -o switch.
111
112By default, memleak prints its output every 5 seconds. To change this
Sasha Goldshtein75ba13f2016-02-09 06:03:46 -0800113interval, pass the interval as a positional parameter to memleak. You can
114also control the number of times the output will be printed before exiting.
115For example:
116
117# ./memleak.py 1 10
118
119... will print the outstanding allocation statistics every second, for ten
120times, and then exit.
Sasha Goldshteindda47692016-02-08 03:10:13 -0800121
122
123USAGE message:
124
125# ./memleak.py -h
Sasha Goldshtein75ba13f2016-02-09 06:03:46 -0800126usage: memleak.py [-h] [-p PID] [-t] [-a] [-o OLDER] [-c COMMAND]
127 [-s SAMPLE_RATE]
128 [interval] [count]
Sasha Goldshteindda47692016-02-08 03:10:13 -0800129
130Trace outstanding memory allocations that weren't freed.
131Supports both user-mode allocations made with malloc/free and kernel-mode
132allocations made with kmalloc/kfree.
133
Sasha Goldshtein75ba13f2016-02-09 06:03:46 -0800134 interval interval in seconds to print outstanding allocations
135 count number of times to print the report before exiting
136
Sasha Goldshteindda47692016-02-08 03:10:13 -0800137optional arguments:
138 -h, --help show this help message and exit
139 -p PID, --pid PID the PID to trace; if not specified, trace kernel
140 allocs
141 -t, --trace print trace messages for each alloc/free call
Sasha Goldshteindda47692016-02-08 03:10:13 -0800142 -a, --show-allocs show allocation addresses and sizes as well as call
143 stacks
144 -o OLDER, --older OLDER
145 prune allocations younger than this age in
146 milliseconds
147 -c COMMAND, --command COMMAND
148 execute and trace the specified command
Sasha Goldshtein521ab4f2016-02-08 05:48:31 -0800149 -s SAMPLE_RATE, --sample-rate SAMPLE_RATE
150 sample every N-th allocation to decrease the overhead
Sasha Goldshteindda47692016-02-08 03:10:13 -0800151
152EXAMPLES:
153
154./memleak.py -p $(pidof allocs)
155 Trace allocations and display a summary of "leaked" (outstanding)
156 allocations every 5 seconds
157./memleak.py -p $(pidof allocs) -t
158 Trace allocations and display each individual call to malloc/free
Sasha Goldshtein75ba13f2016-02-09 06:03:46 -0800159./memleak.py -ap $(pidof allocs) 10
160 Trace allocations and display allocated addresses, sizes, and stacks
161 every 10 seconds for outstanding allocations
162./memleak.py -c "./allocs"
163 Run the specified command and trace its allocations
164./memleak.py
165 Trace allocations in kernel mode and display a summary of outstanding
166 allocations every 5 seconds
167./memleak.py -o 60000
168 Trace allocations in kernel mode and display a summary of outstanding
169 allocations that are at least one minute (60 seconds) old
Sasha Goldshtein521ab4f2016-02-08 05:48:31 -0800170./memleak.py -s 5
171 Trace roughly every 5th allocation, to reduce overhead