blob: 9930c4bddc6fa7cb1221cee9279d31ed6a0c05f2 [file] [log] [blame]
Ingo Molnare7bc62b2008-12-04 20:13:45 +01001
2Performance Counters for Linux
3------------------------------
4
5Performance counters are special hardware registers available on most modern
6CPUs. These registers count the number of certain types of hw events: such
7as instructions executed, cachemisses suffered, or branches mis-predicted -
8without slowing down the kernel or applications. These registers can also
9trigger interrupts when a threshold number of events have passed - and can
10thus be used to profile the code that runs on that CPU.
11
12The Linux Performance Counter subsystem provides an abstraction of these
Ingo Molnar447557a2008-12-11 20:40:18 +010013hardware capabilities. It provides per task and per CPU counters, counter
Paul Mackerrasf66c6b22009-03-23 10:29:36 +110014groups, and it provides event capabilities on top of those. It
15provides "virtual" 64-bit counters, regardless of the width of the
16underlying hardware counters.
Ingo Molnare7bc62b2008-12-04 20:13:45 +010017
18Performance counters are accessed via special file descriptors.
19There's one file descriptor per virtual counter used.
20
21The special file descriptor is opened via the perf_counter_open()
22system call:
23
Ingo Molnar447557a2008-12-11 20:40:18 +010024 int sys_perf_counter_open(struct perf_counter_hw_event *hw_event_uptr,
Paul Mackerrasf66c6b22009-03-23 10:29:36 +110025 pid_t pid, int cpu, int group_fd,
26 unsigned long flags);
Ingo Molnare7bc62b2008-12-04 20:13:45 +010027
28The syscall returns the new fd. The fd can be used via the normal
29VFS system calls: read() can be used to read the counter, fcntl()
30can be used to set the blocking mode, etc.
31
32Multiple counters can be kept open at a time, and the counters
33can be poll()ed.
34
Ingo Molnar447557a2008-12-11 20:40:18 +010035When creating a new counter fd, 'perf_counter_hw_event' is:
Ingo Molnare7bc62b2008-12-04 20:13:45 +010036
Ingo Molnar447557a2008-12-11 20:40:18 +010037struct perf_counter_hw_event {
Peter Zijlstrae5791a82009-05-01 12:23:19 +020038 /*
39 * The MSB of the config word signifies if the rest contains cpu
40 * specific (raw) counter configuration data, if unset, the next
41 * 7 bits are an event type and the rest of the bits are the event
42 * identifier.
43 */
44 __u64 config;
Ingo Molnar447557a2008-12-11 20:40:18 +010045
Peter Zijlstrae5791a82009-05-01 12:23:19 +020046 __u64 irq_period;
47 __u32 record_type;
48 __u32 read_format;
Ingo Molnar447557a2008-12-11 20:40:18 +010049
Peter Zijlstrae5791a82009-05-01 12:23:19 +020050 __u64 disabled : 1, /* off by default */
51 nmi : 1, /* NMI sampling */
52 inherit : 1, /* children inherit it */
53 pinned : 1, /* must always be on PMU */
54 exclusive : 1, /* only group on PMU */
55 exclude_user : 1, /* don't count user */
56 exclude_kernel : 1, /* ditto kernel */
57 exclude_hv : 1, /* ditto hypervisor */
58 exclude_idle : 1, /* don't count when idle */
59 mmap : 1, /* include mmap data */
60 munmap : 1, /* include munmap data */
61 comm : 1, /* include comm data */
Ingo Molnar447557a2008-12-11 20:40:18 +010062
Peter Zijlstrae5791a82009-05-01 12:23:19 +020063 __reserved_1 : 52;
Paul Mackerrasf66c6b22009-03-23 10:29:36 +110064
Peter Zijlstrae5791a82009-05-01 12:23:19 +020065 __u32 extra_config_len;
66 __u32 wakeup_events; /* wakeup every n events */
Paul Mackerrasf66c6b22009-03-23 10:29:36 +110067
Peter Zijlstrae5791a82009-05-01 12:23:19 +020068 __u64 __reserved_2;
69 __u64 __reserved_3;
Ingo Molnar447557a2008-12-11 20:40:18 +010070};
71
Peter Zijlstrae5791a82009-05-01 12:23:19 +020072The 'config' field specifies what the counter should count. It
Paul Mackerrasf66c6b22009-03-23 10:29:36 +110073is divided into 3 bit-fields:
74
Peter Zijlstrae5791a82009-05-01 12:23:19 +020075raw_type: 1 bit (most significant bit) 0x8000_0000_0000_0000
76type: 7 bits (next most significant) 0x7f00_0000_0000_0000
77event_id: 56 bits (least significant) 0x00ff_ffff_ffff_ffff
Paul Mackerrasf66c6b22009-03-23 10:29:36 +110078
79If 'raw_type' is 1, then the counter will count a hardware event
80specified by the remaining 63 bits of event_config. The encoding is
81machine-specific.
82
83If 'raw_type' is 0, then the 'type' field says what kind of counter
84this is, with the following encoding:
85
86enum perf_event_types {
87 PERF_TYPE_HARDWARE = 0,
88 PERF_TYPE_SOFTWARE = 1,
89 PERF_TYPE_TRACEPOINT = 2,
90};
91
92A counter of PERF_TYPE_HARDWARE will count the hardware event
93specified by 'event_id':
94
Ingo Molnar447557a2008-12-11 20:40:18 +010095/*
Paul Mackerrasf66c6b22009-03-23 10:29:36 +110096 * Generalized performance counter event types, used by the hw_event.event_id
Ingo Molnar447557a2008-12-11 20:40:18 +010097 * parameter of the sys_perf_counter_open() syscall:
98 */
Paul Mackerrasf66c6b22009-03-23 10:29:36 +110099enum hw_event_ids {
Ingo Molnar447557a2008-12-11 20:40:18 +0100100 /*
101 * Common hardware events, generalized by the kernel:
102 */
Paul Mackerrasf66c6b22009-03-23 10:29:36 +1100103 PERF_COUNT_CPU_CYCLES = 0,
104 PERF_COUNT_INSTRUCTIONS = 1,
105 PERF_COUNT_CACHE_REFERENCES = 2,
106 PERF_COUNT_CACHE_MISSES = 3,
107 PERF_COUNT_BRANCH_INSTRUCTIONS = 4,
108 PERF_COUNT_BRANCH_MISSES = 5,
109 PERF_COUNT_BUS_CYCLES = 6,
Ingo Molnar447557a2008-12-11 20:40:18 +0100110};
Ingo Molnare7bc62b2008-12-04 20:13:45 +0100111
Paul Mackerrasf66c6b22009-03-23 10:29:36 +1100112These are standardized types of events that work relatively uniformly
113on all CPUs that implement Performance Counters support under Linux,
114although there may be variations (e.g., different CPUs might count
115cache references and misses at different levels of the cache hierarchy).
116If a CPU is not able to count the selected event, then the system call
117will return -EINVAL.
Ingo Molnare7bc62b2008-12-04 20:13:45 +0100118
Paul Mackerrasf66c6b22009-03-23 10:29:36 +1100119More hw_event_types are supported as well, but they are CPU-specific
120and accessed as raw events. For example, to count "External bus
121cycles while bus lock signal asserted" events on Intel Core CPUs, pass
122in a 0x4064 event_id value and set hw_event.raw_type to 1.
Ingo Molnare7bc62b2008-12-04 20:13:45 +0100123
Paul Mackerrasf66c6b22009-03-23 10:29:36 +1100124A counter of type PERF_TYPE_SOFTWARE will count one of the available
125software events, selected by 'event_id':
126
127/*
128 * Special "software" counters provided by the kernel, even if the hardware
129 * does not support performance counters. These counters measure various
130 * physical and sw events of the kernel (and allow the profiling of them as
131 * well):
132 */
133enum sw_event_ids {
134 PERF_COUNT_CPU_CLOCK = 0,
135 PERF_COUNT_TASK_CLOCK = 1,
136 PERF_COUNT_PAGE_FAULTS = 2,
137 PERF_COUNT_CONTEXT_SWITCHES = 3,
138 PERF_COUNT_CPU_MIGRATIONS = 4,
139 PERF_COUNT_PAGE_FAULTS_MIN = 5,
140 PERF_COUNT_PAGE_FAULTS_MAJ = 6,
141};
142
Peter Zijlstrae5791a82009-05-01 12:23:19 +0200143Counters of the type PERF_TYPE_TRACEPOINT are available when the ftrace event
144tracer is available, and event_id values can be obtained from
145/debug/tracing/events/*/*/id
146
147
Paul Mackerrasf66c6b22009-03-23 10:29:36 +1100148Counters come in two flavours: counting counters and sampling
149counters. A "counting" counter is one that is used for counting the
150number of events that occur, and is characterised by having
Peter Zijlstrae5791a82009-05-01 12:23:19 +0200151irq_period = 0.
152
153
154A read() on a counter returns the current value of the counter and possible
155additional values as specified by 'read_format', each value is a u64 (8 bytes)
156in size.
157
158/*
159 * Bits that can be set in hw_event.read_format to request that
160 * reads on the counter should return the indicated quantities,
161 * in increasing order of bit value, after the counter value.
162 */
163enum perf_counter_read_format {
164 PERF_FORMAT_TOTAL_TIME_ENABLED = 1,
165 PERF_FORMAT_TOTAL_TIME_RUNNING = 2,
166};
167
168Using these additional values one can establish the overcommit ratio for a
169particular counter allowing one to take the round-robin scheduling effect
170into account.
171
Paul Mackerrasf66c6b22009-03-23 10:29:36 +1100172
173A "sampling" counter is one that is set up to generate an interrupt
174every N events, where N is given by 'irq_period'. A sampling counter
Peter Zijlstrae5791a82009-05-01 12:23:19 +0200175has irq_period > 0. The record_type controls what data is recorded on each
176interrupt:
Ingo Molnare7bc62b2008-12-04 20:13:45 +0100177
Ingo Molnar447557a2008-12-11 20:40:18 +0100178/*
Peter Zijlstrae5791a82009-05-01 12:23:19 +0200179 * Bits that can be set in hw_event.record_type to request information
180 * in the overflow packets.
Ingo Molnar447557a2008-12-11 20:40:18 +0100181 */
Peter Zijlstrae5791a82009-05-01 12:23:19 +0200182enum perf_counter_record_format {
183 PERF_RECORD_IP = 1U << 0,
184 PERF_RECORD_TID = 1U << 1,
185 PERF_RECORD_TIME = 1U << 2,
186 PERF_RECORD_ADDR = 1U << 3,
187 PERF_RECORD_GROUP = 1U << 4,
188 PERF_RECORD_CALLCHAIN = 1U << 5,
Ingo Molnar447557a2008-12-11 20:40:18 +0100189};
Ingo Molnare7bc62b2008-12-04 20:13:45 +0100190
Peter Zijlstrae5791a82009-05-01 12:23:19 +0200191Such (and other) events will be recorded in a ring-buffer, which is
192available to user-space using mmap() (see below).
Ingo Molnare7bc62b2008-12-04 20:13:45 +0100193
Paul Mackerrasf66c6b22009-03-23 10:29:36 +1100194The 'disabled' bit specifies whether the counter starts out disabled
195or enabled. If it is initially disabled, it can be enabled by ioctl
196or prctl (see below).
Ingo Molnar447557a2008-12-11 20:40:18 +0100197
Paul Mackerrasf66c6b22009-03-23 10:29:36 +1100198The 'nmi' bit specifies, for hardware events, whether the counter
199should be set up to request non-maskable interrupts (NMIs) or normal
200interrupts. This bit is ignored if the user doesn't have
201CAP_SYS_ADMIN privilege (i.e. is not root) or if the CPU doesn't
202generate NMIs from hardware counters.
203
204The 'inherit' bit, if set, specifies that this counter should count
205events on descendant tasks as well as the task specified. This only
206applies to new descendents, not to any existing descendents at the
207time the counter is created (nor to any new descendents of existing
208descendents).
209
210The 'pinned' bit, if set, specifies that the counter should always be
211on the CPU if at all possible. It only applies to hardware counters
212and only to group leaders. If a pinned counter cannot be put onto the
213CPU (e.g. because there are not enough hardware counters or because of
214a conflict with some other event), then the counter goes into an
215'error' state, where reads return end-of-file (i.e. read() returns 0)
216until the counter is subsequently enabled or disabled.
217
218The 'exclusive' bit, if set, specifies that when this counter's group
219is on the CPU, it should be the only group using the CPU's counters.
220In future, this will allow sophisticated monitoring programs to supply
221extra configuration information via 'extra_config_len' to exploit
222advanced features of the CPU's Performance Monitor Unit (PMU) that are
223not otherwise accessible and that might disrupt other hardware
224counters.
225
226The 'exclude_user', 'exclude_kernel' and 'exclude_hv' bits provide a
227way to request that counting of events be restricted to times when the
228CPU is in user, kernel and/or hypervisor mode.
229
Peter Zijlstrae5791a82009-05-01 12:23:19 +0200230The 'mmap' and 'munmap' bits allow recording of PROT_EXEC mmap/munmap
231operations, these can be used to relate userspace IP addresses to actual
232code, even after the mapping (or even the whole process) is gone,
233these events are recorded in the ring-buffer (see below).
234
235The 'comm' bit allows tracking of process comm data on process creation.
236This too is recorded in the ring-buffer (see below).
Paul Mackerrasf66c6b22009-03-23 10:29:36 +1100237
238The 'pid' parameter to the perf_counter_open() system call allows the
239counter to be specific to a task:
Ingo Molnare7bc62b2008-12-04 20:13:45 +0100240
241 pid == 0: if the pid parameter is zero, the counter is attached to the
242 current task.
243
244 pid > 0: the counter is attached to a specific task (if the current task
245 has sufficient privilege to do so)
246
247 pid < 0: all tasks are counted (per cpu counters)
248
Paul Mackerrasf66c6b22009-03-23 10:29:36 +1100249The 'cpu' parameter allows a counter to be made specific to a CPU:
Ingo Molnare7bc62b2008-12-04 20:13:45 +0100250
251 cpu >= 0: the counter is restricted to a specific CPU
252 cpu == -1: the counter counts on all CPUs
253
Ingo Molnar447557a2008-12-11 20:40:18 +0100254(Note: the combination of 'pid == -1' and 'cpu == -1' is not valid.)
Ingo Molnare7bc62b2008-12-04 20:13:45 +0100255
256A 'pid > 0' and 'cpu == -1' counter is a per task counter that counts
257events of that task and 'follows' that task to whatever CPU the task
258gets schedule to. Per task counters can be created by any user, for
259their own tasks.
260
261A 'pid == -1' and 'cpu == x' counter is a per CPU counter that counts
262all events on CPU-x. Per CPU counters need CAP_SYS_ADMIN privilege.
263
Paul Mackerrasf66c6b22009-03-23 10:29:36 +1100264The 'flags' parameter is currently unused and must be zero.
265
266The 'group_fd' parameter allows counter "groups" to be set up. A
267counter group has one counter which is the group "leader". The leader
268is created first, with group_fd = -1 in the perf_counter_open call
269that creates it. The rest of the group members are created
270subsequently, with group_fd giving the fd of the group leader.
271(A single counter on its own is created with group_fd = -1 and is
272considered to be a group with only 1 member.)
273
274A counter group is scheduled onto the CPU as a unit, that is, it will
275only be put onto the CPU if all of the counters in the group can be
276put onto the CPU. This means that the values of the member counters
277can be meaningfully compared, added, divided (to get ratios), etc.,
278with each other, since they have counted events for the same set of
279executed instructions.
280
Peter Zijlstrae5791a82009-05-01 12:23:19 +0200281
282Like stated, asynchronous events, like counter overflow or PROT_EXEC mmap
283tracking are logged into a ring-buffer. This ring-buffer is created and
284accessed through mmap().
285
286The mmap size should be 1+2^n pages, where the first page is a meta-data page
287(struct perf_counter_mmap_page) that contains various bits of information such
288as where the ring-buffer head is.
289
290/*
291 * Structure of the page that can be mapped via mmap
292 */
293struct perf_counter_mmap_page {
294 __u32 version; /* version number of this structure */
295 __u32 compat_version; /* lowest version this is compat with */
296
297 /*
298 * Bits needed to read the hw counters in user-space.
299 *
300 * u32 seq;
301 * s64 count;
302 *
303 * do {
304 * seq = pc->lock;
305 *
306 * barrier()
307 * if (pc->index) {
308 * count = pmc_read(pc->index - 1);
309 * count += pc->offset;
310 * } else
311 * goto regular_read;
312 *
313 * barrier();
314 * } while (pc->lock != seq);
315 *
316 * NOTE: for obvious reason this only works on self-monitoring
317 * processes.
318 */
319 __u32 lock; /* seqlock for synchronization */
320 __u32 index; /* hardware counter identifier */
321 __s64 offset; /* add to hardware counter value */
322
323 /*
324 * Control data for the mmap() data buffer.
325 *
326 * User-space reading this value should issue an rmb(), on SMP capable
327 * platforms, after reading this value -- see perf_counter_wakeup().
328 */
329 __u32 data_head; /* head in the data section */
330};
331
332NOTE: the hw-counter userspace bits are arch specific and are currently only
333 implemented on powerpc.
334
335The following 2^n pages are the ring-buffer which contains events of the form:
336
337#define PERF_EVENT_MISC_KERNEL (1 << 0)
338#define PERF_EVENT_MISC_USER (1 << 1)
339#define PERF_EVENT_MISC_OVERFLOW (1 << 2)
340
341struct perf_event_header {
342 __u32 type;
343 __u16 misc;
344 __u16 size;
345};
346
347enum perf_event_type {
348
349 /*
350 * The MMAP events record the PROT_EXEC mappings so that we can
351 * correlate userspace IPs to code. They have the following structure:
352 *
353 * struct {
354 * struct perf_event_header header;
355 *
356 * u32 pid, tid;
357 * u64 addr;
358 * u64 len;
359 * u64 pgoff;
360 * char filename[];
361 * };
362 */
363 PERF_EVENT_MMAP = 1,
364 PERF_EVENT_MUNMAP = 2,
365
366 /*
367 * struct {
368 * struct perf_event_header header;
369 *
370 * u32 pid, tid;
371 * char comm[];
372 * };
373 */
374 PERF_EVENT_COMM = 3,
375
376 /*
377 * When header.misc & PERF_EVENT_MISC_OVERFLOW the event_type field
378 * will be PERF_RECORD_*
379 *
380 * struct {
381 * struct perf_event_header header;
382 *
383 * { u64 ip; } && PERF_RECORD_IP
384 * { u32 pid, tid; } && PERF_RECORD_TID
385 * { u64 time; } && PERF_RECORD_TIME
386 * { u64 addr; } && PERF_RECORD_ADDR
387 *
388 * { u64 nr;
389 * { u64 event, val; } cnt[nr]; } && PERF_RECORD_GROUP
390 *
391 * { u16 nr,
392 * hv,
393 * kernel,
394 * user;
395 * u64 ips[nr]; } && PERF_RECORD_CALLCHAIN
396 * };
397 */
398};
399
400NOTE: PERF_RECORD_CALLCHAIN is arch specific and currently only implemented
401 on x86.
402
403Notification of new events is possible through poll()/select()/epoll() and
404fcntl() managing signals.
405
406Normally a notification is generated for every page filled, however one can
407additionally set perf_counter_hw_event.wakeup_events to generate one every
408so many counter overflow events.
409
410Future work will include a splice() interface to the ring-buffer.
411
412
Paul Mackerrasf66c6b22009-03-23 10:29:36 +1100413Counters can be enabled and disabled in two ways: via ioctl and via
414prctl. When a counter is disabled, it doesn't count or generate
415events but does continue to exist and maintain its count value.
416
417An individual counter or counter group can be enabled with
418
419 ioctl(fd, PERF_COUNTER_IOC_ENABLE);
420
421or disabled with
422
423 ioctl(fd, PERF_COUNTER_IOC_DISABLE);
424
425Enabling or disabling the leader of a group enables or disables the
426whole group; that is, while the group leader is disabled, none of the
427counters in the group will count. Enabling or disabling a member of a
428group other than the leader only affects that counter - disabling an
429non-leader stops that counter from counting but doesn't affect any
430other counter.
431
Peter Zijlstrae5791a82009-05-01 12:23:19 +0200432Additionally, non-inherited overflow counters can use
433
434 ioctl(fd, PERF_COUNTER_IOC_REFRESH, nr);
435
436to enable a counter for 'nr' events, after which it gets disabled again.
437
Paul Mackerrasf66c6b22009-03-23 10:29:36 +1100438A process can enable or disable all the counter groups that are
439attached to it, using prctl:
440
441 prctl(PR_TASK_PERF_COUNTERS_ENABLE);
442
443 prctl(PR_TASK_PERF_COUNTERS_DISABLE);
444
445This applies to all counters on the current process, whether created
446by this process or by another, and doesn't affect any counters that
447this process has created on other processes. It only enables or
448disables the group leaders, not any other members in the groups.
Ingo Molnar447557a2008-12-11 20:40:18 +0100449