blob: cde23b4a12a19a9306464ec1296cb4fee19eb9c0 [file] [log] [blame]
Pekka Paalanenc6c67c12008-05-12 21:20:59 +02001 In-kernel memory-mapped I/O tracing
2
3
4Home page and links to optional user space tools:
5
6 http://nouveau.freedesktop.org/wiki/MmioTrace
7
8MMIO tracing was originally developed by Intel around 2003 for their Fault
9Injection Test Harness. In Dec 2006 - Jan 2007, using the code from Intel,
10Jeff Muizelaar created a tool for tracing MMIO accesses with the Nouveau
11project in mind. Since then many people have contributed.
12
13Mmiotrace was built for reverse engineering any memory-mapped IO device with
14the Nouveau project as the first real user. Only x86 and x86_64 architectures
15are supported.
16
17Out-of-tree mmiotrace was originally modified for mainline inclusion and
18ftrace framework by Pekka Paalanen <pq@iki.fi>.
19
20
21Preparation
22-----------
23
24Mmiotrace feature is compiled in by the CONFIG_MMIOTRACE option. Tracing is
25disabled by default, so it is safe to have this set to yes. SMP systems are
26supported, but tracing is unreliable and may miss events if more than one CPU
27is on-line, therefore mmiotrace takes all but one CPU off-line during run-time
Pekka Paalanen6f6f3942008-05-12 21:21:03 +020028activation. You can re-enable CPUs by hand, but you have been warned, there
29is no way to automatically detect if you are losing events due to CPUs racing.
Pekka Paalanenc6c67c12008-05-12 21:20:59 +020030
31
32Usage Quick Reference
33---------------------
34
35$ mount -t debugfs debugfs /debug
36$ echo mmiotrace > /debug/tracing/current_tracer
37$ cat /debug/tracing/trace_pipe > mydump.txt &
38Start X or whatever.
Pekka Paalanen5bf9a1e2008-09-16 22:06:42 +030039$ echo "X is up" > /debug/tracing/trace_marker
Pekka Paalanenfb91ee62008-11-23 21:24:59 +020040$ echo nop > /debug/tracing/current_tracer
Pekka Paalanen6f6f3942008-05-12 21:21:03 +020041Check for lost events.
Pekka Paalanenc6c67c12008-05-12 21:20:59 +020042
43
44Usage
45-----
46
47Make sure debugfs is mounted to /debug. If not, (requires root privileges)
48$ mount -t debugfs debugfs /debug
49
50Check that the driver you are about to trace is not loaded.
51
52Activate mmiotrace (requires root privileges):
53$ echo mmiotrace > /debug/tracing/current_tracer
54
55Start storing the trace:
56$ cat /debug/tracing/trace_pipe > mydump.txt &
57The 'cat' process should stay running (sleeping) in the background.
58
59Load the driver you want to trace and use it. Mmiotrace will only catch MMIO
60accesses to areas that are ioremapped while mmiotrace is active.
61
Pekka Paalanenc6c67c12008-05-12 21:20:59 +020062During tracing you can place comments (markers) into the trace by
Pekka Paalanen5bf9a1e2008-09-16 22:06:42 +030063$ echo "X is up" > /debug/tracing/trace_marker
Pekka Paalanenc6c67c12008-05-12 21:20:59 +020064This makes it easier to see which part of the (huge) trace corresponds to
65which action. It is recommended to place descriptive markers about what you
66do.
67
68Shut down mmiotrace (requires root privileges):
Pekka Paalanenfb91ee62008-11-23 21:24:59 +020069$ echo nop > /debug/tracing/current_tracer
Pekka Paalanen6f6f3942008-05-12 21:21:03 +020070The 'cat' process exits. If it does not, kill it by issuing 'fg' command and
71pressing ctrl+c.
Pekka Paalanenc6c67c12008-05-12 21:20:59 +020072
Pekka Paalanen6f6f3942008-05-12 21:21:03 +020073Check that mmiotrace did not lose events due to a buffer filling up. Either
74$ grep -i lost mydump.txt
75which tells you exactly how many events were lost, or use
76$ dmesg
77to view your kernel log and look for "mmiotrace has lost events" warning. If
78events were lost, the trace is incomplete. You should enlarge the buffers and
79try again. Buffers are enlarged by first seeing how large the current buffers
80are:
81$ cat /debug/tracing/trace_entries
82gives you a number. Approximately double this number and write it back, for
83instance:
Pekka Paalanenfb91ee62008-11-23 21:24:59 +020084$ echo 0 > /debug/tracing/tracing_enabled
Pekka Paalanen6f6f3942008-05-12 21:21:03 +020085$ echo 128000 > /debug/tracing/trace_entries
Pekka Paalanenfb91ee62008-11-23 21:24:59 +020086$ echo 1 > /debug/tracing/tracing_enabled
Pekka Paalanen6f6f3942008-05-12 21:21:03 +020087Then start again from the top.
Pekka Paalanenc6c67c12008-05-12 21:20:59 +020088
89If you are doing a trace for a driver project, e.g. Nouveau, you should also
90do the following before sending your results:
91$ lspci -vvv > lspci.txt
92$ dmesg > dmesg.txt
93$ tar zcf pciid-nick-mmiotrace.tar.gz mydump.txt lspci.txt dmesg.txt
94and then send the .tar.gz file. The trace compresses considerably. Replace
95"pciid" and "nick" with the PCI ID or model name of your piece of hardware
96under investigation and your nick name.
97
98
99How Mmiotrace Works
100-------------------
101
102Access to hardware IO-memory is gained by mapping addresses from PCI bus by
103calling one of the ioremap_*() functions. Mmiotrace is hooked into the
104__ioremap() function and gets called whenever a mapping is created. Mapping is
105an event that is recorded into the trace log. Note, that ISA range mappings
106are not caught, since the mapping always exists and is returned directly.
107
108MMIO accesses are recorded via page faults. Just before __ioremap() returns,
109the mapped pages are marked as not present. Any access to the pages causes a
110fault. The page fault handler calls mmiotrace to handle the fault. Mmiotrace
111marks the page present, sets TF flag to achieve single stepping and exits the
112fault handler. The instruction that faulted is executed and debug trap is
113entered. Here mmiotrace again marks the page as not present. The instruction
114is decoded to get the type of operation (read/write), data width and the value
115read or written. These are stored to the trace log.
116
117Setting the page present in the page fault handler has a race condition on SMP
118machines. During the single stepping other CPUs may run freely on that page
119and events can be missed without a notice. Re-enabling other CPUs during
120tracing is discouraged.
121
122
123Trace Log Format
124----------------
125
126The raw log is text and easily filtered with e.g. grep and awk. One record is
127one line in the log. A record starts with a keyword, followed by keyword
128dependant arguments. Arguments are separated by a space, or continue until the
129end of line. The format for version 20070824 is as follows:
130
131Explanation Keyword Space separated arguments
132---------------------------------------------------------------------------
133
134read event R width, timestamp, map id, physical, value, PC, PID
135write event W width, timestamp, map id, physical, value, PC, PID
136ioremap event MAP timestamp, map id, physical, virtual, length, PC, PID
137iounmap event UNMAP timestamp, map id, PC, PID
138marker MARK timestamp, text
139version VERSION the string "20070824"
140info for reader LSPCI one line from lspci -v
141PCI address map PCIDEV space separated /proc/bus/pci/devices data
142unk. opcode UNKNOWN timestamp, map id, physical, data, PC, PID
143
144Timestamp is in seconds with decimals. Physical is a PCI bus address, virtual
145is a kernel virtual address. Width is the data width in bytes and value is the
146data value. Map id is an arbitrary id number identifying the mapping that was
147used in an operation. PC is the program counter and PID is process id. PC is
148zero if it is not recorded. PID is always zero as tracing MMIO accesses
149originating in user space memory is not yet supported.
150
151For instance, the following awk filter will pass all 32-bit writes that target
152physical addresses in the range [0xfb73ce40, 0xfb800000[
153
154$ awk '/W 4 / { adr=strtonum($5); if (adr >= 0xfb73ce40 &&
155adr < 0xfb800000) print; }'
156
157
158Tools for Developers
159--------------------
160
161The user space tools include utilities for:
162- replacing numeric addresses and values with hardware register names
163- replaying MMIO logs, i.e., re-executing the recorded writes
164
165