blob: f959909d7154971d50f64cf773adea2b8afe7b6f [file] [log] [blame]
Jason Baron86151fd2009-02-05 11:53:15 -05001
2Introduction
3============
4
5This document describes how to use the dynamic debug (ddebug) feature.
6
7Dynamic debug is designed to allow you to dynamically enable/disable kernel
8code to obtain additional kernel information. Currently, if
Jonathan Corbet9cad7962011-03-25 10:42:17 -06009CONFIG_DYNAMIC_DEBUG is set, then all pr_debug()/dev_dbg() calls can be
Jason Baron86151fd2009-02-05 11:53:15 -050010dynamically enabled per-callsite.
11
12Dynamic debug has even more useful features:
13
14 * Simple query language allows turning on and off debugging statements by
15 matching any combination of:
16
17 - source filename
18 - function name
19 - line number (including ranges of line numbers)
20 - module name
21 - format string
22
23 * Provides a debugfs control file: <debugfs>/dynamic_debug/control which can be
24 read to display the complete list of known debug statements, to help guide you
25
26Controlling dynamic debug Behaviour
Thomas Renningera648ec02010-08-06 16:11:02 +020027===================================
Jason Baron86151fd2009-02-05 11:53:15 -050028
Jonathan Corbet9cad7962011-03-25 10:42:17 -060029The behaviour of pr_debug()/dev_dbg()s are controlled via writing to a
Jason Baron86151fd2009-02-05 11:53:15 -050030control file in the 'debugfs' filesystem. Thus, you must first mount the debugfs
31filesystem, in order to make use of this feature. Subsequently, we refer to the
32control file as: <debugfs>/dynamic_debug/control. For example, if you want to
33enable printing from source file 'svcsock.c', line 1603 you simply do:
34
35nullarbor:~ # echo 'file svcsock.c line 1603 +p' >
36 <debugfs>/dynamic_debug/control
37
38If you make a mistake with the syntax, the write will fail thus:
39
40nullarbor:~ # echo 'file svcsock.c wtf 1 +p' >
41 <debugfs>/dynamic_debug/control
42-bash: echo: write error: Invalid argument
43
44Viewing Dynamic Debug Behaviour
45===========================
46
47You can view the currently configured behaviour of all the debug statements
48via:
49
50nullarbor:~ # cat <debugfs>/dynamic_debug/control
51# filename:lineno [module]function flags format
Greg Banks9898abb2009-02-06 12:54:26 +110052/usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svc_rdma.c:323 [svcxprt_rdma]svc_rdma_cleanup - "SVCRDMA Module Removed, deregister RPC RDMA transport\012"
53/usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svc_rdma.c:341 [svcxprt_rdma]svc_rdma_init - "\011max_inline : %d\012"
54/usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svc_rdma.c:340 [svcxprt_rdma]svc_rdma_init - "\011sq_depth : %d\012"
55/usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svc_rdma.c:338 [svcxprt_rdma]svc_rdma_init - "\011max_requests : %d\012"
Jason Baron86151fd2009-02-05 11:53:15 -050056...
57
58
59You can also apply standard Unix text manipulation filters to this
60data, e.g.
61
62nullarbor:~ # grep -i rdma <debugfs>/dynamic_debug/control | wc -l
6362
64
65nullarbor:~ # grep -i tcp <debugfs>/dynamic_debug/control | wc -l
6642
67
68Note in particular that the third column shows the enabled behaviour
69flags for each debug statement callsite (see below for definitions of the
70flags). The default value, no extra behaviour enabled, is "-". So
71you can view all the debug statement callsites with any non-default flags:
72
73nullarbor:~ # awk '$3 != "-"' <debugfs>/dynamic_debug/control
74# filename:lineno [module]function flags format
Greg Banks9898abb2009-02-06 12:54:26 +110075/usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svcsock.c:1603 [sunrpc]svc_send p "svc_process: st_sendto returned %d\012"
Jason Baron86151fd2009-02-05 11:53:15 -050076
77
78Command Language Reference
79==========================
80
81At the lexical level, a command comprises a sequence of words separated
82by whitespace characters. Note that newlines are treated as word
83separators and do *not* end a command or allow multiple commands to
84be done together. So these are all equivalent:
85
86nullarbor:~ # echo -c 'file svcsock.c line 1603 +p' >
87 <debugfs>/dynamic_debug/control
88nullarbor:~ # echo -c ' file svcsock.c line 1603 +p ' >
89 <debugfs>/dynamic_debug/control
90nullarbor:~ # echo -c 'file svcsock.c\nline 1603 +p' >
91 <debugfs>/dynamic_debug/control
92nullarbor:~ # echo -n 'file svcsock.c line 1603 +p' >
93 <debugfs>/dynamic_debug/control
94
95Commands are bounded by a write() system call. If you want to do
96multiple commands you need to do a separate "echo" for each, like:
97
98nullarbor:~ # echo 'file svcsock.c line 1603 +p' > /proc/dprintk ;\
99> echo 'file svcsock.c line 1563 +p' > /proc/dprintk
100
101or even like:
102
103nullarbor:~ # (
104> echo 'file svcsock.c line 1603 +p' ;\
105> echo 'file svcsock.c line 1563 +p' ;\
106> ) > /proc/dprintk
107
108At the syntactical level, a command comprises a sequence of match
109specifications, followed by a flags change specification.
110
111command ::= match-spec* flags-spec
112
113The match-spec's are used to choose a subset of the known dprintk()
114callsites to which to apply the flags-spec. Think of them as a query
115with implicit ANDs between each pair. Note that an empty list of
116match-specs is possible, but is not very useful because it will not
117match any debug statement callsites.
118
119A match specification comprises a keyword, which controls the attribute
120of the callsite to be compared, and a value to compare against. Possible
121keywords are:
122
123match-spec ::= 'func' string |
124 'file' string |
125 'module' string |
126 'format' string |
127 'line' line-range
128
129line-range ::= lineno |
130 '-'lineno |
131 lineno'-' |
132 lineno'-'lineno
133// Note: line-range cannot contain space, e.g.
134// "1-30" is valid range but "1 - 30" is not.
135
136lineno ::= unsigned-int
137
138The meanings of each keyword are:
139
140func
141 The given string is compared against the function name
142 of each callsite. Example:
143
144 func svc_tcp_accept
145
146file
147 The given string is compared against either the full
148 pathname or the basename of the source file of each
149 callsite. Examples:
150
151 file svcsock.c
152 file /usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svcsock.c
153
154module
155 The given string is compared against the module name
156 of each callsite. The module name is the string as
157 seen in "lsmod", i.e. without the directory or the .ko
158 suffix and with '-' changed to '_'. Examples:
159
160 module sunrpc
161 module nfsd
162
163format
164 The given string is searched for in the dynamic debug format
165 string. Note that the string does not need to match the
166 entire format, only some part. Whitespace and other
167 special characters can be escaped using C octal character
168 escape \ooo notation, e.g. the space character is \040.
Greg Banks9898abb2009-02-06 12:54:26 +1100169 Alternatively, the string can be enclosed in double quote
170 characters (") or single quote characters (').
Jason Baron86151fd2009-02-05 11:53:15 -0500171 Examples:
172
173 format svcrdma: // many of the NFS/RDMA server dprintks
174 format readahead // some dprintks in the readahead cache
Greg Banks9898abb2009-02-06 12:54:26 +1100175 format nfsd:\040SETATTR // one way to match a format with whitespace
176 format "nfsd: SETATTR" // a neater way to match a format with whitespace
177 format 'nfsd: SETATTR' // yet another way to match a format with whitespace
Jason Baron86151fd2009-02-05 11:53:15 -0500178
179line
180 The given line number or range of line numbers is compared
181 against the line number of each dprintk() callsite. A single
182 line number matches the callsite line number exactly. A
183 range of line numbers matches any callsite between the first
184 and last line number inclusive. An empty first number means
185 the first line in the file, an empty line number means the
186 last number in the file. Examples:
187
188 line 1603 // exactly line 1603
189 line 1600-1605 // the six lines from line 1600 to line 1605
190 line -1605 // the 1605 lines from line 1 to line 1605
191 line 1600- // all lines from line 1600 to the end of the file
192
193The flags specification comprises a change operation followed
194by one or more flag characters. The change operation is one
195of the characters:
196
197-
198 remove the given flags
199
200+
201 add the given flags
202
203=
204 set the flags to the given flags
205
206The flags are:
207
Bart Van Assche8ba6ebf52011-01-23 17:17:24 +0100208f
209 Include the function name in the printed message
210l
211 Include line number in the printed message
212m
213 Include module name in the printed message
Jason Baron86151fd2009-02-05 11:53:15 -0500214p
215 Causes a printk() message to be emitted to dmesg
Bart Van Assche8ba6ebf52011-01-23 17:17:24 +0100216t
217 Include thread ID in messages not generated from interrupt context
Jason Baron86151fd2009-02-05 11:53:15 -0500218
Bart Van Assche8ba6ebf52011-01-23 17:17:24 +0100219Note the regexp ^[-+=][flmpt]+$ matches a flags specification.
Jason Baron86151fd2009-02-05 11:53:15 -0500220Note also that there is no convenient syntax to remove all
Bart Van Assche8ba6ebf52011-01-23 17:17:24 +0100221the flags at once, you need to use "-flmpt".
Jason Baron86151fd2009-02-05 11:53:15 -0500222
Thomas Renningera648ec02010-08-06 16:11:02 +0200223
224Debug messages during boot process
225==================================
226
227To be able to activate debug messages during the boot process,
228even before userspace and debugfs exists, use the boot parameter:
229ddebug_query="QUERY"
230
231QUERY follows the syntax described above, but must not exceed 1023
232characters. The enablement of debug messages is done as an arch_initcall.
233Thus you can enable debug messages in all code processed after this
234arch_initcall via this boot parameter.
235On an x86 system for example ACPI enablement is a subsys_initcall and
236ddebug_query="file ec.c +p"
237will show early Embedded Controller transactions during ACPI setup if
238your machine (typically a laptop) has an Embedded Controller.
239PCI (or other devices) initialization also is a hot candidate for using
240this boot parameter for debugging purposes.
241
242
Jason Baron86151fd2009-02-05 11:53:15 -0500243Examples
244========
245
246// enable the message at line 1603 of file svcsock.c
247nullarbor:~ # echo -n 'file svcsock.c line 1603 +p' >
248 <debugfs>/dynamic_debug/control
249
250// enable all the messages in file svcsock.c
251nullarbor:~ # echo -n 'file svcsock.c +p' >
252 <debugfs>/dynamic_debug/control
253
254// enable all the messages in the NFS server module
255nullarbor:~ # echo -n 'module nfsd +p' >
256 <debugfs>/dynamic_debug/control
257
258// enable all 12 messages in the function svc_process()
259nullarbor:~ # echo -n 'func svc_process +p' >
260 <debugfs>/dynamic_debug/control
261
262// disable all 12 messages in the function svc_process()
263nullarbor:~ # echo -n 'func svc_process -p' >
264 <debugfs>/dynamic_debug/control
Greg Banks9898abb2009-02-06 12:54:26 +1100265
266// enable messages for NFS calls READ, READLINK, READDIR and READDIR+.
267nullarbor:~ # echo -n 'format "nfsd: READ" +p' >
268 <debugfs>/dynamic_debug/control