| Brendan Gregg | cd1cad1 | 2016-02-12 02:27:19 -0800 | [diff] [blame] | 1 | .TH ext4slower 8 "2016-02-11" "USER COMMANDS" |
| 2 | .SH NAME |
| 3 | ext4slower \- Trace slow ext4 file operations, with per-event details. |
| 4 | .SH SYNOPSIS |
| 5 | .B ext4slower [\-h] [\-j] [\-p PID] [min_ms] |
| 6 | .SH DESCRIPTION |
| 7 | This tool traces common ext4 file operations: reads, writes, opens, and |
| 8 | syncs. It measures the time spent in these operations, and prints details |
| 9 | for each that exceeded a threshold. |
| 10 | |
| 11 | WARNING: See the OVERHEAD section. |
| 12 | |
| 13 | By default, a minimum millisecond threshold of 10 is used. If a threshold of 0 |
| 14 | is used, all events are printed (warning: verbose). |
| 15 | |
| 16 | Since this works by tracing the ext4_file_operations interface functions, it |
| 17 | will need updating to match any changes to these functions. |
| 18 | |
| 19 | Since this uses BPF, only the root user can use this tool. |
| 20 | .SH REQUIREMENTS |
| 21 | CONFIG_BPF and bcc. |
| 22 | .SH OPTIONS |
| 23 | \-p PID |
| 24 | Trace this PID only. |
| 25 | .TP |
| 26 | min_ms |
| 27 | Minimum I/O latency (duration) to trace, in milliseconds. Default is 10 ms. |
| 28 | .SH EXAMPLES |
| 29 | .TP |
| 30 | Trace synchronous file reads and writes slower than 10 ms: |
| 31 | # |
| 32 | .B ext4slower |
| 33 | .TP |
| 34 | Trace slower than 1 ms: |
| 35 | # |
| 36 | .B ext4slower 1 |
| 37 | .TP |
| 38 | Trace slower than 1 ms, and output just the fields in parsable format (csv): |
| 39 | # |
| 40 | .B ext4slower \-j 1 |
| 41 | .TP |
| 42 | Trace all file reads and writes (warning: the output will be verbose): |
| 43 | # |
| 44 | .B ext4slower 0 |
| 45 | .TP |
| 46 | Trace slower than 1 ms, for PID 181 only: |
| 47 | # |
| 48 | .B ext4slower \-p 181 1 |
| 49 | .SH FIELDS |
| 50 | .TP |
| 51 | TIME(s) |
| 52 | Time of I/O completion since the first I/O seen, in seconds. |
| 53 | .TP |
| 54 | COMM |
| 55 | Process name. |
| 56 | .TP |
| 57 | PID |
| 58 | Process ID. |
| 59 | .TP |
| 60 | T |
| 61 | Type of operation. R == read, W == write, O == open, S == fsync. |
| 62 | .TP |
| 63 | OFF_KB |
| 64 | File offset for the I/O, in Kbytes. |
| 65 | .TP |
| 66 | BYTES |
| 67 | Size of I/O, in bytes. |
| 68 | .TP |
| 69 | LAT(ms) |
| 70 | Latency (duration) of I/O, measured from when it was issued by VFS to the |
| 71 | filesystem, to when it completed. This time is inclusive of block device I/O, |
| 72 | file system CPU cycles, file system locks, run queue latency, etc. It's a more |
| 73 | accurate measure of the latency suffered by applications performing file |
| 74 | system I/O, than to measure this down at the block device interface. |
| 75 | .TP |
| 76 | FILENAME |
| 77 | A cached kernel file name (comes from dentry->d_iname). |
| 78 | .TP |
| 79 | ENDTIME_us |
| 80 | Completion timestamp, microseconds (\-j only). |
| 81 | .TP |
| 82 | OFFSET_b |
| 83 | File offset, bytes (\-j only). |
| 84 | .TP |
| 85 | LATENCY_us |
| 86 | Latency (duration) of the I/O, in microseconds (\-j only). |
| 87 | .SH OVERHEAD |
| 88 | This adds low-overhead instrumentation to these ext4 operations, |
| 89 | including reads and writes from the file system cache. Such reads and writes |
| 90 | can be very frequent (depending on the workload; eg, 1M/sec), at which |
| 91 | point the overhead of this tool (even if it prints no "slower" events) can |
| 92 | begin to become significant. Measure and quantify before use. If this |
| 93 | continues to be a problem, consider switching to a tool that prints in-kernel |
| 94 | summaries only. |
| 95 | .PP |
| 96 | Note that the overhead of this tool should be less than fileslower(8), as |
| 97 | this tool targets ext4 functions only, and not all file read/write paths |
| 98 | (which can include socket I/O). |
| 99 | .SH SOURCE |
| 100 | This is from bcc. |
| 101 | .IP |
| 102 | https://github.com/iovisor/bcc |
| 103 | .PP |
| 104 | Also look in the bcc distribution for a companion _examples.txt file containing |
| 105 | example usage, output, and commentary for this tool. |
| 106 | .SH OS |
| 107 | Linux |
| 108 | .SH STABILITY |
| 109 | Unstable - in development. |
| 110 | .SH AUTHOR |
| 111 | Brendan Gregg |
| 112 | .SH SEE ALSO |
| 113 | biosnoop(8), funccount(8), fileslower(8) |