| Brendan Gregg | bea3430 | 2016-02-13 21:07:23 -0800 | [diff] [blame] | 1 | Demonstrations of mdflush, the Linux eBPF/bcc version. |
| 2 | |
| 3 | |
| 4 | The mdflush tool traces flushes at the md driver level, and prints details |
| 5 | including the time of the flush: |
| 6 | |
| 7 | # ./mdflush |
| 8 | Tracing md flush requests... Hit Ctrl-C to end. |
| 9 | TIME PID COMM DEVICE |
| 10 | 03:13:49 16770 sync md0 |
| 11 | 03:14:08 16864 sync md0 |
| 12 | 03:14:49 496 kworker/1:0H md0 |
| 13 | 03:14:49 488 xfsaild/md0 md0 |
| 14 | 03:14:54 488 xfsaild/md0 md0 |
| 15 | 03:15:00 488 xfsaild/md0 md0 |
| 16 | 03:15:02 85 kswapd0 md0 |
| 17 | 03:15:02 488 xfsaild/md0 md0 |
| 18 | 03:15:05 488 xfsaild/md0 md0 |
| 19 | 03:15:08 488 xfsaild/md0 md0 |
| 20 | 03:15:10 488 xfsaild/md0 md0 |
| 21 | 03:15:11 488 xfsaild/md0 md0 |
| 22 | 03:15:11 488 xfsaild/md0 md0 |
| 23 | 03:15:11 488 xfsaild/md0 md0 |
| 24 | 03:15:11 488 xfsaild/md0 md0 |
| 25 | 03:15:11 488 xfsaild/md0 md0 |
| 26 | 03:15:12 488 xfsaild/md0 md0 |
| 27 | 03:15:13 488 xfsaild/md0 md0 |
| 28 | 03:15:15 488 xfsaild/md0 md0 |
| 29 | 03:15:19 496 kworker/1:0H md0 |
| 30 | 03:15:49 496 kworker/1:0H md0 |
| 31 | 03:15:55 18840 sync md0 |
| 32 | 03:16:49 496 kworker/1:0H md0 |
| 33 | 03:17:19 496 kworker/1:0H md0 |
| 34 | 03:20:19 496 kworker/1:0H md0 |
| 35 | 03:21:19 496 kworker/1:0H md0 |
| 36 | 03:21:49 496 kworker/1:0H md0 |
| 37 | 03:25:19 496 kworker/1:0H md0 |
| 38 | [...] |
| 39 | |
| 40 | This can be useful for correlation with latency outliers or spikes in disk |
| 41 | latency, as measured using another tool (eg, system monitoring). If spikes in |
| 42 | disk latency often coincide with md flush events, then it would make flushing |
| 43 | a target for tuning. |
| 44 | |
| 45 | Note that the flush events are likely to originate from higher in the I/O |
| 46 | stack, such as from file systems. This traces md processing them, and the |
| 47 | timestamp corresponds with when md began to issue the flush to disks. |