blob: d63d1f2db56275c82f5216dcdba0128c5e45dd50 [file] [log] [blame]
Aaron Carrolld60e92d2007-09-17 10:32:59 +02001.TH fio 1 "September 2007" "User Manual"
2.SH NAME
3fio \- flexible I/O tester
4.SH SYNOPSIS
5.B fio
6[\fIoptions\fR] [\fIjobfile\fR]...
7.SH DESCRIPTION
8.B fio
9is a tool that will spawn a number of threads or processes doing a
10particular type of I/O action as specified by the user.
11The typical use of fio is to write a job file matching the I/O load
12one wants to simulate.
13.SH OPTIONS
14.TP
Jens Axboe49da1242011-10-13 20:17:02 +020015.BI \-\-debug \fR=\fPtype
16Enable verbose tracing of various fio actions. May be `all' for all types
17or individual types separated by a comma (eg \-\-debug=io,file). `help' will
18list all available tracing options.
19.TP
Aaron Carrolld60e92d2007-09-17 10:32:59 +020020.BI \-\-output \fR=\fPfilename
21Write output to \fIfilename\fR.
22.TP
23.BI \-\-timeout \fR=\fPtimeout
24Limit run time to \fItimeout\fR seconds.
25.TP
26.B \-\-latency\-log
27Generate per-job latency logs.
28.TP
29.B \-\-bandwidth\-log
30Generate per-job bandwidth logs.
31.TP
32.B \-\-minimal
Aaron Carrolld1429b52007-09-18 08:10:57 +020033Print statistics in a terse, semicolon-delimited format.
Aaron Carrolld60e92d2007-09-17 10:32:59 +020034.TP
Jens Axboe49da1242011-10-13 20:17:02 +020035.B \-\-version
36Display version information and exit.
37.TP
Jens Axboe065248b2011-10-13 20:51:05 +020038.BI \-\-terse\-version \fR=\fPversion
Jens Axboe4d658652011-10-17 15:05:47 +020039Set terse version output format (Current version 3, or older version 2).
Jens Axboe49da1242011-10-13 20:17:02 +020040.TP
41.B \-\-help
42Display usage information and exit.
43.TP
44.BI \-\-cmdhelp \fR=\fPcommand
45Print help information for \fIcommand\fR. May be `all' for all commands.
46.TP
Aaron Carrolld60e92d2007-09-17 10:32:59 +020047.BI \-\-showcmd \fR=\fPjobfile
48Convert \fIjobfile\fR to a set of command-line options.
49.TP
50.B \-\-readonly
51Enable read-only safety checks.
52.TP
53.BI \-\-eta \fR=\fPwhen
54Specifies when real-time ETA estimate should be printed. \fIwhen\fR may
55be one of `always', `never' or `auto'.
56.TP
Jens Axboe49da1242011-10-13 20:17:02 +020057.BI \-\-readonly
58Turn on safety read-only checks, preventing any attempted write.
59.TP
Aaron Carrollc0a5d352008-02-26 23:10:39 +010060.BI \-\-section \fR=\fPsec
Jens Axboe49da1242011-10-13 20:17:02 +020061Only run section \fIsec\fR from job file. Multiple of these options can be given, adding more sections to run.
Aaron Carrollc0a5d352008-02-26 23:10:39 +010062.TP
Jens Axboe49da1242011-10-13 20:17:02 +020063.BI \-\-alloc\-size \fR=\fPkb
64Set the internal smalloc pool size to \fIkb\fP kilobytes.
Aaron Carrolld60e92d2007-09-17 10:32:59 +020065.TP
Jens Axboe49da1242011-10-13 20:17:02 +020066.BI \-\-warnings\-fatal
67All fio parser warnings are fatal, causing fio to exit with an error.
Jens Axboe91837882008-02-05 12:02:07 +010068.TP
Jens Axboe49da1242011-10-13 20:17:02 +020069.BI \-\-max\-jobs \fR=\fPnr
70Set the maximum allowed number of jobs (threads/processes) to suport.
Aaron Carrolld60e92d2007-09-17 10:32:59 +020071.TP
Jens Axboe49da1242011-10-13 20:17:02 +020072.BI \-\-server \fR=\fPargs
73Start a backend server, with \fIargs\fP specifying what to listen to. See client/server section.
Jens Axboef57a9c52011-09-09 21:01:37 +020074.TP
Jens Axboe49da1242011-10-13 20:17:02 +020075.BI \-\-daemonize \fR=\fPpidfile
76Background a fio server, writing the pid to the given pid file.
77.TP
78.BI \-\-client \fR=\fPhost
79Instead of running the jobs locally, send and run them on the given host.
Aaron Carrolld60e92d2007-09-17 10:32:59 +020080.SH "JOB FILE FORMAT"
81Job files are in `ini' format. They consist of one or more
82job definitions, which begin with a job name in square brackets and
83extend to the next job name. The job name can be any ASCII string
84except `global', which has a special meaning. Following the job name is
85a sequence of zero or more parameters, one per line, that define the
86behavior of the job. Any line starting with a `;' or `#' character is
Aaron Carrolld1429b52007-09-18 08:10:57 +020087considered a comment and ignored.
Aaron Carrolld9956b62007-11-16 12:12:45 +010088.P
89If \fIjobfile\fR is specified as `-', the job file will be read from
90standard input.
Aaron Carrolld60e92d2007-09-17 10:32:59 +020091.SS "Global Section"
92The global section contains default parameters for jobs specified in the
93job file. A job is only affected by global sections residing above it,
94and there may be any number of global sections. Specific job definitions
95may override any parameter set in global sections.
96.SH "JOB PARAMETERS"
97.SS Types
98Some parameters may take arguments of a specific type. The types used are:
99.TP
100.I str
101String: a sequence of alphanumeric characters.
102.TP
103.I int
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200104SI integer: a whole number, possibly containing a suffix denoting the base unit
Jens Axboeb09da8f2009-07-17 23:16:17 +0200105of the value. Accepted suffixes are `k', 'M', 'G', 'T', and 'P', denoting
106kilo (1024), mega (1024^2), giga (1024^3), tera (1024^4), and peta (1024^5)
107respectively. The suffix is not case sensitive. If prefixed with '0x', the
Martin Steigerwald5982a922011-06-27 16:07:24 +0200108value is assumed to be base 16 (hexadecimal). A suffix may include a trailing 'b',
109for instance 'kb' is identical to 'k'. You can specify a base 10 value
Jens Axboe57fc29f2010-06-23 22:24:07 +0200110by using 'KiB', 'MiB', 'GiB', etc. This is useful for disk drives where
111values are often given in base 10 values. Specifying '30GiB' will get you
11230*1000^3 bytes.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200113.TP
114.I bool
115Boolean: a true or false value. `0' denotes false, `1' denotes true.
116.TP
117.I irange
118Integer range: a range of integers specified in the format
Aaron Carrolld1429b52007-09-18 08:10:57 +0200119\fIlower\fR:\fIupper\fR or \fIlower\fR\-\fIupper\fR. \fIlower\fR and
120\fIupper\fR may contain a suffix as described above. If an option allows two
121sets of ranges, they are separated with a `,' or `/' character. For example:
122`8\-8k/8M\-4G'.
Yu-ju Hong83349192011-08-13 00:53:44 +0200123.TP
124.I float_list
125List of floating numbers: A list of floating numbers, separated by
126a ':' charcater.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200127.SS "Parameter List"
128.TP
129.BI name \fR=\fPstr
Aaron Carrolld9956b62007-11-16 12:12:45 +0100130May be used to override the job name. On the command line, this parameter
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200131has the special purpose of signalling the start of a new job.
132.TP
133.BI description \fR=\fPstr
134Human-readable description of the job. It is printed when the job is run, but
135otherwise has no special purpose.
136.TP
137.BI directory \fR=\fPstr
138Prefix filenames with this directory. Used to place files in a location other
139than `./'.
140.TP
141.BI filename \fR=\fPstr
142.B fio
143normally makes up a file name based on the job name, thread number, and file
Aaron Carrolld1429b52007-09-18 08:10:57 +0200144number. If you want to share files between threads in a job or several jobs,
145specify a \fIfilename\fR for each of them to override the default. If the I/O
146engine used is `net', \fIfilename\fR is the host and port to connect to in the
147format \fIhost\fR/\fIport\fR. If the I/O engine is file-based, you can specify
148a number of files by separating the names with a `:' character. `\-' is a
149reserved name, meaning stdin or stdout, depending on the read/write direction
150set.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200151.TP
Jens Axboe3ce9dca2009-06-10 08:55:21 +0200152.BI lockfile \fR=\fPstr
153Fio defaults to not locking any files before it does IO to them. If a file or
154file descriptor is shared, fio can serialize IO to that file to make the end
155result consistent. This is usual for emulating real workloads that share files.
156The lock modes are:
157.RS
158.RS
159.TP
160.B none
161No locking. This is the default.
162.TP
163.B exclusive
164Only one thread or process may do IO at the time, excluding all others.
165.TP
166.B readwrite
167Read-write locking on the file. Many readers may access the file at the same
168time, but writes get exclusive access.
169.RE
170.P
171The option may be post-fixed with a lock batch number. If set, then each
172thread/process may do that amount of IOs to the file before giving up the lock.
173Since lock acquisition is expensive, batching the lock/unlocks will speed up IO.
174.RE
175.P
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200176.BI opendir \fR=\fPstr
177Recursively open any files below directory \fIstr\fR.
178.TP
179.BI readwrite \fR=\fPstr "\fR,\fP rw" \fR=\fPstr
180Type of I/O pattern. Accepted values are:
181.RS
182.RS
183.TP
184.B read
Aaron Carrolld1429b52007-09-18 08:10:57 +0200185Sequential reads.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200186.TP
187.B write
Aaron Carrolld1429b52007-09-18 08:10:57 +0200188Sequential writes.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200189.TP
190.B randread
Aaron Carrolld1429b52007-09-18 08:10:57 +0200191Random reads.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200192.TP
193.B randwrite
Aaron Carrolld1429b52007-09-18 08:10:57 +0200194Random writes.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200195.TP
196.B rw
Aaron Carrolld1429b52007-09-18 08:10:57 +0200197Mixed sequential reads and writes.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200198.TP
199.B randrw
Aaron Carrolld1429b52007-09-18 08:10:57 +0200200Mixed random reads and writes.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200201.RE
202.P
Jens Axboe38dad622010-07-20 14:46:00 -0600203For mixed I/O, the default split is 50/50. For certain types of io the result
204may still be skewed a bit, since the speed may be different. It is possible to
205specify a number of IO's to do before getting a new offset, this is one by
206appending a `:\fI<nr>\fR to the end of the string given. For a random read, it
207would look like \fBrw=randread:8\fR for passing in an offset modifier with a
Jens Axboe059b0802011-08-25 09:09:37 +0200208value of 8. If the postfix is used with a sequential IO pattern, then the value
209specified will be added to the generated offset for each IO. For instance,
210using \fBrw=write:4k\fR will skip 4k for every write. It turns sequential IO
211into sequential IO with holes. See the \fBrw_sequencer\fR option.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200212.RE
213.TP
Jens Axboe38dad622010-07-20 14:46:00 -0600214.BI rw_sequencer \fR=\fPstr
215If an offset modifier is given by appending a number to the \fBrw=<str>\fR line,
216then this option controls how that number modifies the IO offset being
217generated. Accepted values are:
218.RS
219.RS
220.TP
221.B sequential
222Generate sequential offset
223.TP
224.B identical
225Generate the same offset
226.RE
227.P
228\fBsequential\fR is only useful for random IO, where fio would normally
229generate a new random offset for every IO. If you append eg 8 to randread, you
230would get a new random offset for every 8 IO's. The result would be a seek for
231only every 8 IO's, instead of for every IO. Use \fBrw=randread:8\fR to specify
232that. As sequential IO is already sequential, setting \fBsequential\fR for that
233would not result in any differences. \fBidentical\fR behaves in a similar
234fashion, except it sends the same offset 8 number of times before generating a
235new offset.
236.RE
237.P
238.TP
Jens Axboe90fef2d2009-07-17 22:33:32 +0200239.BI kb_base \fR=\fPint
240The base unit for a kilobyte. The defacto base is 2^10, 1024. Storage
241manufacturers like to use 10^3 or 1000 as a base ten unit instead, for obvious
242reasons. Allow values are 1024 or 1000, with 1024 being the default.
243.TP
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200244.BI randrepeat \fR=\fPbool
245Seed the random number generator in a predictable way so results are repeatable
Aaron Carrolld1429b52007-09-18 08:10:57 +0200246across runs. Default: true.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200247.TP
Jens Axboe2615cc42011-03-28 09:35:09 +0200248.BI use_os_rand \fR=\fPbool
249Fio can either use the random generator supplied by the OS to generator random
250offsets, or it can use it's own internal generator (based on Tausworthe).
251Default is to use the internal generator, which is often of better quality and
252faster. Default: false.
253.TP
Eric Gourioua596f042011-06-17 09:11:45 +0200254.BI fallocate \fR=\fPstr
255Whether pre-allocation is performed when laying down files. Accepted values
256are:
257.RS
258.RS
259.TP
260.B none
261Do not pre-allocate space.
262.TP
263.B posix
264Pre-allocate via posix_fallocate().
265.TP
266.B keep
267Pre-allocate via fallocate() with FALLOC_FL_KEEP_SIZE set.
268.TP
269.B 0
270Backward-compatible alias for 'none'.
271.TP
272.B 1
273Backward-compatible alias for 'posix'.
274.RE
275.P
276May not be available on all supported platforms. 'keep' is only
277available on Linux. If using ZFS on Solaris this must be set to 'none'
278because ZFS doesn't support it. Default: 'posix'.
279.RE
Jens Axboe7bc8c2c2010-01-28 11:31:31 +0100280.TP
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200281.BI fadvise_hint \fR=\fPbool
Aaron Carrolld1429b52007-09-18 08:10:57 +0200282Disable use of \fIposix_fadvise\fR\|(2) to advise the kernel what I/O patterns
283are likely to be issued. Default: true.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200284.TP
Jens Axboef7fa2652009-03-09 14:20:20 +0100285.BI size \fR=\fPint
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200286Total size of I/O for this job. \fBfio\fR will run until this many bytes have
287been transfered, unless limited by other options (\fBruntime\fR, for instance).
Jens Axboed7c8be02010-11-25 08:21:39 +0100288Unless \fBnrfiles\fR and \fBfilesize\fR options are given, this amount will be
Jens Axboed6667262010-06-25 11:32:48 +0200289divided between the available files for the job. If not set, fio will use the
290full size of the given files or devices. If the the files do not exist, size
Jens Axboe7bb59102011-07-12 19:47:03 +0200291must be given. It is also possible to give size as a percentage between 1 and
292100. If size=20% is given, fio will use 20% of the full size of the given files
293or devices.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200294.TP
Jens Axboe74586c12011-01-20 10:16:03 -0700295.BI fill_device \fR=\fPbool "\fR,\fB fill_fs" \fR=\fPbool
Jens Axboe3ce9dca2009-06-10 08:55:21 +0200296Sets size to something really large and waits for ENOSPC (no space left on
297device) as the terminating condition. Only makes sense with sequential write.
298For a read workload, the mount point will be filled first then IO started on
Jens Axboe4f124322011-01-19 15:35:26 -0700299the result. This option doesn't make sense if operating on a raw device node,
300since the size of that is already known by the file system. Additionally,
301writing beyond end-of-device will not return ENOSPC there.
Jens Axboe3ce9dca2009-06-10 08:55:21 +0200302.TP
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200303.BI filesize \fR=\fPirange
304Individual file sizes. May be a range, in which case \fBfio\fR will select sizes
Aaron Carrolld1429b52007-09-18 08:10:57 +0200305for files at random within the given range, limited to \fBsize\fR in total (if
306that is given). If \fBfilesize\fR is not specified, each created file is the
307same size.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200308.TP
Jens Axboef7fa2652009-03-09 14:20:20 +0100309.BI blocksize \fR=\fPint[,int] "\fR,\fB bs" \fR=\fPint[,int]
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200310Block size for I/O units. Default: 4k. Values for reads and writes can be
Jens Axboe656ebab2010-04-13 10:39:14 +0200311specified separately in the format \fIread\fR,\fIwrite\fR, either of
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200312which may be empty to leave that value at its default.
313.TP
Jens Axboe91837882008-02-05 12:02:07 +0100314.BI blocksize_range \fR=\fPirange[,irange] "\fR,\fB bsrange" \fR=\fPirange[,irange]
Aaron Carrolld1429b52007-09-18 08:10:57 +0200315Specify a range of I/O block sizes. The issued I/O unit will always be a
316multiple of the minimum size, unless \fBblocksize_unaligned\fR is set. Applies
Jens Axboe91837882008-02-05 12:02:07 +0100317to both reads and writes if only one range is given, but can be specified
Jens Axboe656ebab2010-04-13 10:39:14 +0200318separately with a comma seperating the values. Example: bsrange=1k-4k,2k-8k.
Jens Axboe91837882008-02-05 12:02:07 +0100319Also (see \fBblocksize\fR).
320.TP
321.BI bssplit \fR=\fPstr
322This option allows even finer grained control of the block sizes issued,
323not just even splits between them. With this option, you can weight various
324block sizes for exact control of the issued IO for a job that has mixed
325block sizes. The format of the option is bssplit=blocksize/percentage,
Martin Steigerwald5982a922011-06-27 16:07:24 +0200326optionally adding as many definitions as needed separated by a colon.
Jens Axboe91837882008-02-05 12:02:07 +0100327Example: bssplit=4k/10:64k/50:32k/40 would issue 50% 64k blocks, 10% 4k
Jens Axboec83cdd32009-04-24 14:23:59 +0200328blocks and 40% 32k blocks. \fBbssplit\fR also supports giving separate
329splits to reads and writes. The format is identical to what the
330\fBbs\fR option accepts, the read and write parts are separated with a
331comma.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200332.TP
333.B blocksize_unaligned\fR,\fP bs_unaligned
Aaron Carrolld1429b52007-09-18 08:10:57 +0200334If set, any size in \fBblocksize_range\fR may be used. This typically won't
335work with direct I/O, as that normally requires sector alignment.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200336.TP
Jens Axboe2b7a01d2009-03-11 11:00:13 +0100337.BI blockalign \fR=\fPint[,int] "\fR,\fB ba" \fR=\fPint[,int]
Martin Steigerwald639ce0f2009-05-20 11:33:49 +0200338At what boundary to align random IO offsets. Defaults to the same as 'blocksize'
339the minimum blocksize given. Minimum alignment is typically 512b
Jens Axboe2b7a01d2009-03-11 11:00:13 +0100340for using direct IO, though it usually depends on the hardware block size.
341This option is mutually exclusive with using a random map for files, so it
342will turn off that option.
Jens Axboe43602662009-03-14 20:08:47 +0100343.TP
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200344.B zero_buffers
345Initialise buffers with all zeros. Default: fill buffers with random data.
346.TP
Jens Axboe901bb992009-03-14 20:17:36 +0100347.B refill_buffers
348If this option is given, fio will refill the IO buffers on every submit. The
349default is to only fill it at init time and reuse that data. Only makes sense
350if zero_buffers isn't specified, naturally. If data verification is enabled,
351refill_buffers is also automatically enabled.
352.TP
Jens Axboefd684182011-09-19 09:24:44 +0200353.BI scramble_buffers \fR=\fPbool
354If \fBrefill_buffers\fR is too costly and the target is using data
355deduplication, then setting this option will slightly modify the IO buffer
356contents to defeat normal de-dupe attempts. This is not enough to defeat
357more clever block compression attempts, but it will stop naive dedupe
358of blocks. Default: true.
359.TP
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200360.BI nrfiles \fR=\fPint
361Number of files to use for this job. Default: 1.
362.TP
363.BI openfiles \fR=\fPint
364Number of files to keep open at the same time. Default: \fBnrfiles\fR.
365.TP
366.BI file_service_type \fR=\fPstr
367Defines how files to service are selected. The following types are defined:
368.RS
369.RS
370.TP
371.B random
372Choose a file at random
373.TP
374.B roundrobin
375Round robin over open files (default).
Jens Axboe6b7f6852009-03-09 14:22:56 +0100376.B sequential
377Do each file in the set sequentially.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200378.RE
379.P
380The number of I/Os to issue before switching a new file can be specified by
381appending `:\fIint\fR' to the service type.
382.RE
383.TP
384.BI ioengine \fR=\fPstr
385Defines how the job issues I/O. The following types are defined:
386.RS
387.RS
388.TP
389.B sync
390Basic \fIread\fR\|(2) or \fIwrite\fR\|(2) I/O. \fIfseek\fR\|(2) is used to
391position the I/O location.
392.TP
gurudas paia31041e2007-10-23 15:12:30 +0200393.B psync
394Basic \fIpread\fR\|(2) or \fIpwrite\fR\|(2) I/O.
395.TP
Jens Axboe91837882008-02-05 12:02:07 +0100396.B vsync
397Basic \fIreadv\fR\|(2) or \fIwritev\fR\|(2) I/O. Will emulate queuing by
398coalescing adjacents IOs into a single submission.
399.TP
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200400.B libaio
Jens Axboec44b1ff2011-08-30 20:43:53 -0600401Linux native asynchronous I/O. This engine also has a sub-option,
402\fBuserspace_reap\fR. To set it, use \fBioengine=libaio:userspace_reap\fR.
403Normally, with the libaio engine in use, fio will use the
404\fIio_getevents\fR\|(3) system call to reap newly returned events. With this
405flag turned on, the AIO ring will be read directly from user-space to reap
406events. The reaping mode is only enabled when polling for a minimum of \fB0\fR
407events (eg when \fBiodepth_batch_complete=0\fR).
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200408.TP
409.B posixaio
Bruce Cran03e20d62011-01-02 20:14:54 +0100410POSIX asynchronous I/O using \fIaio_read\fR\|(3) and \fIaio_write\fR\|(3).
411.TP
412.B solarisaio
413Solaris native asynchronous I/O.
414.TP
415.B windowsaio
416Windows native asynchronous I/O.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200417.TP
418.B mmap
Aaron Carrolld1429b52007-09-18 08:10:57 +0200419File is memory mapped with \fImmap\fR\|(2) and data copied using
420\fImemcpy\fR\|(3).
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200421.TP
422.B splice
Aaron Carrolld1429b52007-09-18 08:10:57 +0200423\fIsplice\fR\|(2) is used to transfer the data and \fIvmsplice\fR\|(2) to
424transfer data from user-space to the kernel.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200425.TP
426.B syslet-rw
427Use the syslet system calls to make regular read/write asynchronous.
428.TP
429.B sg
430SCSI generic sg v3 I/O. May be either synchronous using the SG_IO ioctl, or if
Aaron Carrolld1429b52007-09-18 08:10:57 +0200431the target is an sg character device, we use \fIread\fR\|(2) and
432\fIwrite\fR\|(2) for asynchronous I/O.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200433.TP
434.B null
435Doesn't transfer any data, just pretends to. Mainly used to exercise \fBfio\fR
436itself and for debugging and testing purposes.
437.TP
438.B net
439Transfer over the network. \fBfilename\fR must be set appropriately to
Jens Axboe0fd666b2011-10-06 20:08:53 +0200440`\fIhost\fR,\fIport\fR,\fItype\fR' regardless of data direction. \fItype\fR
441is one of \fBtcp\fR, \fBudp\fR, or \fBunix\fR. For UNIX domain sockets,
442the \fIhost\fR parameter is a file system path.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200443.TP
444.B netsplice
445Like \fBnet\fR, but uses \fIsplice\fR\|(2) and \fIvmsplice\fR\|(2) to map data
446and send/receive.
447.TP
gurudas pai53aec0a2007-10-05 13:20:18 +0200448.B cpuio
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200449Doesn't transfer any data, but burns CPU cycles according to \fBcpuload\fR and
450\fBcpucycles\fR parameters.
451.TP
452.B guasi
453The GUASI I/O engine is the Generic Userspace Asynchronous Syscall Interface
454approach to asycnronous I/O.
Aaron Carrolld1429b52007-09-18 08:10:57 +0200455.br
456See <http://www.xmailserver.org/guasi\-lib.html>.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200457.TP
ren yufei21b8aee2011-08-01 10:01:57 +0200458.B rdma
Bart Van Assche85286c52011-08-07 21:50:51 +0200459The RDMA I/O engine supports both RDMA memory semantics (RDMA_WRITE/RDMA_READ)
460and channel semantics (Send/Recv) for the InfiniBand, RoCE and iWARP protocols.
ren yufei21b8aee2011-08-01 10:01:57 +0200461.TP
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200462.B external
463Loads an external I/O engine object file. Append the engine filename as
464`:\fIenginepath\fR'.
465.RE
466.RE
467.TP
468.BI iodepth \fR=\fPint
Sebastian Kayser8489dae2010-12-01 22:28:47 +0100469Number of I/O units to keep in flight against the file. Note that increasing
470iodepth beyond 1 will not affect synchronous ioengines (except for small
Jens Axboeee72ca02010-12-02 20:05:37 +0100471degress when verify_async is in use). Even async engines my impose OS
472restrictions causing the desired depth not to be achieved. This may happen on
473Linux when using libaio and not setting \fBdirect\fR=1, since buffered IO is
474not async on that OS. Keep an eye on the IO depth distribution in the
475fio output to verify that the achieved depth is as expected. Default: 1.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200476.TP
477.BI iodepth_batch \fR=\fPint
478Number of I/Os to submit at once. Default: \fBiodepth\fR.
479.TP
Jens Axboe3ce9dca2009-06-10 08:55:21 +0200480.BI iodepth_batch_complete \fR=\fPint
481This defines how many pieces of IO to retrieve at once. It defaults to 1 which
482 means that we'll ask for a minimum of 1 IO in the retrieval process from the
483kernel. The IO retrieval will go on until we hit the limit set by
484\fBiodepth_low\fR. If this variable is set to 0, then fio will always check for
485completed events before queuing more IO. This helps reduce IO latency, at the
486cost of more retrieval system calls.
487.TP
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200488.BI iodepth_low \fR=\fPint
489Low watermark indicating when to start filling the queue again. Default:
490\fBiodepth\fR.
491.TP
492.BI direct \fR=\fPbool
493If true, use non-buffered I/O (usually O_DIRECT). Default: false.
494.TP
495.BI buffered \fR=\fPbool
496If true, use buffered I/O. This is the opposite of the \fBdirect\fR parameter.
497Default: true.
498.TP
Jens Axboef7fa2652009-03-09 14:20:20 +0100499.BI offset \fR=\fPint
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200500Offset in the file to start I/O. Data before the offset will not be touched.
501.TP
502.BI fsync \fR=\fPint
Aaron Carrolld1429b52007-09-18 08:10:57 +0200503How many I/Os to perform before issuing an \fBfsync\fR\|(2) of dirty data. If
5040, don't sync. Default: 0.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200505.TP
Jens Axboe5f9099e2009-06-16 22:40:26 +0200506.BI fdatasync \fR=\fPint
507Like \fBfsync\fR, but uses \fBfdatasync\fR\|(2) instead to only sync the
508data parts of the file. Default: 0.
509.TP
Jens Axboee76b1da2010-03-09 20:49:54 +0100510.BI sync_file_range \fR=\fPstr:int
511Use sync_file_range() for every \fRval\fP number of write operations. Fio will
512track range of writes that have happened since the last sync_file_range() call.
513\fRstr\fP can currently be one or more of:
514.RS
515.TP
516.B wait_before
517SYNC_FILE_RANGE_WAIT_BEFORE
518.TP
519.B write
520SYNC_FILE_RANGE_WRITE
521.TP
522.B wait_after
523SYNC_FILE_RANGE_WRITE
524.TP
525.RE
526.P
527So if you do sync_file_range=wait_before,write:8, fio would use
528\fBSYNC_FILE_RANGE_WAIT_BEFORE | SYNC_FILE_RANGE_WRITE\fP for every 8 writes.
529Also see the sync_file_range(2) man page. This option is Linux specific.
530.TP
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200531.BI overwrite \fR=\fPbool
Aaron Carrolld1429b52007-09-18 08:10:57 +0200532If writing, setup the file first and do overwrites. Default: false.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200533.TP
534.BI end_fsync \fR=\fPbool
Aaron Carrolld1429b52007-09-18 08:10:57 +0200535Sync file contents when job exits. Default: false.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200536.TP
537.BI fsync_on_close \fR=\fPbool
538If true, sync file contents on close. This differs from \fBend_fsync\fR in that
Aaron Carrolld1429b52007-09-18 08:10:57 +0200539it will happen on every close, not just at the end of the job. Default: false.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200540.TP
541.BI rwmixcycle \fR=\fPint
542How many milliseconds before switching between reads and writes for a mixed
543workload. Default: 500ms.
544.TP
545.BI rwmixread \fR=\fPint
546Percentage of a mixed workload that should be reads. Default: 50.
547.TP
548.BI rwmixwrite \fR=\fPint
Aaron Carrolld1429b52007-09-18 08:10:57 +0200549Percentage of a mixed workload that should be writes. If \fBrwmixread\fR and
Jens Axboec35dd7a2009-06-10 08:39:16 +0200550\fBrwmixwrite\fR are given and do not sum to 100%, the latter of the two
551overrides the first. This may interfere with a given rate setting, if fio is
552asked to limit reads or writes to a certain rate. If that is the case, then
553the distribution may be skewed. Default: 50.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200554.TP
555.B norandommap
556Normally \fBfio\fR will cover every block of the file when doing random I/O. If
557this parameter is given, a new offset will be chosen without looking at past
558I/O history. This parameter is mutually exclusive with \fBverify\fR.
559.TP
Jens Axboe744492c2011-08-08 09:47:13 +0200560.BI softrandommap \fR=\fPbool
Jens Axboe3ce9dca2009-06-10 08:55:21 +0200561See \fBnorandommap\fR. If fio runs with the random block map enabled and it
562fails to allocate the map, if this option is set it will continue without a
563random block map. As coverage will not be as complete as with random maps, this
564option is disabled by default.
565.TP
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200566.BI nice \fR=\fPint
567Run job with given nice value. See \fInice\fR\|(2).
568.TP
569.BI prio \fR=\fPint
570Set I/O priority value of this job between 0 (highest) and 7 (lowest). See
571\fIionice\fR\|(1).
572.TP
573.BI prioclass \fR=\fPint
574Set I/O priority class. See \fIionice\fR\|(1).
575.TP
576.BI thinktime \fR=\fPint
577Stall job for given number of microseconds between issuing I/Os.
578.TP
579.BI thinktime_spin \fR=\fPint
580Pretend to spend CPU time for given number of microseconds, sleeping the rest
581of the time specified by \fBthinktime\fR. Only valid if \fBthinktime\fR is set.
582.TP
583.BI thinktime_blocks \fR=\fPint
584Number of blocks to issue before waiting \fBthinktime\fR microseconds.
585Default: 1.
586.TP
587.BI rate \fR=\fPint
Jens Axboec35dd7a2009-06-10 08:39:16 +0200588Cap bandwidth used by this job. The number is in bytes/sec, the normal postfix
589rules apply. You can use \fBrate\fR=500k to limit reads and writes to 500k each,
590or you can specify read and writes separately. Using \fBrate\fR=1m,500k would
591limit reads to 1MB/sec and writes to 500KB/sec. Capping only reads or writes
592can be done with \fBrate\fR=,500k or \fBrate\fR=500k,. The former will only
593limit writes (to 500KB/sec), the latter will only limit reads.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200594.TP
595.BI ratemin \fR=\fPint
596Tell \fBfio\fR to do whatever it can to maintain at least the given bandwidth.
Jens Axboec35dd7a2009-06-10 08:39:16 +0200597Failing to meet this requirement will cause the job to exit. The same format
598as \fBrate\fR is used for read vs write separation.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200599.TP
600.BI rate_iops \fR=\fPint
Jens Axboec35dd7a2009-06-10 08:39:16 +0200601Cap the bandwidth to this number of IOPS. Basically the same as rate, just
602specified independently of bandwidth. The same format as \fBrate\fR is used for
603read vs write seperation. If \fBblocksize\fR is a range, the smallest block
604size is used as the metric.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200605.TP
606.BI rate_iops_min \fR=\fPint
Jens Axboec35dd7a2009-06-10 08:39:16 +0200607If this rate of I/O is not met, the job will exit. The same format as \fBrate\fR
608is used for read vs write seperation.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200609.TP
610.BI ratecycle \fR=\fPint
611Average bandwidth for \fBrate\fR and \fBratemin\fR over this number of
612milliseconds. Default: 1000ms.
613.TP
614.BI cpumask \fR=\fPint
615Set CPU affinity for this job. \fIint\fR is a bitmask of allowed CPUs the job
616may run on. See \fBsched_setaffinity\fR\|(2).
617.TP
618.BI cpus_allowed \fR=\fPstr
619Same as \fBcpumask\fR, but allows a comma-delimited list of CPU numbers.
620.TP
621.BI startdelay \fR=\fPint
622Delay start of job for the specified number of seconds.
623.TP
624.BI runtime \fR=\fPint
625Terminate processing after the specified number of seconds.
626.TP
627.B time_based
628If given, run for the specified \fBruntime\fR duration even if the files are
629completely read or written. The same workload will be repeated as many times
630as \fBruntime\fR allows.
631.TP
Jens Axboe901bb992009-03-14 20:17:36 +0100632.BI ramp_time \fR=\fPint
633If set, fio will run the specified workload for this amount of time before
634logging any performance numbers. Useful for letting performance settle before
635logging results, thus minimizing the runtime required for stable results. Note
Jens Axboec35dd7a2009-06-10 08:39:16 +0200636that the \fBramp_time\fR is considered lead in time for a job, thus it will
637increase the total runtime if a special timeout or runtime is specified.
Jens Axboe901bb992009-03-14 20:17:36 +0100638.TP
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200639.BI invalidate \fR=\fPbool
640Invalidate buffer-cache for the file prior to starting I/O. Default: true.
641.TP
642.BI sync \fR=\fPbool
643Use synchronous I/O for buffered writes. For the majority of I/O engines,
Aaron Carrolld1429b52007-09-18 08:10:57 +0200644this means using O_SYNC. Default: false.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200645.TP
646.BI iomem \fR=\fPstr "\fR,\fP mem" \fR=\fPstr
647Allocation method for I/O unit buffer. Allowed values are:
648.RS
649.RS
650.TP
651.B malloc
652Allocate memory with \fImalloc\fR\|(3).
653.TP
654.B shm
655Use shared memory buffers allocated through \fIshmget\fR\|(2).
656.TP
657.B shmhuge
658Same as \fBshm\fR, but use huge pages as backing.
659.TP
660.B mmap
661Use \fImmap\fR\|(2) for allocation. Uses anonymous memory unless a filename
662is given after the option in the format `:\fIfile\fR'.
663.TP
664.B mmaphuge
665Same as \fBmmap\fR, but use huge files as backing.
666.RE
667.P
668The amount of memory allocated is the maximum allowed \fBblocksize\fR for the
669job multiplied by \fBiodepth\fR. For \fBshmhuge\fR or \fBmmaphuge\fR to work,
670the system must have free huge pages allocated. \fBmmaphuge\fR also needs to
Jens Axboe2e266ba2009-09-14 08:56:53 +0200671have hugetlbfs mounted, and \fIfile\fR must point there. At least on Linux,
672huge pages must be manually allocated. See \fB/proc/sys/vm/nr_hugehages\fR
673and the documentation for that. Normally you just need to echo an appropriate
674number, eg echoing 8 will ensure that the OS has 8 huge pages ready for
675use.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200676.RE
677.TP
Jens Axboed3923652011-08-03 12:38:39 +0200678.BI iomem_align \fR=\fPint "\fR,\fP mem_align" \fR=\fPint
Jens Axboed529ee12009-07-01 10:33:03 +0200679This indiciates the memory alignment of the IO memory buffers. Note that the
680given alignment is applied to the first IO unit buffer, if using \fBiodepth\fR
681the alignment of the following buffers are given by the \fBbs\fR used. In
682other words, if using a \fBbs\fR that is a multiple of the page sized in the
683system, all buffers will be aligned to this value. If using a \fBbs\fR that
684is not page aligned, the alignment of subsequent IO memory buffers is the
685sum of the \fBiomem_align\fR and \fBbs\fR used.
686.TP
Jens Axboef7fa2652009-03-09 14:20:20 +0100687.BI hugepage\-size \fR=\fPint
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200688Defines the size of a huge page. Must be at least equal to the system setting.
Jens Axboeb22989b2009-07-17 22:29:23 +0200689Should be a multiple of 1MB. Default: 4MB.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200690.TP
691.B exitall
692Terminate all jobs when one finishes. Default: wait for each job to finish.
693.TP
694.BI bwavgtime \fR=\fPint
695Average bandwidth calculations over the given time in milliseconds. Default:
696500ms.
697.TP
Jens Axboec8eeb9d2011-10-05 14:02:22 +0200698.BI iopsavgtime \fR=\fPint
699Average IOPS calculations over the given time in milliseconds. Default:
700500ms.
701.TP
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200702.BI create_serialize \fR=\fPbool
Aaron Carrolld1429b52007-09-18 08:10:57 +0200703If true, serialize file creation for the jobs. Default: true.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200704.TP
705.BI create_fsync \fR=\fPbool
706\fIfsync\fR\|(2) data file after creation. Default: true.
707.TP
Jens Axboe6b7f6852009-03-09 14:22:56 +0100708.BI create_on_open \fR=\fPbool
709If true, the files are not created until they are opened for IO by the job.
710.TP
Jens Axboee9f48472009-06-03 12:14:08 +0200711.BI pre_read \fR=\fPbool
712If this is given, files will be pre-read into memory before starting the given
713IO operation. This will also clear the \fR \fBinvalidate\fR flag, since it is
Jens Axboe9c0d2242009-07-01 12:26:28 +0200714pointless to pre-read and then drop the cache. This will only work for IO
715engines that are seekable, since they allow you to read the same data
716multiple times. Thus it will not work on eg network or splice IO.
Jens Axboee9f48472009-06-03 12:14:08 +0200717.TP
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200718.BI unlink \fR=\fPbool
719Unlink job files when done. Default: false.
720.TP
721.BI loops \fR=\fPint
722Specifies the number of iterations (runs of the same workload) of this job.
723Default: 1.
724.TP
725.BI do_verify \fR=\fPbool
726Run the verify phase after a write phase. Only valid if \fBverify\fR is set.
727Default: true.
728.TP
729.BI verify \fR=\fPstr
730Method of verifying file contents after each iteration of the job. Allowed
731values are:
732.RS
733.RS
734.TP
Jens Axboeb892dc02009-09-05 20:37:35 +0200735.B md5 crc16 crc32 crc32c crc32c-intel crc64 crc7 sha256 sha512 sha1
Jens Axboe0539d752010-06-21 15:22:56 +0200736Store appropriate checksum in the header of each block. crc32c-intel is
737hardware accelerated SSE4.2 driven, falls back to regular crc32c if
738not supported by the system.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200739.TP
740.B meta
741Write extra information about each I/O (timestamp, block number, etc.). The
Jens Axboe996093b2010-06-24 08:37:13 +0200742block number is verified. See \fBverify_pattern\fR as well.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200743.TP
744.B null
745Pretend to verify. Used for testing internals.
746.RE
Jens Axboeb892dc02009-09-05 20:37:35 +0200747
748This option can be used for repeated burn-in tests of a system to make sure
749that the written data is also correctly read back. If the data direction given
750is a read or random read, fio will assume that it should verify a previously
751written file. If the data direction includes any form of write, the verify will
752be of the newly written data.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200753.RE
754.TP
755.BI verify_sort \fR=\fPbool
756If true, written verify blocks are sorted if \fBfio\fR deems it to be faster to
757read them back in a sorted manner. Default: true.
758.TP
Jens Axboef7fa2652009-03-09 14:20:20 +0100759.BI verify_offset \fR=\fPint
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200760Swap the verification header with data somewhere else in the block before
Aaron Carrolld1429b52007-09-18 08:10:57 +0200761writing. It is swapped back before verifying.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200762.TP
Jens Axboef7fa2652009-03-09 14:20:20 +0100763.BI verify_interval \fR=\fPint
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200764Write the verification header for this number of bytes, which should divide
765\fBblocksize\fR. Default: \fBblocksize\fR.
766.TP
Jens Axboe996093b2010-06-24 08:37:13 +0200767.BI verify_pattern \fR=\fPstr
768If set, fio will fill the io buffers with this pattern. Fio defaults to filling
769with totally random bytes, but sometimes it's interesting to fill with a known
770pattern for io verification purposes. Depending on the width of the pattern,
771fio will fill 1/2/3/4 bytes of the buffer at the time(it can be either a
772decimal or a hex number). The verify_pattern if larger than a 32-bit quantity
773has to be a hex number that starts with either "0x" or "0X". Use with
774\fBverify\fP=meta.
775.TP
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200776.BI verify_fatal \fR=\fPbool
777If true, exit the job on the first observed verification failure. Default:
778false.
779.TP
Jens Axboeb463e932011-01-12 09:03:23 +0100780.BI verify_dump \fR=\fPbool
781If set, dump the contents of both the original data block and the data block we
782read off disk to files. This allows later analysis to inspect just what kind of
783data corruption occurred. On by default.
784.TP
Jens Axboee8462bd2009-07-06 12:59:04 +0200785.BI verify_async \fR=\fPint
786Fio will normally verify IO inline from the submitting thread. This option
787takes an integer describing how many async offload threads to create for IO
788verification instead, causing fio to offload the duty of verifying IO contents
Jens Axboec85c3242009-07-06 14:12:57 +0200789to one or more separate threads. If using this offload option, even sync IO
790engines can benefit from using an \fBiodepth\fR setting higher than 1, as it
791allows them to have IO in flight while verifies are running.
Jens Axboee8462bd2009-07-06 12:59:04 +0200792.TP
793.BI verify_async_cpus \fR=\fPstr
794Tell fio to set the given CPU affinity on the async IO verification threads.
795See \fBcpus_allowed\fP for the format used.
796.TP
Jens Axboe6f874182010-06-21 12:53:26 +0200797.BI verify_backlog \fR=\fPint
798Fio will normally verify the written contents of a job that utilizes verify
799once that job has completed. In other words, everything is written then
800everything is read back and verified. You may want to verify continually
801instead for a variety of reasons. Fio stores the meta data associated with an
802IO block in memory, so for large verify workloads, quite a bit of memory would
David Nellans092f7072010-10-26 08:08:42 -0600803be used up holding this meta data. If this option is enabled, fio will write
804only N blocks before verifying these blocks.
Jens Axboe6f874182010-06-21 12:53:26 +0200805.TP
806.BI verify_backlog_batch \fR=\fPint
807Control how many blocks fio will verify if verify_backlog is set. If not set,
808will default to the value of \fBverify_backlog\fR (meaning the entire queue is
David Nellans092f7072010-10-26 08:08:42 -0600809read back and verified). If \fBverify_backlog_batch\fR is less than
810\fBverify_backlog\fR then not all blocks will be verified, if
811\fBverify_backlog_batch\fR is larger than \fBverify_backlog\fR, some blocks
812will be verified more than once.
Jens Axboe6f874182010-06-21 12:53:26 +0200813.TP
Jens Axboed3923652011-08-03 12:38:39 +0200814.B stonewall "\fR,\fP wait_for_previous"
Martin Steigerwald5982a922011-06-27 16:07:24 +0200815Wait for preceding jobs in the job file to exit before starting this one.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200816\fBstonewall\fR implies \fBnew_group\fR.
817.TP
818.B new_group
819Start a new reporting group. If not given, all jobs in a file will be part
820of the same reporting group, unless separated by a stonewall.
821.TP
822.BI numjobs \fR=\fPint
823Number of clones (processes/threads performing the same workload) of this job.
824Default: 1.
825.TP
826.B group_reporting
827If set, display per-group reports instead of per-job when \fBnumjobs\fR is
828specified.
829.TP
830.B thread
831Use threads created with \fBpthread_create\fR\|(3) instead of processes created
832with \fBfork\fR\|(2).
833.TP
Jens Axboef7fa2652009-03-09 14:20:20 +0100834.BI zonesize \fR=\fPint
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200835Divide file into zones of the specified size in bytes. See \fBzoneskip\fR.
836.TP
Jens Axboef7fa2652009-03-09 14:20:20 +0100837.BI zoneskip \fR=\fPint
Aaron Carrolld1429b52007-09-18 08:10:57 +0200838Skip the specified number of bytes when \fBzonesize\fR bytes of data have been
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200839read.
840.TP
841.BI write_iolog \fR=\fPstr
Stefan Hajnoczi5b42a482011-01-08 20:28:41 +0100842Write the issued I/O patterns to the specified file. Specify a separate file
843for each job, otherwise the iologs will be interspersed and the file may be
844corrupt.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200845.TP
846.BI read_iolog \fR=\fPstr
847Replay the I/O patterns contained in the specified file generated by
848\fBwrite_iolog\fR, or may be a \fBblktrace\fR binary file.
849.TP
David Nellans64bbb862010-08-24 22:13:30 +0200850.BI replay_no_stall \fR=\fPint
851While replaying I/O patterns using \fBread_iolog\fR the default behavior
852attempts to respect timing information between I/Os. Enabling
853\fBreplay_no_stall\fR causes I/Os to be replayed as fast as possible while
854still respecting ordering.
855.TP
David Nellansd1c46c02010-08-31 21:20:47 +0200856.BI replay_redirect \fR=\fPstr
857While replaying I/O patterns using \fBread_iolog\fR the default behavior
858is to replay the IOPS onto the major/minor device that each IOP was recorded
859from. Setting \fBreplay_redirect\fR causes all IOPS to be replayed onto the
860single specified device regardless of the device it was recorded from.
861.TP
Steven Noonan836bad52011-09-14 09:21:33 +0200862.BI write_bw_log \fR=\fPstr
Jens Axboe901bb992009-03-14 20:17:36 +0100863If given, write a bandwidth log of the jobs in this job file. Can be used to
864store data of the bandwidth of the jobs in their lifetime. The included
865fio_generate_plots script uses gnuplot to turn these text files into nice
866graphs. See \fBwrite_log_log\fR for behaviour of given filename. For this
867option, the postfix is _bw.log.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200868.TP
Steven Noonan836bad52011-09-14 09:21:33 +0200869.BI write_lat_log \fR=\fPstr
Jens Axboe901bb992009-03-14 20:17:36 +0100870Same as \fBwrite_bw_log\fR, but writes I/O completion latencies. If no
871filename is given with this option, the default filename of "jobname_type.log"
872is used. Even if the filename is given, fio will still append the type of log.
873.TP
Jens Axboec8eeb9d2011-10-05 14:02:22 +0200874.BI write_iops_log \fR=\fPstr
875Same as \fBwrite_bw_log\fR, but writes IOPS. If no filename is given with this
876option, the default filename of "jobname_type.log" is used. Even if the
877filename is given, fio will still append the type of log.
878.TP
Steven Noonan836bad52011-09-14 09:21:33 +0200879.BI disable_lat \fR=\fPbool
Jens Axboe02af0982010-06-24 09:59:34 +0200880Disable measurements of total latency numbers. Useful only for cutting
Jens Axboe901bb992009-03-14 20:17:36 +0100881back the number of calls to gettimeofday, as that does impact performance at
882really high IOPS rates. Note that to really get rid of a large amount of these
883calls, this option must be used with disable_slat and disable_bw as well.
884.TP
Steven Noonan836bad52011-09-14 09:21:33 +0200885.BI disable_clat \fR=\fPbool
Steven Noonanc95f9da2011-06-22 09:47:09 +0200886Disable measurements of completion latency numbers. See \fBdisable_lat\fR.
Jens Axboe02af0982010-06-24 09:59:34 +0200887.TP
Steven Noonan836bad52011-09-14 09:21:33 +0200888.BI disable_slat \fR=\fPbool
Jens Axboe02af0982010-06-24 09:59:34 +0200889Disable measurements of submission latency numbers. See \fBdisable_lat\fR.
Jens Axboe901bb992009-03-14 20:17:36 +0100890.TP
Steven Noonan836bad52011-09-14 09:21:33 +0200891.BI disable_bw_measurement \fR=\fPbool
Jens Axboe02af0982010-06-24 09:59:34 +0200892Disable measurements of throughput/bandwidth numbers. See \fBdisable_lat\fR.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200893.TP
Jens Axboef7fa2652009-03-09 14:20:20 +0100894.BI lockmem \fR=\fPint
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200895Pin the specified amount of memory with \fBmlock\fR\|(2). Can be used to
896simulate a smaller amount of memory.
897.TP
898.BI exec_prerun \fR=\fPstr
899Before running the job, execute the specified command with \fBsystem\fR\|(3).
900.TP
901.BI exec_postrun \fR=\fPstr
902Same as \fBexec_prerun\fR, but the command is executed after the job completes.
903.TP
904.BI ioscheduler \fR=\fPstr
905Attempt to switch the device hosting the file to the specified I/O scheduler.
906.TP
907.BI cpuload \fR=\fPint
908If the job is a CPU cycle-eater, attempt to use the specified percentage of
909CPU cycles.
910.TP
911.BI cpuchunks \fR=\fPint
912If the job is a CPU cycle-eater, split the load into cycles of the
913given time in milliseconds.
914.TP
915.BI disk_util \fR=\fPbool
Aaron Carrolld1429b52007-09-18 08:10:57 +0200916Generate disk utilization statistics if the platform supports it. Default: true.
Jens Axboe901bb992009-03-14 20:17:36 +0100917.TP
918.BI gtod_reduce \fR=\fPbool
919Enable all of the gettimeofday() reducing options (disable_clat, disable_slat,
920disable_bw) plus reduce precision of the timeout somewhat to really shrink the
921gettimeofday() call count. With this option enabled, we only do about 0.4% of
922the gtod() calls we would have done if all time keeping was enabled.
923.TP
924.BI gtod_cpu \fR=\fPint
925Sometimes it's cheaper to dedicate a single thread of execution to just getting
926the current time. Fio (and databases, for instance) are very intensive on
927gettimeofday() calls. With this option, you can set one CPU aside for doing
928nothing but logging current time to a shared memory location. Then the other
929threads/processes that run IO workloads need only copy that segment, instead of
930entering the kernel with a gettimeofday() call. The CPU set aside for doing
931these time calls will be excluded from other uses. Fio will manually clear it
932from the CPU mask of other jobs.
Radha Ramachandranf2bba182009-06-15 08:40:16 +0200933.TP
Jens Axboea696fa22009-12-04 10:05:02 +0100934.BI cgroup \fR=\fPstr
935Add job to this control group. If it doesn't exist, it will be created.
Jens Axboe6adb38a2009-12-07 08:01:26 +0100936The system must have a mounted cgroup blkio mount point for this to work. If
937your system doesn't have it mounted, you can do so with:
938
Martin Steigerwald5982a922011-06-27 16:07:24 +0200939# mount \-t cgroup \-o blkio none /cgroup
Jens Axboea696fa22009-12-04 10:05:02 +0100940.TP
941.BI cgroup_weight \fR=\fPint
942Set the weight of the cgroup to this value. See the documentation that comes
943with the kernel, allowed values are in the range of 100..1000.
Jens Axboee0b0d892009-12-08 10:10:14 +0100944.TP
Vivek Goyal7de87092010-03-31 22:55:15 +0200945.BI cgroup_nodelete \fR=\fPbool
946Normally fio will delete the cgroups it has created after the job completion.
947To override this behavior and to leave cgroups around after the job completion,
948set cgroup_nodelete=1. This can be useful if one wants to inspect various
949cgroup files after job completion. Default: false
950.TP
Jens Axboee0b0d892009-12-08 10:10:14 +0100951.BI uid \fR=\fPint
952Instead of running as the invoking user, set the user ID to this value before
953the thread/process does any work.
954.TP
955.BI gid \fR=\fPint
956Set group ID, see \fBuid\fR.
Yu-ju Hong83349192011-08-13 00:53:44 +0200957.TP
958.BI clat_percentiles \fR=\fPbool
959Enable the reporting of percentiles of completion latencies.
960.TP
961.BI percentile_list \fR=\fPfloat_list
962Overwrite the default list of percentiles for completion
963latencies. Each number is a floating number in the range (0,100], and
964the maximum length of the list is 20. Use ':' to separate the
Martin Steigerwald3eb07282011-10-05 11:41:54 +0200965numbers. For example, \-\-percentile_list=99.5:99.9 will cause fio to
Yu-ju Hong83349192011-08-13 00:53:44 +0200966report the values of completion latency below which 99.5% and 99.9% of
967the observed latencies fell, respectively.
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200968.SH OUTPUT
Aaron Carrolld1429b52007-09-18 08:10:57 +0200969While running, \fBfio\fR will display the status of the created jobs. For
970example:
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200971.RS
Aaron Carrolld1429b52007-09-18 08:10:57 +0200972.P
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200973Threads: 1: [_r] [24.8% done] [ 13509/ 8334 kb/s] [eta 00h:01m:31s]
974.RE
975.P
Aaron Carrolld1429b52007-09-18 08:10:57 +0200976The characters in the first set of brackets denote the current status of each
977threads. The possible values are:
978.P
979.PD 0
Aaron Carrolld60e92d2007-09-17 10:32:59 +0200980.RS
981.TP
982.B P
983Setup but not started.
984.TP
985.B C
986Thread created.
987.TP
988.B I
989Initialized, waiting.
990.TP
991.B R
992Running, doing sequential reads.
993.TP
994.B r
995Running, doing random reads.
996.TP
997.B W
998Running, doing sequential writes.
999.TP
1000.B w
1001Running, doing random writes.
1002.TP
1003.B M
1004Running, doing mixed sequential reads/writes.
1005.TP
1006.B m
1007Running, doing mixed random reads/writes.
1008.TP
1009.B F
1010Running, currently waiting for \fBfsync\fR\|(2).
1011.TP
1012.B V
1013Running, verifying written data.
1014.TP
1015.B E
1016Exited, not reaped by main thread.
1017.TP
1018.B \-
1019Exited, thread reaped.
1020.RE
Aaron Carrolld1429b52007-09-18 08:10:57 +02001021.PD
Aaron Carrolld60e92d2007-09-17 10:32:59 +02001022.P
1023The second set of brackets shows the estimated completion percentage of
1024the current group. The third set shows the read and write I/O rate,
1025respectively. Finally, the estimated run time of the job is displayed.
1026.P
1027When \fBfio\fR completes (or is interrupted by Ctrl-C), it will show data
1028for each thread, each group of threads, and each disk, in that order.
1029.P
1030Per-thread statistics first show the threads client number, group-id, and
1031error code. The remaining figures are as follows:
1032.RS
Aaron Carrolld60e92d2007-09-17 10:32:59 +02001033.TP
1034.B io
1035Number of megabytes of I/O performed.
1036.TP
1037.B bw
1038Average data rate (bandwidth).
1039.TP
1040.B runt
1041Threads run time.
1042.TP
1043.B slat
1044Submission latency minimum, maximum, average and standard deviation. This is
1045the time it took to submit the I/O.
1046.TP
1047.B clat
1048Completion latency minimum, maximum, average and standard deviation. This
1049is the time between submission and completion.
1050.TP
1051.B bw
1052Bandwidth minimum, maximum, percentage of aggregate bandwidth received, average
1053and standard deviation.
1054.TP
1055.B cpu
1056CPU usage statistics. Includes user and system time, number of context switches
1057this thread went through and number of major and minor page faults.
1058.TP
1059.B IO depths
1060Distribution of I/O depths. Each depth includes everything less than (or equal)
1061to it, but greater than the previous depth.
1062.TP
1063.B IO issued
1064Number of read/write requests issued, and number of short read/write requests.
1065.TP
1066.B IO latencies
1067Distribution of I/O completion latencies. The numbers follow the same pattern
1068as \fBIO depths\fR.
1069.RE
Aaron Carrolld60e92d2007-09-17 10:32:59 +02001070.P
1071The group statistics show:
Aaron Carrolld1429b52007-09-18 08:10:57 +02001072.PD 0
Aaron Carrolld60e92d2007-09-17 10:32:59 +02001073.RS
1074.TP
1075.B io
1076Number of megabytes I/O performed.
1077.TP
1078.B aggrb
1079Aggregate bandwidth of threads in the group.
1080.TP
1081.B minb
1082Minimum average bandwidth a thread saw.
1083.TP
1084.B maxb
1085Maximum average bandwidth a thread saw.
1086.TP
1087.B mint
Aaron Carrolld1429b52007-09-18 08:10:57 +02001088Shortest runtime of threads in the group.
Aaron Carrolld60e92d2007-09-17 10:32:59 +02001089.TP
1090.B maxt
1091Longest runtime of threads in the group.
1092.RE
Aaron Carrolld1429b52007-09-18 08:10:57 +02001093.PD
Aaron Carrolld60e92d2007-09-17 10:32:59 +02001094.P
1095Finally, disk statistics are printed with reads first:
Aaron Carrolld1429b52007-09-18 08:10:57 +02001096.PD 0
Aaron Carrolld60e92d2007-09-17 10:32:59 +02001097.RS
1098.TP
1099.B ios
1100Number of I/Os performed by all groups.
1101.TP
1102.B merge
1103Number of merges in the I/O scheduler.
1104.TP
1105.B ticks
1106Number of ticks we kept the disk busy.
1107.TP
1108.B io_queue
1109Total time spent in the disk queue.
1110.TP
1111.B util
1112Disk utilization.
1113.RE
Aaron Carrolld1429b52007-09-18 08:10:57 +02001114.PD
Aaron Carrolld60e92d2007-09-17 10:32:59 +02001115.SH TERSE OUTPUT
1116If the \fB\-\-minimal\fR option is given, the results will be printed in a
David Nellans562c2d22010-09-23 08:38:17 +02001117semicolon-delimited format suitable for scripted use - a job description
1118(if provided) follows on a new line. Note that the first
Jens Axboe525c2bf2010-06-30 15:22:21 +02001119number in the line is the version number. If the output has to be changed
1120for some reason, this number will be incremented by 1 to signify that
1121change. The fields are:
Aaron Carrolld60e92d2007-09-17 10:32:59 +02001122.P
1123.RS
Jens Axboe5e726d02011-10-14 08:08:10 +02001124.B terse version, fio version, jobname, groupid, error
Aaron Carrolld60e92d2007-09-17 10:32:59 +02001125.P
1126Read status:
1127.RS
Jens Axboe312b4af2011-10-13 13:11:42 +02001128.B Total I/O \fR(KB)\fP, bandwidth \fR(KB/s)\fP, IOPS, runtime \fR(ms)\fP
Aaron Carrolld60e92d2007-09-17 10:32:59 +02001129.P
1130Submission latency:
1131.RS
1132.B min, max, mean, standard deviation
1133.RE
1134Completion latency:
1135.RS
1136.B min, max, mean, standard deviation
1137.RE
Jens Axboe1db92cb2011-10-13 13:43:36 +02001138Completion latency percentiles (20 fields):
1139.RS
1140.B Xth percentile=usec
1141.RE
Jens Axboe525c2bf2010-06-30 15:22:21 +02001142Total latency:
1143.RS
1144.B min, max, mean, standard deviation
1145.RE
Aaron Carrolld60e92d2007-09-17 10:32:59 +02001146Bandwidth:
1147.RS
1148.B min, max, aggregate percentage of total, mean, standard deviation
1149.RE
1150.RE
1151.P
1152Write status:
1153.RS
Jens Axboe312b4af2011-10-13 13:11:42 +02001154.B Total I/O \fR(KB)\fP, bandwidth \fR(KB/s)\fP, IOPS, runtime \fR(ms)\fP
Aaron Carrolld60e92d2007-09-17 10:32:59 +02001155.P
1156Submission latency:
1157.RS
1158.B min, max, mean, standard deviation
1159.RE
1160Completion latency:
1161.RS
1162.B min, max, mean, standard deviation
1163.RE
Jens Axboe1db92cb2011-10-13 13:43:36 +02001164Completion latency percentiles (20 fields):
1165.RS
1166.B Xth percentile=usec
1167.RE
Jens Axboe525c2bf2010-06-30 15:22:21 +02001168Total latency:
1169.RS
1170.B min, max, mean, standard deviation
1171.RE
Aaron Carrolld60e92d2007-09-17 10:32:59 +02001172Bandwidth:
1173.RS
1174.B min, max, aggregate percentage of total, mean, standard deviation
1175.RE
1176.RE
1177.P
Aaron Carrolld1429b52007-09-18 08:10:57 +02001178CPU usage:
Aaron Carrolld60e92d2007-09-17 10:32:59 +02001179.RS
Carl Henrik Lundebd2626f2008-06-12 09:17:46 +02001180.B user, system, context switches, major page faults, minor page faults
Aaron Carrolld60e92d2007-09-17 10:32:59 +02001181.RE
1182.P
1183IO depth distribution:
1184.RS
1185.B <=1, 2, 4, 8, 16, 32, >=64
1186.RE
1187.P
David Nellans562c2d22010-09-23 08:38:17 +02001188IO latency distribution:
Aaron Carrolld60e92d2007-09-17 10:32:59 +02001189.RS
David Nellans562c2d22010-09-23 08:38:17 +02001190Microseconds:
1191.RS
1192.B <=2, 4, 10, 20, 50, 100, 250, 500, 750, 1000
1193.RE
1194Milliseconds:
1195.RS
1196.B <=2, 4, 10, 20, 50, 100, 250, 500, 750, 1000, 2000, >=2000
1197.RE
Aaron Carrolld60e92d2007-09-17 10:32:59 +02001198.RE
1199.P
Jens Axboef2f788d2011-10-13 14:03:52 +02001200Disk utilization (1 for each disk used):
1201.RS
1202.B name, read ios, write ios, read merges, write merges, read ticks, write ticks, read in-queue time, write in-queue time, disk utilization percentage
1203.RE
1204.P
Martin Steigerwald5982a922011-06-27 16:07:24 +02001205Error Info (dependent on continue_on_error, default off):
David Nellans562c2d22010-09-23 08:38:17 +02001206.RS
1207.B total # errors, first error code
1208.RE
1209.P
1210.B text description (if provided in config - appears on newline)
Aaron Carrolld60e92d2007-09-17 10:32:59 +02001211.RE
Jens Axboe49da1242011-10-13 20:17:02 +02001212.SH CLIENT / SERVER
1213Normally you would run fio as a stand-alone application on the machine
1214where the IO workload should be generated. However, it is also possible to
1215run the frontend and backend of fio separately. This makes it possible to
1216have a fio server running on the machine(s) where the IO workload should
1217be running, while controlling it from another machine.
1218
1219To start the server, you would do:
1220
1221\fBfio \-\-server=args\fR
1222
1223on that machine, where args defines what fio listens to. The arguments
1224are of the form 'type:hostname or IP:port'. 'type' is either 'ip' for
1225TCP/IP, or 'sock' for a local unix domain socket. 'hostname' is either
1226a hostname or IP address, and 'port' is the port to listen to (only valid
1227for TCP/IP, not a local socket). Some examples:
1228
12291) fio --server
1230
1231 Start a fio server, listening on all interfaces on the default port (8765).
1232
12332) fio --server=ip:hostname:4444
1234
1235 Start a fio server, listening on IP belonging to hostname and on port 4444.
1236
12373) fio --server=:4444
1238
1239 Start a fio server, listening on all interfaces on port 4444.
1240
12414) fio --server=1.2.3.4
1242
1243 Start a fio server, listening on IP 1.2.3.4 on the default port.
1244
12455) fio --server=sock:/tmp/fio.sock
1246
1247 Start a fio server, listening on the local socket /tmp/fio.sock.
1248
1249When a server is running, you can connect to it from a client. The client
1250is run with:
1251
1252fio --local-args --client=server --remote-args <job file(s)>
1253
1254where --local-args are arguments that are local to the client where it is
1255running, 'server' is the connect string, and --remote-args and <job file(s)>
1256are sent to the server. The 'server' string follows the same format as it
1257does on the server side, to allow IP/hostname/socket and port strings.
1258You can connect to multiple clients as well, to do that you could run:
1259
1260fio --client=server2 --client=server2 <job file(s)>
Aaron Carrolld60e92d2007-09-17 10:32:59 +02001261.SH AUTHORS
Jens Axboe49da1242011-10-13 20:17:02 +02001262
Aaron Carrolld60e92d2007-09-17 10:32:59 +02001263.B fio
Jens Axboeaa58d252010-06-09 09:49:38 +02001264was written by Jens Axboe <jens.axboe@oracle.com>,
1265now Jens Axboe <jaxboe@fusionio.com>.
Aaron Carrolld1429b52007-09-18 08:10:57 +02001266.br
1267This man page was written by Aaron Carroll <aaronc@cse.unsw.edu.au> based
Aaron Carrolld60e92d2007-09-17 10:32:59 +02001268on documentation by Jens Axboe.
1269.SH "REPORTING BUGS"
Jens Axboe482900c2009-06-02 12:15:51 +02001270Report bugs to the \fBfio\fR mailing list <fio@vger.kernel.org>.
Aaron Carrolld1429b52007-09-18 08:10:57 +02001271See \fBREADME\fR.
Aaron Carrolld60e92d2007-09-17 10:32:59 +02001272.SH "SEE ALSO"
Aaron Carrolld1429b52007-09-18 08:10:57 +02001273For further documentation see \fBHOWTO\fR and \fBREADME\fR.
1274.br
1275Sample jobfiles are available in the \fBexamples\fR directory.
Aaron Carrolld60e92d2007-09-17 10:32:59 +02001276