blob: f1ee9aa7676ebf08e0060d94fd34f7acd1746698 [file] [log] [blame]
Jens Axboeebac4652005-12-08 15:25:21 +01001fio
2---
3
Jens Axboe79809112006-06-09 10:14:54 +02004fio is a tool that will spawn a number of threads or processes doing a
5particular type of io action as specified by the user. fio takes a
6number of global parameters, each inherited by the thread unless
7otherwise parameters given to them overriding that setting is given.
8The typical use of fio is to write a job file matching the io load
9one wants to simulate.
Jens Axboeebac4652005-12-08 15:25:21 +010010
Jens Axboe2b02b542005-12-08 15:29:14 +010011
12Source
13------
14
15fio resides in a git repo, the canonical place is:
16
Jens Axboe6b3eccb2007-07-18 13:51:19 +020017git://git.kernel.dk/fio.git
Jens Axboe97f049c2007-04-23 18:54:55 +020018
19The http protocol also works, path is the same.
Jens Axboe2b02b542005-12-08 15:29:14 +010020
Jens Axboe79809112006-06-09 10:14:54 +020021Snapshots are frequently generated and they include the git meta data as
22well. You can download them here:
Jens Axboe2b02b542005-12-08 15:29:14 +010023
24http://brick.kernel.dk/snaps/
25
Jens Axboe1053a102006-06-06 09:23:13 +020026
Sebastian Kayserd85b1ad2010-11-30 20:49:18 +010027Binary packages
28---------------
29
30Debian:
31Starting with Debian "Squeeze", fio packages are part of the official
32Debian repository. http://packages.debian.org/search?keywords=fio
33
34Ubuntu:
35Starting with Ubuntu 10.04 LTS (aka "Lucid Lynx"), fio packages are part
36of the Ubuntu "universe" repository.
37http://packages.ubuntu.com/search?keywords=fio
38
39SUSE:
40Pascal Bleser <guru@unixtech.be> has fio RPMs in his repository for SUSE
41variants, you can find them here:
Jens Axboe1053a102006-06-06 09:23:13 +020042http://linux01.gwdg.de/~pbleser/rpm-navigation.php?cat=System/fio
43
Sebastian Kayserd85b1ad2010-11-30 20:49:18 +010044Red Hat, CentOS & Co:
Jens Axboea68594c2007-03-05 12:37:54 +010045Dag Wieërs has RPMs for Red Hat related distros, find them here:
Jens Axboea68594c2007-03-05 12:37:54 +010046http://dag.wieers.com/rpm/packages/fio/
47
Sebastian Kayserd85b1ad2010-11-30 20:49:18 +010048Mandriva:
Jens Axboe244e1702007-07-19 14:21:08 +020049Mandriva has integrated fio into their package repository, so installing
50on that distro should be as easy as typing 'urpmi fio'.
51
Sebastian Kayserd85b1ad2010-11-30 20:49:18 +010052Solaris:
53Packages for Solaris are available from OpenCSW. Install their pkgutil
54tool (http://www.opencsw.org/get-it/pkgutil/) and then install fio via
55'pkgutil -i fio'.
56
Jens Axboe2b02b542005-12-08 15:29:14 +010057
Jens Axboe726f6ff2007-01-03 21:02:41 +010058Mailing list
59------------
60
61There's a mailing list associated with fio. It's meant for general
Jens Axboe2e8552b2008-10-01 09:04:28 +020062discussion, bug reporting, questions, and development - basically anything
63that has to do with fio. An automated mail detailing recent commits is
64automatically sent to the list at most daily. The list address is
65fio@vger.kernel.org, subscribe by sending an email to
66majordomo@vger.kernel.org with
67
68subscribe fio
69
Erik Inge Bolsø4f5d1522010-04-14 10:13:57 +020070in the body of the email. Archives can be found here:
71
72http://www.spinics.net/lists/fio/
73
74and archives for the old list can be found here:
Jens Axboe2e8552b2008-10-01 09:04:28 +020075
76http://maillist.kernel.dk/fio-devel/
Jens Axboe726f6ff2007-01-03 21:02:41 +010077
78
Jens Axboebbfd6b02006-06-07 19:42:54 +020079Building
80--------
81
82Just type 'make' and 'make install'. If on FreeBSD, for now you have to
Jens Axboe5f421002008-06-02 16:15:59 +020083specify the FreeBSD Makefile with -f and use gmake (not make), eg:
Jens Axboebbfd6b02006-06-07 19:42:54 +020084
Jens Axboe5f421002008-06-02 16:15:59 +020085$ gmake -f Makefile.Freebsd && gmake -f Makefile.FreeBSD install
Jens Axboebbfd6b02006-06-07 19:42:54 +020086
Cigy Cyriacbf2e8212010-08-10 19:51:11 -040087Same goes for AIX:
88
89$ gmake -f Makefile.aix && gmake -f Makefile.aix install
90
Jens Axboeedffcb92006-06-08 13:40:18 +020091Likewise with OpenSolaris, use the Makefile.solaris to compile there.
Jens Axboe5f421002008-06-02 16:15:59 +020092The OpenSolaris make should work fine. This might change in the
93future if I opt for an autoconf type setup.
Jens Axboebbfd6b02006-06-07 19:42:54 +020094
Jens Axboe6de43c12008-04-11 09:16:59 +020095If your compile fails with an error like this:
96
97 CC gettime.o
98In file included from fio.h:23,
99 from gettime.c:8:
100os/os.h:15:20: error: libaio.h: No such file or directory
101In file included from gettime.c:8:
102fio.h:119: error: field 'iocb' has incomplete type
103make: *** [gettime.o] Error 1
104
105Check that you have the libaio development package installed. On RPM
106based distros, it's typically called libaio-devel.
107
Jens Axboebbfd6b02006-06-07 19:42:54 +0200108
Jens Axboe972cfd22006-06-09 11:08:56 +0200109Command line
110------------
Jens Axboeebac4652005-12-08 15:25:21 +0100111
112$ fio
Jens Axboeee56ad52008-02-01 10:30:20 +0100113 --debug Enable some debugging options (see below)
Jens Axboeb4692822006-10-27 13:43:22 +0200114 --output Write output to file
Jens Axboe062c6022008-02-18 20:15:03 +0100115 --timeout Runtime in seconds
Jens Axboeb4692822006-10-27 13:43:22 +0200116 --latency-log Generate per-job latency logs
117 --bandwidth-log Generate per-job bandwidth logs
118 --minimal Minimal (terse) output
119 --version Print version info and exit
Jens Axboefd28ca42007-01-09 21:20:13 +0100120 --help Print this page
121 --cmdhelp=cmd Print command help, "all" for all of them
Jens Axboecca73aa2007-04-04 11:09:19 +0200122 --showcmd Turn a job file into command line options
Jens Axboe062c6022008-02-18 20:15:03 +0100123 --readonly Turn on safety read-only checks, preventing writes
Aaron Carrolle592a062007-09-14 09:49:41 +0200124 --eta=when When ETA estimate should be printed
125 May be "always", "never" or "auto"
Jens Axboe01f06b62008-02-18 20:53:47 +0100126 --section=name Only run specified section in job file
Jens Axboe2b386d22008-03-26 10:32:57 +0100127 --alloc-size=kb Set smalloc pool to this size in kb (def 1024)
Aaron Carrolle592a062007-09-14 09:49:41 +0200128
Jens Axboeebac4652005-12-08 15:25:21 +0100129
Jens Axboeb4692822006-10-27 13:43:22 +0200130Any parameters following the options will be assumed to be job files,
131unless they match a job file parameter. You can add as many as you want,
132each job file will be regarded as a separate group and fio will stonewall
133its execution.
Jens Axboe972cfd22006-06-09 11:08:56 +0200134
Jens Axboe724e4432007-09-11 20:02:05 +0200135The --readonly switch is an extra safety guard to prevent accidentically
136turning on a write setting when that is not desired. Fio will only write
137if rw=write/randwrite/rw/randrw is given, but this extra safety net can
138be used as an extra precaution. It will also enable a write check in the
139io engine core to prevent an accidental write due to a fio bug.
140
Jens Axboeee56ad52008-02-01 10:30:20 +0100141The debug switch allows adding options that trigger certain logging
142options in fio. Currently the options are:
143
144 process Dump info related to processes
145 file Dump info related to file actions
146 io Dump info related to IO queuing
147 mem Dump info related to memory allocations
Jens Axboebd6f78b2008-02-01 20:27:52 +0100148 blktrace Dump info related to blktrace setup
149 verify Dump info related to IO verification
150 all Enable all debug options
Jens Axboe811a0d02008-02-19 20:11:41 +0100151 random Dump info related to random offset generation
Jens Axboea3d741f2008-02-27 18:32:33 +0100152 parse Dump info related to option matching and parsing
Jens Axboecd991b92008-03-07 13:19:35 +0100153 diskutil Dump info related to disk utilization updates
Jens Axboe5e1d3062008-05-23 11:55:53 +0200154 job:x Dump info only related to job number x
Jens Axboe29adda32009-01-05 19:06:39 +0100155 mutex Dump info only related to mutex up/down ops
Jens Axboec223da82010-03-24 13:23:53 +0100156 profile Dump info related to profile extensions
157 time Dump info related to internal time keeping
Jens Axboebd6f78b2008-02-01 20:27:52 +0100158 ? or help Show available debug options.
Jens Axboeee56ad52008-02-01 10:30:20 +0100159
160You can specify as many as you want, eg --debug=file,mem will enable
Jens Axboebd6f78b2008-02-01 20:27:52 +0100161file and memory debugging.
Jens Axboeee56ad52008-02-01 10:30:20 +0100162
Jens Axboe01f06b62008-02-18 20:53:47 +0100163The section switch is meant to make it easier to ship a bigger job file
164instead of several smaller ones. Say you define a job file with light,
165moderate, and heavy parts. Then you can ask fio to run the given part
166only by giving it a --section=heavy command line option. The section
167option only applies to job sections, the reserved 'global' section is
168always parsed and taken into account.
169
Jens Axboe2b386d22008-03-26 10:32:57 +0100170Fio has an internal allocator for shared memory called smalloc. It
171allocates shared structures from this pool. The pool defaults to 1024k
Jens Axboe931823c2009-03-05 21:23:18 +0100172in size, and can grow to 128 pools. If running large jobs with randommap
Jens Axboe2b386d22008-03-26 10:32:57 +0100173enabled it can run out of memory, in which case the --alloc-size switch
Jens Axboe931823c2009-03-05 21:23:18 +0100174is handy for starting with a larger pool size. The backing store is
175files in /tmp. Fio cleans up after itself, while it is running you
176may see .fio_smalloc.* files in /tmp.
Jens Axboe2b386d22008-03-26 10:32:57 +0100177
Jens Axboe79809112006-06-09 10:14:54 +0200178
179Job file
180--------
181
Jens Axboe71bfa162006-10-25 11:08:19 +0200182See the HOWTO file for a more detailed description of parameters and what
Jens Axboe4661f3d2007-08-15 09:32:21 +0200183they mean. This file contains the terse version. You can describe big and
184complex setups with the command line, but generally it's a lot easier to
Jens Axboe71bfa162006-10-25 11:08:19 +0200185just write a simple job file to describe the workload. The job file format
Jens Axboe4661f3d2007-08-15 09:32:21 +0200186is in the ini style format, as that is easy to read and write for the user.
Jens Axboe79809112006-06-09 10:14:54 +0200187
188The job file parameters are:
Jens Axboeebac4652005-12-08 15:25:21 +0100189
Jens Axboe01452052006-06-07 10:29:47 +0200190 name=x Use 'x' as the identifier for this job.
Jens Axboe61697c32007-02-05 15:04:46 +0100191 description=x 'x' is a text description of the job.
Jens Axboeebac4652005-12-08 15:25:21 +0100192 directory=x Use 'x' as the top level directory for storing files
Jens Axboeb50b8752006-10-24 19:05:53 +0200193 filename=x Force the use of 'x' as the filename for all files
194 in this thread. If not given, fio will make up
195 a suitable filename based on the thread and file
196 number.
Jens Axboe3d60d1e2006-05-25 06:31:06 +0200197 rw=x 'x' may be: read, randread, write, randwrite,
198 rw (read-write mix), randrw (read-write random mix)
Jens Axboea6ccc7b2006-06-02 10:14:15 +0200199 rwmixcycle=x Base cycle for switching between read and write
200 in msecs.
201 rwmixread=x 'x' percentage of rw mix ios will be reads. If
202 rwmixwrite is also given, the last of the two will
203 be used if they don't add up to 100%.
204 rwmixwrite=x 'x' percentage of rw mix ios will be writes. See
205 rwmixread.
Jens Axboe9ebc27e2006-06-12 10:21:50 +0200206 rand_repeatable=x The sequence of random io blocks can be repeatable
207 across runs, if 'x' is 1.
Jens Axboeebac4652005-12-08 15:25:21 +0100208 size=x Set file size to x bytes (x string can include k/m/g)
209 ioengine=x 'x' may be: aio/libaio/linuxaio for Linux aio,
Jens Axboe78e7b3e2008-06-02 13:28:21 +0200210 posixaio for POSIX aio, solarisaio for Solaris
Bruce Cran03e20d62011-01-02 20:14:54 +0100211 native async IO, windowsaio for Windows native async IO,
212 sync for regular read/write io,
Jens Axboe1d2af022008-02-04 10:59:07 +0100213 psync for regular pread/pwrite io, vsync for regular
214 readv/writev (with queuing emulation) mmap for mmap'ed
215 io, syslet-rw for syslet driven read/write, splice for
Gurudas Paid0c70932008-05-30 13:35:00 +0200216 using splice/vmsplice, sg for direct SG_IO io, net
217 for network io, or cpuio for a cycler burner load. sg
Jens Axboe1d2af022008-02-04 10:59:07 +0100218 only works on Linux on SCSI (or SCSI-like devices, such
219 as usb-storage or sata/libata driven) devices. Fio also
220 has a null io engine, which is mainly used for testing
221 fio itself.
222
Jens Axboeebac4652005-12-08 15:25:21 +0100223 iodepth=x For async io, allow 'x' ios in flight
224 overwrite=x If 'x', layout a write file first.
Jens Axboe53cdc682006-10-18 11:50:58 +0200225 nrfiles=x Spread io load over 'x' number of files per job,
226 if possible.
Jens Axboeebac4652005-12-08 15:25:21 +0100227 prio=x Run io at prio X, 0-7 is the kernel allowed range
228 prioclass=x Run io at prio class X
229 bs=x Use 'x' for thread blocksize. May include k/m postfix.
230 bsrange=x-y Mix thread block sizes randomly between x and y. May
231 also include k/m postfix.
232 direct=x 1 for direct IO, 0 for buffered IO
233 thinktime=x "Think" x usec after each io
Jens Axboeb22989b2009-07-17 22:29:23 +0200234 rate=x Throttle rate to x KB/sec
235 ratemin=x Quit if rate of x KB/sec can't be met
Jens Axboeebac4652005-12-08 15:25:21 +0100236 ratecycle=x ratemin averaged over x msecs
237 cpumask=x Only allow job to run on CPUs defined by mask.
Jens Axboed2e268b2007-06-15 10:33:49 +0200238 cpus_allowed=x Like 'cpumask', but allow text setting of CPU affinity.
Jens Axboe795407c2006-10-24 19:42:44 +0200239 fsync=x If writing with buffered IO, fsync after every
240 'x' blocks have been written.
241 end_fsync=x If 'x', run fsync() after end-of-job.
Jens Axboeebac4652005-12-08 15:25:21 +0100242 startdelay=x Start this thread x seconds after startup
Jens Axboe03b74b32007-01-11 11:04:31 +0100243 runtime=x Terminate x seconds after startup. Can include a
Jens Axboe906c8d72006-06-13 09:37:56 +0200244 normal time suffix if not given in seconds, such as
245 'm' for minutes, 'h' for hours, and 'd' for days.
Jens Axboeebac4652005-12-08 15:25:21 +0100246 offset=x Start io at offset x (x string can include k/m/g)
247 invalidate=x Invalidate page cache for file prior to doing io
Jens Axboe795407c2006-10-24 19:42:44 +0200248 sync=x Use sync writes if x and writing buffered IO.
Jens Axboeebac4652005-12-08 15:25:21 +0100249 mem=x If x == malloc, use malloc for buffers. If x == shm,
Jens Axboe795407c2006-10-24 19:42:44 +0200250 use shared memory for buffers. If x == mmap, use
251 anonymous mmap.
Jens Axboeebac4652005-12-08 15:25:21 +0100252 exitall When one thread quits, terminate the others
253 bwavgtime=x Average bandwidth stats over an x msec window.
254 create_serialize=x If 'x', serialize file creation.
255 create_fsync=x If 'x', run fsync() after file creation.
Jens Axboef6cbb262006-10-18 14:16:42 +0200256 unlink If set, unlink files when done.
Jens Axboeebac4652005-12-08 15:25:21 +0100257 loops=x Run the job 'x' number of times.
258 verify=x If 'x' == md5, use md5 for verifies. If 'x' == crc32,
259 use crc32 for verifies. md5 is 'safer', but crc32 is
260 a lot faster. Only makes sense for writing to a file.
Jens Axboebac39e02008-06-11 20:46:19 +0200261 For other types of checksumming, see HOWTO.
Jens Axboeebac4652005-12-08 15:25:21 +0100262 stonewall Wait for preceeding jobs to end before running.
263 numjobs=x Create 'x' similar entries for this job
264 thread Use pthreads instead of forked jobs
Jens Axboe20dc95c2005-12-09 10:29:35 +0100265 zonesize=x
266 zoneskip=y Zone options must be paired. If given, the job
267 will skip y bytes for every x read/written. This
268 can be used to gauge hard drive speed over the entire
269 platter, without reading everything. Both x/y can
270 include k/m/g suffix.
Jens Axboeaea47d42006-05-26 19:27:29 +0200271 iolog=x Open and read io pattern from file 'x'. The file must
272 contain one io action per line in the following format:
273 rw, offset, length
274 where with rw=0/1 for read/write, and the offset
275 and length entries being in bytes.
Jens Axboe843a7412006-06-01 21:14:21 -0700276 write_iolog=x Write an iolog to file 'x' in the same format as iolog.
277 The iolog options are exclusive, if both given the
278 read iolog will be performed.
Jens Axboeec94ec52006-10-20 10:59:19 +0200279 write_bw_log Write a bandwidth log.
280 write_lat_log Write a latency log.
Jens Axboec04f7ec2006-05-31 10:13:16 +0200281 lockmem=x Lock down x amount of memory on the machine, to
282 simulate a machine with less memory available. x can
283 include k/m/g suffix.
Jens Axboeb6f4d882006-06-02 10:32:51 +0200284 nice=x Run job at given nice value.
Jens Axboe4e0ba8a2006-06-06 09:36:28 +0200285 exec_prerun=x Run 'x' before job io is begun.
286 exec_postrun=x Run 'x' after job io has finished.
Jens Axboeda867742006-06-06 10:39:10 -0700287 ioscheduler=x Use ioscheduler 'x' for this job.
Jens Axboeb990b5c2006-09-14 09:48:22 +0200288 cpuload=x For a CPU io thread, percentage of CPU time to attempt
289 to burn.
Jens Axboeba0fbe12007-03-09 14:34:23 +0100290 cpuchunks=x Split burn cycles into pieces of x usecs.
Jens Axboeebac4652005-12-08 15:25:21 +0100291
Jens Axboe79809112006-06-09 10:14:54 +0200292
Jens Axboe217bc042009-01-07 10:05:14 +0100293
294Platforms
295---------
296
Jens Axboe04924a12010-08-11 11:13:56 -0400297Fio works on (at least) Linux, Solaris, AIX, OSX, NetBSD, and FreeBSD. Some
298features and/or options may only be available on some of the platforms,
299typically because those features only apply to that platform (like the
300solarisaio engine, or the splice engine on Linux).
Jens Axboe217bc042009-01-07 10:05:14 +0100301
302Some features are not available on FreeBSD/Solaris even if they could be
303implemented, I'd be happy to take patches for that. An example of that is
304disk utility statistics and (I think) huge page support, support for that
305does exist in FreeBSD/Solaris.
306
307Fio uses pthread mutexes for signalling and locking and FreeBSD does not
308support process shared pthread mutexes. As a result, only threads are
309supported on FreeBSD. This could be fixed with sysv ipc locking or
310other locking alternatives.
311
312Other *BSD platforms are untested, but fio should work there almost out
313of the box. Since I don't do test runs or even compiles on those platforms,
314your mileage may vary. Sending me patches for other platforms is greatly
315appreciated. There's a lot of value in having the same test/benchmark tool
316available on all platforms.
317
Cigy Cyriacbf2e8212010-08-10 19:51:11 -0400318Note that POSIX aio is not enabled by default on AIX. If you get messages like:
319
320 Symbol resolution failed for /usr/lib/libc.a(posix_aio.o) because:
321 Symbol _posix_kaio_rdwr (number 2) is not exported from dependent module /unix.
322
323you need to enable POSIX aio. Run the following commands as root:
324
325 # lsdev -C -l posix_aio0
326 posix_aio0 Defined Posix Asynchronous I/O
327 # cfgmgr -l posix_aio0
328 # lsdev -C -l posix_aio0
329 posix_aio0 Available Posix Asynchronous I/O
330
331POSIX aio should work now. To make the change permanent:
332
333 # chdev -l posix_aio0 -P -a autoconfig='available'
334 posix_aio0 changed
Jens Axboe217bc042009-01-07 10:05:14 +0100335
336
Jens Axboe79809112006-06-09 10:14:54 +0200337Author
338------
339
Jens Axboeaae22ca2006-09-05 10:46:22 +0200340Fio was written by Jens Axboe <axboe@kernel.dk> to enable flexible testing
Jens Axboe79809112006-06-09 10:14:54 +0200341of the Linux IO subsystem and schedulers. He got tired of writing
342specific test applications to simulate a given workload, and found that
343the existing io benchmark/test tools out there weren't flexible enough
344to do what he wanted.
345
Jens Axboeaae22ca2006-09-05 10:46:22 +0200346Jens Axboe <axboe@kernel.dk> 20060905
Jens Axboe79809112006-06-09 10:14:54 +0200347