Add low water mark for queuing depth

Current fio will attempt to keep the queue full at all times,
but sometimes that's not what you want. Add iodepth_low to
indicate a low water mark for queuing depth, so that when we
see a FIO_Q_BUSY or run out of free requests, let the queue
drain down to the iodepth_low setting before building it up
again.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
diff --git a/HOWTO b/HOWTO
index ac15fc8..ea98810 100644
--- a/HOWTO
+++ b/HOWTO
@@ -294,6 +294,13 @@
 		job, can be overridden with a larger value for higher
 		concurrency.
 
+iodepth_low=int	The low water mark indicating when to start filling
+		the queue again. Defaults to the same as iodepth, meaning
+		that fio will attempt to keep the queue full at all times.
+		If iodepth is set to eg 16 and iodepth_low is set to 4, then
+		after fio has filled the queue of 16 requests, it will let
+		the depth drain down to 4 before starting to fill it again.
+
 direct=bool	If value is true, use non-buffered io. This is usually
 		O_DIRECT.