Add low water mark for queuing depth

Current fio will attempt to keep the queue full at all times,
but sometimes that's not what you want. Add iodepth_low to
indicate a low water mark for queuing depth, so that when we
see a FIO_Q_BUSY or run out of free requests, let the queue
drain down to the iodepth_low setting before building it up
again.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
diff --git a/fio.c b/fio.c
index e4fec4f..1c2748e 100644
--- a/fio.c
+++ b/fio.c
@@ -287,9 +287,13 @@
 		 * completed io_u's first.
 		 */
 		min_events = 0;
-		if (queue_full(td) || ret == FIO_Q_BUSY)
+		if (queue_full(td) || ret == FIO_Q_BUSY) {
 			min_events = 1;
 
+			if (td->cur_depth > td->iodepth_low)
+				min_events = td->cur_depth - td->iodepth_low;
+		}
+
 		/*
 		 * Reap required number of io units, if any, and do the
 		 * verification on them through the callback handler
@@ -411,9 +415,13 @@
 		 */
 		if (ret == FIO_Q_QUEUED || ret == FIO_Q_BUSY) {
 			min_evts = 0;
-			if (queue_full(td) || ret == FIO_Q_BUSY)
+			if (queue_full(td) || ret == FIO_Q_BUSY) {
 				min_evts = 1;
 
+				if (td->cur_depth > td->iodepth_low)
+					min_evts = td->cur_depth - td->iodepth_low;
+			}
+
 			fio_gettime(&comp_time, NULL);
 			bytes_done = io_u_queued_complete(td, min_evts, NULL);
 			if (bytes_done < 0)