Add low water mark for queuing depth

Current fio will attempt to keep the queue full at all times,
but sometimes that's not what you want. Add iodepth_low to
indicate a low water mark for queuing depth, so that when we
see a FIO_Q_BUSY or run out of free requests, let the queue
drain down to the iodepth_low setting before building it up
again.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
diff --git a/init.c b/init.c
index 6dc221e..5630c9f 100644
--- a/init.c
+++ b/init.c
@@ -91,6 +91,12 @@
 		.def	= "1",
 	},
 	{
+		.name	= "iodepth_low",
+		.type	= FIO_OPT_INT,
+		.off1	= td_var_offset(iodepth_low),
+		.help	= "Low water mark for queuing depth",
+	},
+	{
 		.name	= "size",
 		.type	= FIO_OPT_STR_VAL,
 		.off1	= td_var_offset(total_file_size),
@@ -644,6 +650,14 @@
 	 */
 	if (td->thinktime_spin > td->thinktime)
 		td->thinktime_spin = td->thinktime;
+
+	/*
+	 * The low water mark cannot be bigger than the iodepth
+	 */
+	if (td->iodepth_low > td->iodepth || !td->iodepth_low)
+		td->iodepth_low = td->iodepth;
+
+	printf("io depth %d/%d\n", td->iodepth_low, td->iodepth);
 }
 
 /*