IPoIB: Move ipoib_ib_dev_flush() to ipoib workqueue

Move ipoib_ib_dev_flush() to ipoib's workqueue.  This keeps it ordered
with respect to other work scheduled by the ipoib driver.  This fixes
problems with races, for example:
 - ipoib_ib_dev_flush() has started running because of an IB event
 - user does ifconfig ib0 down
 - ipoib_mcast_stop_thread() gets called twice and waits for the same
   completion twice

Signed-off-by: Jack Morgenstein <jackm@mellanox.co.il>
Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c
index c3b5f79..1633aad 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_main.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c
@@ -133,7 +133,13 @@
 
 	netif_stop_queue(dev);
 
-	ipoib_ib_dev_down(dev);
+	/*
+	 * Now flush workqueue to make sure a scheduled task doesn't
+	 * bring our internal state back up.
+	 */
+	flush_workqueue(ipoib_workqueue);
+
+	ipoib_ib_dev_down(dev, 1);
 	ipoib_ib_dev_stop(dev);
 
 	if (!test_bit(IPOIB_FLAG_SUBINTERFACE, &priv->flags)) {