phy-sun4i-usb: Fix irq free conditions to match request conditions

commit 5cf700ac9d50 ("phy: phy-sun4i-usb: Fix optional gpios failing
probe")
changed the condition under which irqs are requested, but omitted matching
changes to sun4i_usb_phy_remove(). This commit fixes this.

Fixes: 5cf700ac9d50 ("phy: phy-sun4i-usb: Fix optional gpios failing probe")
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
diff --git a/drivers/phy/phy-sun4i-usb.c b/drivers/phy/phy-sun4i-usb.c
index 7390294..de3101f 100644
--- a/drivers/phy/phy-sun4i-usb.c
+++ b/drivers/phy/phy-sun4i-usb.c
@@ -514,9 +514,9 @@
 
 	if (data->vbus_power_nb_registered)
 		power_supply_unreg_notifier(&data->vbus_power_nb);
-	if (data->id_det_irq >= 0)
+	if (data->id_det_irq > 0)
 		devm_free_irq(dev, data->id_det_irq, data);
-	if (data->vbus_det_irq >= 0)
+	if (data->vbus_det_irq > 0)
 		devm_free_irq(dev, data->vbus_det_irq, data);
 
 	cancel_delayed_work_sync(&data->detect);