ASoC: rmmod fixes for audio drivers

For remove module(rmmod), update drivers
to cleanup resources and allow insert module
again without any issue.

Change-Id: Iddc6e5e11d986359afd3100bf3c5eab70cb1c1eb
Signed-off-by: Laxminath Kasam <lkasam@codeaurora.org>
diff --git a/dsp/audio_slimslave.c b/dsp/audio_slimslave.c
index 25866d1..55982f3 100644
--- a/dsp/audio_slimslave.c
+++ b/dsp/audio_slimslave.c
@@ -1,4 +1,4 @@
-/* Copyright (c) 2013-2014, 2017 The Linux Foundation. All rights reserved.
+/* Copyright (c) 2013-2014, 2017-2018 The Linux Foundation. All rights reserved.
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License version 2 and
@@ -167,7 +167,7 @@
 
 void audio_slimslave_exit(void)
 {
-
+	slim_driver_unregister(&audio_slimslave_driver);
 }
 
 /* Module information */