Revert "Don't release vk memory allocator until context is deleted."

This reverts commit 11afa63693fa65e7dcf2f2f0c0315316825d60e0.

Reason for revert: breaking perf bots

Original change's description:
> Don't release vk memory allocator until context is deleted.
> 
> Bug: chromium:1081806
> 
> Change-Id: I0a7a92408a3f2119e6b70d8eef6aa4f2d1982d4b
> Reviewed-on: https://skia-review.googlesource.com/c/skia/+/297357
> Reviewed-by: Brian Salomon <bsalomon@google.com>
> Commit-Queue: Greg Daniel <egdaniel@google.com>

TBR=egdaniel@google.com,bsalomon@google.com

Change-Id: I48d200cb72eb30c119359e89c5a9123213e62a5e
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:1081806
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/297457
Reviewed-by: Greg Daniel <egdaniel@google.com>
Commit-Queue: Greg Daniel <egdaniel@google.com>
4 files changed