Remember to start pending client if enumerate takes too long

Fixes: 121185474

Test: No obvious changes in behavior
Test: Fixes the issue based on bug logs + code path inspection
Change-Id: Ia8ad6ad57a76eeaff783097b10b93436eae68e94
1 file changed