Merge "Fix a bug of not deleting previously added config files on disk"