#1778443 robotparser fixes from Aristotelis Mikropoulos
2 files changed