| commit | 960e848f0d32399824d61dce2ff5736bb596ed44 | [log] [tgz] |
|---|---|---|
| author | Berker Peksag <berker.peksag@gmail.com> | Thu Oct 08 12:27:06 2015 +0300 |
| committer | Berker Peksag <berker.peksag@gmail.com> | Thu Oct 08 12:27:06 2015 +0300 |
| tree | 7274633785f0e6b8d0ce64375bdca6351b0d32db | |
| parent | 2137dc15737e286e576d69ad10124973c3f9ba1e [diff] |
Issue #16099: RobotFileParser now supports Crawl-delay and Request-rate extensions. Patch by Nikolay Bogoychev.