diff options
| author | Andreas Baumann <abaumann@yahoo.com> | 2014-07-23 16:44:07 +0200 |
|---|---|---|
| committer | Andreas Baumann <abaumann@yahoo.com> | 2014-07-23 16:44:07 +0200 |
| commit | a57788acee59705418b96525410b84fbee2f405a (patch) | |
| tree | 660a828a29f0f769638ce18c1b45c62bd602e012 /makefiles | |
| parent | ce77513807f47bc7af59c8320932d3348aeb99ea (diff) | |
| download | crawler-a57788acee59705418b96525410b84fbee2f405a.tar.gz crawler-a57788acee59705418b96525410b84fbee2f405a.tar.bz2 | |
added parsing of Sitemap in robots.txt
Diffstat (limited to 'makefiles')
0 files changed, 0 insertions, 0 deletions