http://3oamv2de6ooacy72hv3ulym3i3vvlprlkqas7hhzo4pu5nyxnji37zad.onion/projects/micro-search
The default delay for this bot is 2 seconds per URL, and it is configurable by anyone hosting it. For example, if you specifiy a crawl delay as 5 seconds like below, and the bot is configured to delay the crawl to 10 seconds, the bot will run every 10 seconds instead of 5. User-agent: IW-microsearch
Crawl-delay: 5 How to run this project?