This command is a part of the crawling politeness policy and prevents overloading of the remote servers.
When crawling, indexer counts the number of requests to every unique IP address it has downloaded documents from during the current crawling session, as well as remembers the timestamp of the first request made to every IP address.
Before downloading a document from a remote host, indexer calculates the current request density by dividing the number of requests to the current IP address to the time left from the first request to this IP address. If the current request density is higher than IPRequestPerMinLimit, indexer puts the current document back to the crawling queue to visit this document later, according to Period value associated with this document.
The default value is 10.