Neevabot is the web crawler for the search engine neeva.com.
Requests from Neevabot set the User-Agent to:
Mozilla/5.0 (compatible; Neevabot/1.0; +https://neeva.com/neevabot) and originate from IPs where reverse DNS shows it in the *.neevabot.com domain. For example:
$ host 22.214.171.124 126.96.36.199.in-addr.arpa domain name pointer 100-26-127-17.neevabot.com
Standard directives in robots.txt that target Neevabot are respected. For example, the following will allow Neevabot to crawl all pages, except those under /private/:
User-Agent: Neevabot Allow: / Disallow: /private/
Neevabot also supports the crawl-delay directive in robots.txt files. It interprets the value as the minimum duration, in seconds, between the start of consecutive requests. For example, assume you have specified the following in your robots.txt file:
User-Agent: Neevabot Crawl-delay: 5 Neevabot will split each day into 5 second intervals and issue at most one request to your domain inside each interval.
If there is no rule targeting Neevabot, but there is a rule targeting Googlebot, then Neevabot will follow the Googlebot directives. For example, Neevabot will fetch all pages, except those under /private/ with the following robots.txt: