Bot Directory
Identify crawlers, AI agents, search bots, monitoring services, automation tools, and potentially unwanted traffic by name, operator, purpose, and user-agent.
- User-agent and robots.txt data
- Operator, family, type, and purpose
- Verification notes and source links