Botcrawl is an independently operated cybersecurity publication and public directory project founded in 2011.
Botcrawl maintains structured public information about web bots, internet networks, and physical data centers, and publishes reporting on cybersecurity incidents, online threats, artificial intelligence, and internet infrastructure.
The website is free to use. Its directories and reporting are intended to make technical information easier to locate and interpret without overstating what the available evidence proves.
Documents crawlers, AI agents, scrapers, scanners, monitoring services, preview bots, user-triggered agents, and other automated systems that interact with websites.
Records may include documented names, User-Agent patterns, robots.txt tokens, operators, classifications, verification methods, behavior, and handling information.
Documents internet providers, autonomous systems, network operators, and representative IP ranges.
Network records provide technical context for addresses that may appear in server logs, analytics, firewall events, abuse reports, and technical investigations.
Documents physical data centers and facility projects, including operational sites, proposed developments, facilities under construction, and historical locations when sufficient information is available.
Records may include operators, addresses, coordinates, development status, facility type, location precision, and other available infrastructure details.
Reports on data breaches, malware, scams, ransomware, vulnerabilities, artificial intelligence, internet infrastructure, and other security issues affecting people and organizations.
Developing stories may involve incomplete, disputed, or changing information. Coverage is updated when additional credible information materially changes what is known.