Bot Directory Record
Colly
Review firstColly is a crawler framework used for Colly is used for Automated website access, content retrieval, or integration activity associated with the listed operator; it appears in server logs as `colly`.
- Operator
- Colly
- Family
- Colly
- Type
- Crawler Framework
- Source type
- Observed
- Last checked
- 2026-06-22
User-Agent Signal
Collycolly
User-agent strings are identification signals, not proof of identity. Confirm important allow, block, or rate-limit decisions with logs, DNS or IP evidence, request behavior, or operator documentation when available.
Robots.txt Snippet
Click snippet to copyUser-agent: colly
Disallow: /Click the snippet to copy it, or highlight the text manually.
Handling Guidance
MonitorUse this record as bot intelligence, then verify the request source and behavior before allowing, blocking, or rate limiting.
Colly is used for Automated website access, content retrieval, or integration activity associated with the listed operator.
Intelligence Details
Public record fields- Operator
- Colly
- Family
- Colly
- Type
- Crawler Framework
- Purpose
- Automated Web Crawling
- Identity type
- Documented
- Confidence
- Low
- Last verified
- 2026-06-22
- Last checked
- 2026-06-22
- Source type
- Observed
- Verification
- Verify Colly by matching `colly` to observed traffic patterns and user-agent evidence, then checking reverse DNS, IP ownership, request behavior, and crawl consistency.
- Spoofing risk
- Colly has high spoofing risk because the pattern is low-confidence or observation-based; do not trust the user-agent by itself.
Notes
- Colly is a crawler framework used for Colly is used for Automated website access, content retrieval, or integration activity associated with the listed operator.
- Its primary user-agent pattern is
colly. - Colly is verified with Low confidence. The identity type is Documented, and the evidence basis is observed traffic patterns and user-agent evidence.
- Colly does not have confirmed robots.txt behavior in the available public evidence.
- Colly should be monitored first, then rate-limited or blocked if the crawl rate, paths, or behavior are unwanted.
Evidence and Source
- Verify Colly by matching `colly` to observed traffic patterns and user-agent evidence, then checking reverse DNS, IP ownership, request behavior, and crawl consistency.
- Colly traffic is primarily detected by the `colly` user-agent pattern. Compare source IPs, reverse DNS, request paths, and crawl cadence before trusting the traffic.
- Colly is used for Automated website access, content retrieval, or integration activity associated with the listed operator.
- Colly has high spoofing risk because the pattern is low-confidence or observation-based; do not trust the user-agent by itself.
Use in Botcrawl Edge
Botcrawl EdgeMatch this signal against live requests, related datacenter traffic, and rule actions in Edge.