Overview
MetaJobBot searches public employer sites for job vacancies to include in METAJob results.
METAJob documents robots.txt and noindex controls, crawl pacing, and operator IP addresses.
Identification: MetaJobBot. Robots.txt token: metajobbot.
Operator documentation: https://www.metajob.de/the/crawler.
Identity
- User-Agent Pattern
-
MetaJobBot - Robots Token
- metajobbot
- Identity Type
- Officially documented
- Evidence Method
- Verify the documented request signature or provider IP/network where available; a matching User-Agent string alone is not proof of origin.
Classification
- Type
- Search
- Kind
- Job search crawler
- Family
- METAJob
- Purpose
- Job listing indexing
Behavior and handling
- Common Use
- finding and indexing job advertisements
- Detection Notes
- Match HTTP User-Agent token MetaJobBot; The operator describes this as a focused jobs crawler rather than a general search spider.
- Respects robots.txt
- Yes
- Spoofing Risk
- The MetaJobBot token can be spoofed in an HTTP User-Agent. Do not trust the token without independent corroboration.
- Risk
- Neutral
- Recommended Handling
- Depends
Rules and controls
- Robots.txt Snippet
-
User-agent: metajobbot Disallow: /
Relationships
- Operator
- METAJob Checked 2026-10-22
Relationships without an Evidence link are normalized from the canonical directory record. They should not be interpreted as independent proof of physical presence or request origin.