scraper

Klaviyo/1.0 Neutral

klaviyo

The integration with Klaviyo will automatically capture information about who visits your site and views your products including details on what they viewed so you can send super personal.

Operator and family: Klaviyo Type: Scraper Purpose: Scraping

Verified robots.txt: No Block: Depends Verified Bot

LoomlyBot Neutral

Loomly Bot

LoomlyBot is used to extract metadata from web pages in order to show a social media post preview within Loomly so that clients can see what their social media posts will look like when.

Operator and family: Loomly Type: Scraper Purpose: Scraping

Verified robots.txt: No Block: Depends Verified Bot

Mail.RU_Bot Neutral

MailRUBot

The mail.ru bot is a mail fetcher on behalf of the Mail.ru email service.

Operator and family: Mail Russia Type: Scraper Purpose: Scraping

Verified robots.txt: Yes Block: Depends Verified Bot

MarketGoo Neutral

marketgoo

marketgoo is a crawler used for scraping, SEO analysis, or data collection from marketgoo.

Operator and family: marketgoo Type: Scraper Purpose: Scraping

Verified robots.txt: Yes Block: Depends Verified Bot

MauiBot Neutral

MauiBot

MauiBot web-scraper identifier observed in user-agent databases and server logs.

Family: MauiBot Type: Scraper Purpose: Web Scraping

Not verified robots.txt: Unknown Block: Depends Observed

MgidBot Neutral

MgidBot

MgidBot is used for detecting context categories of the content for advertising recommendations.

Operator and family: MGID Type: Scraper Purpose: Scraping

Verified robots.txt: Yes Block: Depends Verified Bot

MicrosoftPreview Neutral

Microsoft Preview

MicrosoftPreview generates page snapshots for Microsoft products.

Operator and family: Microsoft Type: Scraper Purpose: Scraping

Verified robots.txt: No Block: Depends Verified Bot

MissinglettrBot Neutral

Missinglettr Bot

Missinglettr will crawl specific blog posts on customers' sites to help turn them into social media campaigns.

Operator and family: Missinglettr Type: Scraper Purpose: Scraping

Verified robots.txt: No Block: Depends Verified Bot

MRGbot Safe

MRGbot

MRGbot is a crawler used for scraping, SEO analysis, or data collection from MRG Web Services srl.

Operator and family: MRG Web Services srl Type: Scraper Purpose: Scraping

Verified robots.txt: Yes Block: Depends Verified Bot

scraping@nytimes.com Neutral

New York Times Newsgathering

New York Times Newsgathering is a crawler used for scraping, SEO analysis, or data collection from The New York Times.

Operator and family: The New York Times Type: Scraper Purpose: Scraping

Verified robots.txt: No Block: Depends Verified Bot

Nostra Neutral

Nostra

Nostra accelerates site speed for managed web platforms.

Operator and family: Nostra Type: Scraper Purpose: Scraping

Verified robots.txt: No Block: Depends Verified Bot

omgili Neutral

Omgili Bot

Omgili/Webz.io crawler used to index forums, comments, discussion boards, and other public web data.

Operator: Webz.io / Omgili Family: Omgili Type: Scraper Purpose: Web Data Indexing

Verified robots.txt: Yes Block: Depends Verified Bot

online-webceo-bot Neutral

Online Webceo Bot

A bot associated with WebCEO, a company that provides SEO tools and services.

Operator and family: Online Webceo Type: Scraper Purpose: Scraping

Verified robots.txt: No Block: Depends Verified Bot

OpenGraph.io/ Neutral

Opengraph Bot

Our API is used by mostly consumer facing products to preview links when sharing them on their platforms. For example, how when a link is shared on Facebook or Slack, those platforms.

Operator and family: Opengraph Type: Scraper Purpose: Scraping

Verified robots.txt: No Block: Depends Verified Bot

OdklBot Neutral

Ozon Web Grabber

A component that serves to load previews for external and internal links. For external links, whenever possible, information from the open graph tags specified on the page (title, descr.

Operator and family: Ozon Type: Scraper Purpose: Scraping

Verified robots.txt: No Block: Depends Verified Bot

Prerender Neutral

prerender

prerender is a crawler used for scraping, SEO analysis, or data collection from Prerender, LLC.

Operator: Prerender, LLC Family: LLC. Type: Scraper Purpose: Scraping

Verified robots.txt: No Block: Depends Verified Bot

rakutenusabot-image Neutral

Rakuten Image extraction bot

Rakuten uses this bot to crawl product images so that we can display cashback deals for our merchants.

Operator and family: Rakuten Type: Scraper Purpose: Scraping

Verified robots.txt: No Block: Depends Verified Bot

Reelevant Neutral

Reelevant

Reelavant allows customers to dynamically update content inside their emails. Needs to fetch images of their product at runtime and send them back to the users.

Operator and family: Reelevant Type: Scraper Purpose: Scraping

Verified robots.txt: No Block: Depends Verified Bot

SFDC-Callout Neutral

Salesforce

Salesforce is a crawler used for scraping, SEO analysis, or data collection from Salseforce.

Operator and family: Salseforce Type: Scraper Purpose: Scraping

Verified robots.txt: No Block: Depends Verified Bot

page-preview-tool Neutral

SalesViewerBot

Since we offer sales and marketing information we need to enrich the company information. To provide crucial company information inside our service we need to provide a preview of visitor.

Operator and family: SalesViewer GmbH Type: Scraper Purpose: Scraping

Verified robots.txt: No Block: Depends Verified Bot

seo-audit-check-bot Neutral

SEO Audit Check Bot

SEO audit check bot is likely an automated tool within the WebCEO platform that performs comprehensive SEO audits on websites.

Operator and family: SEO Audit Check Type: Scraper Purpose: Scraping

Verified robots.txt: No Block: Depends Verified Bot

serpstatbot Neutral

SerpstatBot

SerpstatBot is the Serpstat bot collects data for Serpstat's Backlink Analysis tool.

Operator and family: Serpstat Type: Scraper Purpose: Scraping

Verified robots.txt: No Block: Depends Verified Bot

Shortwave Image Fetcher Neutral

Shortwave Image Fetcher

Shortwave Image Fetcher is a crawler used for scraping, SEO analysis, or data collection from Shortwave Communications Inc..

Operator and family: Shortwave Communications Inc. Type: Scraper Purpose: Scraping

Verified robots.txt: No Block: Depends Verified Bot

Skroutz ImageBot Neutral

Skroutz ImageBot

Skroutz ImageBot to fetch the individual product images.

Operator and family: Skroutz S.A. Type: Scraper Purpose: Scraping

Verified robots.txt: No Block: Depends Verified Bot