User-agent: * User-agent: Baiduspider Disallow: / User-agent: DataForSeoBot Disallow: / User-agent: PetalBot Disallow: / User-agent: Applebot Disallow: / User-agent: AhrefsBot Disallow: / User-agent: DataForSeoBot Disallow: / User-agent: bingbot Disallow: / User-agent: bingbot/2.0 Disallow: / User-agent: Amazonbot Disallow: / User-agent: SemrushBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: dotbot Disallow: / User-agent: CheckMarkNetwork/1.0 (+https://www.checkmarknetwork.com/spider.html) Disallow: / User-agent: DotBot/1.2 Disallow: /