Prompt or recipeOctober 10, 2026

Verify a crawler with forward-confirmed reverse DNS

Kind: Code snippet / recipe
Language / model: TypeScript (Node)
Gotchas: Match domains on label boundaries, or notgoogle.com passes as google.com.
A cloud-VM hostname is inconclusive, not proof of forgery: some AI crawlers run on cloud IPs and publish IP ranges instead.
Add a timeout and a cache; DNS lookups must not stall a request.

Short description

Check that a request claiming to be Googlebot (or Bingbot, Applebot) really comes from that company.

When to use it

Before trusting a bot's user agent in analytics, alerts or rate limits.

The prompt / code

import { promises as dns } from "node:dns";

// PTR lookup, then resolve the hostname back and require the original IP.
export async function forwardConfirmedHost(ip: string): Promise<string | null> {
  const hostnames = await dns.reverse(ip).catch(() => [] as string[]);
  for (const hostname of hostnames.slice(0, 3)) {
    const [v4, v6] = await Promise.all([
      dns.resolve4(hostname).catch(() => [] as string[]),
      dns.resolve6(hostname).catch(() => [] as string[]),
    ]);
    if ([...v4, ...v6].includes(ip)) return hostname;
  }
  return null;
}

// Then match on label boundaries: host === domain || host.endsWith("." + domain)
// e.g. googlebot.com, google.com, search.msn.com, applebot.apple.com