OratureBot

OratureBot is Orature's podcast crawler. It fetches podcast feeds, their enclosures (episode audio), and chapter data, and it fetches robots.txt before fetching anything else from an origin.

What it fetches

• Podcast feeds (RSS/Atom)
• Episode enclosures (the audio files a feed links to)
• Chapter data, where a feed or file exposes it
• robots.txt, read before anything else on an origin

robots.txt

OratureBot obeys robots.txt. An explicit disallow is never bypassed. To write a rule for it specifically, use its token:

User-agent: OratureBot
Disallow: /private/

When a host does not publish a robots.txt file, we assume crawling is allowed. Temporary failures are retried, and normal request pacing still applies.

How it identifies itself

Every request OratureBot makes carries an HTTP Message Signature (RFC 9421) over its Signature-Agent header, so an origin (or the CDN in front of it) can verify the request really came from OratureBot and not from anyone claiming its user agent. The signing keys are published at /.well-known/http-message-signatures-directory, the same directory Cloudflare's verified-bot program checks against.

Rate-limiting or blocking it

A robots.txt Disallow or Crawl-delay group scoped to User-agent: OratureBot is the preferred way to slow or stop it. It can also be matched by its verified identity in any system, such as a CDN's bot-management product, that checks a Web Bot Auth signature rather than only the user agent string.

Contact

Questions about this crawler: [email protected]