HamletBot

HamletBot is the web crawler operated by Hamlet. It identifies itself with this User-Agent:

HamletBot/1.0 (+https://api.myhamlet.com/bot)

What it fetches, and why

HamletBot fetches two things from government websites: the pages that list a public body's elected and appointed officials, and the official headshots of those officials. Hamlet uses them to identify who is speaking in the transcript of a public meeting, so that a reader can tell one voice from another.

It fetches on demand rather than on a schedule, and at low volume: a handful of pages for a government body, when someone is working on that body's roster.

How it behaves

• It reads /robots.txt before every request and obeys it, matching the User-agent: HamletBot group when there is one and falling back to User-agent: * when there is not. A disallowed path is never requested.
• It waits out Crawl-delay, and skips a host entirely rather than fetching faster than the delay asks.
• It requests public pages only, and never submits a form, signs in, or follows a link that changes anything.

How to verify a request really came from us

Every request HamletBot sends carries a RFC 9421 HTTP message signature under the Web Bot Auth profile: a Signature-Agent header naming https://api.myhamlet.com, plus Signature-Input and Signature. Verify it against the Ed25519 public key we publish here:

https://api.myhamlet.com/.well-known/http-message-signatures-directory

A request carrying our User-Agent but no valid signature against that key did not come from us.

If you would rather we did not

Disallowing HamletBot in your robots.txt stops it, and it takes effect on our next request to your site. If you would rather write to a person, or you think HamletBot is behaving badly, mail abuse@myhamlet.com and a human will read it.