Gay Book List

About gaybooklistbot

If you found this page from your server logs: hello! Here's exactly what our crawler does.

gaybooklistbot/1.0 is the fetcher for Gay Book List (https://www.gaybooklist.com), a community-run LGBTQ+ book catalog maintained by one person.

  • What it fetches: published RSS feeds of a small, hand-picked list of LGBTQ+ book sites (once per day, with conditional requests so unchanged feeds cost you nothing), and occasionally individual public pages (publisher product pages, review pages) while researching a specific book.
  • What we keep from feeds: post title, link, date, and the short excerpt your feed itself provides. We never store or republish full article text.
  • robots.txt: honored on every fetch, including re-checking your file daily. Blocking gaybooklistbot (or all agents) stops us within a day. We never bypass paywalls, challenges, or blocks — a block is treated as permanent.
  • To block us: User-agent: gaybooklistbot · Disallow: / — or just email us and we'll remove your site immediately.

Questions, requests, or a “please stop”: contact@gaybooklist.com. See also where our data comes from.