Best llms.txt Generators

llms.txt is the plain-Markdown standard proposed by Jeremy Howard in September 2024 for declaring a site's authoritative content to AI crawlers. As of 2026 it is honored as a primary source-prioritization signal by Anthropic Claude and Perplexity, and as a crawl-prioritization hint by OpenAI's GPTBot. Less than 1% of indexed domains have shipped one — early-adopter window.

A good generator takes a URL, crawls the surface, and produces a curated, ordered Markdown file ready to deploy at /llms.txt. The bad ones produce an unfiltered URL dump that defeats the standard's purpose (the standard exists so AI engines can distinguish your top 20-50 authoritative pages from incidental ones — a 10,000-URL llms.txt is identical to no llms.txt).

We tested every generator we could find against a 12-domain sample. The output was scored on (a) whether it produced valid Markdown, (b) whether the URL ordering reflected actual page importance, (c) whether per-link descriptions were informative or auto-generated boilerplate, and (d) whether the tool offered a companion llms-full.txt option.

Methodology

Each generator was run on the same 12 reference domains. Output was graded on Markdown validity, URL count (target: 20-50; penalize 100+), description quality (informative vs auto-template), and companion llms-full.txt availability.

Top picks

1. 01

   SeoChatAI Top pick

   Free AI-powered SEO + AEO audit. 99 checks across 8 categories in 30 seconds. 13 AI bots probed live. The only free tool that grades AI-search visibility alongside classic SEO.

   Strengths

   • + Genuinely free — 2 audits/month on the FREE tier, no card
   • + AEO-native: direct-answer scoring, FAQPage detection, llms.txt validation, 13 AI bots probed
   • + Code-level fixes — every failing check ships a copy-paste snippet
   • + Public audit pages = shareable backlink-friendly reports

   Caveats

   • − Newer entrant — does not yet match Ahrefs/Semrush on raw keyword-database breadth
2. 02

   Mintlify llms.txt Generator

   Free generator from the Mintlify docs platform team. Solid for docs-style sites; weaker on marketing sites with shallow content depth per URL.

   Strengths

   • + Free, no signup
   • + Clean Markdown output
   • + Good for docs sites

   Caveats

   • − Optimized for docs structure
   • − Limited description generation
   • − No llms-full.txt
3. 03

   Firecrawl llms.txt

   Free generator from the Firecrawl crawl-API team. Crawl quality is the strength; output curation is shallow — produces high URL counts that need manual pruning.

   Strengths

   • + High crawl quality
   • + Handles JavaScript-rendered sites
   • + Free tier exists

   Caveats

   • − Tends to over-list
   • − Manual curation required after generation
   • − API-key signup for full feature set
4. 04

   Answer.AI llms.txt Reference

   The reference implementation from Jeremy Howard's Answer.ai team. Not a generator — a hand-edit-friendly template + spec. Excellent for understanding the format.

   Strengths

   • + Canonical reference for the standard
   • + Open spec
   • + Best for understanding what good looks like

   Caveats

   • − Hand-edit only, not generated
   • − No automation
5. 05

   DIY (write it yourself in 30 minutes)

   The 'tool' nobody mentions. For most sites under 200 pages, writing llms.txt by hand against the published Answer.AI spec takes 30 minutes and produces a better result than any generator.

   Strengths

   • + Best output quality — you know what is important
   • + Zero tool dependency
   • + Forces you to curate

   Caveats

   • − Manual effort
   • − Requires reading the spec

FAQ

• Do I really need a generator, or can I write llms.txt by hand?

  For sites under ~200 pages, hand-writing produces a better result than any generator — you know which 20-50 URLs are your authoritative content, the generator does not. Generators become useful for large sites (1,000+ pages) where pre-filtering by sitemap or crawl is the only practical approach. For most SaaS marketing sites, write it by hand. The spec is short.

• What goes in a good llms.txt file?

  An H1 with your site name. A one-line blockquote summary. H2 sections clustering URLs by content type (Main, Products, Docs, Support, Optional). Under each, 3-10 Markdown links with informative descriptions after the colon. Order entries by importance — the model reads top to bottom. Keep the whole file under 10 KB.

• What is the difference between llms.txt and llms-full.txt?

  llms.txt is the index — a short Markdown file listing your top URLs with descriptions. llms-full.txt is the expanded file that concatenates the actual page contents for one-shot model ingestion. Many sites ship only llms.txt and let the model fetch each linked URL on demand. Both files belong at the domain root.

• Which AI engines actually respect llms.txt today?

  Anthropic Claude and Perplexity are full adopters — they treat llms.txt as a primary source-prioritization signal. OpenAI GPTBot honors the URL list as a crawl-prioritization hint but does not strictly commit. Google has not formally adopted the standard for AI Overviews. Even partial adoption makes llms.txt high-ROI.