What is llms.txt?
llms.txt is a plain-text file placed at the root of a website (at /llms.txt) that gives AI language model systems a structured, human-readable summary of the site. It was proposed by Jeremy Howard in September 2024 as a simple convention for helping AI tools understand a site's purpose without having to crawl and parse every page individually.
The file is modelled on the simplicity of robots.txt but serves a different purpose: where robots.txt controls access, llms.txt provides context. It tells an AI system what the site is about, which pages carry the most important content, and whether AI training use is permitted.
Unlike robots.txt, which is a technical standard with decades of crawler adoption, llms.txt is an emerging convention. Perplexity has publicly confirmed it reads the file. OpenAI has not confirmed it for ChatGPT. Google has not confirmed it for Gemini. That said, the file costs almost nothing to add - a 15-minute task for most sites - and benefits the engines that do read it while adding a positive AI-readiness signal.
Important: llms.txt is a supporting signal
The highest-impact AI search improvements are AI crawler access (robots.txt), FAQPage structured data, and E-E-A-T signals - in that order. llms.txt is worth adding but will not compensate for blocked crawlers or missing schema. Run a free AI Search audit to see your full score before deciding where to start.
How llms.txt differs from robots.txt
| Property | robots.txt | llms.txt |
|---|---|---|
| Purpose | Controls crawler access to pages | Provides context about the site to AI systems |
| Format | Structured rules (User-agent, Allow, Disallow) | Free-form Markdown |
| Standard | Established (RFC 9309, 1994) | Proposed convention (2024) |
| Adoption | Universal - every major crawler reads it | Partial - Perplexity confirmed; ChatGPT unconfirmed |
| Effect of absence | Crawlers assume full access is permitted | AI tools summarise from page content instead |
| Effect of presence | Restricts or permits crawler access explicitly | Helps AI tools understand site purpose faster |
How to write an llms.txt file
The file uses Markdown formatting. It has three sections: a site name and description, a list of key pages with brief descriptions, and an optional usage policy. You do not need to list every page - five to ten of your most important pages is enough.
Don't want to write it manually?
Use the free llms.txt Generator - it scans your site and builds a complete, structured file automatically from your sitemap and page metadata.
Template (copy and customise)
# yourdomain.com > AI-readable overview of yourdomain.com ## About yourdomain.com is [describe what your site does in 1-2 sentences. Be specific about who it helps and what problem it solves]. ## Key Pages - [Homepage](https://yourdomain.com/): [brief description of what visitors find here] - [About](https://yourdomain.com/about/): [brief description of company and team] - [Services](https://yourdomain.com/services/): [brief description of services offered] - [Blog](https://yourdomain.com/blog/): [topics covered in the blog] - [Contact](https://yourdomain.com/contact/): [how to get in touch] ## Optional: Usage Policy This content may be used by AI systems to summarise and answer user queries. For commercial training use, contact hello@yourdomain.com.
Create the file
Create a plain text file named llms.txt. Use Markdown formatting as shown in the template above. Keep it under 500 words - AI tools prefer concise, scannable summaries over long narratives.
Place it at your site root
The file must be accessible at https://yourdomain.com/llms.txt. For Next.js: add it to the /public folder. For WordPress: upload via the file manager or Appearance - Theme File Editor to the theme root. For Webflow: use the File Manager. For Squarespace: use Code Injection to serve static files, or confirm your hosting provider supports root-level file placement.
Verify it is accessible
Open https://yourdomain.com/llms.txt in a browser. The raw Markdown should appear as plain text - no redirect, no 404, no login required. The AI Search Checker also tests this automatically as part of its 32-signal audit.
Keep it updated
Update llms.txt whenever you add a major new section or product to your site. AI tools that cache the file will re-read it on their next crawl cycle. Outdated llms.txt files are not penalised - they are simply less useful than accurate ones.
What to include in your llms.txt
Include
- One-sentence description of what the site does
- Who the site serves (target audience)
- 5-10 most important pages with descriptions
- Your contact or about page
- Usage policy if you have a position on AI training use
Skip
- Every page on the site - be selective
- SEO keyword stuffing
- Duplicate content from your homepage meta description
- Technical implementation details
- Information that changes weekly (pricing, stock)
Frequently asked questions
What is llms.txt?
llms.txt is a plain-text file placed at the root of a website (at /llms.txt) that gives AI language model systems a structured, human-readable summary of the site - what it does, its key pages, and optionally a usage policy. It was proposed by Jeremy Howard in September 2024 as a simple convention for helping AI tools understand a site's purpose without having to crawl and parse every page.
Does ChatGPT read llms.txt?
OpenAI has not publicly confirmed that ChatGPT or its crawlers read llms.txt. Perplexity has indicated it reads the file when present. Several smaller AI tools and assistants have also adopted it. Because it is a low-effort addition and emerging as a standard, adding it is recommended even without confirmed adoption by every major engine - it costs nothing and benefits the engines that do read it.
Is llms.txt the same as robots.txt?
No. robots.txt tells crawlers which pages they are and are not allowed to access. llms.txt does not control access - it provides context. A page not listed in llms.txt can still be crawled if robots.txt permits it. The two files serve different purposes: robots.txt is a technical access control file, llms.txt is an informational summary for AI systems.
How important is llms.txt for AI search visibility?
It is a supporting signal rather than a primary one. The most important signals for AI search visibility are AI crawler access (robots.txt), structured data (FAQPage, HowTo schema), E-E-A-T attribution, and content structure. llms.txt adds 3 points to the AI Search Checker score and signals that you are AI-aware, but it will not compensate for blocked crawlers or missing schema. Fix those first, then add llms.txt.
Where does the llms.txt file go?
At the root of your domain, accessible at https://yourdomain.com/llms.txt. For most CMS platforms: place it in the /public folder (Next.js, Nuxt, Gatsby), the /static folder (Hugo, Eleventy), or upload it as a static file through the platform's file manager (WordPress, Squarespace, Webflow). The file must be publicly accessible without a redirect.
Is there an llms.txt checker tool?
The GetInlytics AI Search Checker tests for the presence of llms.txt as part of its 32-signal audit. It also checks whether the file is accessible without a redirect and whether its content is parseable. You can run a free check on any URL at getinlytics.com/tools/ai-search-checker.
Check if your llms.txt is set up correctly
The free AI Search Visibility Checker tests for llms.txt presence and accessibility as part of 32 AI-readiness signals. It also checks AI crawler access in robots.txt, FAQPage schema, E-E-A-T signals, and content structure - the higher-priority fixes that compound alongside llms.txt.
Run free AI search audit