llms.txt Implementation: The Complete Guide — Designing Your Site for AI Crawlers

Executive summary

• llms.txt is a text file that tells AI crawlers, in summary form, "here is how this site is organized." It is conceptually close to an AI version of robots.txt, but its goal is not to block — it is to win preferential treatment as a citation candidate.
• In GEO Meter's smoke analysis (2 topics x roughly 20 domains), we observed a +20–30pp gap in llms.txt deployment rates between top-cited and bottom-cited groups. The implementation cost is only 1–2 hours, while the effect is large, making this the first measure you should tackle.
• Place a single file at https://<your-domain>/llms.txt. No HTML is needed — it is plain text in a Markdown-like format.
• "Just put it up" is not enough. The recommended structure is four blocks: title / summary / key URLs / content intent. This directly affects how accurately AI reads it.

1. Why llms.txt works

A typical flow when an AI (LLM) generates an answer looks like this:

1. The user asks a question
2. The AI searches the Web and/or references its existing knowledge
3. It fetches a set of relevant URLs and reads the content
4. It summarizes and cites the information it ingested to produce an answer

llms.txt provides a hint at step 3 — "which URLs should I read?" — at the moment the AI is deciding.

A /llms.txt placed at the site root acts as the first place the AI checks to learn "what does this site offer?" Its role is close to sitemap.xml in SEO terms, but it differs in that it is a summary tailored for AI alone, stripped of human-facing navigation information.

Backed by GEO Meter observation data

In a smoke analysis run by GEO Meter (2 topics x roughly 20 domains):

A consistent +20–30pp gap was observed across multiple topics. The correlation is strong enough that simply deploying it tends to land you in the top group.

2. Basic structure (four blocks)

The recommended minimum structure is the following four blocks:

# Site name (equivalent to H1)

> 1-2 sentence overview of the site (whose problem does it solve?)

## Key sections

- [Page title](https://example.com/page-1): One-line description
- [Page title](https://example.com/page-2): One-line description

## Notes

- Target audience / intended reader
- The context in which you want to be cited (optional)

Example (B2B SaaS)

# Sencha Lab — Sencha tea subscription for businesses

> Sencha Lab is a sencha tea subscription service used by 200 companies.
> Direct from origin, pesticide-free, flexible 3-month plans. From ¥3,800/month.

## Key sections

- [Service overview](https://sencha-lab.example.com/): What the corporate sencha subscription is
- [Pricing plans](https://sencha-lab.example.com/pricing): Monthly plans and how to sign up
- [Customer stories](https://sencha-lab.example.com/cases): Interviews with 5 corporate customers
- [FAQ](https://sencha-lab.example.com/faq): Delivery / cancellation / tea-leaf FAQs

## Notes

- Intended readers: HR, reception, and employee-benefits leads
- Citation context: corporate tea gifts / employee benefits / office drinks

Example (sole proprietor)

# Tanaka Tax Accountant Office

> A tax accountant office in Akasaka, Minato-ku, Tokyo. 25 years of experience
> specializing in corporate closing, sole-proprietor tax filing, and inheritance.
> First consultation is free.

## Key sections

- [Office profile](https://tanaka-tax.example.com/): Principal's profile and supported work
- [Services and fees](https://tanaka-tax.example.com/services): Corporate closing / filing / inheritance pricing
- [Consultation flow](https://tanaka-tax.example.com/flow): From first consultation to contract
- [FAQ](https://tanaka-tax.example.com/faq): Fees / scope / timeline FAQ

## Notes

- Intended readers: SMBs near Akasaka, sole proprietors, people anticipating inheritance
- Citation context: queries combining region + profession, e.g. "Akasaka tax accountant" or "Tokyo inheritance tax accountant"

3. Step-by-step implementation

STEP 1: List 5–10 key URLs (15 minutes)

Order the pages you want the AI to cite by priority. Listing every page is counterproductive (the AI treats it as noise), so narrow it to 5–10.

How to prioritize:

1. Service / product / pricing pages (closest to conversion)
2. FAQ pages (Q&A format the AI can quote directly)
3. Customer stories / track record pages (concrete, easy to cite)
4. Company / profile information (basis for trust)

STEP 2: Write a one-line description per URL (20 minutes)

Use the "Page title: one-line description" format. The AI reads these descriptions to decide whether to read the page, so avoid abstract phrasing.

❌ Bad:
- [Service](https://...): About our service

✅ Good:
- [Pricing plans](https://...): Two plans at ¥9,800 / ¥49,800 per month, plus the sign-up steps

STEP 3: Write a 1–2 sentence overview of the whole site (15 minutes)

In the leading > block (blockquote), write a 1–2 sentence summary of the entire site. It works well if you cover three points: target / what you offer / differentiator.

> [Company name] is [service description] for [target audience].
> [Differentiator] (number of customers / price range / specialty, etc.).

STEP 4: Place it at the site root (10 minutes)

Make it reachable at https://<your-domain>/llms.txt. The deployment method depends on your server setup:

Vercel / Netlify (static / Next.js)

Just place it at public/llms.txt. It is published automatically on deploy.

WordPress

Upload llms.txt to the root of public_html via FTP or SSH.

Apache / nginx (hand-rolled)

Place it directly under DocumentRoot. It is readable as long as your .htaccess or server config sets the text/plain MIME type.

Via CDN (Cloudflare, etc.)

Place it on the origin and purge the CDN cache.

STEP 5: Verify (10 minutes)

Open https://<your-domain>/llms.txt in a browser and confirm the text is shown. The critical thing is that no HTML error page is returned.

Verification command:

curl -I https://<your-domain>/llms.txt
# → HTTP/2 200 is OK
# → Content-Type: text/plain; charset=utf-8 is preferable

4. Common pitfalls

❌ Listing every page

Copying your entire sitemap and listing 200 pages looks like noise from the AI's perspective. Narrow to 5–10 and make priorities explicit.

❌ Pasting HTML

AI will still read <div> or <a> tags if you include them, but the recommendation is plain text + Markdown. Strip the tags.

❌ Writing customer-facing "marketing copy"

Abstract marketing prose like "An innovative solution to transform your future" lacks the specificity the AI needs to cite you. Write in a fact-based, number-based style.

❌ Omitting a date

Including a last-updated date inside llms.txt makes the AI more likely to judge the site as "fresh." Adding something like > Last updated: 2026-05-25 at the top is recommended.

❌ Trailing blank lines

This is a minor point, but some AIs have been reported to misread a trailing blank line as a "section break." The safe convention is to end the last line with a newline (POSIX convention) but with no blank line after it.

5. AI-by-AI behavior differences (as of 2026-05)

From GEO Meter's observations, each AI handles llms.txt slightly differently:

→ It works for any AI, but it especially helps ChatGPT / Claude. Gemini tends to prioritize Schema.org and FAQ pages instead.

6. Difference from llms-full.txt

Recently you may see "llms-full.txt" talked about as a related spec. It is an extension of llms.txt — a long-form file bundling the full text of every page.

A general corporate site is fully served by llms.txt alone. llms-full.txt is limited to services with API references or technical documentation.

7. What to do after publishing

Just publishing llms.txt is not enough — AI needs to actually come and read it. Next steps:

1. Allow AI crawlers in robots.txt (Allow GPTBot / ClaudeBot / Google-Extended, etc.)
2. Earn inbound links (the AI needs to discover llms.txt via links from other sites)
3. Measure the impact with GEO Meter (track SOV and citation counts monthly)

→ Run a free 3-minute GEO Meter diagnostic for your site

8. Summary

• llms.txt is a text file that hands AI a "site summary + key URL list"
• Recommended four-block structure: site name / overview / key URLs / notes
• Implementation: 1–2 hours; effect: up to +30pp (GEO Meter observation)
• The golden rules for good content: do not list every page, do not write abstract copy, do include numbers
• After deployment, combine AI-crawler Allow rules + inbound links + measurement to maximize impact

Related resources

• GEO Basics: The Complete Guide — Big picture and positioning of GEO
• GEO Implementation: The Complete Guide — Priority order of the five main measures
• AI Platform Optimization: The Complete Guide — Tuning for Claude / ChatGPT / Gemini
• Free GEO Meter Diagnostic — Measure your site's current state