llms.txt: what it is and whether to publish one

By KYN AI Advisory Team — AI implementation specialists, Singapore

llms.txt is a proposed Markdown file, placed at your site root, that gives language models a short summary of your site and a curated list of links to the pages that matter. Jeremy Howard proposed it in September 2024. It is not an agreed web standard, and Google says no special AI text file is needed to appear in AI Overviews or AI Mode. Publishing one costs an hour and does little harm. Treat it as optional, low priority, and no substitute for clear pages.

What is llms.txt?

The proposal at llmstxt.org describes a file named llms.txt, written in Markdown, that sits at the site root (/llms.txt) or at a subpath, where it covers URLs under that path. The idea: a model, at the moment it answers a question, can read one compact file and find the right pages without crawling a whole site.

It differs from other site files:

| File | Audience | Purpose | |---|---|---| | robots.txt | Crawlers | Which paths bots may fetch | | sitemap.xml | Search engines | A full list of URLs to index | | llms.txt | Language models, at answer time | A short, curated guide to the pages that matter |

The proposal page itself describes llms.txt as a tool for LLMs used on demand, and does not claim that search engines have adopted it.

What goes inside the file?

According to the specification, the sections appear in this order:

An H1 with the site or project name. This is the only required part.
A blockquote with a short summary.
Optional body text with more detail, without headings.
H2 sections that hold lists of links, each as a Markdown link with an optional note.
An "Optional" section, by convention, for secondary links an agent can skip when it needs less context.

A minimal example for an SME:

# Example Co

> Example Co builds custom finance systems for SMEs in Singapore and Malaysia. Systems run in the client's own cloud account.

## Services
- Finance systems: bank reconciliation, GST, payroll
- Pricing: how fees are set

## Guides
- What is generative engine optimisation: definition and first steps

## Optional
- Case studies

Some sites also publish llms-full.txt, a longer file with page content inline. This is a convention by site owners, not part of the minimal specification.

Do AI assistants use it?

Be precise about what is known.

Google states in its AI features guidance that you do not need to create new machine-readable files, AI text files or markup to appear in AI Overviews or AI Mode.
OpenAI documents its crawlers and robots.txt controls in its bot documentation. That page, as fetched, describes robots.txt guidance, not llms.txt.
Perplexity documents PerplexityBot and Perplexity-User in its bot guide, also in terms of robots.txt.
Documentation platforms and developer tools are where adoption is reported, according to the proposal page.

We found no vendor statement saying a major assistant ranks or cites pages because of llms.txt. If you see a claim that it does, ask for the test. Our check method, in how to check if ChatGPT, Perplexity and Gemini cite your site, lets you test it on your own site.

Should an SME publish one?

Use this rule: publish it after the basics, and only if upkeep is near zero.

Publish if:

your site has a clear set of 10 to 30 key pages;
you have an automated process that regenerates the file when pages change;
you have developer-style content such as documentation or API guides, where the file is most used.

Skip it for now if:

your first paragraphs are still vague, your robots.txt blocks AI search bots, or your pages do not render without scripts. Fix those first;
nobody will maintain the file. A stale list that points to deleted pages does harm.

What matters more than llms.txt?

In order:

Access. Allow OAI-SearchBot and PerplexityBot in robots.txt if you want to appear in those search features. Each vendor's documentation says so. Check firewall rules too.
Indexing. Pages should be indexed and eligible for Search with a snippet, for Google's features.
Answer-first pages. The first paragraph answers the question. See how to write an answer-first article.
Sourced facts. Numbers linked to authorities.
Consistent business facts. Who you are, where you operate, how to contact you.

Then measure. Our introduction to the whole field is what is generative engine optimisation.

How do you publish one safely?

Write the file by hand for your 10 to 30 key pages. Use only public URLs that return 200.
Do not include anything private, staged or login-gated.
Place it at /llms.txt and confirm it loads as plain text.
Regenerate it whenever a key page is added, moved or removed. Better, generate it from your page list in the build.
Add it to your monthly checks, with a test for broken links.
Log the date you published, then compare your citation check before and after. Expect a small or no effect, and decide from the data.

What does a maintenance routine look like?

A file that is wrong is worse than no file. A short routine keeps it right.

| When | Task | Owner | |---|---|---| | On each publish or removal | Update the page list; regenerate the file in the build | Whoever publishes | | Monthly | Fetch every link in the file and confirm a 200 status | Marketing lead | | Quarterly | Re-read the summary blockquote; check it matches your homepage wording | Founder | | After a site move | Rebuild the file from the new URLs and test it | Developer |

Keep the summary to two sentences that state what you do, for whom and where. Do not copy your tagline. A model reading the file has no use for a slogan, and a plain statement is easier to quote accurately.

Choose the links by one test: would you send a new client to this page? If yes, list it. If it is a thin tag page, an old campaign or a login screen, leave it out. Ten well-chosen links beat a hundred. Put the pages a buyer reads first, such as the service page and the pricing page, at the top, and push older articles into the Optional section. Add a one-line note to each link saying what the page answers, in the words a buyer would use.

Where KYN fits

KYN's marketing system builds the page list your site already needs, and can generate an llms.txt from it as part of the build if you want one. Publishing stays behind a named person's approval, everything runs in your own cloud account, and every action is logged with the actor. The AI search check, about S$0.05 per check across 22 checks on our systems, is how we test whether a change moved anything. See marketing systems and the AI search check.

The first 14 days are a review of your stack and interviews with your team. A full build ships in under 90 days.

Frequently asked questions

Is llms.txt an official standard?

No. It is a proposal published at llmstxt.org. The page describes the format and its uses, and does not claim adoption by search engines.

Will llms.txt make ChatGPT cite my site?

We found no official statement that it does. OpenAI's documentation points to allowing OAI-SearchBot in robots.txt for ChatGPT search. Test on your own site before you rely on any claim.

Can llms.txt hurt my site?

Rarely. The risk is a stale file that lists dead or private pages, or one that contradicts your site. Keep it short, accurate and generated from the real page list.

Checked 30 October 2026.

Curious whether Marketing agents fits your business? Talk to KYN on WhatsApp — no forms, just a conversation.

Start Building →