What llms.txt is and why Repkit checks for it

llms.txt is a plain-text file published at the root of a site, usually at /llms.txt, that points assistants and crawlers to the pages you most want them to understand. It is not a replacement for robots.txt or your sitemap. Think of it as a short reading list with notes: which URLs matter, what they contain, and how you prefer automated tools to treat them.

The idea spread because large language models and new crawlers do not share one manual. A sitemap lists URLs for indexing; llms.txt adds intent — "start here," "this is our pricing," "this PDF is the menu." Repkit checks whether the file exists, responds with a success code, and is non-empty. Missing llms.txt is not a failure of your business; it is simply a missing hint that Repkit reports honestly.

A useful llms.txt is short and maintained. Link to stable pages, not campaign query strings that expire next week. Prefer HTTPS URLs on your own domain. If you have both a marketing site and a booking subdomain, say which one is authoritative for hours and contact. When you rename services, update llms.txt in the same pull request as the homepage.

llms.txt does not bypass paywalls or private areas. If a URL requires a login, do not list it as public guidance. Repkit's fetcher behaves like a polite crawler; it will not invent credentials. Keep sensitive client portals off the list and ensure robots.txt reflects the same boundary.

Some teams worry that llms.txt "invites" AI training. Your robots.txt and terms already define much of that posture. llms.txt is optional transparency about what you are willing to show on the open web anyway. If you block a crawler in robots.txt, listing a URL in llms.txt does not override that block, and Repkit reports both files separately so you can see the combined effect.

Repkit scores llms.txt alongside other checks — titles, contact signals, heading structure — because assistants rarely use one signal in isolation. A great llms.txt with an empty homepage still reads poorly. A solid homepage with no llms.txt may still do well; the check simply marks the hint as absent, which is easier to fix than rewriting ten blog posts.

Writing the first file takes minutes. Start with your homepage, services or menu page, pricing or fees page if public, contact page, and any canonical guide you want cited. Add one line each describing purpose in plain language. Avoid marketing superlatives; describe facts such as emergency call-out areas, new patient registration, or the postcode patch you actively market, then re-run Repkit to confirm the file is served with a success response.

The £29 PDF plan includes guidance on llms.txt and discovery files you can publish yourself. Validate by re-running the free check after publish so the checker confirms the file responds.

Format stays deliberately boring: UTF-8 text, stable URLs, optional markdown links. Fancy layouts break parsers. Store the file in version control beside robots.txt, and curate a handful of defining pages rather than every tag archive.

Treat llms.txt like a signpost, not SEO magic. Combined with a clear public site, it reduces the chance that an assistant summarises you from a stale blog post instead of your current offer — the same outcome Repkit's visibility bands reward when you re-run the free URL check after publish, with no card required on the homepage flow.