GEO

llms.txt, explained: the file that guides AI crawlers

llms.txt is a simple file that tells AI crawlers what your site is about and what to prioritise. Here is what it does and how to start one.

If you have set up a robots.txt or a sitemap, the idea behind llms.txt will feel familiar. It is a small, plain-text file that helps AI crawlers understand your site: what it is about, which pages matter most, and how you would like to be represented. Think of it as a friendly briefing for the models that read you.

What it is for

Search crawlers have long used robots.txt for access and sitemaps for structure. llms.txt adds a layer aimed at assistants, pointing them at your most important, most quotable pages so they spend their attention where it counts. It does not force anything, but it makes you easier to read well.

What goes in it

A good file leads with a short description of your brand, then a curated list of key links with a line on why each matters: your core product pages, your best explainers, your documentation. Keep it honest and current, and avoid stuffing it with everything, since the value is in the priority you signal.

Where it fits

llms.txt is one piece of access, not the whole job. It works alongside clean robots rules, structured data and the answer-first writing that earns a citation. On its own it will not win you a mention, but without readable, well-marked content behind it, a crawler has little to reward. It is part of the same access layer we describe in what GEO is.

Not sure whether crawlers can read your site today, or whether your llms.txt is doing its job? A free scan checks crawler access and the rest of your GEO foundations, and labels every finding by how it was measured.

See what AI says about your brand

Free, no card, takes a few seconds.

AI crawlersllms.txt