# How to make a Docusaurus site agent-ready > Generate llms.txt in sidebar order, a markdown twin of every page, AGENTS.md, and the OpenAPI document from a Docusaurus plugin, and fail the build on any drift. Source: https://voxgig.com/howto/make-a-docusaurus-site-agent-ready - Audience: devrel - Level: intermediate - Languages: typescript - Verified: 2026-10-15 - Published: 2026-10-15 ## Short answer Write a small Docusaurus plugin. Its postBuild hook reads the docs plugin's loaded content from the plugins list it is handed, then writes llms.txt in sidebar order, a markdown twin per page, AGENTS.md, and the OpenAPI document. Leave out older versions and unlisted pages, then compare the build's HTML pages with llms.txt and throw on any difference, so the build fails instead of shipping a stale index. --- ## You will need Node 22 or later and a Docusaurus 3 site. Verified 2026-10-15 against Node 22.22.0, Docusaurus 3.10.2 with `@docusaurus/faster`, React 19.2.0, and `@signalwire/docusaurus-plugin-llms-txt` 1.2.2. The sample is a real site, built by every command on this page. What goes in the site-root `AGENTS.md` is a separate task; this page only publishes the file. ## Approaches compared | Approach | When it fits | What it costs you | When to pick something else | | --- | --- | --- | --- | | [`@signalwire/docusaurus-plugin-llms-txt`](https://github.com/signalwire/docusaurus-plugins/blob/6cab586686567b9b04db97ee48d0843814af0523/packages/docusaurus-plugin-llms-txt/README.md) | You want `llms.txt` and twins from one install and accept its grouping | Groups by URL path rather than sidebar, and lists unlisted pages, unless you configure around it | The sidebar order is the reading order you want agents to follow | | [A custom `postBuild` plugin](https://docusaurus.io/docs/api/plugin-methods/lifecycle-apis) | You want sidebar order, your own exclusions, and a check that can fail the build | About 200 lines to own, reading a docs plugin content shape the lifecycle page does not document | You have one small site and no versioned docs | | [Static files in `static/`](https://docusaurus.io/docs/static-assets) | A site that changes rarely, with no build step you control | Every page added or renamed is a hand edit, and nothing notices when one is missed | The docs change more often than someone remembers the index | A plugin from npm is the least code, and its upgrades arrive on its maintainers' schedule. Version 1.2.2 groups pages by URL and lists versioned docs unless told otherwise, which is measured below. The custom plugin is code you own, and it reads Docusaurus's own sidebar. Static files cost nothing to set up, but they drift the first time a page is added without an edit to them. ## Write the files from the plugin system The sidebars and the doc metadata belong to the docs plugin. The [lifecycle page](https://docusaurus.io/docs/api/plugin-methods/lifecycle-apis) lists a `plugins` array among the props `postBuild` receives, and in 3.10.2 the docs plugin's entry carries its loaded content. The page does not describe that content's shape, so pin the version and treat an upgrade as a change to test. The 3.10.2 [type definitions](https://github.com/facebook/docusaurus/blob/v3.10.2/packages/docusaurus-types/src/plugin.d.ts) also declare an `allContentLoaded` hook that the lifecycle page leaves out, which this plugin does not need. ```js title="plugins/agent-surfaces.mjs" async postBuild({ outDir, plugins }) { const docsPlugin = plugins.find((p) => p.name === 'docusaurus-plugin-content-docs' && p.options.id === 'default') if (!docsPlugin) throw new Error('agent-surfaces needs the default instance of the docs plugin') const versions = docsPlugin.content.loadedVersions const current = versions.find((v) => v.versionName === 'current') const sections = sidebarSections(current) const write = async (path, text) => { await mkdir(dirname(join(outDir, path)), { recursive: true }) await writeFile(join(outDir, path), text) } for (const doc of sections.flatMap((s) => s.docs)) { const source = await readFile(join(siteDir, doc.source.replace(/^@site\//, '')), 'utf8') await write(twinPath(doc.permalink), markdownTwin(source, doc.title)) } ``` A twin is the page's source with the frontmatter and MDX imports removed, where an import inside a code fence is part of a sample and stays. That keeps what the writer wrote, and it keeps a JSX component as JSX. The npm plugin converts the rendered HTML instead, which resolves components and loses the source. The twin sits at the page's URL with `.md` appended, where the [`llms.txt` proposal](https://llmstxt.org/) puts it.[^1] The order comes from the sidebar. `sidebarSections` walks it, turns each top-level category into an `llms.txt` section, and drops unlisted docs. It puts any doc no sidebar names in an "Other pages" section, because an agent can land on it from search all the same: ```js title="surfaces.mjs" const walk = (items, section) => { for (const item of items) { if (item.type === 'doc' || item.type === 'ref') take(item.id, section) if (item.type === 'category') { // A top-level category becomes a section; a nested one stays in its parent's. const inner = section === sections[0] ? { title: item.label, docs: [] } : section if (inner !== section) sections.push(inner) if (item.link?.type === 'doc') take(item.link.id, inner) walk(item.items, inner) } } } ``` The sample site has a current version, a frozen 1.0 under `/1.0/`, and one page marked [`unlisted: true`](https://docusaurus.io/docs/api/plugins/@docusaurus/plugin-content-docs), which Docusaurus builds with a `noindex` robots tag. Build it: ```bash npm run build ``` ```text output [agent-surfaces] llms.txt: 5 pages in 2 sections, sidebar order [agent-surfaces] markdown twins: 5, AGENTS.md, openapi.yaml [agent-surfaces] left out: /1.0/ and 1 unlisted [agent-surfaces] check: llms.txt and the build agree ``` The index it wrote follows the sidebar, Paginate before Authenticate: ```bash cat build/llms.txt ``` ```text output # Meterco API > The Meterco energy metering API. ## Docs - [Meterco API](https://docs.meterco.example/index.md): What the Meterco API does and where to start. - [Quickstart](https://docs.meterco.example/quickstart.md): Install the client, authenticate, and read one meter. ## Guides - [Paginate](https://docs.meterco.example/guides/paginate.md): Follow the next cursor until it is absent. - [Authenticate](https://docs.meterco.example/guides/authenticate.md): Send an API key as a bearer token, and rotate it without downtime. - [Errors](https://docs.meterco.example/guides/errors.md): Problem details on every error, and which ones to retry. ## Optional - [OpenAPI document](https://docs.meterco.example/openapi.yaml): every operation and schema ``` ## Compare it with the plugin from npm The same site, built with `@signalwire/docusaurus-plugin-llms-txt` and its `includeVersionedDocs` option off, then put through the same check: ```bash SURFACES=signalwire npx docusaurus build --out-dir build-signalwire > /dev/null 2>&1 && cat build-signalwire/llms.txt && node check-surfaces.mjs build-signalwire --exclude /1.0/ --require llms.txt ``` ```text output # Meterco API > The Meterco energy metering API. - [Meterco API](/index.md) ## guides - [Authenticate](/guides/authenticate.md): Send an API key as a bearer token, and rotate it without downtime. - [Errors](/guides/errors.md): Problem details on every error, and which ones to retry. - [Legacy session tokens](/guides/legacy-tokens.md): The session tokens removed in version 2. - [Paginate](/guides/paginate.md): Follow the next cursor until it is absent. ## quickstart - [Quickstart](/quickstart.md): Install the client, authenticate, and read one meter. build-signalwire: 1 disagreement llms.txt lists /guides/legacy-tokens.md, which is not a page the site offers ``` The npm plugin's index differs in three ways, and each can be configured away. Sections are URL path segments, sorted, so Authenticate comes before Paginate. The unlisted page is listed. With the option left at its default, a separate run listed both 1.0 pages under a `## 1.0` section as well. Its README offers `includeOrder` and `excludeRoutes` for the first two, which is a glob list to keep in step with the sidebar by hand. ## Fail the build when the index and the pages disagree The check reads only the build directory. Every HTML page that is not a 404, not under an excluded version path, and not `noindex` must be in `llms.txt` and have a twin. Every `llms.txt` entry must be such a page. It knows nothing of the plugin, which is why the same function checks all three routes. The plugin calls it last and throws on any finding, which fails the build. `add-page.mjs` copies the site, adds a webhooks guide to the sidebar, and builds it twice: with the plugin, and with yesterday's hand-written files in `static/`. Then it adds a pricing page outside the docs: ```bash node add-page.mjs ``` ```text output added docs/guides/webhooks.md to the Guides category plugin route llms.txt entry: - [Webhooks](https://docs.meterco.example/guides/webhooks.md): Receive a signed event when a reading arrives. markdown twin: guides/webhooks.md check: agrees static route llms.txt entry: none markdown twin: none check: 2 disagreements /guides/webhooks is built but not in llms.txt /guides/webhooks has no markdown twin at /guides/webhooks.md plugin route, after adding src/pages/pricing.md docusaurus build exited 1 agent surfaces disagree with the build: /pricing is built but not in llms.txt /pricing has no markdown twin at /pricing.md ``` The static build passed, so only a check run after the build can catch the missing page. For a static route, that check is a CI step after the build: `node check-surfaces.mjs build --exclude /1.0/`. The pricing page fails the plugin's build because the docs plugin does not own it. List it, or name it in the plugin's `exclude` option if agents should not have it.[^2] ## Check it worked The tests build small sites on disk, so they cover cases the sample lacks, such as a nested category and a doc no sidebar names. They also cover a link to another site, an `exclude` of `/pricing` that must not swallow `/pricing-faq`, and an `import` line inside a code sample. ```bash npm test ``` ```text output # tests 10 # suites 0 # pass 10 # fail 0 ``` ## When it goes wrong The build fails with `require.resolveWeak is not a function`. The site's `package.json` has `"type": "module"`, which broke this sample's 3.10.2 build with `future.faster` on and with it off. Remove the field and keep ESM in `.mjs` files, as the plugin here does. The build is too slow to run on every pull request. `future.faster` with `@docusaurus/faster` cut a cold build of this sample from 26 seconds to 7 in the container the page was checked in. Measure your own site before you put a full build in front of every pull request. ## When not to do this Do not write a custom plugin for a site with a handful of pages and one version. Install the npm plugin, set `includeVersionedDocs: false`, name unlisted pages in `excludeRoutes`, and run the check script in CI. That covers the drift without code of your own to upgrade. Do not keep the agent files in `static/` on a site that changes weekly. They are right on the day they are written, and this page's check is the only thing that will say when they stop being so. Do not read the twins as a rendering of the page. They are the source, so an MDX component reaches the agent as JSX. If your pages lean on components for their content, convert the built HTML instead, which is what the npm plugin does. ## Related how-tos - [Publish an OpenAPI document at a stable URL for agents](/howto/publish-openapi-at-a-stable-url) - [Set robots.txt rules for each AI crawler](/howto/set-robots-rules-for-ai-crawlers) - [Detect a reference site drifting from its OpenAPI spec](/howto/check-a-reference-site-in-ci) - [Install an API skill and check a coding agent uses it](/howto/install-an-api-skill-for-a-coding-agent) - [Organize a large OpenAPI reference by tags and tag groups](/howto/organise-a-large-openapi-reference) ## Last verified Verified 2026-10-15 against Node 22.22.0, `@docusaurus/core` 3.10.2, `@docusaurus/faster` 3.10.2, React 19.2.0, and `@signalwire/docusaurus-plugin-llms-txt` 1.2.2. Every output block is what the command preceding it printed, from real Docusaurus builds of the sample site. The build times are from one machine and are not captured. [^1]: The proposal is [Jeremy Howard's](https://llmstxt.org/), and its second version offers two spellings for a twin's URL: `.md` appended, as in `page.html.md`, or the extension replaced, as in `page.md`. A Docusaurus URL such as `/guides/paginate` has no extension, so the two spellings name the same file and the choice makes itself. [^2]: The robots tag on an unlisted page is how the check tells it apart from a page to list. The built page carries it as ``, with quotes left only on the value that needs them. The [docs plugin reference](https://docusaurus.io/docs/api/plugins/@docusaurus/plugin-content-docs) promises that an unlisted page is not indexed, and the tag is how a static file keeps that promise.