Add llms.txt with Mapbox/MapLibre split - #2593
Conversation
a398492 to
1c01abd
Compare
Generate stack-specific LLM documentation files so agents load only the relevant API reference, plus a root router and example links with technique blurbs from example READMEs. Co-Authored-By: Cursor <cursoragent@cursor.com>
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Reviewed by Cursor Bugbot for commit 13514f5. Configure here.
| docsDir: [{path: '../docs', routeBasePath: 'docs', label: 'Docs'}], | ||
| generateLLMsTxt: false, | ||
| generateLLMsFullTxt: false, | ||
| generateMarkdownFiles: false, |
There was a problem hiding this comment.
Index links point to missing markdown
Medium Severity
generateMarkdownFiles is set to false, but docusaurus-plugin-llms v0.4.0 still appends .md to doc links in the generated index files. Those URLs do not exist in the static build (live pages are HTML paths without .md), so agents following llms.txt / llms-mapbox.txt / llms-maplibre.txt links get 404s unless they only use the full-content bundles.
Reviewed by Cursor Bugbot for commit 13514f5. Configure here.
There was a problem hiding this comment.
This seems to be true - the index links end in .md, while the deployed Docusaurus routes generally do not.
|
Thank you for this contribution! How do libraries handle pointing LLMs to the correct version of their docs? We have a pattern of maintaining release branches for each version with complete docs (e.g. https://github.com/visgl/react-map-gl/tree/7.1-release/docs) |
I am not aware of a lot of standardization in this space, yet. I think it would be fine to have this kind of feature only for the pages that are generated to the website. AFAIK older release docs are not generated but only accessible via Github? NextJS recently made it that each page has a ".md" version Site note: I am now using https://github.com/FixMyBerlin/fixmyskills/tree/main/skills/react-map-gl to work with react map gl and LLM which I find give better results because it is more tailored to errors the LLM does which are not covered by the docs directly. Maybe someone can use this and set up https://tanstack.com/intent/latest to have the LLM support (entry) as part of the package. |
|
A clean build still puts opposite-stack API links in the stack-specific indexes: |
|
Thanks for sharing these ideas! I'm open to publishing old versions of the docs in a more formal way.. One complication is that a lot of pages include interactive demos, which I suspect will lead to some odd compatibility issues. Oh course, not an issue for markdown files FYI this URL returns a special 404. |
|
Hey @tordans, latest-only support for the website sounds reasonable for this PR, provided the generated files identify the current release and all links are validated. The FixMyBerlin skill contains useful lessons, though we would need to extract the library-generic guidance from the project-specific conventions. I think that deserves a separate issue to explore bundled docs or an official versioned skill, potentially using Intent. |
I created #2603 to track this. |
2 participants
Summary
docusaurus-plugin-llmsto generate LLM-friendly documentation during the website buildllms.txt— lightweight router pointing to stack-specific bundlesllms-mapbox.txt/llms-maplibre.txt— link indexes (shared guides + stack API + examples)llms-mapbox-full.txt/llms-maplibre-full.txt— full markdown content per stackllms-full.txt(~3700 lines of near-duplicate Mapbox/MapLibre API pages)examples/{stack}/{name}/README.mdAfter deploy, files will be available at:
Preview
Preview
Test plan
cd website && yarn buildsucceedswebsite/build/llms.txtis a router with no Mapbox/MapLibre API pagesllms-mapbox.txtcontains Mapbox API + 16 examples with blurbs; noapi-reference/maplibrelinksllms-maplibre.txtcontains MapLibre API + 16 examples; noapi-reference/mapboxlinksllms-full.txtgenerated*-releasebranch pushMade with Cursor