llms.txt for Developer Blogs: What It Does and How to Ship One

The short answer

llms.txt is a plain Markdown file at the root of your site that tells AI agents what the site is and which pages matter most. For a developer blog it takes about twenty lines of code to generate from the same Markdown files that already render your posts, it costs nothing to serve, and it will not move your Google rankings.

That last part matters, because a lot of advice about llms.txt in 2026 oversells it. This guide covers what the spec actually says, what the adoption data shows, where the file helps a small developer blog, and the code to ship one on Next.js or Astro without maintaining it by hand.

What the llms.txt spec says

The format was proposed by Jeremy Howard in September 2024 and is documented at llmstxt.org. The rules are short:

• The file lives at /llms.txt (a subpath such as /docs/llms.txt is allowed and covers URLs under it).
• The only required element is an H1 with the name of the site or project.
• After the H1 comes a blockquote with a one-paragraph summary, then optional paragraphs or lists with more detail.
• Then zero or more H2 sections that are "file lists": Markdown list items, each a link with an optional note after a colon.
• An H2 named Optional holds links an agent can skip when it needs a shorter context.

The spec also suggests that pages worth reading should have a clean Markdown version at the same URL with .md added, so an agent can fetch your post without the navigation, scripts and footer. For a blog whose posts already are Markdown files in a repo, that part is unusually cheap, which we come back to below.

Here is a minimal, valid llms.txt for a small SaaS with a blog:

# Acme Deploy



> Acme Deploy is a preview-environment tool for Next.js teams. This file lists the docs and blog posts most useful for understanding the product.



Pricing starts at $20/month. Docs are versioned with the product.



## Docs



- [Getting started](https://acme.dev/docs/getting-started): install and first preview in five minutes

- [Pricing](https://acme.dev/pricing): plans and limits



## Blog



- [Preview environments for monorepos](https://acme.dev/blog/monorepo-previews): how we isolate per-package builds

- [Cutting preview cold starts](https://acme.dev/blog/cold-starts): measured results from 40 repos



## Optional



- [Changelog](https://acme.dev/changelog)

What the 2026 data actually shows

The honest picture has three parts, and you should know all of them before spending time on this.

Adoption grew fast. Originality.ai, which monitors more than 3 million websites, counted 36,120 sites with an llms.txt file in May 2026, up 8.8x from about 4,000 in June 2025 (PPC Land summary).

Almost nobody reads them yet. The same report cites Ahrefs server-log data from 137,000 domains: 97% of llms.txt files received zero requests in May 2026, and AI retrieval bots made up only 1.1% of the requests that did arrive. SEO audit tools were a much larger share.

Google treats it as an agent feature, not a search signal. On May 5, 2026, Chrome Lighthouse added llms.txt to a new set of agentic browsing audits (PPC Land). The audit only fails if your server returns an error for the file; a 404 is marked not applicable, because the file is optional. Separately, Google's John Mueller described llms.txt on Reddit as "purely speculative for now" and said AI systems were not using it (TechWyse report).

So: it is not a ranking factor, most crawlers ignore it today, and the one Google tool that checks it only cares that it does not error. If someone sells you llms.txt as an AI-traffic growth lever, that is not what the data says.

Why ship one anyway

For a developer blog the case is about cost, not upside. Three reasons hold up:

• It is nearly free. If you generate it from your posts, there is nothing to maintain. A static text route costs nothing to serve.
• Agents that do read it get a better summary. Some agent tools look for a site's llms.txt when a user points them at a domain, and Lighthouse now checks it. When a tool does read it, a hand-picked list of your best posts beats a crawl of your navigation.
• The Markdown versions are useful on their own. Serving /blog/my-post.md gives any tool, including your own scripts, a clean copy of the post. For a Markdown blog that is the source file you already have.

What it does not replace: the basics that get a post discovered and cited in the first place. Server-rendered HTML, a correct canonical, a sitemap that includes every post, and plain links between posts matter far more. Our Next.js blog SEO checklist covers those. Do that first, then add llms.txt in ten minutes. For what does move AI citations, see how to get a developer blog cited in AI answers.

Ship it on Next.js (App Router)

On the App Router, a folder named llms.txt with a route.ts inside serves the file. Read the posts with the same loader your blog pages use, so the file can never drift out of sync with what is published.

// app/llms.txt/route.ts

import { getBlogPosts } from "@/lib/blog-content";



const SITE = "https://example.com";



export const dynamic = "force-static";



export function GET() {

  const posts = getBlogPosts().slice(0, 30);

  const lines = [

    "# Example",

    "",

    "> Example is a tool for X. This file lists the pages most useful for understanding it.",

    "",

    "## Product",

    `- [Pricing](${SITE}/pricing): plans and what each includes`,

    "",

    "## Blog",

    ...posts.map((post) => `- [${post.title}](${SITE}/blog/${post.slug}.md): ${post.description}`),

    "",

    "## Optional",

    `- [All posts](${SITE}/blog)`,

  ];

  return new Response(lines.join("\n"), {

    headers: { "content-type": "text/plain; charset=utf-8" },

  });

}

Two details are worth copying. First, force-static makes Next.js render the file at build time, so it is just a static asset. Second, the description after each link comes straight from the post frontmatter; if your descriptions are vague, this file will show it, which is a useful nudge to write better ones.

Blogger-dog.com serves its own llms.txt from a route like this. Ours started as a product and API description for agents, without a blog section, which is the most common gap we see: the file exists but lists none of the content that answers real questions.

Serve each post as Markdown

The spec's .md convention fits a Markdown blog well. Next.js does not allow a dynamic segment with a fixed suffix like [slug].md, so the simplest approach is a rewrite to a route handler that returns the raw file.

// next.config.mjs

export default {

  async rewrites() {

    return [{ source: "/blog/:slug.md", destination: "/api/post-markdown/:slug" }];

  },

};

// app/api/post-markdown/[slug]/route.ts

import { getBlogPost } from "@/lib/blog-content";



// Next.js 15+: params is a Promise. On Next.js 13/14, drop the Promise and the await.

export async function GET(_req: Request, { params }: { params: Promise<{ slug: string }> }) {

  const { slug } = await params;

  const post = getBlogPost(slug);

  if (!post) return new Response("Not found", { status: 404 });

  return new Response(`# ${post.title}\n\n${post.body}\n`, {

    headers: { "content-type": "text/markdown; charset=utf-8" },

  });

}

Keep the canonical on the HTML page. The Markdown copy is for agents; you do not want it competing with the real post in search, so add an X-Robots-Tag: noindex header to the Markdown response if you are cautious, and do not put the .md URLs in your sitemap.

Ship it on Astro

Astro serves any file in src/pages as an endpoint, including src/pages/llms.txt.ts. With content collections the whole thing is a few lines:

// src/pages/llms.txt.ts

import type { APIRoute } from "astro";

import { getCollection } from "astro:content";



export const GET: APIRoute = async ({ site }) => {

  const posts = (await getCollection("blog", ({ data }) => !data.draft))

    .sort((a, b) => b.data.pubDate.valueOf() - a.data.pubDate.valueOf())

    .slice(0, 30);

  const base = site?.toString().replace(/\/$/, "") ?? "";

  const body = [

    "# Example",

    "",

    "> Example is a tool for X. This file lists the pages most useful for understanding it.",

    "",

    "## Blog",

    ...posts.map((p) => `- [${p.data.title}](${base}/blog/${p.id}/): ${p.data.description}`),

  ].join("\n");

  return new Response(body, { headers: { "content-type": "text/plain; charset=utf-8" } });

};

Set site in astro.config.mjs so the links are absolute. On a static build, Astro writes the output to dist/llms.txt at build time. Adjust pubDate, draft and the collection name to match your schema; those follow the default blog template. If you want to see how Blogger Dog writes into an Astro content collection without breaking your schema, the Astro integration page lists the paths and frontmatter it supports.

Hugo and Jekyll, briefly

Static generators without code endpoints can still produce the file from a template. In Hugo, define a custom output format with baseName = "llms", mediaType = "text/plain" and isPlainText = true, add it to the home page outputs, and range over .Site.RegularPages in layouts/index.llms.txt. In Jekyll, a file at the root named llms.txt with empty front matter (two lines of three dashes) is processed by Liquid, so you can loop over site.posts in it. Either way the rule is the same: generate it from your content, never hand-maintain it.

Check it after every deploy

A file nobody checks will break quietly, and the only thing the Lighthouse audit cares about is that it does not return an error. Add one line to whatever post-deploy check you run:

curl -sf https://example.com/llms.txt | head -5 || echo "llms.txt is broken"

It is also worth confirming that every link in the file resolves, because a post you renamed or redirected will otherwise sit there as a dead entry:

curl -s https://example.com/llms.txt \

  | grep -o 'https://[^)]*' \

  | while read -r url; do

      code=$(curl -s -o /dev/null -w '%{http_code}' "$url")

      [ "$code" = "200" ] || echo "$code $url"

    done

This is the same idea we apply to publishing in general: a post is not done until its public URL returns the real page. We wrote about that in live URL verification for GitHub blog publishing.

What to put in it, and what to leave out

The file is a curated index, not a sitemap. A few rules keep it useful:

• Lead with the summary. The blockquote is often the only part an agent keeps. Say what the product does and for whom, in one or two plain sentences.
• List your best 10 to 30 pages, not all of them. Pricing, getting started, and the posts that explain how your product works or answer the questions customers ask most.
• Write descriptions that state a fact. "How we cut preview cold starts by 60% across 40 repos" helps an agent; "Thoughts on performance" does not.
• Put archive and legal pages under Optional, or leave them out.
• Do not stuff keywords. Nothing ranks on this file. It is read by tools that summarise; repetition only makes the summary worse.

FAQ

Is llms.txt a ranking factor in Google?

No. Google has not said it uses the file for Search, its Search Advocate has called it speculative, and the Lighthouse audit added in May 2026 only checks that the file does not return a server error. Treat it as agent documentation, not SEO.

Do I need llms-full.txt as well?

Only if you have documentation an agent might want in one fetch. llms-full.txt is a convention for concatenating the full Markdown of your key pages into a single file. For a blog with a few dozen posts, linking the per-post .md versions is usually enough.

Should llms.txt be blocked or allowed in robots.txt?

Allowed. Blocking it defeats the purpose, and robots.txt rules for AI crawlers are a separate decision about whether they may read your HTML pages at all.

Will an llms.txt get my posts cited in ChatGPT or Perplexity?

There is no evidence of that today; the server-log data shows AI retrieval bots rarely fetch the file. Citations come from posts that are crawlable, specific, well-sourced and linked from elsewhere. Our guide to GitHub blog publishing for SaaS teams covers the publishing habits that make that possible.

Keep the posts coming

An llms.txt file is only as good as the posts it lists, and on most developer-run blogs the real problem is that the last post is months old. Blogger Dog researches and writes cited SEO and AEO posts, commits them straight into your Next.js, Astro, Hugo, Jekyll, Gatsby or Docusaurus repo in the format your site already reads, and only counts a post once its public URL is live. If you generate llms.txt from your content as shown above, every new post shows up there automatically.

You get 3 posts free, then plans start at $19/month. See pricing and plans.