How to Get Your Developer Blog Cited in AI Answers

The short answer

AI answer engines cite pages they can crawl, have indexed, and can quote cleanly. For a blog that lives in a GitHub repo, that comes down to four things you control:

1. Let the search crawlers of each AI product read your site in robots.txt, separately from their training crawlers.
2. Get every post indexed by Google and Bing, because AI Overviews and several assistants answer from search indexes.
3. Write each section so its first two or three sentences answer one question on their own.
4. Server-render the text, keep structured data consistent with it, and check the live page, not the build.

None of this needs a special file or a new tool. Google says so directly: "You don't need to create new machine readable files, AI text files, or markup to appear in these features" (Google Search Central, AI features and your website, updated 2025-12-10). Answer engine optimization (AEO) for a developer blog is mostly good technical SEO, plus writing that is easy to lift out and quote.

The rest of this guide goes step by step, with the config and code for a Next.js, Astro or Hugo blog.

What the AI products actually say about citations

Most "get cited by ChatGPT" advice is guesswork. The primary docs are short, so start there.

Google AI Overviews and AI Mode. A page must be "indexed and eligible to be shown in Google Search with a snippet" to appear as a supporting link, and "there are no additional requirements" beyond that. There is "no special schema.org structured data that you need to add." Traffic from these features is counted in Search Console's Performance report under the Web search type (Google).

ChatGPT search (OpenAI). OpenAI runs three agents. OAI-SearchBot surfaces sites in ChatGPT search, and "sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers." GPTBot collects training data and is controlled separately. ChatGPT-User fetches a page when a person asks for it, and robots.txt rules "may not apply" to it. Robots.txt changes take about 24 hours to apply (OpenAI crawlers overview).

Perplexity. PerplexityBot builds the index Perplexity answers from. Its docs recommend allowing it if you want to appear in results, and changes can take up to 24 hours (Perplexity crawlers).

Claude (Anthropic). ClaudeBot collects training data, Claude-SearchBot indexes pages to improve search results, and Claude-User fetches pages when a user asks. Anthropic notes that blocking the search and user agents may reduce your visibility in Claude's answers (Anthropic support).

The pattern is the same everywhere. Each product has a search agent that decides whether you can be cited and a training agent that does not. You can allow one and block the other.

Step 1: Allow the search crawlers in robots.txt

Many developer sites still ship a robots.txt copied from a 2023 "block the AI bots" thread. That blocks the crawlers that would cite you. Decide on training separately, then write the file on purpose.

A robots.txt that allows citation and opts out of training looks like this:

# Search and answer engines: allowed

User-agent: Googlebot

User-agent: Bingbot

User-agent: OAI-SearchBot

User-agent: PerplexityBot

User-agent: Claude-SearchBot

Allow: /



# Model training: opted out

User-agent: GPTBot

User-agent: ClaudeBot

User-agent: Google-Extended

Disallow: /



# Everyone else

User-agent: *

Allow: /

Disallow: /app/



Sitemap: https://example.com/sitemap.xml

If you are happy for your posts to be used in training, a plain User-agent: * / Allow: / covers everything. That is what blogger-dog.com/robots.txt does, with Disallow lines only for the logged-in app.

In Next.js App Router, generate it from app/robots.ts so it ships with every deploy:

// app/robots.ts

import type { MetadataRoute } from "next";



export default function robots(): MetadataRoute.Robots {

  return {

    rules: [

      {

        userAgent: ["Googlebot", "Bingbot", "OAI-SearchBot", "PerplexityBot", "Claude-SearchBot"],

        allow: "/",

      },

      { userAgent: ["GPTBot", "ClaudeBot", "Google-Extended"], disallow: "/" },

      { userAgent: "*", allow: "/", disallow: "/app/" },

    ],

    sitemap: "https://example.com/sitemap.xml",

  };

}

In Astro, Hugo, Jekyll, Gatsby or Docusaurus, a static robots.txt in public/ or static/ does the same job. Hugo can also generate one: set enableRobotsTXT = true and, to customise it, add a layouts/robots.txt template.

Check for a firewall or CDN rule as well. Some hosting dashboards have a one-click "block AI bots" toggle that returns 403 to these agents no matter what robots.txt says. Curl your own post with each user agent:

for ua in OAI-SearchBot PerplexityBot Claude-SearchBot Bingbot; do

  printf "%-18s " "$ua"

  curl -s -o /dev/null -w "%{http_code}\n" -A "Mozilla/5.0 (compatible; $ua)" \

    https://example.com/blog/your-post

done

Every line should print 200. A 403 or a challenge page means the edge is blocking the bot before robots.txt is ever read.

Step 2: Get every post indexed by Google and Bing

You cannot be a supporting link in AI Overviews unless the page is indexed in Google with a snippet. Several assistants also lean on traditional search indexes when they browse, and Bing's index powers Microsoft Copilot's web answers. Indexing is the step most stale developer blogs fail first.

The basics for a repo-backed blog:

• A sitemap with every post and nothing that 404s or redirects. In Next.js, generate app/sitemap.ts from the same function that lists your Markdown files. Only set lastModified when you know it. A build timestamp on every URL teaches crawlers to ignore the field. (Our programmatic SEO guide for the App Router shows a sitemap built from one data file.)
• Plain <a href> links to every post from the blog index and from related posts. A post only reachable through a client-side search box is close to invisible.
• Google Search Console and Bing Webmaster Tools both verified, with the sitemap submitted to each. After you publish an important post, use URL Inspection and request indexing.
• IndexNow for Bing. IndexNow lets you notify participating engines the moment a URL changes. You host a key file at your site root and send one request per deploy (IndexNow documentation):

curl -s -X POST "https://api.indexnow.org/indexnow" \

  -H "Content-Type: application/json; charset=utf-8" \

  -d '{

    "host": "example.com",

    "key": "your-indexnow-key",

    "urlList": ["https://example.com/blog/your-new-post"]

  }'

A 200 or 202 response means the submission was received. Put that call in the CI step that runs after a deploy, and pass it only the URLs that changed.

Step 3: Write sections an answer engine can quote

Once a page is crawlable and indexed, the content decides whether it gets picked. Answer engines pull short passages that directly answer the user's question, then link the source. A passage is easy to quote when it stands on its own.

Write each H2 section like this:

• Question-shaped heading. "How do I add a sitemap to an Astro blog?" matches what people type into ChatGPT better than "Sitemaps".
• Answer in the first two or three sentences. Give the conclusion, the command or the number first, then explain. If someone read only those sentences, they should have the answer.
• One idea per section. A section that answers three questions gets quoted for none of them.
• Specifics over adjectives. "Robots.txt changes take about 24 hours to apply in ChatGPT search" is quotable. "Changes can take a while" is not.
• Name things exactly. Use the real user-agent string, the real config key, the real version. Assistants favor passages that match the entities in the question.
• Show the code. For developer queries, a correct, copy-pasteable snippet is often the whole answer, and it is what gets cited.

Here is the difference on a real question, "Does Next.js generate a sitemap automatically?"

Hard to quote: Sitemaps are an important part of SEO, and modern frameworks offer a range of options. Next.js is no exception, and there are several approaches developers can consider depending on their needs.

Easy to quote: No. Next.js does not create a sitemap by default. Add an app/sitemap.ts file that returns an array of URLs, and Next.js serves it at /sitemap.xml.

The second version answers the question, names the file, and gives the result in under 40 words.

A short FAQ at the end of a post helps for the same reason: each question and answer pair is a self-contained passage. Keep the answers factual and specific. Do not repeat the headings above.

Step 4: Keep structured data honest and consistent

Google says no special schema is needed for AI features. Structured data still helps classic search understand what a page is, and its guidance asks that structured data match the visible content of the page (Google). For a blog post, BlogPosting with a real headline, dates and author is enough:

// app/blog/[slug]/page.tsx (excerpt)

const jsonLd = {

  "@context": "https://schema.org",

  "@type": "BlogPosting",

  headline: post.title,

  description: post.description,

  datePublished: post.date,

  dateModified: post.updated ?? post.date,

  author: { "@type": "Organization", name: "Example" },

  mainEntityOfPage: `https://example.com/blog/${post.slug}`,

};



return (

  <article>

    <script

      type="application/ld+json"

      dangerouslySetInnerHTML={{ __html: JSON.stringify(jsonLd) }}

    />

    {/* post body */}

  </article>

);

Two rules matter more than the markup itself:

• dateModified must be true. Bumping it on every build, without changing the post, is a freshness signal you have not earned. Set it when the content actually changes.
• Headline and description must match the page. If the JSON-LD says one thing and the H1 says another, nothing gains trust.

The Astro blog SEO checklist and the Next.js blog SEO checklist both include full BlogPosting examples for their stacks.

Step 5: Server-render the text and check what bots receive

Google renders JavaScript, but you should not assume every AI crawler does. If your post body or headings only appear after client-side JavaScript runs, some agents will see an empty shell. Static output from Astro, Hugo, Jekyll, Gatsby or Docusaurus is safe by default. In Next.js, keep posts as server components and avoid dynamic(() => import(...), { ssr: false }) for anything that contains text.

Check the raw HTML the way a crawler gets it:

curl -s https://example.com/blog/your-post | grep -c "<h2"

curl -s https://example.com/blog/your-post | grep -o "<title>[^<]*</title>"

If the h2 count is 0 while the browser shows several headings, the text is client-rendered. Also check that the page returns 200, has a self-referencing canonical, and does not carry a stray noindex from a preview environment.

Step 6: Keep publishing, and keep old posts accurate

A blog with one post from last spring is rarely the best source for anything. Answer engines and search engines both favor sites that cover a topic in depth and keep it current. For a small SaaS team, that means:

• A steady cadence. One useful post a week builds topical depth much faster than a burst of ten followed by silence.
• Clusters, not one-offs. Write several posts around the questions your buyers ask, and link them to each other with descriptive anchor text.
• Updates to posts that rank. When a framework API changes, fix the post, update dateModified, and say what changed.

This is where most developer blogs stall. The setup above takes an afternoon. Writing and shipping a cited, well-structured post every week, for months, is the hard part.

Step 7: Measure AI referrals

You cannot see "AI citations" in one dashboard, but you can see their traffic.

• Google Search Console: AI Overviews and AI Mode clicks are included in the Performance report under Web search. They are not split out, so watch impressions and clicks on the question-shaped queries your posts answer.
• Referrers in your analytics: look for chatgpt.com, perplexity.ai, claude.ai, copilot.microsoft.com and gemini.google.com. Links from ChatGPT search often arrive with utm_source=chatgpt.com.
• Server logs: grep for OAI-SearchBot, PerplexityBot and Claude-SearchBot to confirm they fetch your new posts after you publish.

grep -E "OAI-SearchBot|PerplexityBot|Claude-SearchBot" access.log \

  | awk '{print $7}' | sort | uniq -c | sort -rn | head

If the bots never request a post, go back to Steps 1 and 2. If they fetch it but you get no referrals, the content is the problem: go back to Step 3.

What you can skip

• llms.txt is optional. Google says it does not need AI text files, and the evidence that assistants rely on it is thin. It is cheap to add, and our llms.txt guide for developer blogs shows how, but it will not get an unindexed post cited.
• "AEO schema" plugins that add FAQ markup to every page. Use FAQPage markup only where there is a visible FAQ.
• Rewriting every post for AI. Fix the crawl and index problems first. They affect every page at once.

The checklist

• robots.txt allows Googlebot, Bingbot, OAI-SearchBot, PerplexityBot and Claude-SearchBot; training bots decided on purpose
• No CDN or firewall rule returns 403 to those agents (curl each one)
• Sitemap lists every post, with honest lastModified
• Every post linked with a plain <a href> from the blog index
• Search Console and Bing Webmaster Tools verified, sitemap submitted; IndexNow ping on deploy
• Question-shaped H2s, each answered in its first two or three sentences
• Specific numbers, exact names and working code in the answers
• Server-rendered text; curl shows the headings
• BlogPosting JSON-LD that matches the page, with a truthful dateModified
• A weekly publishing cadence, plus updates to posts that rank
• AI referrers and bot fetches tracked

FAQ

Do I need to block GPTBot to stay out of training but still appear in ChatGPT?

Yes, that combination works. OpenAI treats GPTBot (training) and OAI-SearchBot (search) as independent settings, so you can disallow the first and allow the second. Changes take about 24 hours to apply.

Does Google need special markup for AI Overviews?

No. Google says there are no additional requirements and no special schema.org markup. The page needs to be indexed and eligible to show in Search with a snippet.

How long until a new post can be cited?

It depends on when each engine crawls and indexes it. Submitting the sitemap, requesting indexing in Search Console and sending an IndexNow ping all shorten the wait. Expect days for an established site and longer for a new one.

Does a static site have an advantage?

For crawling, yes. Static HTML from Astro, Hugo, Jekyll, Gatsby or Docusaurus gives every crawler the full text without running JavaScript. What gets you cited is still the content and the indexing.

Which stacks does Blogger Dog publish to?

Next.js, Astro, Hugo, Jekyll, Gatsby and Docusaurus blogs in GitHub or GitLab repos. Each integration page explains where posts land and which frontmatter is written.

Keep the posts coming

The technical part of AEO is a one-time fix. The part that compounds is publishing well-structured, cited posts every week. Blogger Dog researches each topic, writes the post with sources and answer-first sections, commits it as Markdown into your repo, and only counts it once the public URL is live.

You get 3 posts free, then plans start at $19/month for 10 posts. See pricing and plans.