---
title: "Markdown mirrors"
description: "Every page on this site has a .md twin at the same path, so an AI agent can read the content instead of parsing the layout."
canonical: https://yonnia.com/en/projects/markdown-mirrors
language: en
status: active
year: 2026
stack: ["Node.js"]
---
# Markdown mirrors

Every page on this site has a .md twin at the same path, so an AI agent can read the content instead of parsing the layout.

Append `.md` to any URL on this site and you get the same page as clean Markdown. The footer link on each page points at it.

## The problem it solves

An increasing share of the traffic that matters is not a browser. It is a model fetching the page to answer a question. That client pays the full cost of my layout — nav, footer, breadcrumbs, structured data — to extract the one thing it wants, which is the text.

The mirror is the same content with the chrome removed: front matter with the canonical URL and dates, then the body. It is generated from the same source in the same build, so it cannot drift out of sync with the HTML.

## The one decision worth explaining

The obvious way to keep mirrors out of search results is `Disallow: /*.md$` in robots.txt. That is wrong, and I wrote it that way first before catching it.

`Disallow` blocks *fetching*. It would block the exact agents the files exist for, while doing nothing that a `noindex` does not do better. Search engines would still not index them, and the AI clients would get a 403-shaped answer from a file built specifically for them.

The correct tool is an `X-Robots-Tag: noindex, nofollow` header on `/*.md`, which is what the Cloudflare Pages `_headers` file does. The files stay fetchable and stay out of the index. Two separate questions — *can you read this* and *should this appear in results* — and one directive for each.

## Also there

`/llms.txt` at the root: a single plain-text index of every page with its description, so an agent can see the whole site without crawling it.