How we build AI-readable websites
Search engines have had structured-data conventions for over a decade. AI models and agents are still catching up — and most websites are invisible to them by default, hidden behind JavaScript that never gets rendered. This page is the technique-by-technique breakdown of how we fix that, using this exact site as the example.
1. Structured data (JSON-LD) on every page
Every page carries schema.org structured data as a plain, invisible <script type="application/ld+json"> tag — Organization on every page, Service on each service page, FAQPage anywhere there's an FAQ, BreadcrumbList for navigation context, Article for blog posts, and Person for team members. This is the same mechanism Google has used for rich results for years; AI agents increasingly parse it too, because it's unambiguous — no inference required to know a price, a rating, or an author.
We generate these from typed builder functions in lib/schema.ts, sourced from the same data files (lib/data/*.ts) that render the human-visible page — so the structured data can never say something different from what a visitor actually sees.
// lib/schema.ts
export function serviceSchema(service) {
return {
'@context': 'https://schema.org',
'@type': 'Service',
name: service.title,
description: service.longDesc,
offers: {
'@type': 'Offer',
priceCurrency: 'GHS',
priceSpecification: {
minPrice: service.priceFrom,
maxPrice: service.priceTo,
},
},
};
}2. A generated llms.txt at the site root
/llms.txt is a plain-text summary of the entire site — company overview, every service with pricing, the full portfolio, blog index, FAQ, and contact details — written for an LLM to read in one pass, no rendering required. It follows the emerging llms.txt convention.
Ours is generated at request time from the same service, portfolio, and pricing data as the rest of the site, via a Next.js route handler — not hand-maintained, so it can never go stale.
// app/llms.txt/route.ts
export async function GET() {
return new Response(buildLlmsTxt(), {
headers: { 'Content-Type': 'text/plain; charset=utf-8' },
});
}3. A markdown mirror of every page
Every page on this site has a plain-Markdown twin at the same URL plus .md — for example /about.md or /services/ai-chatbots.md. A Next.js rewrite routes any *.md request to a handler that renders the page's content as clean Markdown, with no CSS, navigation chrome, or client-side JavaScript to execute — ideal for simple fetch-based agents that don't run a browser.
// next.config.mjs
async rewrites() {
return [
{ source: '/:path*.md', destination: '/api/markdown/:path*' },
];
}4. A dedicated JSON API for agents
/api/crawl returns the entire business — services, pricing, portfolio, blog, FAQ, contact — as one structured JSON document, CORS-enabled, cached, and versioned. This is for agents that would rather call an API than parse HTML or Markdown at all: a shopping assistant comparing agencies, a RAG pipeline indexing vendor data, or an autonomous browsing agent fact-checking a claim about us.
GET https://webwavegh.eu.cc/api/crawl
{
"organization": { "name": "WebWave Ghana", ... },
"services": [ { "slug": "ai-chatbots", "priceRange": {...} } ],
"portfolio": [ ... ],
"faq": [ ... ]
}5. Semantic HTML, not div soup
Case studies render as <article>, sections use <section>, navigation uses <nav aria-label="Breadcrumb">, and heading levels follow a real document outline (one h1 per page, nested h2/h3). This matters for AI readability for the same reason it matters for accessibility: a parser that understands document structure needs far less guesswork than one reading an undifferentiated tree of styled <div>s.
<article>
<nav aria-label="Breadcrumb">Home / Portfolio / Kumasi AI Tutor</nav>
<h1>Kumasi AI Tutor</h1>
<section>
<h2>The problem</h2>
<p>...</p>
</section>
</article>7. A transparent AI-Readability Score
The badge you see on portfolio pages isn't a marketing prop with a hardcoded number — it's computed from five inspectable, equally-weighted criteria: JSON-LD present, structured fields complete, a markdown mirror exists, semantic HTML is used, and metadata is specific rather than boilerplate. No industry standard for this exists yet; we built it as an honest internal checklist that doubles as a way to show clients exactly what "AI-readable" means in practice, rather than asking them to take our word for it.
const CRITERIA = [
{ key: 'hasJsonLd', points: 30 },
{ key: 'hasStructuredFields', points: 25 },
{ key: 'hasMarkdownMirror', points: 20 },
{ key: 'hasSemanticHtml', points: 15 },
{ key: 'hasDescriptiveMetadata', points: 10 },
];8. An AI-crawler-aware robots.txt and a live sitemap
Our robots.txt explicitly allows the major AI-agent crawlers — GPTBot, ClaudeBot, PerplexityBot, Google-Extended, and others — by name, alongside the standard wildcard rule. The sitemap is generated at build/request time from the same route data as the rest of the site (services, portfolio, blog posts), so newly published pages are never missing from it.
const AI_AGENTS = [
'GPTBot', 'ClaudeBot', 'PerplexityBot',
'Google-Extended', 'CCBot', 'anthropic-ai',
];9. Multilingual readiness
The /locations/accra page ships with an hreflang alternate slot already wired up for a future Twi (Akan) translation at /tw/locations/accra — commented out until the translation exists, but the routing and metadata structure is in place so adding a language later is a content task, not an architecture change.
alternates: {
languages: {
'en-GH': 'https://webwavegh.eu.cc/locations/accra',
// 'tw-GH': 'https://webwavegh.eu.cc/tw/locations/accra',
},
}Want this for your website?
Every technique on this page ships as standard practice on WebWave Ghana projects — not an upsell.
Start a project