Under the hood

How we build AI-readable websites

Search engines have had structured-data conventions for over a decade. AI models and agents are still catching up — and most websites are invisible to them by default, hidden behind JavaScript that never gets rendered. This page is the technique-by-technique breakdown of how we fix that, using this exact site as the example.

1. Structured data (JSON-LD) on every page

Every page carries schema.org structured data as a plain, invisible <script type="application/ld+json"> tag — Organization on every page, Service on each service page, FAQPage anywhere there's an FAQ, BreadcrumbList for navigation context, Article for blog posts, and Person for team members. This is the same mechanism Google has used for rich results for years; AI agents increasingly parse it too, because it's unambiguous — no inference required to know a price, a rating, or an author.

We generate these from typed builder functions in lib/schema.ts, sourced from the same data files (lib/data/*.ts) that render the human-visible page — so the structured data can never say something different from what a visitor actually sees.

// lib/schema.ts
export function serviceSchema(service) {
  return {
    '@context': 'https://schema.org',
    '@type': 'Service',
    name: service.title,
    description: service.longDesc,
    offers: {
      '@type': 'Offer',
      priceCurrency: 'GHS',
      priceSpecification: {
        minPrice: service.priceFrom,
        maxPrice: service.priceTo,
      },
    },
  };
}

2. A generated llms.txt at the site root

/llms.txt is a plain-text summary of the entire site — company overview, every service with pricing, the full portfolio, blog index, FAQ, and contact details — written for an LLM to read in one pass, no rendering required. It follows the emerging llms.txt convention.

Ours is generated at request time from the same service, portfolio, and pricing data as the rest of the site, via a Next.js route handler — not hand-maintained, so it can never go stale.

// app/llms.txt/route.ts
export async function GET() {
  return new Response(buildLlmsTxt(), {
    headers: { 'Content-Type': 'text/plain; charset=utf-8' },
  });
}

3. A markdown mirror of every page

Every page on this site has a plain-Markdown twin at the same URL plus .md — for example /about.md or /services/ai-chatbots.md. A Next.js rewrite routes any *.md request to a handler that renders the page's content as clean Markdown, with no CSS, navigation chrome, or client-side JavaScript to execute — ideal for simple fetch-based agents that don't run a browser.

// next.config.mjs
async rewrites() {
  return [
    { source: '/:path*.md', destination: '/api/markdown/:path*' },
  ];
}

4. A dedicated JSON API for agents

/api/crawl returns the entire business — services, pricing, portfolio, blog, FAQ, contact — as one structured JSON document, CORS-enabled, cached, and versioned. This is for agents that would rather call an API than parse HTML or Markdown at all: a shopping assistant comparing agencies, a RAG pipeline indexing vendor data, or an autonomous browsing agent fact-checking a claim about us.

GET https://webwavegh.eu.cc/api/crawl

{
  "organization": { "name": "WebWave Ghana", ... },
  "services": [ { "slug": "ai-chatbots", "priceRange": {...} } ],
  "portfolio": [ ... ],
  "faq": [ ... ]
}

5. Semantic HTML, not div soup

Case studies render as <article>, sections use <section>, navigation uses <nav aria-label="Breadcrumb">, and heading levels follow a real document outline (one h1 per page, nested h2/h3). This matters for AI readability for the same reason it matters for accessibility: a parser that understands document structure needs far less guesswork than one reading an undifferentiated tree of styled <div>s.

<article>
  <nav aria-label="Breadcrumb">Home / Portfolio / Kumasi AI Tutor</nav>
  <h1>Kumasi AI Tutor</h1>
  <section>
    <h2>The problem</h2>
    <p>...</p>
  </section>
</article>

6. Hidden, structured case-study data

Each portfolio page carries a CreativeWork JSON-LD block with an additionalProperty bag for fields schema.org has no native slot for — tech stack, client industry, the problem, the solution, and the measured result. It sits alongside identical content that's fully visible in the page body; nothing here is display:none-hidden from a human reader. The point is giving an agent a clean, typed field to read instead of extracting "40% conversion increase" out of a paragraph of prose.

additionalProperty: [
  { name: 'clientIndustry', value: 'Grocery & Retail' },
  { name: 'problem', value: '...' },
  { name: 'result', value: 'Conversion rate increased 40%...' },
  { name: 'techStack', value: 'React, Node.js, MoMo API, PostgreSQL' },
]

7. A transparent AI-Readability Score

The badge you see on portfolio pages isn't a marketing prop with a hardcoded number — it's computed from five inspectable, equally-weighted criteria: JSON-LD present, structured fields complete, a markdown mirror exists, semantic HTML is used, and metadata is specific rather than boilerplate. No industry standard for this exists yet; we built it as an honest internal checklist that doubles as a way to show clients exactly what "AI-readable" means in practice, rather than asking them to take our word for it.

const CRITERIA = [
  { key: 'hasJsonLd', points: 30 },
  { key: 'hasStructuredFields', points: 25 },
  { key: 'hasMarkdownMirror', points: 20 },
  { key: 'hasSemanticHtml', points: 15 },
  { key: 'hasDescriptiveMetadata', points: 10 },
];

8. An AI-crawler-aware robots.txt and a live sitemap

Our robots.txt explicitly allows the major AI-agent crawlers — GPTBot, ClaudeBot, PerplexityBot, Google-Extended, and others — by name, alongside the standard wildcard rule. The sitemap is generated at build/request time from the same route data as the rest of the site (services, portfolio, blog posts), so newly published pages are never missing from it.

const AI_AGENTS = [
  'GPTBot', 'ClaudeBot', 'PerplexityBot',
  'Google-Extended', 'CCBot', 'anthropic-ai',
];

9. Multilingual readiness

The /locations/accra page ships with an hreflang alternate slot already wired up for a future Twi (Akan) translation at /tw/locations/accra — commented out until the translation exists, but the routing and metadata structure is in place so adding a language later is a content task, not an architecture change.

alternates: {
  languages: {
    'en-GH': 'https://webwavegh.eu.cc/locations/accra',
    // 'tw-GH': 'https://webwavegh.eu.cc/tw/locations/accra',
  },
}

Want this for your website?

Every technique on this page ships as standard practice on WebWave Ghana projects — not an upsell.

Start a project