Documentation sites, blogs, and knowledge bases need a content format that balances authoring ergonomics with rendering flexibility. Plain Markdown is easy to write but limited—it cannot embed interactive components, data visualizations, or custom callout boxes. Raw React components are fully flexible but terrible for writing long-form prose. MDX bridges this gap by extending Markdown with JSX: authors write natural Markdown for headings, paragraphs, lists, and code blocks, then drop in React components wherever they need interactivity or custom formatting.
The power of MDX comes from the unified ecosystem: a pipeline of remark plugins (operating on Markdown AST) and rehype plugins (operating on HTML AST) that transform content between parsing and rendering. remark-gfm adds GitHub Flavored Markdown extensions like tables and task lists. rehype-slug generates anchor IDs for headings. rehype-autolink-headings wraps those headings in clickable links. gray-matter extracts YAML frontmatter into structured metadata. Together, these plugins form a content pipeline that turns a directory of .mdx files into a fully featured documentation site.
A documentation site can use this pipeline to power its public-facing documentation, blog, and changelog. Content authors write MDX files with frontmatter metadata, the build pipeline compiles them with next-mdx-remote’s compileMDX, and the resulting React components render inside the Next.js App Router with full access to the site’s design system components.
The unified ecosystem processes content through three phases. Parsing converts raw text to an abstract syntax tree (AST): remark-parse produces an MDAST (Markdown AST). Transformation applies plugins that modify the AST: remark plugins work on MDAST, then remark-rehype converts to HAST (HTML AST), and rehype plugins work on HAST. Stringifying converts the final AST to output: rehype-stringify produces HTML strings, or in MDX’s case, the AST is compiled to a React component function.
YAML frontmatter at the top of an MDX file (between --- delimiters) defines structured metadata: title, date, author, tags, description, and any custom fields. gray-matter parses this block and separates it from the content body. This metadata drives page titles, SEO tags, blog post listings, and content filtering without requiring a separate CMS or database.
remark-gfm extends the Markdown parser to support GitHub’s syntax extensions: pipe tables, task list checkboxes (- [x] done), strikethrough (~~text~~), and autolinked URLs. These features are expected by developers who write Markdown on GitHub daily. Without this plugin, these syntaxes render as literal text rather than their intended HTML elements.
rehype-slug generates URL-friendly id attributes for every heading element (e.g., <h2 id="getting-started">). rehype-autolink-headings inserts an anchor link inside or around each heading that points to its own id, enabling users to click a heading to get a shareable deep link. Together, they make every section of a documentation page individually linkable—essential for support teams, changelogs, and long reference docs.
next-mdx-remote compiles MDX content at runtime (in a server component or at build time via generateStaticParams) rather than at webpack compilation time. compileMDX takes a raw MDX string, applies the remark/rehype plugin chain, and returns a serialized React component plus extracted frontmatter. The component is rendered on the client with a components prop that maps HTML element names to custom React components—this is how <h1> becomes a styled heading and <pre> becomes a syntax-highlighted code block.
Each content file combines YAML metadata with MDX body content. The frontmatter is extracted before compilation; the body is processed through the remark/rehype pipeline.
---
title: "Building Search Indexes with MySearchDB"
description: "A step-by-step guide to creating, configuring, and populating search indexes"
date: "2026-03-15"
author: "Engineering Team"
tags: ["search", "indexing", "getting-started"]
category: "guides"
---
# Building Search Indexes with MySearchDB
Search indexes are the foundation of any search experience. This guide walks
through creating an index, defining field mappings, and ingesting documents.
## Prerequisites
Before starting, make sure you have:
- [x] A search database account with API access
- [x] The `mydb` CLI installed (`npm install -g @mydb/cli`)
- [ ] At least 100 sample documents to index
## Creating an Index
<Callout type="info">
Index names must be lowercase alphanumeric with hyphens. They cannot be
changed after creation.
</Callout>
```bash
mydb index create product-search \
--primary-key id \
--max-documents 1000000
```
<ApiPlayground endpoint="/v1/indexes" method="POST" />The compilation function wires together gray-matter for frontmatter extraction, remark plugins for Markdown processing, and rehype plugins for HTML-level transformations.
// lib/mdx.ts
import { compileMDX } from "next-mdx-remote/rsc";
import remarkGfm from "remark-gfm";
import rehypeSlug from "rehype-slug";
import rehypeAutolinkHeadings from "rehype-autolink-headings";
import rehypePrism from "rehype-prism-plus";
import { mdxComponents } from "@/components/mdx";
export interface Frontmatter {
title: string;
description: string;
date: string;
author: string;
tags: string[];
category: string;
}
export async function compileDoc(source: string) {
const { content, frontmatter } = await compileMDX<Frontmatter>({
source,
options: {
parseFrontmatter: true,
mdxOptions: {
remarkPlugins: [
remarkGfm, // tables, task lists, strikethrough, autolinks
],
rehypePlugins: [
rehypeSlug, // add id attributes to headings
[rehypeAutolinkHeadings, {
behavior: "wrap", // wrap heading text in an <a> tag
properties: {
className: ["heading-anchor"],
ariaLabel: "Link to this section",
},
}],
[rehypePrism, {
ignoreMissing: true, // don't throw on unknown languages
showLineNumbers: true,
}],
],
},
},
components: mdxComponents,
});
return { content, frontmatter };
}The component map replaces default HTML elements with styled React components and adds custom elements like <Callout> and <ApiPlayground> that MDX authors can use inline.
// components/mdx/index.tsx
import type { MDXComponents } from "mdx/types";
import { Callout } from "./Callout";
import { ApiPlayground } from "./ApiPlayground";
import { CodeBlock } from "./CodeBlock";
import { Heading } from "./Heading";
import Image from "next/image";
export const mdxComponents: MDXComponents = {
// Override default HTML elements with styled versions
h1: (props) => <Heading level={1} {...props} />,
h2: (props) => <Heading level={2} {...props} />,
h3: (props) => <Heading level={3} {...props} />,
pre: ({ children, ...props }) => (
<CodeBlock {...props}>{children}</CodeBlock>
),
img: ({ src, alt, ...props }) => (
<Image
src={src ?? ""}
alt={alt ?? ""}
width={800}
height={400}
className="rounded-lg"
{...props}
/>
),
table: (props) => (
<div className="table-wrapper">
<table {...props} />
</div>
),
// Custom components available in MDX files
Callout,
ApiPlayground,
};A set of utility functions reads MDX files from the filesystem, extracts frontmatter with gray-matter for listing pages (without compiling the full MDX), and loads individual documents for rendering.
// lib/content.ts
import fs from "fs/promises";
import path from "path";
import matter from "gray-matter";
import { compileDoc, Frontmatter } from "./mdx";
const CONTENT_DIR = path.join(process.cwd(), "content");
export interface ContentEntry {
slug: string;
frontmatter: Frontmatter;
}
/**
* List all content entries in a category directory.
* Uses gray-matter for fast frontmatter extraction without full MDX compilation.
*/
export async function listContent(category: string): Promise<ContentEntry[]> {
const dir = path.join(CONTENT_DIR, category);
const files = await fs.readdir(dir);
const entries = await Promise.all(
files
.filter((f) => f.endsWith(".mdx"))
.map(async (filename) => {
const raw = await fs.readFile(path.join(dir, filename), "utf-8");
const { data } = matter(raw);
return {
slug: filename.replace(/\.mdx$/, ""),
frontmatter: data as Frontmatter,
};
})
);
// Sort by date, newest first
return entries.sort(
(a, b) =>
new Date(b.frontmatter.date).getTime() -
new Date(a.frontmatter.date).getTime()
);
}
/**
* Load and compile a single MDX document by category and slug.
*/
export async function loadContent(category: string, slug: string) {
const filePath = path.join(CONTENT_DIR, category, `${slug}.mdx`);
const raw = await fs.readFile(filePath, "utf-8");
return compileDoc(raw);
}
/**
* Generate static params for all content in a category.
* Used by Next.js generateStaticParams for ISR.
*/
export async function getContentSlugs(category: string) {
const dir = path.join(CONTENT_DIR, category);
const files = await fs.readdir(dir);
return files
.filter((f) => f.endsWith(".mdx"))
.map((f) => ({ slug: f.replace(/\.mdx$/, "") }));
}The page component loads and renders the compiled MDX content as a React Server Component, with full static generation support.
// app/docs/[category]/[slug]/page.tsx
import { notFound } from "next/navigation";
import { loadContent, getContentSlugs } from "@/lib/content";
interface DocPageProps {
params: Promise<{ category: string; slug: string }>;
}
export default async function DocPage({ params }: DocPageProps) {
const { category, slug } = await params;
try {
const { content, frontmatter } = await loadContent(category, slug);
return (
<article className="prose">
<header>
<h1>{frontmatter.title}</h1>
<p className="description">{frontmatter.description}</p>
<time dateTime={frontmatter.date}>
{new Date(frontmatter.date).toLocaleDateString("en-US", {
year: "numeric",
month: "long",
day: "numeric",
})}
</time>
<div className="tags">
{frontmatter.tags.map((tag) => (
<span key={tag} className="tag">{tag}</span>
))}
</div>
</header>
<div className="content">{content}</div>
</article>
);
} catch {
notFound();
}
}
export async function generateStaticParams() {
const categories = ["guides", "reference", "blog"];
const params = await Promise.all(
categories.map(async (category) => {
const slugs = await getContentSlugs(category);
return slugs.map((s) => ({ category, ...s }));
})
);
return params.flat();
}
export async function generateMetadata({ params }: DocPageProps) {
const { category, slug } = await params;
try {
const { frontmatter } = await loadContent(category, slug);
return {
title: frontmatter.title,
description: frontmatter.description,
};
} catch {
return { title: "Not Found" };
}
}A documentation site can use this pipeline to maintain its entire documentation—API reference, getting-started guides, a changelog, and a blog—as a directory of MDX files in the repository. Content changes go through the same pull request workflow as code changes, with Markdown preview in the GitHub PR diff. The build pipeline compiles all MDX at build time via generateStaticParams, producing static HTML that deploys to a CDN with near-zero server costs.
The custom component mapping is heavily used: <Callout> renders info, warning, and error boxes; <ApiPlayground> embeds an interactive API request builder that pre-fills the endpoint and lets the reader send test requests; <CodeBlock> adds a copy button, language label, and line highlighting. These components are designed by the engineering team but authored by content writers who never touch React code directly.
A companion marketing site can share the same MDX pipeline but use a different component mapping tuned for marketing content: <FeatureGrid>, <Testimonial>, and <PricingTable> components let the marketing team build rich landing pages without developer involvement. The shared compileDoc function and plugin configuration ensure consistent Markdown rendering across both sites.
A common extension is generating a table of contents from the heading structure. A custom remark plugin walks the MDAST, extracts all heading nodes with their text and depth, and exposes them as metadata alongside the frontmatter. The page layout renders this as a sticky sidebar navigation.
next-mdx-remote/rsc produces a React Server Component. If a custom component in the components map uses useState or useEffect, it must be marked with "use client". Forgetting this produces a runtime error about hooks being called in a server component. The component map itself can be defined in a server module; only the individual interactive components need the client directive.--- inside a code block, gray-matter may misparse it as frontmatter. Always ensure frontmatter delimiters are at the very start of the file with no leading whitespace or content.rehype-slug must run before rehype-autolink-headings, because autolink depends on the id attributes that slug generates. Similarly, syntax highlighting plugins must run after any plugin that modifies code blocks. Plugin arrays are processed in order; swapping two entries can produce silent failures.compileMDX is not free—each file runs through the full plugin chain. For sites with hundreds of MDX files, build times can grow to minutes. Use generateStaticParams with ISR (Incremental Static Regeneration) and revalidation to avoid recompiling unchanged pages on every deploy.loadContent: If a slug does not correspond to an actual file, fs.readFile throws. The page component must catch this error and call notFound() rather than letting the error propagate as a 500.) do not resolve correctly when the MDX file is loaded via next-mdx-remote because the content is not part of the webpack module graph. Use absolute paths, import images in the component map, or store images in the public/ directory with full paths.