Filesystem-Backed MDX Blog Utilities

Overview

Many static site generators and Next.js blogs store content as Markdown or MDX files on disk rather than in a database or CMS. Each post is a file in a content/posts/ directory with YAML frontmatter (title, date, tags, description) followed by the body content. A set of utility functions reads these files, parses the frontmatter, sorts posts by date, extracts slugs from filenames, and exposes the data to the site’s rendering pipeline. This “filesystem as database” pattern is simple, version-controlled, and requires no external services.

Teams commonly use this pattern for product documentation sites and engineering blogs. A lib/posts.ts module uses Node’s fs module and the gray-matter library to scan the posts directory, parse each file’s frontmatter, and return sorted post metadata for listing pages and full post content for individual pages. Next.js’s getStaticPaths and getStaticProps (or the App Router equivalents) call these utilities at build time to generate static HTML for every post.

This document covers the architecture of filesystem-backed content utilities: frontmatter parsing, slug extraction, date sorting, integration with static site generation, and patterns for scaling this approach as the content library grows.

Core Concepts

How It Works

Post Data Types

The first step is defining TypeScript types for post metadata and content. These types enforce the frontmatter contract and provide IDE autocomplete throughout the codebase:

// lib/types.ts
export interface PostFrontmatter {
  title: string;
  date: string;        // ISO 8601 date string
  description: string;
  tags: string[];
  draft?: boolean;
  author?: string;
  coverImage?: string;
}

export interface PostMeta extends PostFrontmatter {
  slug: string;
}

export interface Post extends PostMeta {
  content: string;     // raw MDX body (without frontmatter)
}

Reading and Parsing Posts

The core utility reads all MDX files from the posts directory, parses their frontmatter, extracts slugs, and returns sorted metadata:

// lib/posts.ts
import fs from "fs";
import path from "path";
import matter from "gray-matter";
import type { PostMeta, Post } from "./types";

const POSTS_DIR = path.join(process.cwd(), "content", "posts");

export function getAllPostSlugs(): string[] {
  return fs
    .readdirSync(POSTS_DIR)
    .filter((f) => f.endsWith(".mdx") || f.endsWith(".md"))
    .map((f) => f.replace(/\.mdx?$/, ""));
}

export function getPostBySlug(slug: string): Post {
  const mdxPath = path.join(POSTS_DIR, `${slug}.mdx`);
  const mdPath = path.join(POSTS_DIR, `${slug}.md`);
  const filePath = fs.existsSync(mdxPath) ? mdxPath : mdPath;

  const raw = fs.readFileSync(filePath, "utf-8");
  const { data, content } = matter(raw);

  return {
    slug,
    title: data.title,
    date: data.date,
    description: data.description,
    tags: data.tags ?? [],
    draft: data.draft ?? false,
    author: data.author,
    coverImage: data.coverImage,
    content,
  };
}

export function getAllPosts(): PostMeta[] {
  const slugs = getAllPostSlugs();
  return slugs
    .map((slug) => {
      const { content, ...meta } = getPostBySlug(slug);
      return meta;
    })
    .filter((post) => !post.draft)
    .sort((a, b) => new Date(b.date).getTime() - new Date(a.date).getTime());
}

Next.js Integration

The utilities plug directly into Next.js’s data fetching functions. For the Pages Router, getStaticPaths generates paths from slugs and getStaticProps fetches individual posts. For the App Router, these become generateStaticParams and direct function calls in Server Components:

// Pages Router: pages/blog/[slug].tsx
import { getAllPostSlugs, getPostBySlug } from "@/lib/posts";
import type { GetStaticPaths, GetStaticProps } from "next";

export const getStaticPaths: GetStaticPaths = async () => {
  const slugs = getAllPostSlugs();
  return {
    paths: slugs.map((slug) => ({ params: { slug } })),
    fallback: false,
  };
};

export const getStaticProps: GetStaticProps = async ({ params }) => {
  const post = getPostBySlug(params!.slug as string);
  return { props: { post } };
};

// App Router: app/blog/[slug]/page.tsx
import { getAllPostSlugs, getPostBySlug } from "@/lib/posts";

export function generateStaticParams() {
  return getAllPostSlugs().map((slug) => ({ slug }));
}

export default function BlogPost({ params }: { params: { slug: string } }) {
  const post = getPostBySlug(params.slug);
  // render post.content with MDX compiler
}

Blog Listing Page

The listing page uses getAllPosts() to render a sorted, filtered list of post cards:

// app/blog/page.tsx
import { getAllPosts } from "@/lib/posts";
import Link from "next/link";

export default function BlogIndex() {
  const posts = getAllPosts();

  return (
    <main>
      <h1>Blog</h1>
      {posts.map((post) => (
        <article key={post.slug}>
          <Link href={`/blog/${post.slug}`}>
            <h2>{post.title}</h2>
          </Link>
          <time dateTime={post.date}>
            {new Date(post.date).toLocaleDateString()}
          </time>
          <p>{post.description}</p>
          <div>
            {post.tags.map((tag) => (
              <span key={tag}>{tag}</span>
            ))}
          </div>
        </article>
      ))}
    </main>
  );
}

Practical Applications

Filesystem-backed content utilities are commonly used for several purposes:

• Blog posts: Engineering blog posts about product features, performance benchmarks, and architecture decisions are written as MDX files with embedded code examples and benchmark charts.
• Documentation pages: API documentation and guides are stored as MDX files, allowing authors to embed interactive API explorers and code playgrounds directly in the docs.
• Changelog entries: Each release has a corresponding MDX file in content/changelog/, and the changelog page renders them in reverse chronological order.
• Landing page content: Feature descriptions, testimonials, and pricing information are stored as frontmatter-only MDX files, making it easy for non-developers to update marketing copy by editing text files.

Implementation Patterns

Caching Parsed Posts

Reading and parsing every MDX file on every page request (even at build time) can be slow with hundreds of posts. A simple in-memory cache stores parsed posts keyed by slug and invalidates on file modification time changes. In development mode, the cache checks fs.statSync(filePath).mtimeMs on each access; in production builds, the cache is populated once and never invalidated.

Tag and Category Indexes

Building tag pages (/blog/tags/vector-search) requires an inverted index from tags to posts. The utility scans all posts, collects unique tags, and groups posts by tag. This runs at build time and generates a static page per tag:

export function getPostsByTag(): Map<string, PostMeta[]> {
  const posts = getAllPosts();
  const tagMap = new Map<string, PostMeta[]>();
  for (const post of posts) {
    for (const tag of post.tags) {
      const existing = tagMap.get(tag) ?? [];
      existing.push(post);
      tagMap.set(tag, existing);
    }
  }
  return tagMap;
}

RSS Feed Generation

The same post utilities power RSS feed generation. A build script calls getAllPosts(), formats each post as an RSS <item>, and writes the result to public/feed.xml. Because the utilities return structured data, generating Atom, JSON Feed, or Sitemap formats is equally straightforward.

Common Pitfalls

• Synchronous fs calls in production: Using fs.readFileSync is fine at build time but blocks the event loop in server-side rendering. If posts are loaded at request time (ISR, SSR), switch to fs.promises.readFile or pre-load all posts into memory at startup.
• Missing frontmatter validation: If a post file is missing required frontmatter fields (like date), the build fails with a cryptic error deep in the sorting function. Validate frontmatter early and surface clear error messages pointing to the problematic file.
• Date parsing inconsistencies: YAML parses dates differently depending on the format. 2024-01-15 is parsed as a Date object by js-yaml, while "2024-01-15" (quoted) remains a string. Always quote dates in frontmatter or normalize them in the utility function to avoid timezone-related sorting bugs.
• Large content directories: With 500+ posts, fs.readdirSync and sequential parsing become slow. Batch read with Promise.all and consider a build-time manifest file that caches parsed frontmatter so the listing page doesn’t re-parse every file.
• Draft posts in production: Forgetting to filter draft: true posts from getAllPosts() publishes unfinished content. Always filter drafts in the listing function and conditionally include them only in development mode.