Per-Project Cloudflare DNS Automation

Overview

When a platform manages multiple projects, each with its own custom domain, DNS record management becomes an operational bottleneck. Manually creating A records, CNAME entries, and TXT verification records for each project deployment is error-prone, slow, and impossible to audit at scale. Automating DNS through Cloudflare’s API transforms domain provisioning into a deterministic, code-driven process that runs as part of the deployment pipeline.

In the deploy platform, each project configuration stores its Cloudflare zone ID, an API token reference (pointing to GCP Secret Manager), and a domain template using the {project-slug} placeholder. When the deploy controller processes a project, it resolves the domain template, fetches the Cloudflare API token from Secret Manager, and upserts a proxied A record pointing to the project’s load balancer IP. The controller persists the Cloudflare record ID back to the project configuration, enabling future updates and clean deletion when a project is decommissioned.

This topic covers the architecture of per-project DNS automation, the Cloudflare API integration patterns, the security model for token scoping, and the reconciliation logic that keeps DNS records in sync with deployment state.

Core Concepts

How It Works

The DNS automation pipeline is triggered as part of the project deployment flow. After the Kubernetes resources are applied and the load balancer IP is allocated, the deploy controller enters the DNS reconciliation phase. It reads the project’s DNS configuration, resolves the domain template, fetches the API token, and upserts the DNS record via the Cloudflare API.

Project DNS Configuration Schema

Each project stores its DNS configuration as a structured object. This is typically part of the project’s infrastructure configuration in the deploy platform’s database or configuration file.

// types/project.ts
interface ProjectDNSConfig {
    // Cloudflare zone ID for the parent domain.
    zoneId: string;

    // Secret Manager secret name containing the Cloudflare API token.
    tokenSecretRef: string;

    // Domain template with {project-slug} placeholder.
    domainTemplate: string;

    // Cloudflare record ID, populated after first creation.
    recordId?: string;

    // Whether to proxy traffic through Cloudflare.
    proxied: boolean;
}

// Example:
// {
//   zoneId: "a1b2c3d4e5f6...",
//   tokenSecretRef: "prod-cloudflare-token-zone-example",
//   domainTemplate: "{project-slug}.app.example.com",
//   recordId: "f6e5d4c3b2a1...",
//   proxied: true
// }

Fetching the API Token from Secret Manager

The deploy controller retrieves the Cloudflare API token from GCP Secret Manager at runtime. This avoids embedding tokens in configuration files or environment variables.

// controllers/dns.go
package controllers

import (
    "context"
    "fmt"

    secretmanager "cloud.google.com/go/secretmanager/apiv1"
    secretmanagerpb "cloud.google.com/go/secretmanager/apiv1/secretmanagerpb"
)

func fetchCloudflareToken(ctx context.Context, projectID, secretName string) (string, error) {
    client, err := secretmanager.NewClient(ctx)
    if err != nil {
        return "", fmt.Errorf("creating secret manager client: %w", err)
    }
    defer client.Close()

    result, err := client.AccessSecretVersion(ctx, &secretmanagerpb.AccessSecretVersionRequest{
        Name: fmt.Sprintf("projects/%s/secrets/%s/versions/latest", projectID, secretName),
    })
    if err != nil {
        return "", fmt.Errorf("accessing secret %s: %w", secretName, err)
    }

    return string(result.Payload.Data), nil
}

Upserting DNS Records via the Cloudflare API

The core reconciliation logic checks for an existing record ID. If present, it updates the record; if absent, it creates a new one and persists the returned ID.

// controllers/dns.go (continued)
import (
    "bytes"
    "encoding/json"
    "io"
    "net/http"
    "strings"
)

type CloudflareDNSRecord struct {
    ID      string `json:"id"`
    Type    string `json:"type"`
    Name    string `json:"name"`
    Content string `json:"content"`
    Proxied bool   `json:"proxied"`
    TTL     int    `json:"ttl"`
}

type CloudflareResponse struct {
    Success bool                `json:"success"`
    Result  CloudflareDNSRecord `json:"result"`
}

func upsertDNSRecord(
    token, zoneID, domain, ip string,
    proxied bool, existingRecordID string,
) (string, error) {
    payload := CloudflareDNSRecord{
        Type:    "A",
        Name:    domain,
        Content: ip,
        Proxied: proxied,
        TTL:     1, // Auto TTL when proxied
    }

    body, _ := json.Marshal(payload)

    var method, url string
    if existingRecordID != "" {
        // Update existing record.
        method = "PUT"
        url = fmt.Sprintf(
            "https://api.cloudflare.com/client/v4/zones/%s/dns_records/%s",
            zoneID, existingRecordID,
        )
    } else {
        // Create new record.
        method = "POST"
        url = fmt.Sprintf(
            "https://api.cloudflare.com/client/v4/zones/%s/dns_records",
            zoneID,
        )
    }

    req, _ := http.NewRequest(method, url, bytes.NewReader(body))
    req.Header.Set("Authorization", "Bearer "+token)
    req.Header.Set("Content-Type", "application/json")

    resp, err := http.DefaultClient.Do(req)
    if err != nil {
        return "", fmt.Errorf("cloudflare API request: %w", err)
    }
    defer resp.Body.Close()

    respBody, _ := io.ReadAll(resp.Body)
    var cfResp CloudflareResponse
    if err := json.Unmarshal(respBody, &cfResp); err != nil {
        return "", fmt.Errorf("parsing cloudflare response: %w", err)
    }

    if !cfResp.Success {
        return "", fmt.Errorf("cloudflare API error: %s", string(respBody))
    }

    return cfResp.Result.ID, nil
}

func resolveDomain(template, projectSlug string) string {
    return strings.ReplaceAll(template, "{project-slug}", projectSlug)
}

Full Reconciliation Flow

The deploy controller orchestrates the full flow: resolve the domain, fetch the token, upsert the record, and persist the record ID back to the project configuration.

// controllers/dns.go (continued)
func ReconcileDNS(ctx context.Context, project *Project, loadBalancerIP string) error {
    // Resolve domain from template.
    domain := resolveDomain(project.DNS.DomainTemplate, project.Slug)

    // Fetch the Cloudflare API token from Secret Manager.
    token, err := fetchCloudflareToken(ctx, gcpProjectID, project.DNS.TokenSecretRef)
    if err != nil {
        return fmt.Errorf("fetching cloudflare token: %w", err)
    }

    // Upsert the DNS record.
    recordID, err := upsertDNSRecord(
        token, project.DNS.ZoneID, domain, loadBalancerIP,
        project.DNS.Proxied, project.DNS.RecordID,
    )
    if err != nil {
        return fmt.Errorf("upserting DNS record for %s: %w", domain, err)
    }

    // Persist the record ID for future reconciliation.
    project.DNS.RecordID = recordID
    if err := saveProject(ctx, project); err != nil {
        return fmt.Errorf("saving project DNS state: %w", err)
    }

    log.Printf("DNS reconciled: %s -> %s (record: %s)", domain, loadBalancerIP, recordID)
    return nil
}

Practical Applications

The deploy platform’s DNS automation eliminates manual Cloudflare dashboard interactions for project provisioning. When a new project is created, the deploy pipeline automatically provisions its domain, sets up the proxied A record, and stores the record ID for lifecycle management. Project deletion triggers a cleanup phase that removes the DNS record, preventing stale entries from accumulating.

The per-zone token scoping is critical for multi-tenant scenarios. If the deploy platform manages projects across multiple customers, each with their own Cloudflare zone, the per-zone token ensures that a compromise of one customer’s token cannot affect another customer’s DNS records. The tokens are stored in GCP Secret Manager with environment prefixes (following the ESO pattern from the previous topic), so staging deployments use staging tokens that point to staging zones.

The template-based domain system supports complex domain hierarchies. Projects on the deploy platform can have multiple domains (e.g., {project-slug}.app.example.com for the application and {project-slug}.api.example.com for the API endpoint), each managed as a separate DNS record with its own record ID. The reconciliation loop processes all domain templates for each project, ensuring complete domain coverage.

Typical Integration Points

• Project deployment pipeline — DNS records are upserted after load balancer IP allocation and before health checks.
• Project deletion workflow — DNS records are deleted using the stored record ID before Kubernetes resources are torn down.
• Certificate issuance — Cloudflare’s proxied mode handles SSL termination, but for origin certificates, DNS-01 challenges use the same Cloudflare API token.
• Custom domain configuration — Users can override the domain template with a fully custom domain, triggering CNAME record creation instead of A records.

Implementation Patterns

Idempotent Upsert with Record ID Caching

The upsert pattern (create if absent, update if present) relies on the persisted record ID. If the record ID is lost (database reset, migration error), the controller falls back to listing existing records by name and type to find a match. This prevents duplicate record creation in recovery scenarios.

// controllers/dns.go — fallback lookup
func findExistingRecord(token, zoneID, domain string) (string, error) {
    url := fmt.Sprintf(
        "https://api.cloudflare.com/client/v4/zones/%s/dns_records?type=A&name=%s",
        zoneID, domain,
    )

    req, _ := http.NewRequest("GET", url, nil)
    req.Header.Set("Authorization", "Bearer "+token)

    resp, err := http.DefaultClient.Do(req)
    if err != nil {
        return "", err
    }
    defer resp.Body.Close()

    var result struct {
        Result []CloudflareDNSRecord `json:"result"`
    }
    json.NewDecoder(resp.Body).Decode(&result)

    if len(result.Result) > 0 {
        return result.Result[0].ID, nil
    }
    return "", nil
}

Token Rotation Without Downtime

Cloudflare API tokens can be rotated in Secret Manager by creating a new secret version. The deploy controller always fetches the latest version, so the next deployment automatically uses the new token. The old token remains valid until explicitly revoked in Cloudflare, providing a grace period for in-flight deployments.

Batch DNS Operations

When deploying multiple projects simultaneously, the controller batches DNS operations to respect Cloudflare’s API rate limits (1200 requests per 5 minutes per zone). The deploy platform implements a per-zone rate limiter using a token bucket that limits to 200 requests per minute, leaving headroom for other Cloudflare operations.

Common Pitfalls

Proxied Record TTL Behavior

When a record is proxied, Cloudflare ignores the TTL value and uses its own caching logic (typically 300 seconds at the edge). Setting TTL: 1 (auto) is the correct approach for proxied records. If you set a specific TTL (e.g., 3600) and later disable proxying, the TTL takes effect and clients may cache the old IP for up to an hour during a migration.

Zone ID vs Account ID Confusion

Cloudflare’s API uses both zone IDs and account IDs, and they look identical (32-character hex strings). Using the account ID where the zone ID is expected returns a 7003: Could not route to /zones/... error. Always verify the zone ID by calling GET /zones?name=example.com and checking the returned id field rather than copying IDs from the Cloudflare dashboard, where zone and account IDs appear on the same page.

CNAME Flattening Edge Cases

Cloudflare automatically flattens CNAME records at the zone apex (e.g., example.com without a subdomain) into A records. If the deploy controller creates a CNAME for an apex domain, Cloudflare resolves it to IPs at creation time. If the CNAME target’s IP changes later, the flattened record becomes stale until the next DNS reconciliation. Avoid apex CNAME records; use A records with explicit IP addresses for zone apex domains.

Orphaned Records After Failed Deployments

If a deployment fails after creating the DNS record but before persisting the record ID, the record becomes orphaned — it exists in Cloudflare but the controller has no reference to it. The next deployment creates a duplicate. Implement a pre-creation check (the findExistingRecord fallback) to detect and reclaim orphaned records instead of blindly creating new ones.

API Token Permission Scope Creep

Cloudflare API tokens support granular permissions (DNS:Read, DNS:Edit, Zone:Read, etc.). A common mistake is creating tokens with Zone:Edit permission, which allows modifying zone settings (page rules, security levels, SSL modes) in addition to DNS records. The deploy controller only needs DNS:Edit and Zone:Read for the specific zone. Audit token permissions regularly and rotate any tokens with excess permissions.

Further Reading