Scoped API Key Design with AES-256-GCM

Overview

API keys are one of the oldest and most widely used authentication mechanisms for programmatic access. While simpler than OAuth flows, a production-grade API key system requires careful design around key generation, encryption at rest, scope enforcement, and usage tracking. A flat, unscopeable key is a liability—any compromise grants unlimited access. A well-designed multi-level key system limits blast radius by constraining each key to specific resources and operations.

A well-designed API key architecture uses two distinct key tiers: project-scoped keys with the prj_ prefix that grant broad access across an entire project’s resources, and instance-scoped keys with the db_ prefix that are limited to a single database instance. Both tiers encrypt the key material at rest using AES-256-GCM, use constant-time comparison during validation to prevent timing attacks, and support fine-grained restrictions on which tables and operations (read, write, admin) the key permits.

This design enables a layered security model: infrastructure automation uses project-scoped keys to provision and manage instances, while application code uses instance-scoped keys that can only read from or write to specific tables. Monthly usage quotas add a further safety net, automatically disabling keys that exceed their allocation—useful for limiting the damage from a leaked key before it can be manually revoked.

Core Concepts

How It Works

Key Generation

When a user requests a new API key, the system generates 32 bytes of cryptographically random data using crypto/rand. This raw material is base62-encoded and prefixed with either prj_ or db_ depending on the requested scope. The prefix serves as a visual and programmatic discriminator—developers can immediately identify the key tier, and middleware can route validation logic based on the prefix without decoding the full key.

package apikey

import (
    "crypto/rand"
    "encoding/base64"
    "fmt"
    "strings"
)

type KeyTier string

const (
    TierProject  KeyTier = "prj_"
    TierInstance KeyTier = "db_"
)

type KeyScope struct {
    ProjectID      string
    InstanceID     string
    AllowedTables  []string
    AllowedOps     []string
    MonthlyQuota   int64
}

func GenerateKey(tier KeyTier) (string, error) {
    raw := make([]byte, 32)
    if _, err := rand.Read(raw); err != nil {
        return "", fmt.Errorf("generating key material: %w", err)
    }
    encoded := base64.RawURLEncoding.EncodeToString(raw)
    return string(tier) + encoded, nil
}

Encryption at Rest

Before the key is persisted, it is encrypted with AES-256-GCM. The encryption key (the “key encryption key” or KEK) is loaded from a secrets manager at startup and never written to disk. A fresh 12-byte nonce is generated for each encryption to ensure uniqueness. The nonce is prepended to the ciphertext so that it can be extracted during decryption.

package apikey

import (
    "crypto/aes"
    "crypto/cipher"
    "crypto/rand"
    "fmt"
    "io"
)

type Encryptor struct {
    gcm cipher.AEAD
}

func NewEncryptor(kek []byte) (*Encryptor, error) {
    block, err := aes.NewCipher(kek)
    if err != nil {
        return nil, fmt.Errorf("creating AES cipher: %w", err)
    }
    gcm, err := cipher.NewGCM(block)
    if err != nil {
        return nil, fmt.Errorf("creating GCM: %w", err)
    }
    return &Encryptor{gcm: gcm}, nil
}

func (e *Encryptor) Encrypt(plaintext []byte) ([]byte, error) {
    nonce := make([]byte, e.gcm.NonceSize())
    if _, err := io.ReadFull(rand.Reader, nonce); err != nil {
        return nil, fmt.Errorf("generating nonce: %w", err)
    }
    return e.gcm.Seal(nonce, nonce, plaintext, nil), nil
}

func (e *Encryptor) Decrypt(ciphertext []byte) ([]byte, error) {
    nonceSize := e.gcm.NonceSize()
    if len(ciphertext) < nonceSize {
        return nil, fmt.Errorf("ciphertext too short")
    }
    nonce, ct := ciphertext[:nonceSize], ciphertext[nonceSize:]
    return e.gcm.Open(nil, nonce, ct, nil)
}

Key Validation Middleware

When an API request arrives, the middleware extracts the key from the Authorization header, determines the tier from the prefix, decrypts the stored key for the matching key ID, and performs a constant-time comparison. If the comparison succeeds, the middleware loads the key’s scope and checks whether the requested operation is permitted.

package middleware

import (
    "crypto/subtle"
    "net/http"
    "strings"
)

func (m *AuthMiddleware) ValidateAPIKey(r *http.Request) (*KeyScope, error) {
    header := r.Header.Get("Authorization")
    if !strings.HasPrefix(header, "Bearer ") {
        return nil, ErrMissingKey
    }
    providedKey := strings.TrimPrefix(header, "Bearer ")

    var tier KeyTier
    switch {
    case strings.HasPrefix(providedKey, string(TierProject)):
        tier = TierProject
    case strings.HasPrefix(providedKey, string(TierInstance)):
        tier = TierInstance
    default:
        return nil, ErrInvalidKeyFormat
    }

    record, err := m.store.LookupKeyByPrefix(r.Context(), providedKey[:12])
    if err != nil {
        return nil, ErrKeyNotFound
    }

    storedKey, err := m.encryptor.Decrypt(record.EncryptedKey)
    if err != nil {
        return nil, ErrDecryptionFailed
    }

    if subtle.ConstantTimeCompare([]byte(providedKey), storedKey) != 1 {
        return nil, ErrInvalidKey
    }

    if record.MonthlyUsage >= record.MonthlyQuota {
        return nil, ErrQuotaExceeded
    }

    return &record.Scope, nil
}

Scope Enforcement

After successful key validation, the request enters the scope enforcement layer. This layer checks three dimensions: (1) whether the target resource (project or instance) matches the key’s scope, (2) whether the target table is in the key’s allowed tables list (or if the list is empty, all tables are permitted), and (3) whether the HTTP method maps to an allowed operation. A key with ["read"] permissions can only perform GET requests; a key with ["read", "write"] can also POST and PUT, but not DELETE.

Quota Tracking

Each successful request increments a per-key counter stored in a fast key-value store (Redis or the database’s own counters). A background job resets all counters on the first of each month. The quota check happens after authentication but before the request is processed, so quota enforcement adds minimal latency—a single counter read and comparison.

Implementation Patterns

Key Rotation

A robust key system supports rotation without downtime. The pattern is: generate a new key, associate it with the same scope, notify the client, and set a grace period during which both old and new keys are valid. After the grace period, the old key is marked as revoked. This is especially important for project-scoped keys that may be embedded in CI pipelines or infrastructure automation.

Key ID Indexing

Rather than scanning all encrypted keys to find a match, each key includes a non-secret “key ID” component—typically the first 8–12 characters after the prefix. The key ID is stored as an indexed column in the database, enabling O(1) lookups. Only the matching encrypted key is then decrypted and compared, keeping validation fast even with millions of active keys.

Audit Logging

Every key operation—creation, rotation, revocation, and quota breach—is logged to an append-only audit trail. Each log entry includes the key ID (never the key material), the acting user, a timestamp, and the operation performed. This trail is essential for compliance and for investigating security incidents.

Common Pitfalls

• Storing keys in plaintext: Even in a private database, plaintext keys are one SQL injection away from full compromise. Always encrypt at rest with a KEK stored in a separate secrets manager.
• Using standard string comparison: A byte-by-byte comparison leaks timing information. Use crypto/subtle.ConstantTimeCompare in Go or hmac.compare_digest in Python for key validation.
• Overly broad key scopes: Issuing project-scoped keys when instance-scoped keys would suffice violates the principle of least privilege. Default to the narrowest scope and widen only when justified.
• Forgetting to revoke on personnel changes: When a team member leaves or a service is decommissioned, all associated keys must be revoked. Automate this by linking key lifecycle to identity provider events.
• No quota or rate limiting: A key without quotas can be used to exfiltrate unlimited data or run up infrastructure costs. Always set a reasonable default quota and alert on unusual usage patterns.