HuggingFace PEFT Safetensors Interoperability

Overview

The HuggingFace PEFT (Parameter-Efficient Fine-Tuning) ecosystem stores fine-tuned adapter weights in the safetensors format alongside a JSON configuration file. When integrating adapters trained with or exported from the Python PEFT library into a Go-native inference pipeline — such as a Go PEFT library — engineers must bridge two distinct worlds: the Python naming conventions and tensor layout assumptions baked into every HuggingFace checkpoint, and the idiomatic Go tensor structures that the library uses at runtime. This is not a simple file-copy operation; it requires careful parsing of both the metadata header and the raw binary payload of each .safetensors file, followed by systematic renaming and potential transposition of weight tensors.

At its core, the interoperability problem has three distinct phases. First, the loader must parse adapter_config.json to understand the adapter’s architecture — which modules are targeted, what LoRA rank and alpha values were used, and which base model the adapter was originally trained against. Second, it must open adapter_model.safetensors, decode the binary header to recover per-tensor metadata (dtype, shape, byte offsets), and then stream each tensor’s raw bytes into Go-native memory buffers while applying any required dtype conversion. Third, and most subtly, the tensor key names written by the Python PEFT library follow PyTorch module path conventions — dotted strings like base_model.model.transformer.h.0.attn.c_attn.lora_A.weight — which must be translated into whatever hierarchical naming scheme the library uses internally before the weights can be wired into the correct adapter slots.

Exporting adapters back out to .safetensors is the inverse of this pipeline and is equally important for a plug-and-play experience: users who fine-tune a model through the library’s training facilities, or who merge a LoRA adapter’s low-rank matrices into the base model’s full-rank weights, need to write the result in a format that Python PEFT, the Transformers library, and the broader HuggingFace Hub tooling can load without modification. This means the exporter must reproduce the exact header structure, tensor key naming conventions, and dtype choices that the reference Python implementation would produce, including generating a valid adapter_config.json alongside the weights file. Together, these inbound and outbound pipelines make the library a first-class citizen in the HuggingFace fine-tuning ecosystem from within any Go-based inference stack.

Core Concepts

How It Works

The Go interoperability pipeline is composed of a small set of tightly coupled components: a safetensors parser, a config reader, a path translator, a dtype converter, a transposer, and a safetensors writer. The following code examples show the key mechanisms as implemented in Go within a Go PEFT library.

Parsing the Safetensors Binary Header

The first step when loading an adapter checkpoint is to open the .safetensors file, read the 8-byte header length, decode the JSON descriptor map, and compute the absolute byte offsets for each tensor’s data region. The loader deliberately avoids loading all tensor bytes at once; instead it builds an index that is consulted lazily when individual tensors are requested.

// safetensors/loader.go
package safetensors

import (
    "encoding/binary"
    "encoding/json"
    "fmt"
    "io"
    "os"
)

// TensorDescriptor holds the parsed metadata for a single tensor
// exactly as stored in the safetensors JSON header.
type TensorDescriptor struct {
    DType       string  `json:"dtype"`
    Shape       []int64 `json:"shape"`
    DataOffsets [2]int64 `json:"data_offsets"`
}

// Index is the result of parsing the safetensors header.
// DataStart is the file offset at which tensor bytes begin.
type Index struct {
    Tensors   map[string]TensorDescriptor
    DataStart int64
}

// ParseHeader opens the file at path and decodes the safetensors header.
// It does NOT read any tensor data; callers use ReadTensor to fetch bytes.
func ParseHeader(path string) (*Index, error) {
    f, err := os.Open(path)
    if err != nil {
        return nil, fmt.Errorf("safetensors: open %s: %w", path, err)
    }
    defer f.Close()

    // Read the 8-byte little-endian header length.
    var headerLen uint64
    if err := binary.Read(f, binary.LittleEndian, &headerLen); err != nil {
        return nil, fmt.Errorf("safetensors: read header length: %w", err)
    }
    if headerLen > 100*1024*1024 { // guard against malformed files
        return nil, fmt.Errorf("safetensors: implausible header length %d", headerLen)
    }

    // Decode the JSON header.
    headerBytes := make([]byte, headerLen)
    if _, err := io.ReadFull(f, headerBytes); err != nil {
        return nil, fmt.Errorf("safetensors: read header bytes: %w", err)
    }

    // The header may contain a special "__metadata__" key; skip it.
    raw := make(map[string]json.RawMessage)
    if err := json.Unmarshal(headerBytes, &raw); err != nil {
        return nil, fmt.Errorf("safetensors: unmarshal header: %w", err)
    }

    idx := &Index{
        Tensors:   make(map[string]TensorDescriptor, len(raw)),
        DataStart: int64(8 + headerLen),
    }
    for key, msg := range raw {
        if key == "__metadata__" {
            continue
        }
        var desc TensorDescriptor
        if err := json.Unmarshal(msg, &desc); err != nil {
            return nil, fmt.Errorf("safetensors: parse descriptor for %q: %w", key, err)
        }
        idx.Tensors[key] = desc
    }
    return idx, nil
}

// ReadTensor reads the raw bytes for the named tensor from the open file.
// Bytes are returned in the dtype and C-contiguous layout stored on disk;
// callers are responsible for dtype conversion and transposition.
func ReadTensor(f *os.File, idx *Index, name string) ([]byte, TensorDescriptor, error) {
    desc, ok := idx.Tensors[name]
    if !ok {
        return nil, TensorDescriptor{}, fmt.Errorf("safetensors: tensor %q not found", name)
    }
    start := idx.DataStart + desc.DataOffsets[0]
    length := desc.DataOffsets[1] - desc.DataOffsets[0]
    buf := make([]byte, length)
    if _, err := f.ReadAt(buf, start); err != nil {
        return nil, desc, fmt.Errorf("safetensors: read tensor %q: %w", name, err)
    }
    return buf, desc, nil
}

Loading an Adapter Config and Translating Tensor Paths

Once the header is parsed, the library reads adapter_config.json to build the adapter’s structural description. It then iterates over all tensor keys in the safetensors index, applies the path translation rules for the detected base model family, and loads each tensor with any required dtype conversion and optional transposition.

// peft/loader.go
package peft

import (
    "encoding/json"
    "fmt"
    "math"
    "os"
    "path/filepath"
    "strings"

    "github.com/example/peft/safetensors"
    "github.com/example/peft/tensor"
)

// AdapterConfig mirrors the fields of adapter_config.json that the library uses.
type AdapterConfig struct {
    PeftType          string   `json:"peft_type"`
    Rank              int      `json:"r"`
    LoraAlpha         float64  `json:"lora_alpha"`
    TargetModules     []string `json:"target_modules"`
    LoraDropout       float64  `json:"lora_dropout"`
    Bias              string   `json:"bias"`
    BaseModelPath     string   `json:"base_model_name_or_path"`
    TaskType          string   `json:"task_type"`
}

// LoadAdapterConfig reads and validates adapter_config.json from dir.
func LoadAdapterConfig(dir string) (*AdapterConfig, error) {
    data, err := os.ReadFile(filepath.Join(dir, "adapter_config.json"))
    if err != nil {
        return nil, fmt.Errorf("peft: read adapter_config.json: %w", err)
    }
    var cfg AdapterConfig
    if err := json.Unmarshal(data, &cfg); err != nil {
        return nil, fmt.Errorf("peft: parse adapter_config.json: %w", err)
    }
    if cfg.PeftType != "LORA" {
        return nil, fmt.Errorf("peft: unsupported peft_type %q (only LORA is supported)", cfg.PeftType)
    }
    return &cfg, nil
}

// translateKey converts a HuggingFace PEFT tensor key into the native
// internal path format for LLaMA-family models.
//
// Example input:  "base_model.model.model.layers.4.self_attn.q_proj.lora_A.weight"
// Example output: "transformer/layer_4/attention/query/lora_a"
func translateKey(hfKey string) (string, error) {
    // Strip the mandatory "base_model.model." wrapper prefix.
    key := strings.TrimPrefix(hfKey, "base_model.model.")

    // Split on dots.
    parts := strings.Split(key, ".")

    // Walk the parts and build the native path.
    var out []string
    i := 0
    for i < len(parts) {
        switch parts[i] {
        case "model":
            // top-level model wrapper — skip
            i++
        case "layers":
            // "layers." -> "layer_"
            if i+1 >= len(parts) {
                return "", fmt.Errorf("translateKey: truncated layers segment in %q", hfKey)
            }
            out = append(out, "layer_"+parts[i+1])
            i += 2
        case "self_attn":
            out = append(out, "attention")
            i++
        case "q_proj":
            out = append(out, "query")
            i++
        case "k_proj":
            out = append(out, "key")
            i++
        case "v_proj":
            out = append(out, "value")
            i++
        case "o_proj":
            out = append(out, "out_proj")
            i++
        case "mlp":
            out = append(out, "mlp")
            i++
        case "gate_proj":
            out = append(out, "gate")
            i++
        case "up_proj":
            out = append(out, "up")
            i++
        case "down_proj":
            out = append(out, "down")
            i++
        case "lora_A":
            out = append(out, "lora_a")
            i++
        case "lora_B":
            out = append(out, "lora_b")
            i++
        case "weight":
            // "weight" is implicit in native paths; drop it
            i++
        default:
            out = append(out, parts[i])
            i++
        }
    }

    if len(out) == 0 {
        return "", fmt.Errorf("translateKey: empty result for %q", hfKey)
    }
    return strings.Join(out, "/"), nil
}

// convertBF16toF32 converts a slice of bfloat16 bytes (little-endian pairs)
// to a slice of float32 values.  BF16 shares the exponent bits with F32
// but has only 7 mantissa bits; conversion is a zero-extend of the 2 bytes
// into the upper 2 bytes of a 4-byte IEEE 754 float32.
func convertBF16toF32(src []byte) []float32 {
    n := len(src) / 2
    dst := make([]float32, n)
    for i := 0; i < n; i++ {
        // Reconstruct the BF16 as the upper 16 bits of a uint32.
        lo := src[i*2]
        hi := src[i*2+1]
        bits := uint32(hi)<<24 | uint32(lo)<<16
        dst[i] = math.Float32frombits(bits)
    }
    return dst
}

// LoadAdapter reads adapter_config.json and adapter_model.safetensors from dir,
// applies path translation and dtype conversion, and returns a populated Adapter.
func LoadAdapter(dir string, targetDType tensor.DType) (*Adapter, error) {
    cfg, err := LoadAdapterConfig(dir)
    if err != nil {
        return nil, err
    }

    stPath := filepath.Join(dir, "adapter_model.safetensors")
    idx, err := safetensors.ParseHeader(stPath)
    if err != nil {
        return nil, err
    }

    f, err := os.Open(stPath)
    if err != nil {
        return nil, fmt.Errorf("peft: open safetensors: %w", err)
    }
    defer f.Close()

    adapter := &Adapter{
        Config:  cfg,
        Weights: make(map[string]*tensor.Tensor),
    }

    for hfKey, desc := range idx.Tensors {
        nativeKey, err := translateKey(hfKey)
        if err != nil {
            return nil, fmt.Errorf("peft: translate key %q: %w", hfKey, err)
        }

        raw, _, err := safetensors.ReadTensor(f, idx, hfKey)
        if err != nil {
            return nil, err
        }

        var f32data []float32
        switch desc.DType {
        case "F32":
            f32data = bytesToF32(raw)
        case "BF16":
            f32data = convertBF16toF32(raw)
        case "F16":
            f32data = convertF16toF32(raw)
        default:
            return nil, fmt.Errorf("peft: unsupported dtype %q for tensor %q", desc.DType, hfKey)
        }

        // Determine whether this tensor requires transposition.
        // lora_A matrices are stored by PEFT as [rank, in_features] and do not
        // need transposition.  lora_B matrices are stored as [out_features, rank]
        // and also do not need transposition for standard matmul in the library.
        // However, if the native layer expects fan-in layout, both are transposed.
        needsTranspose := layerExpectsFanIn(nativeKey)
        t := tensor.NewFromF32(f32data, desc.Shape, targetDType)
        if needsTranspose && len(desc.Shape) == 2 {
            t = tensor.Transpose2D(t)
        }
        adapter.Weights[nativeKey] = t
    }

    return adapter, nil
}

Merging LoRA Weights and Exporting to Safetensors

After loading an adapter, the library can optionally merge the low-rank decomposition back into the base model’s full-rank weight tensors. The merged weights are then exported as a standalone .safetensors file with a regenerated JSON header, ready for use with the HuggingFace Transformers library.

// peft/merge.go
package peft

import (
    "encoding/binary"
    "encoding/json"
    "fmt"
    "os"

    "github.com/example/peft/tensor"
)

// MergeLoRA computes W_merged = W + (alpha/rank) * lora_B @ lora_A
// for each target module in the adapter.  It modifies baseWeights in place.
func MergeLoRA(baseWeights map[string]*tensor.Tensor, adapter *Adapter) error {
    scale := adapter.Config.LoraAlpha / float64(adapter.Config.Rank)

    for nativeKey, loraA := range adapter.Weights {
        // Only process lora_a keys; pair with the corresponding lora_b.
        if !strings.HasSuffix(nativeKey, "/lora_a") {
            continue
        }
        baseKey := strings.TrimSuffix(nativeKey, "/lora_a")
        loraBKey := baseKey + "/lora_b"

        loraB, ok := adapter.Weights[loraBKey]
        if !ok {
            return fmt.Errorf("merge: missing lora_b for module %q", baseKey)
        }
        base, ok := baseWeights[baseKey]
        if !ok {
            return fmt.Errorf("merge: base model has no weight for %q", baseKey)
        }

        // delta = loraB @ loraA  (shapes: [out, rank] @ [rank, in] = [out, in])
        delta, err := tensor.MatMul(loraB, loraA)
        if err != nil {
            return fmt.Errorf("merge: matmul for %q: %w", baseKey, err)
        }

        // W_merged = W + scale * delta
        if err := tensor.AddScaledInPlace(base, delta, scale); err != nil {
            return fmt.Errorf("merge: addscaled for %q: %w", baseKey, err)
        }
    }
    return nil
}

// safetensorsHeader is the JSON object written at the top of a .safetensors file.
type safetensorsHeader map[string]safetensorsEntry

type safetensorsEntry struct {
    DType       string   `json:"dtype"`
    Shape       []int64  `json:"shape"`
    DataOffsets [2]int64 `json:"data_offsets"`
}

// ExportSafetensors writes weights to path in safetensors format.
// keyTranslator converts native keys back to HuggingFace PEFT keys.
func ExportSafetensors(
    path string,
    weights map[string]*tensor.Tensor,
    keyTranslator func(string) string,
    exportDType string,
) error {
    // Phase 1: collect tensor bytes and build header entries.
    header := make(safetensorsHeader, len(weights))
    allData := make([][]byte, 0, len(weights))
    var offset int64

    // Sort keys for deterministic output.
    keys := sortedKeys(weights)
    for _, nativeKey := range keys {
        t := weights[nativeKey]
        hfKey := keyTranslator(nativeKey)

        raw, err := tensorToBytes(t, exportDType)
        if err != nil {
            return fmt.Errorf("export: serialize %q: %w", nativeKey, err)
        }
        entry := safetensorsEntry{
            DType: exportDType,
            Shape: t.Shape(),
            DataOffsets: [2]int64{offset, offset + int64(len(raw))},
        }
        header[hfKey] = entry
        allData = append(allData, raw)
        offset += int64(len(raw))
    }

    // Phase 2: serialize the header to JSON and pad to 8-byte alignment.
    headerJSON, err := json.Marshal(header)
    if err != nil {
        return fmt.Errorf("export: marshal header: %w", err)
    }
    // Pad with spaces to align data region to 8 bytes.
    for (8+len(headerJSON))%8 != 0 {
        headerJSON = append(headerJSON, ' ')
    }

    // Phase 3: write the file: 8-byte length + header + tensor data.
    f, err := os.Create(path)
    if err != nil {
        return fmt.Errorf("export: create file: %w", err)
    }
    defer f.Close()

    if err := binary.Write(f, binary.LittleEndian, uint64(len(headerJSON))); err != nil {
        return fmt.Errorf("export: write header length: %w", err)
    }
    if _, err := f.Write(headerJSON); err != nil {
        return fmt.Errorf("export: write header: %w", err)
    }
    for _, chunk := range allData {
        if _, err := f.Write(chunk); err != nil {
            return fmt.Errorf("export: write tensor data: %w", err)
        }
    }
    return nil
}

Practical Applications

The Go safetensors interoperability layer unlocks several high-value workflows. The most common is loading a LoRA adapter checkpoint downloaded directly from the HuggingFace Hub and applying it at inference time to a base model that is already loaded in memory by a model inference server. Because the adapter is stored as a compact set of low-rank matrices (typically less than 1% of the base model’s parameter count), the server can hot-swap between dozens of task-specific adapters with negligible latency and memory overhead compared to maintaining separate full fine-tuned models.

A second application is adapter composition: loading two or more LoRA adapters trained for different tasks and combining them via weighted addition in the weight space before inference. The library’s path translation layer makes this straightforward because all adapter weights are indexed by the same normalized native key scheme after loading, regardless of which Python PEFT version or HuggingFace model family produced them. The merger simply iterates over the union of all adapter keys, scales each tensor by its assigned mixture weight, and sums them into a composite adapter that can optionally be exported back to safetensors for caching.

A third application — particularly relevant for edge and mobile deployment — is merging an adapter into the base model to produce a single monolithic model that requires no adapter-handling code at inference time. The merged safetensors output is fully compatible with Transformers’ AutoModelForCausalLM.from_pretrained, meaning a model can be fine-tuned through the library’s training pipeline, merged, and then deployed in any Python environment without retraining or conversion steps.

Typical Integration Points

• Adapter hot-loading in a model-serving router, where per-request adapter selection is driven by metadata stored in a document store.
• Automated fine-tuning pipelines that invoke it as a library to train adapters on domain-specific corpora, then export the resulting checkpoints directly to a HuggingFace Hub repository for versioned storage.
• Benchmark harnesses that load the same adapter checkpoint in both Python PEFT and the library and compare forward-pass outputs to validate numerical correctness of the translation pipeline, with particular attention to dtype accumulation differences between BF16 and F32 codepaths.
• Adapter quantization workflows that load a BF16 adapter, up-cast to F32 for the merge computation, and then quantize the merged model to INT8 or INT4 for deployment using standard quantization utilities.

Implementation Patterns

Lazy Tensor Loading with an Index Cache

For adapters with many tensors (some large models have hundreds of LoRA-adapted layers), loading all tensors eagerly at startup adds unnecessary latency. The recommended pattern in the library is to build the Index at startup (which only reads the header) and resolve individual tensor bytes on first access, caching them in a sync.Map or a purpose-built LRU. This is especially important in multi-adapter servers where dozens of adapters are registered but only a few are active on any given request.

Architecture-Driven Translation Tables

Rather than hard-coding path translation rules for each model family, the library maintains a small YAML or JSON file per base model architecture (LLaMA, Mistral, Falcon, GPT-NeoX, etc.) that maps HuggingFace module path fragments to native key segments. The translateKey function is then driven by the loaded table rather than a switch statement, making it possible to add support for a new model family by dropping in a new translation file without recompiling the library. The architecture is determined from the base_model_name_or_path field in adapter_config.json combined with the base model’s own config.json file.

Dtype Negotiation at Load Time

The caller of LoadAdapter specifies a targetDType that governs what precision the loaded tensors are stored in within the library’s memory. Common choices are tensor.DTypeF32 for maximum numerical fidelity during development and debugging, and tensor.DTypeBF16 for production serving on hardware with native BF16 support (e.g., modern NVIDIA Ampere and Hopper GPUs via CGo). The dtype conversion happens once at load time, avoiding repeated casts during the forward pass. When merging adapters, the merge computation is always performed at F32 regardless of the storage dtype, and the result is cast back to the target dtype before being written into the base weight tensor.

Round-Trip Validation

Any non-trivial change to the translation or conversion pipeline should be validated by a round-trip test: load a known adapter checkpoint from disk using the library, export it back to a temporary directory, reload it from the exported files, and compare tensor values element-wise against the originals using a maximum absolute error threshold appropriate for the dtype (typically 1e-5 for F32, 5e-3 for BF16). A well-maintained test suite includes a set of reference adapters for each supported model family generated by the canonical Python PEFT library and committed to a test data repository.

Streaming Large Adapters

Very large adapters (e.g., for 70B parameter models with high LoRA rank) may have safetensors files that exceed available RAM on smaller machines. The library’s export path supports streaming writes — it serializes tensors one at a time, writing directly to the file, rather than accumulating all tensor bytes in memory before writing the header. This requires a two-pass approach: the first pass computes tensor sizes and builds the header (without data), and the second pass writes the header and then streams each tensor. The header offsets are computed from tensor shapes and dtypes without materializing the data, which requires the dtype-to-bytes-per-element mapping to be exact for all supported dtypes.

Common Pitfalls

Off-by-One in the Header Offset

The safetensors data offsets stored in the JSON header are relative to the start of the data region (immediately after the header), not relative to the start of the file. A common mistake is to treat them as absolute file offsets, which causes every tensor read to be shifted by 8 + headerLen bytes, producing garbage data that is difficult to diagnose because the byte count is usually correct and only the values are wrong. Always add idx.DataStart to the raw offset values from the descriptor before seeking.

Incorrect BF16 Zero-Extension

BF16 is not natively supported in Go’s math package. The conversion to F32 is simply a zero-extension of the 16-bit pattern into the upper 2 bytes of a 4-byte IEEE 754 float. A subtle mistake is to place the BF16 bytes in the lower 2 bytes of the uint32 (equivalent to multiplying by 2^-16), which produces results 65536x too small. The correct conversion places the BF16 bytes in positions [2] and [3] (big-endian relative to the float32 bit layout), treating the lower 16 bits of the mantissa as zero.

Silent Shape Mismatch After Transposition

After transposing a 2D tensor, the shape stored in the library’s Tensor struct must be updated to reflect the new logical dimensions. Failing to update the shape metadata while transposing only the data bytes results in a tensor that appears to have the pre-transposition shape but whose data is laid out in the post-transposition order. This causes silent numerical errors when the tensor is later used in a matrix multiplication that reads the shape to determine stride lengths. Always use the library’s tensor.Transpose2D, which atomically swaps both the data bytes and the shape field.

Missing lora_B Zero Initialization

The HuggingFace PEFT library initializes lora_B to all zeros at the start of training so that the adapter has no effect on the model’s outputs at initialization. When exporting an adapter that was created programmatically rather than trained from a checkpoint, callers must ensure that lora_B tensors are explicitly zero-initialized before any training steps, or the starting point for fine-tuning will be a random perturbation of the base model rather than an identity delta. The library’s NewAdapter constructor enforces this invariant, but adapters assembled manually from raw tensors bypass the constructor and are susceptible to this mistake.

key Translation Incompleteness for Non-Standard Module Names

Some community fine-tuned models use non-standard target module names — for example, targeting c_attn (GPT-2 style combined QKV projection) instead of separate q_proj, k_proj, v_proj modules. If the translation table does not include entries for these module names, translateKey may silently pass through the original name segment, resulting in native keys that do not match any registered layer slot. The library detects this at load time by checking all translated keys against the base model’s registered layer registry and returns a descriptive error listing any unresolved keys, prompting the caller to extend the translation table.

Further Reading