Audio and video transcription converts spoken language in media files into machine-readable text that can be indexed, searched, and summarized. In the context of a document processing pipeline, transcription bridges the gap between rich media content—meeting recordings, podcasts, screencasts, and conference talks—and text-based search and retrieval systems that operate on string data.
A transcription service provides a /transcribe API that accepts audio data and returns text transcripts using OpenAI’s Whisper model (specifically the Tiny EN variant for speed). The AudioProcessor integrates with this API, handling the complexity of media format conversion: video files must first have their audio tracks stripped using ffmpeg, and all audio must be resampled to 16 kHz mono WAV before submission. The resulting transcripts are stored as indexed documents alongside the source media, making spoken content as searchable as written text.
This pattern is increasingly important as organizations accumulate large volumes of recorded meetings, training sessions, and multimedia documentation. Without transcription, this content is effectively invisible to search—you can only find a recording if you already know it exists. Automatic transcription unlocks the information trapped in audio and video, making it discoverable through the same search interface used for text documents.
Speech recognition models expect audio in a specific format: single-channel (mono), 16-bit PCM, sampled at 16 kHz. Real-world media files come in countless formats—stereo MP3, AAC-encoded MP4, multi-track MKV with multiple audio streams. The preprocessing step uses ffmpeg to normalize all input to the expected format. For video files, this also means extracting only the audio track and discarding the video stream entirely.
Whisper and most speech recognition models require 16 kHz, 16-bit, mono PCM audio. The ffmpeg command -ar 16000 -ac 1 -f wav converts any input format to this standard. This normalization step is non-negotiable—feeding 44.1 kHz stereo audio to a model trained on 16 kHz mono produces degraded transcripts with higher word error rates.
OpenAI’s Whisper comes in multiple sizes: Tiny, Base, Small, Medium, and Large. Each step up roughly doubles the parameters and improves accuracy, but also increases inference time and memory requirements. Whisper Tiny EN is an English-only variant of the smallest model, optimized for speed over accuracy. It processes audio roughly 30x faster than real-time on a modern CPU, making it practical for batch processing large media libraries without GPU acceleration.
Whisper Tiny EN trades transcription accuracy for throughput. For search indexing—where the goal is keyword discoverability, not verbatim transcripts—this tradeoff is often acceptable. A transcript that captures 90% of words correctly still makes the recording findable by topic. For applications requiring high fidelity (legal transcription, medical dictation), larger models or human review are necessary.
Video files contain multiplexed streams: video, audio, subtitles, and metadata. Transcription only needs the audio stream. Extracting audio before sending to the transcription API avoids transmitting potentially gigabytes of video data that will be immediately discarded. The extraction command ffmpeg -i input.mp4 -vn -ar 16000 -ac 1 -f wav pipe:1 strips the video track (-vn), resamples, and outputs raw WAV to stdout for streaming directly to the API.
The -vn flag in ffmpeg means “no video”—it discards all video streams and outputs only audio. Combined with piping to stdout (pipe:1), this enables streaming the extracted audio directly to the transcription API without writing a temporary file to disk. This reduces I/O overhead and disk space requirements, especially important when processing many large video files concurrently.
Long audio files (hour-long meetings, multi-hour recordings) cannot be processed as a single unit due to memory constraints and API timeouts. The audio is split into overlapping chunks (typically 30-second segments with 2-second overlap) that fit within the model’s context window. Overlapping prevents word loss at chunk boundaries. The resulting transcript segments are concatenated with timestamps, producing a time-aligned full transcript.
The transcription pipeline follows a four-stage process: format detection, audio extraction, API submission, and transcript assembly. The AudioProcessor orchestrates these stages, handling both pure audio files and video files with embedded audio tracks.
/transcribe endpoint. The API runs Whisper Tiny EN and returns the transcript as JSON with text and timing information.// Go: Audio processor that extracts and transcribes media files
package transcription
import (
"bytes"
"encoding/json"
"fmt"
"io"
"net/http"
"os/exec"
"path/filepath"
"strings"
)
type AudioProcessor struct {
TranscriberURL string
Client *http.Client
}
type TranscribeResponse struct {
Text string `json:"text"`
Segments []Segment `json:"segments"`
Language string `json:"language"`
Duration float64 `json:"duration"`
}
type Segment struct {
Start float64 `json:"start"`
End float64 `json:"end"`
Text string `json:"text"`
}
// Process handles both audio and video files.
func (ap *AudioProcessor) Process(path string) (*Document, error) {
// Extract audio as 16kHz mono WAV
wavData, err := ap.extractAudio(path)
if err != nil {
return nil, fmt.Errorf("extract audio from %s: %w", path, err)
}
// Send to transcription API
transcript, err := ap.transcribe(wavData)
if err != nil {
return nil, fmt.Errorf("transcribe %s: %w", path, err)
}
// Build document from transcript
doc := &Document{
Content: transcript.Text,
Metadata: map[string]string{
"type": "transcript",
"source": filepath.Base(path),
"language": transcript.Language,
"duration_s": fmt.Sprintf("%.1f", transcript.Duration),
},
}
// Create time-stamped chunks for search
for _, seg := range transcript.Segments {
doc.Chunks = append(doc.Chunks, Chunk{
Text: seg.Text,
Start: seg.Start,
End: seg.End,
})
}
return doc, nil
}
// extractAudio uses ffmpeg to convert any media file to 16kHz mono WAV.
func (ap *AudioProcessor) extractAudio(path string) ([]byte, error) {
cmd := exec.Command("ffmpeg",
"-i", path,
"-vn", // discard video
"-ar", "16000", // 16 kHz sample rate
"-ac", "1", // mono
"-f", "wav", // output format
"pipe:1", // write to stdout
)
var stdout, stderr bytes.Buffer
cmd.Stdout = &stdout
cmd.Stderr = &stderr
if err := cmd.Run(); err != nil {
return nil, fmt.Errorf("ffmpeg: %s", stderr.String())
}
return stdout.Bytes(), nil
}
// transcribe sends WAV audio to the transcription API.
func (ap *AudioProcessor) transcribe(wavData []byte) (*TranscribeResponse, error) {
resp, err := ap.Client.Post(
ap.TranscriberURL+"/transcribe",
"audio/wav",
bytes.NewReader(wavData),
)
if err != nil {
return nil, err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
return nil, fmt.Errorf("transcription service returned %d: %s", resp.StatusCode, body)
}
var result TranscribeResponse
if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
return nil, fmt.Errorf("decode response: %w", err)
}
return &result, nil
}# Shell: ffmpeg commands for common audio extraction scenarios
# Extract audio from MP4 video as 16kHz mono WAV
ffmpeg -i meeting.mp4 -vn -ar 16000 -ac 1 output.wav
# Convert stereo MP3 to mono WAV at 16kHz
ffmpeg -i podcast.mp3 -ar 16000 -ac 1 -f wav output.wav
# Extract audio from MKV, selecting specific audio stream
ffmpeg -i lecture.mkv -map 0:a:0 -vn -ar 16000 -ac 1 -f wav output.wav
# Pipe extracted audio directly (no temp file)
ffmpeg -i recording.mp4 -vn -ar 16000 -ac 1 -f wav pipe:1 | \
curl -X POST -H "Content-Type: audio/wav" \
--data-binary @- http://localhost:8080/transcribe
# Split long audio into 30-second chunks with 2s overlap
ffmpeg -i long_meeting.wav -f segment -segment_time 30 \
-segment_overlap 2 -ar 16000 -ac 1 chunk_%03d.wavOrganizations record hundreds of meetings weekly. Without transcription, finding “the discussion about the API migration timeline” requires someone to remember which meeting it was in and scrub through the recording manually. With automatic transcription, the full text of every meeting is searchable—a query for “API migration timeline” returns the specific meeting and the timestamp where the topic was discussed.
The AudioProcessor enables conference talks, training videos, and screencasts to be searchable alongside written documentation. A developer searching for “database connection pooling” finds not only the docs and code but also the tech talk where the architecture was explained, with a direct link to the relevant timestamp.
Media archives with thousands of audio files become navigable when every file has a searchable transcript. The transcript serves as both a search index and a reading alternative for users who prefer text over audio. Combined with segment timestamps, users can jump directly to the relevant part of a recording.
For pre-recorded files, batch transcription processes the entire file at once (or in large chunks). For live audio—real-time meeting transcription, live captioning—streaming mode sends audio in small windows (e.g., 5-second buffers) and receives partial transcripts with low latency. This implementation uses batch mode because it processes files after they are written, not during recording.
The transcription API may be temporarily unavailable or overloaded. The AudioProcessor retries failed transcription requests with exponential backoff (1s, 2s, 4s, 8s) and a maximum retry count. This absorbs transient failures without overwhelming the service with immediate retries.
Transcription is a pure function: the same audio input always produces the same text output. This makes retries safe—submitting the same audio twice does not create duplicate documents or corrupt state. The pipeline uses content hashing to ensure that even if a file is transcribed multiple times, only one copy of the transcript is stored in the index.
Rather than writing extracted audio to a temporary WAV file on disk, the pipeline streams ffmpeg’s output directly to the transcription API via pipe:1. This avoids disk I/O, reduces cleanup complexity (no temp files to delete on failure), and enables processing on systems with limited disk space. The tradeoff is that the entire audio must fit in memory, which is acceptable for typical file sizes in such a pipeline.