NEXUS: Video Versions Without Manual Editing
A media system that turns long video into platform-ready variants with face tracking, smart crops, GPU encoding, and checks.
Long video gets expensive when every crop, subtitle, and export is manual.
WHERE IT BREAKS
Every platform crop, face frame, subtitle, and encode needs manual editing.
WHAT CHANGED
Face tracking, crop choices, GPU encoding, subtitles, and checks run together.
WHAT YOU CAN SEE
Input size, GPU use, framing, subtitles, and exports are visible.
~3 min / 10-min video
Turnaround
face stays in frame
Framing
9:16, 1:1, 16:9
Formats
Where It Was Risky
Content creators producing video for social media face a production bottleneck that scales linearly with volume:
- Manual Editing Cost: Professional short-form video editing costs $50-200 per video.
- Format Fragmentation: Each platform requires different aspect ratios (9:16, 1:1, 16:9).
- Quality Inconsistency: Human editors produce variable quality depending on fatigue and skill.
- Turnaround Time: Traditional editing takes hours per video.
What Changed in the System
INPUT STAGE
Video Decode (FFmpeg) → Scene Detection → Audio Extraction (Whisper)
↓
ML INFERENCE
Face Detection + Tracking (ONNX, CUDA) | 12,000+ keyframes/video
↓
OUTPUT PATH
Smart Crop (face-centered) → Format Adaptation (9:16/1:1/16:9) → GPU Encode (NVENC)What Has to Work
- Video Decoder: FFmpeg-based frame extraction with hardware-accelerated decoding (CUVID).
- Face Detection & Tracking: ONNX neural network with CUDA. Processes 12,000+ keyframes per video.
- Smart Crop Engine: Face-centered framing with dynamic crop region adjustment.
- Format Adaptation: Automated aspect ratio conversion (16:9 → 9:16/1:1) with intelligent reframing.
- GPU Encoder: NVENC H.264 encoding, 4-10x faster than CPU.
Decisions That Remove Risk
Problem
CPU-based ONNX inference for face tracking takes 15+ minutes per video.
Solution
CUDA Execution Provider for ONNX Runtime. Face detection on GPU reduces inference time 4-10x. Combined with NVENC encoding, the whole video job runs on GPU.
Alternative Rejected
Cloud GPU inference — Network latency, egress costs, vendor dependency. Local GPU amortizes to near-zero cost.
Problem
Raw bounding box detections per-frame produce jittery camera movement.
Solution
Bounding box interpolation with exponential moving average. Detections sampled at keyframes, interpolated for intermediate frames, producing cinematic smooth panning.
What the Output Shows
~3 min / 10-min video
Turnaround
face stays in frame
Framing
9:16, 1:1, 16:9
Formats
12,000+ / video
Face Keyframes
4-10x vs CPU
GPU Acceleration
Constant (streaming)
Memory Usage
~$0 (local GPU)
Cost per Video
Tools Behind It
Core
Rust, FFmpeg, ONNX Runtime
GPU
CUDA (NVENC encode, CUVID decode, ONNX inference)
ML
Face detection neural network (ONNX format)
Audio
Whisper (speech-to-text for subtitles)
Why It Matters
Media Work Without Manual Assembly
Input scale, hardware capability, platform formats, and output checks are coordinated as one production loop.
Hardware-Aware Throughput Ownership
The processing path uses local GPU capacity because turnaround, cost, and repeatability matter more than stack display.
Bounded Processing at Volume
Large videos move through controlled stages without turning memory use into a hidden failure mode.
Output Quality Control
Face tracking and crop smoothing serve one user-visible goal: platform-ready framing that does not look automated.
Cost and Turnaround Compression
Creator video work that used to depend on manual editing becomes repeatable, fast, and locally economical.
> Private repository. Available for code review on request.