- Services
- Labs & Products
- Company
Sando
A 7-Service Event-Driven Media Processing Platform in Rust
Seven independently deployable Rust microservices wired via Apache Kafka — handling chunked uploads with SHA-256 deduplication, GPU-accelerated transcoding (NVENC/QSV), HLS/DASH adaptive streaming, and HMAC-signed URL delivery. Zero shared state.
7
microservices
13×
NVENC real-time
100MB/s
upload throughput
Rust
language
Rust + Tokio
Apache Kafka
FFmpeg + NVENC
PostgreSQL
Redis
HLS / DASH
The Problem We're Solving
Most media pipelines are stitched together from off-the-shelf pieces — a Node.js upload server, a Python transcoding queue, a third-party CDN, a separate streaming service. Each piece has its own failure model, its own language, its own data format. When something goes wrong, no one owns the seam between them.
Sando replaces that entire surface with a single, coherent system — seven Rust services that share a common event schema, enforced by Apache Kafka. From the moment a file upload begins to the moment an HLS segment is served over HTTP/3, the data model never changes shape.
What We Care About
Memory Safety at Scale
Media processing is one of the most resource-intensive workloads you can run. Buffer overflows, use-after-free bugs, and data races are not academic concerns here — they are production incidents. Rust eliminates them at the type level.
Independent Deployability
Each service is packaged as its own container and owns exactly the Kafka topics it produces and consumes. Scaling the transcoding workers independently of the upload layer or the streaming layer is a first-class design goal — not an afterthought.
No Duplicated Work
Uploading the same file twice should cost nothing the second time. Content-addressable storage by hash means identical files are deduplicated before any transcoding begins. This single invariant eliminates a whole class of retry and reprocessing bugs.
GPU Where It Matters
CPU-based transcoding doesn't scale economically for video. Sando is designed around NVIDIA NVENC hardware acceleration from day one — not bolted on later. The software fallback exists, but the hardware path is the primary design target.
Why It's Hard
Addressed
Concurrent Upload Correctness
Chunked uploads from multiple clients arriving out-of-order need to be merged exactly once without race conditions. Getting this wrong means corrupt files or missed uploads under load — exactly the kind of bug that only surfaces in production.
Addressed
Transcoding Job Reliability
A GPU worker can crash mid-transcode. The job queue must detect this, recover the job, and re-process without duplicating the output. Kafka's consumer group semantics handle the delivery side; the locking strategy handles the processing side.
Addressed
Secure Streaming at the Edge
HLS segments cannot be served to unauthenticated clients without proper token validation. URL signing must be fast enough not to add latency to segment playback, and tokens must expire without breaking active streams.
Active
HTTP/3 Segment Delivery
QUIC eliminates TCP head-of-line blocking, which matters significantly for HLS on mobile networks. We're evaluating whether the latency gains under packet loss justify the operational complexity of running a QUIC endpoint in production.
Where We Are
Sando moves in deliberate phases — design first, then build, then ship.
done
System Design & Schema
- #Service boundary and responsibility definitions
- #Database schema across all persistence layers
- #Event topology and inter-service contract design
- #Inter-service API contract definitions
in progress
Core Services
- #Gateway, Upload, and Coordination services
- #Worker: video transcoding and image processing
- #Integration and reliability testing
upcoming
Delivery & Production
- #Adaptive streaming delivery and URL signing
- #Kubernetes deployment and autoscaling
- #Observability and distributed tracing
Interested in Sando?
Sando is a private project currently in active development. If you're building a media platform and want to discuss the design challenges, or if you're interested in working with us on a similar system, we'd be happy to talk.