Design a multimodal image and 1080p video diffusion platform serving 50M+ generations/day (45M images, 5M 1080p video clips; 1,736 peak QPS) across 2,000+ H100/B200 GPUs with <= 4.5s image p95, <= 45s 1080p video latency, progressive live previews, dual-gate safety filtering, and cryptographic C2PA provenance.
From patterns to decisions.
Use a Two-Tier Priority Queue (Redis ZSET interactive + Kafka spot batch) feeding 8-GPU NVLink Ring Attention DiT pods with TinyVAE SSE previews, Dual-Gate CLIP/ViT safety, and HSM-signed C2PA provenance.
Sizing 50M Generations/Day & 202.5 TB/Day Media Egress.
Assume 45M images/day (~1.5s on 1 H100) + 5M 1080p videos/day (~35s on 8x H100 Ring) with 3x peak-to-average ratio (1,736 peak QPS).
- Image Compute: ~780 GPU-days/day.
- Video Compute: ~16,200 GPU-days/day (across 8-GPU rings).
- Storage Ingress: ~202.5 TB/day written to S3 with CDN edge caching.
Fleet & Bandwidth Sizing
1.Stage Objective: Compute GPU fleet size, queue throughput, and S3/CDN bandwidth for 45M images + 5M videos/day.
2.Level (Foundation): Assume 45M images/day (~1.5s on 1 H100) + 5M 1080p videos/day (~35s on 8x H100 Ring) with 3x peak-to-average ratio (1,736 peak QPS).
3.Operational Invariant: Video generation dominates GPU-hours (> 95%), justifying spot-pool isolation from interactive image pods.
Caption: Video generation dominates GPU-hours (> 95%), justifying spot-pool isolation from interactive image pods.
Redis Priority ZSET (Interactive) vs. Kafka Spot Queue (Batch).
Interactive user requests enter a Redis Priority ZSET scored by (tier_weight, enqueue_ts) and dispatch strictly to reserved on-demand GPUs. Bulk API and long video jobs flow into Kafka topics consumed by spot GPU workers.
If interactive queue depth spikes, reserved pods never accept batch work.
Two-Tier Queue Isolation
1.Stage Objective: Eliminate head-of-line blocking between 4.5s image requests and 45s video jobs.
2.Level (Foundation): Interactive user requests enter a **Redis Priority ZSET** scored by `(tier_weight, enqueue_ts)` and dispatch strictly to reserved on-demand
3.Operational Invariant: Separating interactive Redis ZSETs from Kafka spot queues protects the <= 4.5s image p95 SLA.
Caption: Separating interactive Redis ZSETs from Kafka spot queues protects the <= 4.5s image p95 SLA.
Implementing Spot-Resilient 2-Step Latent Checkpointing.
class DiffusionJobRunner:
def __init__(self, checkpoint_store, dit_engine, total_steps: int = 8) -> None:
self.store = checkpoint_store
self.dit = dit_engine
self.total_steps = total_steps
def run_job(self, job_id: str, prompt_emb) -> bytes:
state = self.store.load_latest(job_id)
start_step = state.step if state else 0
latent = state.latent if state else self.dit.sample_noise(job_id)
for step in range(start_step, self.total_steps):
latent = self.dit.denoise_step(latent, prompt_emb, step)
if (step + 1) % 2 == 0 and (step + 1) < self.total_steps:
self.store.save_async(job_id, step + 1, latent)
return self.dit.vae_decode(latent)2-Step Latent Checkpoint Loop
1.Stage Objective: Resume preempted video jobs from the latest 2-step latent checkpoint instead of restarting.
2.Level (Intermediate): class DiffusionJobRunner:
3.Operational Invariant: Saving intermediate latents every 2 steps bounds spot preemption loss to at most 2 denoising steps.
Caption: Saving intermediate latents every 2 steps bounds spot preemption loss to at most 2 denoising steps.
8-GPU NVLink Ring Attention + TinyVAE SSE Streaming.
Each video worker pod binds 8 H100s over NVLink running Ring Attention + FP8 FlashAttention-3. At steps 1, 2, 4, and 8, a side-stream TinyVAE decodes a low-res WebP draft and publishes it to Redis Pub/Sub -> API Gateway SSE.
Ring Worker + SSE Preview
1.Stage Objective: Execute 243K-token video DiTs and stream < 20 ms progressive previews to clients.
2.Level (Intermediate): Each video worker pod binds 8 H100s over NVLink running Ring Attention + FP8 FlashAttention-3. At steps 1, 2, 4, and 8, a side-stream TinyVA
3.Operational Invariant: Side-stream TinyVAE pushes progressive WebP previews over Redis Pub/Sub to SSE clients.
Caption: Side-stream TinyVAE pushes progressive WebP previews over Redis Pub/Sub to SSE clients.
Pre-Gen INT8 CLIP Text Gate & Post-Gen ViT Classifier.
Gate 1 runs an INT8-quantized CLIP text safety classifier (< 3 ms) at ingress before enqueuing. Gate 2 runs a post-generation ViT NSFW/CSAM classifier on the decoded image/video keyframes before uploading to public S3/CDN.
Dual-Gate Safety Pipeline
1.Stage Objective: Block unsafe prompts before wasting GPU compute and catch adversarial visual outputs.
2.Level (Advanced): Gate 1 runs an INT8-quantized CLIP text safety classifier (< 3 ms) at ingress before enqueuing. Gate 2 runs a post-generation ViT NSFW/CSAM
3.Operational Invariant: Pre-gen CLIP saves wasted GPU cycles; post-gen ViT catches adversarial prompt bypasses.
Caption: Pre-gen CLIP saves wasted GPU cycles; post-gen ViT catches adversarial prompt bypasses.
Invisible DCT Watermarking & Merkle-Batched HSM Signing.
Embed an invisible frequency-domain (DCT/SynthID) watermark into pixel output. Because signing 1,736 individual RSA/ECDSA payloads/sec would saturate HSMs, build a Merkle tree over batches of 256 asset SHA-256 hashes every 100 ms and sign only the Merkle root with the HSM, embedding the compact Merkle inclusion proof into each asset's C2PA manifest.
Merkle-Batched HSM C2PA Signing
1.Stage Objective: Sign 50M assets/day cryptographically without bottlenecking on Hardware Security Modules (HSMs).
2.Level (Advanced): Embed an invisible frequency-domain (DCT/SynthID) watermark into pixel output. Because signing 1,736 individual RSA/ECDSA payloads/sec would
3.Operational Invariant: Signing one Merkle root per 256 assets scales C2PA cryptographic provenance to 1,736 QPS.
Caption: Signing one Merkle root per 256 assets scales C2PA cryptographic provenance to 1,736 QPS.
Predictive Queue-Depth Scaling & Cold-Weight Caching.
Cache 30 GB FP8 DiT weights on local NVMe RAID-0 + host page cache across worker nodes and pre-warm standby pods using queue_wait_p95 predictive scaling. Alert if interactive_ttft_p95 > 4.5s or spot_preemption_rate > 15%.
NVMe Weight Cache & SLO Alerts
1.Stage Objective: Prevent 30 GB DiT model-weight cold-start spikes during traffic surges.
2.Level (Advanced): Cache 30 GB FP8 DiT weights on local NVMe RAID-0 + host page cache across worker nodes and pre-warm standby pods using `queue_wait_p95` pred
3.Operational Invariant: Local NVMe model-weight caching eliminates 30s+ S3 download stalls during autoscaling.
Caption: Local NVMe model-weight caching eliminates 30s+ S3 download stalls during autoscaling.
Practice: Make the trade-off yourself.
How does the platform sign 1,736 generated images/videos per second with C2PA provenance without overwhelming HSM hardware?