Add production-shaped video processor runtime

[hansent]

Summary

• add the hosted video-job worker as a focused, reviewable runtime change;
• support one spawned OS process per GPU video job while keeping claims, credentials, heartbeats,
  and platform mutations supervisor-owned;
• echo each claim's rotating processorAccessToken on status and results mutations without exposing
  it to child IPC, environment, logs, metrics, or status;
• expose explicit PyAV versus GStreamer NVDEC and tensor-native Inference 1.4 switches;
• fix source-side FPS limiting for NVDEC/tensor mode;
• preserve annotated MediaMTX output and bounded image-redacted JSON events across process IPC;
• preserve the durable architecture, deployment, and multi-cell design extracted from draft POC
  [POC — do not merge] Video sources: connector agent + warm processor + local media plane #2616.

This supersedes the deployable worker portion of #2616. The benchmark corpus and historical result
artifacts stay on the POC branch; this PR is the production-shaped working branch and does not need
to merge yet.

Architecture documentation

• ARCHITECTURE.md — connector/control/media/result flows, repository ownership, process boundaries, and authentication
• DEPLOYMENT.md — cell topology, runtime configuration, exact image provenance, rollout order, smoke gates, and rollback
• MULTI_CELL_SCALING_RFC.md — sticky home-cell placement, relay sharding, dedicated capacity, admission, and phased rollout

Runtime contract

• PROCESSOR_JOB_EXECUTION_MODE=thread|process
• PROCESSOR_VIDEO_INGEST_MODE=pyav|gstreamer_cuda
• ENABLE_TENSOR_DATA_REPRESENTATION=true|false
• ENABLE_FRAME_DROP_ON_VIDEO_FILE_RATE_LIMITING=true|false
• gstreamer_cuda requires tensor representation;
• pixels and tensors remain inside the job process; only bounded image-redacted JSON crosses IPC;
• managed claims fail closed without a claim token, and each reclaim receives a new proof handle.

Exact image evidence

The proof-aware worker images were built from exact runtime revision
bbf0b89e15d4560a2792ed723b05b87a12e410ea. Current PR head includes only the later deployment
evidence update.

• paired Cloud Build: 4795afc7-337b-4061-a739-7a0b70f37148 (SUCCESS)
• GPU: video-processor-runtime@sha256:85f4395ce395789720eaacc72732cc2d08a07174ffc0a6681218d734009b91a2
• CPU: video-processor-cpu-telemetry@sha256:93517c11390ba1267d77f7e8fb8f6a0f5a584d2f4d850527aa81c88843086530
• exact-image smokes: dd2013a2-085c-4fc2-b735-b18a152671f1 GPU and
  1c41387c-33d0-4f27-b04c-29e8890f15a3 CPU (SUCCESS)

The exact GPU and CPU images are currently healthy on staging. GPU is process-per-job with
capacity 8; CPU remains threaded with capacity 4. Both report ready, zero active jobs, and zero
container restarts.

Control-plane compatibility

The current roboflow #14376 server contract and this worker agree on all proof-bearing mutations:

• /video-jobs/:jobId/status including heartbeat, completion, and failure;
• /video-jobs/:jobId/results/upload-urls;
• /video-jobs/:jobId/results/complete.

The token remains in supervisor-only memory and is removed on finish, cancellation, failure,
removal, or reclaim. The old server ignores the backward-compatible extra request field, enabling
worker-first rollout.

All video control traffic must use https://api.roboflow.one in staging or
https://api.roboflow.com in production. Hosting routes it to light-v2-video; workers and other
video components must not target light-v2-device or a direct Functions/Cloud Run URL.

Validation

• focused worker, claim-proof, IPC, lifecycle, result, security, ingest, and source-limiter tests;
• exact source/server static contract assertion for every proof-bearing route;
• paired image build and credential-free exact-image smokes;
• staging GPU and CPU rollout readiness/status checks;
• Markdown and git diff --check validation;
• fresh exact-head GitHub CI is running for the documentation-only follow-up.

Remaining staging gate

Deploy the matching roboflow #14376 light-v2-video function, Hosting rewrites, and app. Then run
one authenticated GPU and CPU lifecycle smoke proving source list, create/claim, heartbeat,
results, preview, JSON events, cancellation, and activeJobs=0. The stable workspace route returns
404 until that API/Hosting deployment is live.

[github-actions]

👋 Thanks for the pull request! Here is how automated Claude review works here, so you spend credits (and reviewer time) wisely.

🚧 Right now this is a draft, so automated Claude review is paused — nothing is being spent yet. Mark it Ready for review to trigger it.

Warning

💸 The Claude reviewer bills in credits, not vibes

Automated review spins up a real agent that reads real code and spends real credits on every pass. It is glad to help — but it is not a rubber duck, a linter you poke in a loop, or a substitute for reading the contributing guide. Treat it like an expensive senior reviewer whose time you booked, and show up prepared.

Draft when unsure, Ready when you mean it:

• 🌱 Not sure the PR is in good shape yet? Keep it (or set it back) as a draft — drafts pause review, so you can push and iterate without burning credits on a moving target.
• 💪 Feel strong about the contents? Mark it Ready for review and the reviewer will take a look.

However you get there, arrive prepared:

• 🧱 Bring a SOLID, thorough PR. Point your local agent at our skills/ to tune it to our guidelines first — or, if you are one of those fabled carbon-based contributors, read them yourself. A half-baked diff costs exactly the same to review as a finished one.
• ✅ Resolve every comment before you re-request review. Re-requesting with threads still open means paying twice for the same conversation.
• 🔁 Do not use CI review as an inner loop for a local agent. The reviewer is not a step-by-step debugger — do the unfolding locally and arrive with the answer, not the search.
• 🙋 If something looks off, ask a human. One question to a maintainer is cheaper and faster than three rounds of agent re-review chasing a misread.

Reviews are not free. A draft costs nothing to review; a Ready PR is a promise that it is worth reviewing.

• Prefer to skip automated review entirely? Add the skip-claude-review label.

[hansent]

Staging GStreamer cadence follow-up (2026-08-18):

• c12 / 30 FPS Logitech Brio / H.264 1280x720 / YOLO detection-only / one L40S / per-job processes.
• Existing one-frame EAGER/FRESHEST buffer: captured 356.5 FPS aggregate, inferred 316.1 FPS, dropped 40.4 FPS; 29.3% of producer gaps were < half a frame and 22.7% were > 1.5 frames.
• Two-frame LAZY/FIFO buffer: captured 357.5 FPS, inferred 352.3 FPS, dropped 5.2 FPS; per-job median 29.46 FPS. Median mean decode-to-result latency moved from 64.9 ms to 70.0 ms; p95 was 150 ms.
• appsink sync=true was rejected: it collapsed delivery to roughly 6 FPS, so clocking the sink is not the fix.

Conclusion: model execution was not slower in NVDEC mode; bursty producer delivery plus the one-frame latest-only InferencePipeline queue discarded otherwise processable frames. The bounded two-frame FIFO absorbs most bursts for ~5 ms median mean-latency cost.

Code is on this branch in a8b6cfb plus IPC telemetry allowlist fixes f901faa / 3dd6dc4. Infra PR roboflow-infra#2454 now enables 2/lazy only for the staging ingest experiment; chart defaults remain 1/eager and the env block remains disabled by default.

[hansent]

Closing in favor of the extraction: the processor runtime now lives in the dedicated https://github.com/roboflow/rf-video-processor repository (code, tests, image builds, and the ARCHITECTURE/DEPLOYMENT/MULTI_CELL docs are canonical there — imported from this branch at 389a634, with base images pinned to inference-server 1.5.0+ so the merged #2832 tensor-painter fix is included). The one inference-side piece, the GStreamer CUDA producer telemetry delta, is #2868. This branch stays as the pre-extraction history and evidence archive alongside #2616.