fix(stream manager): exit gracefully when another worker already bound the port

[bar-jpg]

Problem

The Stream Manager is started from the app module — docker/config/gpu_http.py and docker/config/cpu_http.py — which uvicorn imports once per worker. It binds a fixed STREAM_MANAGER_HOST:STREAM_MANAGER_PORT (127.0.0.1:7070 by default, manager_app/app.py), through a TCPServer that never set allow_reuse_address.

So with NUM_WORKERS > 1, every worker starts a manager, one wins the bind, and the rest die with an unhandled:

OSError: [Errno 98] Address already in use

The GPU/CPU images ship ENABLE_STREAM_API=True with NUM_WORKERS=1, so the default config is safe — this is only reachable when a user raises NUM_WORKERS, which the docs did not flag as a hazard.

Reported by a customer running roboflow-inference-server-gpu with NUM_WORKERS=4 on an A2, who sees the error at startup along with uvicorn workers dying and respawning, ending up with fewer workers serving than configured and reduced FPS. Their workaround is restarting the container, which just re-runs the same race.

Why a single manager is the right end state

Not a regression, and not fixed by upgrading — docker/config/gpu_http.py is byte-identical from v1.0.1 through main.

inference/core/interfaces/http/http_api.py builds a StreamManagerClient per worker pointing at the same 127.0.0.1:7070. The design is already one manager, many clients — a single manager per container correctly serves all workers. Only the spawn location is wrong.

Changes

• manager_app/app.py — start() catches OSError with errno.EADDRINUSE around the server construction, logs an explanatory line, and returns cleanly instead of raising. The bind now happens before the health-check and warm-up threads start, so an instance that loses the race exits without preloading pipelines (previously STREAM_API_PRELOADED_PROCESSES would have been honoured by all of them). Other OSErrors still propagate.
• manager_app/tcp_server.py — allow_reuse_address = True, matching http.server.HTTPServer, so a restarted manager can rebind while the previous socket lingers in TIME_WAIT. Verified that this does not permit two live listeners, so the port remains an effective mutex and does not mask the condition above.
• docker/config/{cpu,gpu}_http.py — log once at startup when ENABLE_STREAM_API is set with NUM_WORKERS > 1, so the behaviour is greppable in exactly the logs users send us.
• docs/quickstart/docker_configuration_options.md — document the interaction under NUM_WORKERS. Also mentions the existing PRELOAD_HF_IDS constraint already noted in inference/core/env.py, which had never made it into the docs.
• Tests — test_app.py covers both the graceful-exit path (asserting pipelines are not warmed up) and that non-EADDRINUSE errors still surface; test_tcp_server.py covers allow_reuse_address.

Reproduction

No GPU required — cpu_http.py has the identical defect:

docker run --rm -e NUM_WORKERS=4 -e ENABLE_STREAM_API=True roboflow/roboflow-inference-server-cpu

Before: [Errno 98] NUM_WORKERS - 1 times. After: one manager log line, the rest exit quietly.

Notes for reviewers

• Scope: both docker entrypoint configs, and ENABLE_STREAM_API=True ships in 8 images (CPU, CPU.dev, GPU, GPU.dev, GPU.3d, TRT, Jetson 5.1.1, Jetson 6.2.0).
• What this PR does not claim. The port conflict is proven from source. Whether it is also what terminates the uvicorn workers in the customer's logs is not proven — their excerpt is truncated above the OSError, so the traceback frames are missing. This removes the conflict either way; if worker deaths persist there is a second cause to chase.
• Deliberately not included: a file lock or entrypoint-level guard so losing workers never spawn a manager at all. It would save a few wasted spawn imports at startup, but adds stale-lock semantics and a worker-restart edge case — an orphaned manager keeps the port after its parent worker dies, so the lock and the port can disagree. The handling here covers that case correctly. Happy to add it if maintainers prefer.

🤖 Generated with Claude Code

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[github-actions]

👋 Thanks for the pull request! Here is how automated Claude review works here, so you spend credits (and reviewer time) wisely.

🚧 Right now this is a draft, so automated Claude review is paused — nothing is being spent yet. Mark it Ready for review to trigger it.

Warning

💸 The Claude reviewer bills in credits, not vibes

Automated review spins up a real agent that reads real code and spends real credits on every pass. It is glad to help — but it is not a rubber duck, a linter you poke in a loop, or a substitute for reading the contributing guide. Treat it like an expensive senior reviewer whose time you booked, and show up prepared.

Draft when unsure, Ready when you mean it:

• 🌱 Not sure the PR is in good shape yet? Keep it (or set it back) as a draft — drafts pause review, so you can push and iterate without burning credits on a moving target.
• 💪 Feel strong about the contents? Mark it Ready for review and the reviewer will take a look.

However you get there, arrive prepared:

• 🧱 Bring a SOLID, thorough PR. Point your local agent at our skills/ to tune it to our guidelines first — or, if you are one of those fabled carbon-based contributors, read them yourself. A half-baked diff costs exactly the same to review as a finished one.
• ✅ Resolve every comment before you re-request review. Re-requesting with threads still open means paying twice for the same conversation.
• 🔁 Do not use CI review as an inner loop for a local agent. The reviewer is not a step-by-step debugger — do the unfolding locally and arrive with the answer, not the search.
• 🙋 If something looks off, ask a human. One question to a maintainer is cheaper and faster than three rounds of agent re-review chasing a misread.

Reviews are not free. A draft costs nothing to review; a Ready PR is a promise that it is worth reviewing.

• Prefer to skip automated review entirely? Add the skip-claude-review label.

[jarbas-roboflow]

Status: ❌ Request Changes — Gist: https://gist.github.com/jarbasrf/b335d7ab56a46b4d5c6b4725590add50. Resumo: The PR addresses the duplicate GPU worker bind race, but treating every EADDRINUSE as a healthy existing Stream Manager can mask genuine port conflicts and leave stream inference broken.