environmental_variables.md

Environment Variables

Inference behavior can be controlled by set of environmental variables. All environmental variables are listed in inference/core/env.py

Below is a list of some environmental values that require more in-depth explanation.

Environmental variable	Description	Default
ROBOFLOW_REGION	Roboflow region to run against: us or eu. Selects the default API/app hosts across inference, inference-cli, inference-sdk and inference-models (e.g. eu points the API at https://api.roboflow.eu). Explicit URL variables such as API_BASE_URL or ROBOFLOW_API_HOST always take precedence.	us
ROBOFLOW_ENVIRONMENT	Roboflow environment to run against: prod or staging. Combines with ROBOFLOW_REGION to pick default hosts (e.g. eu + staging points the API at https://api.roboflow-eu.one). Takes precedence over the legacy PROJECT variable (roboflow-platform vs roboflow-staging).	prod
ONNXRUNTIME_EXECUTION_PROVIDERS	List of execution providers in priority order, warning message will be displayed if provider is not supported on user platform	See here
SAM2_MAX_EMBEDDING_CACHE_SIZE	The number of sam2 embeddings that will be held in memory. The embeddings will be held in gpu memory. Each embedding takes 16777216 bytes.	100
SAM2_MAX_LOGITS_CACHE_SIZE	The number of sam2 logits that will be held in memory. The the logits will be in cpu memory. Each logit takes 262144 bytes.	1000
DISABLE_SAM2_LOGITS_CACHE	If set to True, disables the caching of SAM2 logits. This can be useful for debugging or in scenarios where memory usage needs to be minimized, but may result in slower performance for repeated similar requests.	False
ENABLE_WORKFLOWS_PROFILING	If set to True, in inference server allows the server to output Workflows profiler traces the client, running in Python package with InferencePipeline it enables profiling.	False
WORKFLOWS_PROFILER_BUFFER_SIZE	Size of profiler buffer (number of consecutive Wrofklows Execution Engine run(...) invocations to trace in buffer.	64
RUNS_ON_JETSON	Boolean flag to tell if inference runs on Jetson device - set to True in all docker builds for Jetson architecture.	False
WORKFLOWS_DEFINITION_CACHE_EXPIRY	Number of seconds to cache Workflows definitions as a result of get_workflow_specification(...) function call	15 * 60 - 15 minutes
DOCKER_SOCKET_PATH	Path to the local socket mounted to the container - by default empty, if provided - enables pooling docker container stats from the docker deamon socket. See more here	Not Set
ENABLE_PROMETHEUS	Boolean flag to enable Prometheus /metrics enpoint.	True for docker images in dockerhub
ENABLE_STREAM_API	Flag to enable Stream Management API in inference server - see more.	False
STREAM_API_PRELOADED_PROCESSES	In context of Stream API - this environment variable controlls how many idle processes are warmed-up ready to be a worker for InferencePipeline - helps speeding up workers processes start on GPU	0
TRANSIENT_ROBOFLOW_API_ERRORS	List of (comma separated) HTTP codes from RF API that should be retried (only applicable to GET endpoints)	None
RETRY_CONNECTION_ERRORS_TO_ROBOFLOW_API	Fleg to decide if connection errors for RF API should be retried (only applicable to GET endpoints)	False
ROBOFLOW_API_REQUEST_TIMEOUT	Timeout (in seconds given as integer) for requests to RF API	None
API_PROXY_BASE_URL	Base URL used for Roboflow API proxy requests to apiproxy/* endpoints. Set this to a direct heavy API Cloud Run service root to bypass Firebase Hosting timeouts for long-running third-party proxy calls.	API_BASE_URL
TRANSIENT_ROBOFLOW_API_ERRORS_RETRIES	Number of times transient errors (connection errors and HTTP transient codes) to RF API will be retried (only applicable to GET endpoints)	3
TRANSIENT_ROBOFLOW_API_ERRORS_RETRY_INTERVAL	Delay interval of retries (for connection errors and HTTP transient codes) of RF API requests (only applicable to GET endpoints)	3
METRICS_ENABLED	Flag to control Roboflow Model Monitoring	True
MODEL_MONITORING_CACHE_BACKEND	Cache backend for inference model-monitoring pingback data. Use default to follow the normal cache selection (REDIS_HOST when configured, otherwise memory), or memory to force process-local buffering and keep Redis off the inference hot path.	default
MODEL_VALIDATION_DISABLED	Flag that can make model loading faster by skipping trial inference	False
DISABLE_VERSION_CHECK	Flag to disable inference version check in background thread. Force-set to True (overriding an explicit False) when SECURE_GATEWAY is configured, since api.github.com is unreachable behind the gateway	False
ALLOW_WORKFLOWS_FONTS_DOWNLOAD	Flag allowing runtime download of the approved fonts used by Workflows visualization blocks (e.g. Rich Label Visualization) when a font is not baked into the installation. Downloads only ever target the registry-pinned URLs and are SHA-256 verified. Official Docker images ship with fonts baked in and set this to False. Downloads are also skipped (with an actionable error) when OFFLINE_MODE is enabled or SECURE_GATEWAY is configured, since the fonts' upstream hosts are unreachable there	True
SECURE_GATEWAY	Address of a Roboflow Secure Gateway for air-gapped deployments (legacy alias: LICENSE_SERVER). Routes Roboflow API and model download traffic through the gateway proxy; also force-disables the version check and falls back to local workflow step execution when remote+hosted is configured — see Docker configuration options	Not Set
OFFLINE_MODE	Startup-only flag for air-gapped deployments. The first inference or inference_models import latches the process-environment value, or the current working directory's .env value when the process environment does not declare the flag; runtime environment changes and module reloads are ignored, and changing modes requires a process restart. Descendants inherit the latch when they inherit the parent environment, but a child launched with a sanitized environment is a new startup boundary. The flag blocks built-in Roboflow API, model artifact, telemetry, monitoring, remote-execution, URL image-hydration, and configured WebRTC STUN/TURN endpoint traffic; enables the Hugging Face and Ultralytics dependency offline controls before Inference imports those libraries; loads models exclusively from a trusted, compatible local cache; disables cache eviction and TensorRT engine-file writes; and makes auto-resolution entries never expire. Install the matching inference-models release and re-warm all required models and Workflows online before disconnecting—ambiguous legacy Workflow entries and model entries without canonical ownership plus the required trust and compatibility metadata are rejected. Credential-free fallback also requires one unambiguous canonical model identity; changed or rotated non-empty keys require an exact cache entry. Offline mode is rejected with LAMBDA, GCP_SERVERLESS, dedicated-workspace auth, a workspace whitelist, or the vLLM HTTP proxy; remote and Modal execution modes are forced local, Workflow file caching is forced on with single-tenant cache attribution, and WebRTC uses host candidates only. WebRTC peer connectivity, custom providers, local-code packages, Workflow integrations, explicitly configured cache infrastructure, and sanitized child environments may still use the network, so enforce OS/network isolation for a hard air gap	False
HF_HOME	Hugging Face cache root. When unset, both inference packages set it before heavy imports to $INFERENCE_HOME/hf_home, $MODEL_CACHE_DIR/hf_home, or /tmp/cache/hf_home in that order. The Hub path respects HF_HUB_CACHE, legacy HUGGINGFACE_HUB_CACHE, then legacy TRANSFORMERS_CACHE before defaulting below HF_HOME; HF_MODULES_CACHE is also derived below HF_HOME unless explicitly configured. Built-in repository-ID loaders pass the resolved Hub path explicitly so an earlier third-party Hugging Face import cannot redirect a warm outside the mounted cache. This makes implicit backbone, processor, and checkpoint downloads survive the supported online-warm to fresh-offline restart when the configured cache volume is mounted	Derived from the configured model cache
YOLO_CONFIG_DIR	Mutable Ultralytics settings directory. Inference sets this before importing Ultralytics and keeps it outside MODEL_CACHE_DIR, allowing the model artifact volume to remain read-only offline. Importing Ultralytics before either Inference package is rejected when OFFLINE_MODE=True, because that import snapshots online state before Inference can disable its connectivity probe	Platform temporary directory under roboflow-inference/ultralytics
ALLOW_OFFLINE_MODEL_CACHE_AUTH_BYPASS	Explicitly permits MODELS_CACHE_AUTH_ENABLED=True with OFFLINE_MODE=True, where remote model authorization cannot be performed. Use only for a trusted single-tenant deployment. Without this opt-in, that configuration fails during startup	False
USE_INFERENCE_MODELS	Flag to select inference-models backend	False
MAX_INFERENCE_MODELS_CACHE_SIZE_MB	Variable to enable inference-models cache watchdog to run scheduled cycles of verification for disk space occupied by models - when set with value >0 - watchdog prunes model artefacts from oldest and biggest - to prevent system running out of space over time. Only applicable when USE_INFERENCE_MODELS=True; the watchdog is disabled in OFFLINE_MODE so an immutable warmed cache is never locked or evicted	-1
INFERENCE_MODELS_CACHE_WATCHDOG_INTERVAL_MINUTES	Variable to controll frequency of inference-models cache watchdog cycles - min is 15 mins.	60
ENABLE_CUDA_MEMORY_RECLAMATION_WATCHDOG	Flag to enable a background daemon that periodically returns cached-but-unused CUDA memory to the driver via torch.cuda.empty_cache(). PyTorch's caching allocator retains freed device blocks in its own pool and never releases them on its own, so on a long-running server the high-water mark of concurrent/batched inference is sticky - reserved VRAM only ever grows. This watchdog reclaims that reclaimable slack on a fixed interval (live allocations are untouched). It relieves the sticky high-water mark accumulated across sequential/batched requests; it does not prevent an OOM caused by a genuinely oversubscribed concurrent peak (that live working set is not reclaimable). Only meaningful with USE_INFERENCE_MODELS=True on CUDA.	False
CUDA_MEMORY_RECLAMATION_WATCHDOG_INTERVAL_SECONDS	Interval (in seconds) between reclamation cycles of the CUDA memory watchdog - min is 5 seconds (lower values are clamped up). Only applies when ENABLE_CUDA_MEMORY_RECLAMATION_WATCHDOG=True.	300
ENABLE_HTTPS	Toggles HTTPS for the inference server. When True the server reads SSL_CERTFILE / SSL_KEYFILE and serves traffic over TLS. See HTTPS configuration.	False
SSL_CERTFILE	Path to a PEM-encoded TLS certificate served when ENABLE_HTTPS=True. Defaults to a sane mount point so customers only need to bind their cert to /etc/inference/certs/server.crt.	/etc/inference/certs/server.crt
SSL_KEYFILE	Path to the PEM-encoded TLS private key paired with SSL_CERTFILE. Defaults to /etc/inference/certs/server.key so a single bind mount is enough to enable HTTPS.	/etc/inference/certs/server.key
SSL_KEYFILE_PASSWORD	Optional passphrase used to decrypt SSL_KEYFILE when the private key is encrypted.	Not set
SSL_CA_CERTS	Optional path to a CA bundle used when client certificate verification (mTLS) is required.	Not set
WORKSPACES_WHITELISTED_FOR_LOCAL_DEPLOYMENT	List of whitelisted workspace urls (separator ,) which are meant to be allowed to execute requests against local server. When enabled - each request (apart from docs, landing page and health / liveness / metrics) will be authorised with Roboflow API key	Not set

Offline deployment note

If the deployment includes the separately released Redis usage offloader, upgrade that process to a release that mirrors this version's startup-only OFFLINE_MODE guard and start it with OFFLINE_MODE=True before entering offline operation. Stop the older offloader first and explicitly choose whether to drain or retain any pre-existing UsageCollector records while the deployment is still online. The inference server cannot prevent an older, separately running offloader from transmitting records that were already queued in Redis.