Cosmos AnomalyGen: package runtime, packaging tool, workflow-block productionization

[probicheaux]

Successor to #2676 (closed when its base stack #2675 squash-merged); the branch was rebuilt on latest main as the anomalygen delta only.

⚠️ Human-blocked open items (called out per the integration plan)

1. Task-type ratification: "image-generation" for cosmos-anomalygen must be what /models/v1/external/stat returns — needs the models-service (GOAT) enum addition; confirm with Inference Core. (On the platform, image-generation is also a billing-meter name in a different namespace — do not reuse it as a meter.)
2. Trust marking: the package ships executable Python (cosmos_anomalygen_runtime.py); it must be registered trusted_source server-side, or callers need ALLOW_INFERENCE_MODELS_UNTRUSTED_PACKAGES=True.
3. NVIDIA license review for paidf-anomalygen-derived packaging (docs page marks it "review pending").
4. Packaging/registration (surface 2) is Core-team tooling; development/cosmos3/pull_anomalygen_weights.py produces the package + --gcs-dest mirror.

What this adds

• cosmos_anomalygen_runtime.py (reference, inference_models/development/cosmos3/) — the self-contained runtime that ships inside the package. Builds the GA config (predict2_anomaly_gen_ddp_2b + the run's frozen ag_config.yaml), repoints every frozen-tower path into the package (DiT, VAE tokenizer, t5-large, NV-DINOv2, guardrail dir), and reuses NVIDIA's SDG entry path 1:1 for generation (temp one-line JSONL → AnomalyInpaintDataset → inpaint_image), so preprocessing/RePaint/crop-paste stay byte-compatible with synthetic_dataset_generation. Requires the paidf-anomalygen:ga container environment (lazy imports; clear error otherwise).
• pull_anomalygen_weights.py — assembles the flat package (base towers + trained adapter + runtime + class_names.txt + inference_config.json) with hardlinks, optional --gcs-dest.
• Production-recipe defaults: guidance 1.5 (≈ CFG 2.5), 35 steps, crop_ratio 4.0, crop-and-paste on, Poisson off — model class + workflow block.
• Workflow block: exposes crop_ratio / poisson_blend; new visibility output (mean abs pixel change inside the placement mask, 0–255) so callers can filter/regenerate empty generations (≥15 worked in the Corning CV study); REMOTE now rejected at construction (compile time); discover_dependent_resources() declared.
• Tests: defaults + missing-runtime-module coverage; block manifest/visibility tests; env-gated integration test (COSMOS_ANOMALYGEN_PACKAGE_DIR).
• Docs: inference_models/docs/models/cosmos-anomalygen.md + nav ("Image Generation" group) + index rows; changelog.

Test evidence

• E2E on real weights (L4 23 GB, GA container, real Corning tube+hole checkpoint @ iter 35k): package assembled by the tool → runtime load() 27 s → generate() 10.1 s/image at 35 steps, output at native input resolution, visibility 49.3 (≥15 threshold), PSNR 12.8.
• inference_models unit suite: 1552 passed (4 pre-existing runtime_introspection failures reproduce on pristine main in the same env).
• Workflows unit suite: 6103 passed, 1 pre-existing-convention failure (test_dependent_resources) fixed by this PR.
• black/isort/flake8 (CI-pinned versions) clean on check_dirs.

🤖 Generated with Claude Code

https://claude.ai/code/session_015qKkor5AsgasrsFz3j7TA2

[github-actions]

👋 Thanks for the pull request! Here is how automated Claude review works here, so you spend credits (and reviewer time) wisely.

🚦 This PR is marked Ready for review, so automated Claude review will run — and every pass spends real credits.

Warning

💸 The Claude reviewer bills in credits, not vibes

Automated review spins up a real agent that reads real code and spends real credits on every pass. It is glad to help — but it is not a rubber duck, a linter you poke in a loop, or a substitute for reading the contributing guide. Treat it like an expensive senior reviewer whose time you booked, and show up prepared.

Draft when unsure, Ready when you mean it:

• 🌱 Not sure the PR is in good shape yet? Keep it (or set it back) as a draft — drafts pause review, so you can push and iterate without burning credits on a moving target.
• 💪 Feel strong about the contents? Mark it Ready for review and the reviewer will take a look.

However you get there, arrive prepared:

• 🧱 Bring a SOLID, thorough PR. Point your local agent at our skills/ to tune it to our guidelines first — or, if you are one of those fabled carbon-based contributors, read them yourself. A half-baked diff costs exactly the same to review as a finished one.
• ✅ Resolve every comment before you re-request review. Re-requesting with threads still open means paying twice for the same conversation.
• 🔁 Do not use CI review as an inner loop for a local agent. The reviewer is not a step-by-step debugger — do the unfolding locally and arrive with the answer, not the search.
• 🙋 If something looks off, ask a human. One question to a maintainer is cheaper and faster than three rounds of agent re-review chasing a misread.

Reviews are not free. A draft costs nothing to review; a Ready PR is a promise that it is worth reviewing.

• Prefer to skip automated review entirely? Add the skip-claude-review label.

[github-actions]

🤖 Claude review started at commit 456946e6c98cc4c93450c3e14959ae1cf9b0834d.

New commits are not auto-reviewed. Add the claude-review label to request a re-review — the label is consumed when the review starts, so just add it again next time.

[github-actions]

Medium/High — declaring a model-manager dependency for a model this block loads via AutoModel, contradicting the sam2_video/sam3_video policy the block claims to follow.

This block loads its weights with AutoModel.from_pretrained(...) in _resolve_model() (in-process, not through the model manager) — exactly like segment_anything2_video/v1.py. Those blocks deliberately do not implement discover_dependent_resources():

# segment_anything2_video/v1.py
# `discover_dependent_resources()` deliberately not implemented: this
# block loads its weights via AutoModel.from_pretrained, not the model
# manager — dependencies stay undeclared (None) for now.

and are listed in NON_MODEL_MANAGER_LOADERS_ALLOWLIST in tests/workflows/unit_tests/core_steps/test_dependent_resources.py.

By returning roboflow_platform_model(model_id=...) here (defaulting to required_action=EXECUTION → execution_location=ENVIRONMENT_DEFINED), this block opts into model-manager pre-loading. Concrete failure path: an InferencePipeline initialized with workflows_dependencies_pre_init=["roboflow_platform_model"] under LOCAL step execution → _is_locally_executed_platform_model() returns True (EXECUTION + ENVIRONMENT_DEFINED + LOCAL) → _pre_load_roboflow_platform_models() calls model_manager.add_model(model_id="cosmos-anomalygen", ...) (inference/core/workflows/execution_engine/v1/core.py:191). But cosmos-anomalygen is served only through the inference_models AutoModel path (custom backend, generate() contract) — it is not a model-manager model, so add_model registers the wrong thing / fails, and the block never uses that registration at runtime anyway (it re-loads via AutoModel).

Note CI is green here: test_every_block_with_resource_kind_fields_declares_dependencies only checks that a roboflow_model_id-field block either overrides discover_dependent_resources() or is allowlisted — so declaring the override silences the guard without making the pre-load path correct.

Recommend matching the cited precedent: drop this override (leave dependencies undeclared / None) and add roboflow_core/cosmos_anomalygen@v1 to NON_MODEL_MANAGER_LOADERS_ALLOWLIST. If instead the intent is genuine model-manager execution, the block must actually run through the model manager rather than AutoModel.

Reviewed at HEAD: 456946e

[github-actions]

Review summary

Skills: review-workflows-blocks, review-inference-models-pkg, review-topic-local-vs-remote-execution, review-topic-input-boundary-security, review-topic-external-contract-and-silent-fallback, review-topic-backward-compat-and-versioning, review-topic-test-hygiene

New Cosmos AnomalyGen surface: inference_models model class + registry entry, package runtime + packaging tool (development/cosmos3/), and a roboflow_core/cosmos_anomalygen@v1 workflow block. The block/model wiring closely mirrors the merged cosmos3_world model and the sam2_video block, and the color/mask contract between the block and the model class is internally consistent (BGR in, BGR out, RGB only inside the runtime). One blocking finding below; the rest are release-coordination items already called out in the PR description.

Blocking finding (inline):

• cosmos_anomalygen/v1.py:192 — the block loads via AutoModel.from_pretrained (not the model manager) yet declares discover_dependent_resources(), opting into model_manager.add_model("cosmos-anomalygen") during opt-in dependency pre-loading (InferencePipeline started with workflows_dependencies_pre_init=["roboflow_platform_model"] under LOCAL step execution). This contradicts the documented sam2_video / sam3_video policy the block cites as its precedent. See the inline comment for the exact path and fix.

Version / changelog notice (maintainers): functional inference_models change — inference-models requires a release-time version change. The contributor correctly added the ## Unreleased entry in inference_models/docs/changelog.md; no contributor version bump is expected. Adding a new workflow block does not by itself force an Execution Engine behavior change, so no EE changelog entry is required here.

Tests to add (behavior-level):

• A workflow compilation unit test exercising the dependency-pre-loading path under LOCAL step execution that asserts this block does NOT trigger model_manager.add_model for cosmos-anomalygen (the path the finding above breaks; the current suite is green despite it). Area: execution-engine dependency-pre-loading unit test.
• The real generate() path is only covered by an env-gated GPU + GA-container integration test — acceptable given runtime constraints, but note it does not run in CI, so the mocked unit tests are the only enforced coverage of the model class.

Minor doubts (non-blocking): discover_dependent_resources() declares execution_location=ENVIRONMENT_DEFINED while the block is LOCAL-only (REMOTE raises at construction) — if the declaration is kept at all it would more accurately be LOCAL; the reference runtime file under development/cosmos3/ executes only inside the GA container and was reviewed for contract shape, not runtime behavior.

Reviewed at HEAD: 456946e

[github-actions]

@probicheaux — a couple of the human-blocked items you called out are release-gating for this block and need confirmation before sign-off.

⏳ This PR is on hold pending your answers — the review will not advance to sign-off until the IMPORTANT questions below are answered (and the blocking finding on cosmos_anomalygen/v1.py:192 is addressed).

IMPORTANT questions

1. Task-type ratification (image-generation). AutoModel.from_pretrained("cosmos-anomalygen", ...) resolves the registry key ("cosmos-anomalygen", "image-generation", BackendType.CUSTOM) from what the weights/stat service returns as taskType. Until /models/v1/external/stat returns image-generation for this model (the GOAT enum addition you note), the model id will not resolve and both CosmosAnomalyGen and the workflow block are non-functional against a registered id. What answer would change the outcome: confirmation (with the models-service change landed/scheduled) that taskType: image-generation is returned for cosmos-anomalygen — otherwise this should not ship as a registered-id path yet. See inference_models/inference_models/models/auto_loaders/models_registry.py (new IMAGE_GENERATION_TASK entry) and pull_anomalygen_weights.py INFERENCE_DEFAULTS["task_type"].

2. Trust marking of the shipped executable runtime. The package ships cosmos_anomalygen_runtime.py, imported and executed via import_class_from_file in runtime_loading.py. Loading it requires the package be registered trusted_source server-side, or ALLOW_INFERENCE_MODELS_UNTRUSTED_PACKAGES=True. What answer would change the outcome: confirmation of which mechanism will gate production loading (trusted registration vs. the env flag) so this executable-code-in-package path is not silently loaded from an untrusted provider. This is a security-sensitive decision for @PawelPeczek-Roboflow / @grzegorz-roboflow / @dkosowski87 to ratify.

(The NVIDIA license review for the paidf-anomalygen-derived packaging — your item #3 — is a maintainer/legal gate, not a code question; flagging it here only so it is not lost. The docs page correctly marks it "review pending".)

New commits are NOT auto-reviewed — add the claude-review label to request a re-review (the label is consumed when the review starts, so just add it again).

Reviewed at HEAD: 456946e