Skip to content

fix(workflows): preserve image dimensions on empty VLM detections - #2892

Open
davidnichols-ops wants to merge 3 commits into
roboflow:mainfrom
davidnichols-ops:fix/vlm-detectors-empty-lineage
Open

fix(workflows): preserve image dimensions on empty VLM detections#2892
davidnichols-ops wants to merge 3 commits into
roboflow:mainfrom
davidnichols-ops:fix/vlm-detectors-empty-lineage

Conversation

@davidnichols-ops

@davidnichols-ops davidnichols-ops commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Summary

The numpy serialiser returned {"width": null, "height": null} for empty VLM-as-detector results, while the tensor-native path returned real image dimensions. This PR fixes the discrepancy.

Root Cause

sv.Detections.data is per-row. For zero-row detections, the serialiser's per-row loop never executes, so IMAGE_DIMENSIONS_KEY stored in data is never read. The tensor-native path avoids this because it stores image dimensions in image_metadata (a dict on the Detections object, not per-row).

What Changed

  1. New helper empty_detections_with_image_metadata() in utils.py — creates a zero-row sv.Detections with metadata={IMAGE_DIMENSIONS_KEY: [h, w]}. sv.Detections.metadata is a free-form dict that survives zero rows (unlike data, which is per-row).

  2. Serialiser fallback in serialise_sv_detections() — after the per-row loop, if image_dimensions is still None, read from detections.metadata.get(IMAGE_DIMENSIONS_KEY).

  3. All 7 VLM-as-detector parsers (anthropic, gemini, muse, openai, qwen, spacexai, v2/llm) — replaced return sv.Detections.empty() with return empty_detections_with_image_metadata(h, w).

Before

>>> serialise_sv_detections(parse_gemini_object_detection_response(image, [], classes, inf))
{'image': {'width': None, 'height': None}, 'predictions': []}

After

>>> serialise_sv_detections(parse_gemini_object_detection_response(image, [], classes, inf))
{'image': {'width': 640, 'height': 480}, 'predictions': []}

Testing

  • 1905 tests pass (0 failures)
  • New regression tests in 7 test files verify that each parser's empty output carries image dimensions in metadata and serialises to real width/height
  • 2 dedicated serialiser tests verify the metadata fallback path and the backward-compatible None path

Notes

  • sv.Detections.metadata is an existing supervision field that was unused in this codebase
  • sv.Detections.merge() filters empty detections before merging, so metadata on empty detections does not interfere with merge operations
  • The int() call in the serialiser replaces .item() to handle both numpy arrays and plain Python lists from metadata

@davidnichols-ops

Copy link
Copy Markdown
Contributor Author

Because this comes from a fork, could someone approve the Actions run before merging? I'm trying to document the actual external-contributor path rather than infer it from the workflow YAML.

The numpy serialiser returned {"width": null, "height": null} for empty
VLM-as-detector results because sv.Detections.data is per-row — zero rows
means the serialiser per-row loop never executes and IMAGE_DIMENSIONS_KEY
is never read. The tensor-native path stores image dimensions in
image_metadata (a dict on the Detections object), so it correctly emits
real width/height for empty detections.

Fix: store IMAGE_DIMENSIONS_KEY in sv.Detections.metadata (a free-form
dict that survives zero rows) and teach the numpy serialiser to read it as
a fallback when the per-row loop yields no rows.

Before: serialise_sv_detections(parse_...(empty_input)) returns
  {image: {width: null, height: null}, predictions: []}

After: serialise_sv_detections(parse_...(empty_input)) returns
  {image: {width: 640, height: 480}, predictions: []}

All 7 VLM-as-detector parsers updated (anthropic, gemini, muse, openai,
qwen, spacexai, v2/llm). 1905 tests pass.
@davidnichols-ops
davidnichols-ops force-pushed the fix/vlm-detectors-empty-lineage branch from 840f480 to 1049093 Compare August 31, 2026 14:36
@davidnichols-ops davidnichols-ops changed the title fix(workflows): keep image lineage on empty VLM detection results fix(workflows): preserve image dimensions on empty VLM detections Aug 31, 2026
SpaceXAI was the only VLM-as-detector parser without a dedicated test
file. Adds 23 tests covering:
- extract_spacexai_detection_entries (list, wrapper, empty, invalid)
- convert_spacexai_detection_to_pixel_xyxy (scaling, clamping, non-integer)
- parse_spacexai_object_detection_response (empty, wrapper, assembly,
  clamping, error cases, image metadata on empty)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant