Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ uv sync --all-groups
See `pyproject.toml` for complete dependency specifications:

- **Core:** PyTorch, torchvision, transformers, supervision, pydantic, pyDeprecate
- **Optional:** `[train]` (minimal training loop dependencies), `[augment]` (custom Albumentations CPU augmentations and Kornia GPU augmentations), `[lora]` (LoRA fine-tuning), `[plus]` (Plus models), `[onnx]` (ONNX export), `[loggers]` (tensorboard, wandb, mlflow, clearml)
- **Optional:** `[train]` (minimal training loop dependencies, including both COCO evaluation backends selectable via `TrainConfig.eval_backend`), `[augment]` (custom Albumentations CPU augmentations and Kornia GPU augmentations), `[lora]` (LoRA fine-tuning), `[plus]` (Plus models), `[onnx]` (ONNX export), `[loggers]` (tensorboard, wandb, mlflow, clearml)
- **Development:** `tests`, `docs`, `build` groups

**Important version constraints:**
Expand Down
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),

### Added

- Added `TrainConfig.eval_backend`, selecting the COCO evaluator used for validation and test mAP. Both options now ship with `rfdetr[train]`; `"faster_coco_eval"` restores the previous evaluator. Keypoint OKS evaluation is unaffected, and the ONNX/TensorRT benchmark evaluator in `rfdetr.evaluation.coco_eval` continues to use `faster-coco-eval` directly.

- Added `TrainConfig.pack_targets` (default `True`), which concatenates each batch's per-sample target dicts into one tensor per field before they cross the DataLoader worker-to-main boundary, and rebuilds them in `transfer_batch_to_device`. Every tensor a worker returns is moved into its own shared-memory segment and passed to the parent as a file descriptor, so a batch of 16 crosses as 114 objects of which 112 carry a few kilobytes in total; packing takes that to 9 without changing a byte of payload. The rebuilt targets are bit-identical, dtypes included, and the training step receives the same plain list of dicts as before. The maintainer selected the default after submitted detection-loader measurements favored packing. All `RFDETRDataModule` loaders yield batches whose targets are `PackedTargets` rather than a tuple of dicts whenever they pack losslessly, which is visible to direct consumers; a batch that cannot pack losslessly falls back to the original tuple of dicts. Representative detection and segmentation host/device-memory and transfer benchmarks have not yet been collected, so that default's full memory envelope remains a follow-up boundary.

- Added `TrainConfig.eval_batch_size`, decoupling the validation, test and predict dataloaders from the training micro-batch size. The three eval loaders previously always reused the resolved `batch_size`, so lowering it to fit an optimizer step also shrank evaluation batches. Evaluation runs under `no_grad`, which avoids autograd activation storage, but in-fit validation still shares device memory with the model and optimizer state and needs memory for its own forward outputs. The default `None` inherits `batch_size` exactly as before. Unlike `batch_size` it accepts no `"auto"`: an explicit `eval_batch_size` is never probed and stays usable on the `batch_size="auto"` path, while leaving it unset keeps the existing "auto was never resolved" error for eval loaders too. The training dataloader, including its `grad_accum_steps` alignment padding, is unaffected.
Expand All @@ -22,6 +24,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),

### Changed

- **Detection and segmentation COCO evaluation now runs on [hotcoco](https://github.com/derekallman/hotcoco) by default** — a Rust COCO evaluator under MIT with `numpy` as its only runtime dependency, added to the `train` extra. Reported metrics do not change: the parity tests compare every aggregate, per-class and class-ID output of both backends for box-only and box-plus-mask evaluation and require exact equality, which they reach. Set `TrainConfig.eval_backend="faster_coco_eval"` to restore the previous evaluator, which remains installed and is still required — torchmetrics resolves its COCO helpers from a closed backend-name enum with no hotcoco member, so the adapter constructs it with the supported name and replaces the resolved modules. What changes is the cost of `compute()`, not the validation forward pass that usually dominates a validation epoch: on synthetic COCO-val-shaped state (5,000 images, 36.6k ground-truth boxes, 300 detections per image, 80 classes, `eval_max_dets=500`) one macOS-CPU `compute()` took 6.4 s before and 1.5 s after. Most of that is not the evaluator: for box-only evaluation the prediction dataset is now handed to the backend as one detection array instead of the million-plus annotation dictionaries TorchMetrics materializes, which on that state costs 0.55 s where the dictionary path costs 2.7 s. Segmentation, the `faster_coco_eval` backend, and states without stored boxes keep the dictionary path. This is a single-machine CPU measurement on generated detections, not a trained-model or multi-hardware figure. Four hotcoco behaviors are silent wrong answers rather than errors and are handled in the adapter, each with a test that fails if the handling is dropped: its `params` and `dataset` getters both return copies, so field-level mutation is discarded — for `dataset` that would leak one IoU type's annotation areas into the other's COCO size buckets, doubling `bbox_map_small` in the shipped regression fixture; its COCO constructor keeps only the COCO fields it recognizes, dropping the per-IoU-type areas entirely on a round-trip; and its own `mask.encode` emits RLE counts as bytes, which that same constructor reads back as an empty mask, collapsing mask AP to `0.0` without raising. Installing hotcoco also puts a generically-named `coco` console script on `PATH`.

- `RFDETR.predict()` now converts PIL and uint8 NumPy inputs from HWC byte storage to contiguous CHW floating-point storage with one dtype/layout allocation and in-place scaling. The default source-image path reuses its already-materialized PIL array, while non-uint8 NumPy inputs retain torchvision's conversion path. The `[0, 1]` range-scan skip for those same two input types is unaffected: the fused conversion divides `uint8` storage by 255 and carries the same guarantee `to_tensor` did.

- The deformable-attention core now reuses its sampled tensor directly for one-level inputs instead of stacking and flattening a one-element list. Multi-level packing is unchanged. This primarily removes allocation work from keypoint cross-attention, whose current configuration uses one feature level and many more queries than the detection decoder.
Expand Down
Loading
Loading