Skip to content

Integrate turbojpeg for accelerated CPU image decoding #1391

Description

@Borda

Context

Currently, our PyTorch DataLoader is bottlenecked by CPU image decoding during the __getitem__ loop. When training RF-DETR on datasets like COCO 2017 (118k images), using standard PIL.Image.open or default OpenCV backends causes heavy CPU contention. Profiling shows high system RAM usage and low CPU compute utilization, indicating that the workers are stalling on single-threaded decoding overhead. This leaves our GPU (e.g., RTX 6000) underutilized due to data starvation.

Proposed Solution

Replace the default JPEG decoder in the dataset class with turbojpeg (or jpeg4py, wrapping libjpeg-turbo).
The implementation should:

  1. Read bytes directly and decode to a NumPy array (RGB).
  2. Pass the decoded array seamlessly into our existing Albumentations CPU pipeline.
  3. Fall back gracefully to cv2 or PIL if the image format is not JPEG or if libjpeg-turbo is not installed on the host machine.

Impact

  • Throughput: libjpeg-turbo is generally 2x to 3x faster than PIL for decoding JPEGs to NumPy arrays.
  • Resource Efficiency: Frees up CPU cycles so workers can spend their compute budget on Albumentations bounding box transforms rather than parsing file bytes.
  • GPU Saturation: Eliminates the dataloader bottleneck, pushing GPU utilization from ~80% to 100%.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions