Context
Currently, our PyTorch DataLoader is bottlenecked by CPU image decoding during the __getitem__ loop. When training RF-DETR on datasets like COCO 2017 (118k images), using standard PIL.Image.open or default OpenCV backends causes heavy CPU contention. Profiling shows high system RAM usage and low CPU compute utilization, indicating that the workers are stalling on single-threaded decoding overhead. This leaves our GPU (e.g., RTX 6000) underutilized due to data starvation.
Proposed Solution
Replace the default JPEG decoder in the dataset class with turbojpeg (or jpeg4py, wrapping libjpeg-turbo).
The implementation should:
- Read bytes directly and decode to a NumPy array (
RGB).
- Pass the decoded array seamlessly into our existing Albumentations CPU pipeline.
- Fall back gracefully to
cv2 or PIL if the image format is not JPEG or if libjpeg-turbo is not installed on the host machine.
Impact
- Throughput:
libjpeg-turbo is generally 2x to 3x faster than PIL for decoding JPEGs to NumPy arrays.
- Resource Efficiency: Frees up CPU cycles so workers can spend their compute budget on Albumentations bounding box transforms rather than parsing file bytes.
- GPU Saturation: Eliminates the dataloader bottleneck, pushing GPU utilization from ~80% to 100%.
Context
Currently, our PyTorch DataLoader is bottlenecked by CPU image decoding during the
__getitem__loop. When training RF-DETR on datasets like COCO 2017 (118k images), using standardPIL.Image.openor default OpenCV backends causes heavy CPU contention. Profiling shows high system RAM usage and low CPU compute utilization, indicating that the workers are stalling on single-threaded decoding overhead. This leaves our GPU (e.g., RTX 6000) underutilized due to data starvation.Proposed Solution
Replace the default JPEG decoder in the dataset class with
turbojpeg(orjpeg4py, wrappinglibjpeg-turbo).The implementation should:
RGB).cv2orPILif the image format is not JPEG or iflibjpeg-turbois not installed on the host machine.Impact
libjpeg-turbois generally 2x to 3x faster than PIL for decoding JPEGs to NumPy arrays.