Skip to content

[Feature]: Pose augmentation cookbook with left/right keypoint remapping #2519

Description

@ternaus

Search before asking

  • I searched the Issues and Discussions and found no similar request.

Feature Description

Would you be open to a separate, self-contained cookbook showing how to augment sv.KeyPoints while preserving left/right landmark semantics?

In pose targets, each position in KeyPoints.xy has a fixed meaning: for example, one row is left_eye and another is right_eye. A horizontal flip must mirror the coordinates and exchange the corresponding left/right rows. Mirroring only the coordinates leaves the target order semantically incorrect.

AlbumentationsX supports transform-aware keypoint row remapping through KeypointParams.label_mapping. The proposed cookbook would:

  • generate a small deterministic image and pose, so it needs no model or dataset download;
  • represent the original pose as sv.KeyPoints;
  • apply HorizontalFlip with explicit mappings for pairs such as left/right eyes and shoulders;
  • set remove_invisible=False so the fixed number of pose landmarks is preserved;
  • rebuild sv.KeyPoints from the transformed coordinates;
  • render the original and transformed poses with Supervision annotators; and
  • verify that each left/right pair retains its canonical row meaning after the flip.

The existing dataset-processing guide and its Albumentations bounding-box section would remain unchanged. This would be a separate pose-specific cookbook, or a short keypoint guide if that fits the documentation structure better.

Example Usage

The core transformation would look like this:

import albumentations as A
import numpy as np
import supervision as sv

keypoint_labels = [
    "left_eye",
    "right_eye",
    "left_shoulder",
    "right_shoulder",
]

augmentation = A.Compose(
    [A.HorizontalFlip(p=1.0)],
    keypoint_params=A.KeypointParams(
        coord_format="xy",
        label_fields=["keypoint_labels"],
        remove_invisible=False,
        label_mapping={
            "HorizontalFlip": {
                "keypoint_labels": {
                    "left_eye": "right_eye",
                    "right_eye": "left_eye",
                    "left_shoulder": "right_shoulder",
                    "right_shoulder": "left_shoulder",
                },
            },
        },
    ),
)

result = augmentation(
    image=image,
    keypoints=key_points.xy[0],
    keypoint_labels=keypoint_labels,
)

augmented_key_points = sv.KeyPoints(
    xy=np.asarray(result["keypoints"], dtype=np.float32)[None, ...],
)

AlbumentationsX reorders the complete keypoint rows for the mapped pairs, so the reconstructed sv.KeyPoints object keeps the fixed pose-target order expected by downstream code.

Dependency scope

AlbumentationsX would be a cookbook-only dependency and would not be added to Supervision's runtime dependencies. The current public package is AGPL-3.0-only. Importing its albumentations module requires an installed PyTorch runtime; users select and install the CPU, CUDA, or MPS build appropriate for their environment.

Contribution

I can prepare the notebook, run it end to end, add the Colab entry and cookbook metadata required by the contribution guide, and keep the example independent of external data and services.

Would this example be welcome? If so, would you prefer a cookbook notebook or a short page in the keypoint documentation?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions