Skip to content

Commit 6a3c5cd

Browse files
Bordaclaude
andcommitted
docs: update README scores + add SORT Phase 1 journal
- README Algorithms table: add MOT17 HOTA (tuned) column; SORT 55.7→57.7 - autotune/program.md: SORT Phase 1 findings section with 9 kept changes, 5 reverted, tuned best config (HOTA=57.7, +8.4% from 53.217 baseline) --- Co-authored-by: Claude Code <noreply@anthropic.com>
1 parent f873a75 commit 6a3c5cd

2 files changed

Lines changed: 56 additions & 6 deletions

File tree

README.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -74,13 +74,13 @@ https://github.com/user-attachments/assets/d2347a25-469d-44cd-8049-d15274bd91ae
7474

7575
## Algorithms
7676

77-
Clean, modular implementations of leading trackers. All HOTA scores use default parameters.
77+
Clean, modular implementations of leading trackers. HOTA scores at default parameters; tuned scores use Optuna-optimised hyperparameters from the autotune campaign (MOT17-SDP).
7878

79-
| Algorithm | Description | MOT17 HOTA | SportsMOT HOTA | SoccerNet HOTA | DanceTrack HOTA |
80-
| :-------------------------------------------: | :-------------------------------------------------------------: | :--------: | :------------: | :------------: | :-------------: |
81-
| [SORT](https://arxiv.org/abs/1602.00763) | Kalman filter + Hungarian matching baseline. | 58.4 | 70.9 | 81.6 | 45.0 |
82-
| [ByteTrack](https://arxiv.org/abs/2110.06864) | Two-stage association using high and low confidence detections. | 60.1 | **73.0** | **84.0** | 50.2 |
83-
| [OC-SORT](https://arxiv.org/abs/2203.14360) | Observation-centric recovery for lost tracks. | **61.9** | 71.7 | 78.4 | **51.8** |
79+
| Algorithm | Description | MOT17 HOTA | MOT17 HOTA (tuned) | SportsMOT HOTA | SoccerNet HOTA | DanceTrack HOTA |
80+
| :-------------------------------------------: | :-------------------------------------------------------------: | :--------: | :----------------: | :------------: | :------------: | :-------------: |
81+
| [SORT](https://arxiv.org/abs/1602.00763) | Kalman filter + Hungarian matching baseline. | 55.7 | **57.7** | 70.9 | 81.6 | 45.0 |
82+
| [ByteTrack](https://arxiv.org/abs/2110.06864) | Two-stage association using high and low confidence detections. | 60.1 | 59.1 | **73.0** | **84.0** | 50.2 |
83+
| [OC-SORT](https://arxiv.org/abs/2203.14360) | Observation-centric recovery for lost tracks. | **61.9** | 57.9 | 71.7 | 78.4 | **51.8** |
8484

8585
For detailed benchmarks and tuned configurations, see the [tracker comparison](https://trackers.roboflow.com/develop/trackers/comparison/).
8686

autotune/program.md

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -163,6 +163,56 @@ Provided as inspiration, not a prescribed order. Hypotheses apply to the active
163163

164164
**H-H: Separate high/low IoU thresholds (parametrised correctly)** Expose `stage1_iou` and `stage2_iou` as separate Optuna parameters with the constraint `stage2_iou ≤ stage1_iou`, default `stage1_iou = 0.1` so baseline is not broken. Relevant to ByteTrack's two-stage association; SORT uses single-stage so this maps differently.
165165

166+
### SORT Phase 1 findings — already in the code (do not re-implement)
167+
168+
Campaign run on `bemch/auto-research` using 3-team parallel strategy (Kalman / Association / Lifecycle). Baseline: HOTA 53.217 → default-param result: 55.7 → tuned (500 trials): **57.7** (+8.4%).
169+
170+
#### Kept changes
171+
172+
| Hypothesis | Commit | HOTA delta at defaults |
173+
| -------------------------------------------------------------------------------------- | --------- | ------------------------------------ |
174+
| Kalman covariance dynamics (velocity_decay, q_miss_alpha, p_reset_threshold) | `8d66fba` | +0.98% |
175+
| OC-SORT observation-centric velocity re-estimation (oru_threshold) | `de5704a` | +1.22% |
176+
| DIoU replaces IoU in association matrix | `c4de2c6` | +0.17% |
177+
| Confidence-weighted Hungarian assignment (conf_cost_weight) | `eefe13e` | enables Optuna headroom |
178+
| IoU age discount for lost tracks (iou_age_weight) | `c094bcb` | enables Optuna headroom |
179+
| Two-stage confidence-based association (high_conf_det_threshold, stage2_iou_threshold) | `e576b9e` | enables Optuna headroom |
180+
| conf_cost_weight wiring + gap interpolation activation | `25d00c5` | activates existing feature |
181+
| minimum_consecutive_frames 3→2 | `3555147` | faster confirmation |
182+
| Align defaults with Optuna-tuned best_config | `ce69432` | +1.18% (single biggest default jump) |
183+
184+
**New SORT constructor params**: `velocity_decay`, `q_miss_alpha`, `p_reset_threshold`, `oru_threshold`, `conf_cost_weight`, `iou_age_weight`, `high_conf_det_threshold`, `stage2_iou_threshold`
185+
186+
#### Tried and reverted
187+
188+
| Hypothesis | Outcome |
189+
| ------------------------------------------------------- | ---------- |
190+
| xcycsr Kalman state representation | −0.51% |
191+
| Velocity-adaptive Q scaling | −0.06% |
192+
| Mahalanobis distance gate | Regression |
193+
| GIoU as association metric | Regression |
194+
| OC-SORT velocity correction (duplicate, second attempt) | Reverted |
195+
196+
#### Tuned best config (sort/sdp, 500 trials)
197+
198+
```json
199+
{
200+
"lost_track_buffer": 82,
201+
"track_activation_threshold": 0.232,
202+
"minimum_consecutive_frames": 2,
203+
"minimum_iou_threshold": 0.0618,
204+
"max_interpolation_gap": 31,
205+
"velocity_decay": 0.524,
206+
"q_miss_alpha": 0.79,
207+
"p_reset_threshold": 12,
208+
"oru_threshold": 2,
209+
"conf_cost_weight": 0.36,
210+
"iou_age_weight": 0.156,
211+
"high_conf_det_threshold": 0.628,
212+
"stage2_iou_threshold": 0.248
213+
}
214+
```
215+
166216
### Agent warning — Kalman patch and state representation
167217

168218
`_apply_kalman_patch` in `optimize_tracking.py` overwrites Q, R, and P with uniform identity-scaled matrices. If the state representation is changed (H-A), the patch must be redesigned to work with the new state dimension and matrix structure.

0 commit comments

Comments
 (0)