Anchor service metadata patterns so foreach paths do not prefix-match - #3357
Anchor service metadata patterns so foreach paths do not prefix-match#3357winklemad wants to merge 2 commits into
Conversation
…r-select
Client.parent_tasks/child_tasks build a filter pattern from a task's
foreach-execution-path and query filter_tasks_by_metadata. The local provider
matched it with regex.match, which only anchors the start, so an exact path
like "middle:1" also matched "middle:10" and "middle:11" once a foreach had
10+ splits, and parent_tasks/child_tasks returned the wrong tasks.
Use regex.fullmatch in the local provider, and anchor the pattern the service
provider forwards to the metadata service as ^(?:pattern)$ so it matches in
full there too (the existing ".*" and "{path},.*" patterns already require
the service to treat this as a regex). The ".*" match-all short-circuit and
the "{path},.*" descendant form are unchanged.
Fixes Netflix#3341
Greptile SummaryThe PR anchors metadata-service task-filter patterns so exact foreach execution paths no longer select tasks whose indices merely share a prefix.
Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains.
|
| Filename | Overview |
|---|---|
| metaflow/plugins/metadata_providers/service.py | Anchors forwarded task-metadata regex patterns while preserving the match-all short circuit. |
| test/unit/test_service_metadata_provider.py | Verifies exact and descendant patterns are anchored and match-all remains unfiltered. |
Reviews (2): Last reviewed commit: "Scope fix to service metadata provider" | Re-trigger Greptile
|
Thanks for the credit and for laying out the overlap so clearly, that was a considerate way to handle it. The split you proposed works well from my side: #3342 has been open since Aug 16 with the local provider fix ( @talsperre happy to go with whatever ordering you prefer. |
|
Thanks @nileshpatil6, likewise, your #3342 nailed the local-provider side. Hoping we can get both landed. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #3357 +/- ##
=========================================
Coverage ? 31.02%
=========================================
Files ? 382
Lines ? 52717
Branches ? 9303
=========================================
Hits ? 16353
Misses ? 35153
Partials ? 1211 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Thanks for the review @talsperre 🙏 |
PR Type
Summary
Handles the OSS service-provider side of #3341.
Task.parent_tasksandTask.child_taskspass foreach execution-path patterns to the metadataservice, whose regex matching is unanchored. An exact path such as
middle:1could therefore also select
middle:10andmiddle:11.The local-provider side is handled separately by #3342.
Root Cause
ServiceMetadataProvider.filter_tasks_by_metadataforwarded the raw patternto the metadata service:
The service applies that pattern as an unanchored regex search.
Fix
Wrap the forwarded pattern in full-string anchors:
This makes
middle:1exact while preserving the descendant patternmiddle:1,.*. The existing.*match-all case remains short-circuited andsends no pattern.
Tests
test/unit/test_service_metadata_provider.pycovers:Run with:
Non-Goals
OSS client boundary.
AI Tool Usage
Tool disclosed by the author: Claude Code, used to help implement the original
change and tests. The service-only rescope retains that implementation.