Skip to content

Update resource detection to be entity-aware. - #5147

Open
jsuereth wants to merge 3 commits into
open-telemetry:mainfrom
jsuereth:wip-entity-detectors
Open

Update resource detection to be entity-aware.#5147
jsuereth wants to merge 3 commits into
open-telemetry:mainfrom
jsuereth:wip-entity-detectors

Conversation

@jsuereth

@jsuereth jsuereth commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Update resource detectors to generate entities vs. raw attributes.

  • Call out the entities that are generated by named detectors vs. namespaces.
  • Add env detector which will support the "environment variable" resource context propagation defined for entities.
  • Updates naming guidance to involve entities.

Note: This should be a non-breaking change as the OTLP produces will only contain additional information, but Resource generated using these named detectors should be exactly the same as before.

Prototype:

Relevant Issues:

What this does NOT do:

  • Figure out how to have env detection built-in. We believe we need to move from OTEL_RESOURCE_ATTRIBUTES to OTEL_ENTITIES to safely allow runtime environments to populate ENV variable in ways that will not clobber other runtimes. We want this migration to be seamless. I expect this change to declare the named resource detector and we'll later specify default-on resource detectors. This name would allow configuration to be used to disable ENV-based propagation.
  • Add necessary SDK components as these are part of [entities] SDK startup specification #5057

@jsuereth
jsuereth requested a review from a team as a code owner June 9, 2026 18:14
Comment thread specification/resource/sdk.md Outdated
Comment thread specification/resource/sdk.md
entities.
* `service`: Populates `service` and `service.instance` entities described
[here](https://opentelemetry.io/docs/specs/semconv/registry/entities/service/).
* `env`: Populates entities based on [Entity Propagation](../entities/entity-propagation.md).

@dmitryax dmitryax Jun 10, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should to define a plan for reconciling OTEL_RESOURCE_ATTRIBUTES and OTEL_ENTITIES before adding this detector. Having separate controls for disabling them can be confusing in my opinion. Maybe we leave this detector out for now and have OTEL_ENTITIES controlled/applied the same way as OTEL_RESOURCE_ATTRIBUTES? Then we can define migration path to this detector?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm concerned that if we don't give users a way to control priority now we'll struggle to add it later

@MatthieuNoirbusson

Copy link
Copy Markdown

This is a welcome change — naming detectors after the entities they populate makes the Resource side much more legible for consumers.

One consumer-side question this amplifies (happy to move it to an issue if out of scope for this PR): with entity-aware detection, the same entity can now reach a consumer through two channels — attached to a Resource on ordinary telemetry, and as entity state events (entity-events.md describes the two as complementary). For cases where both exist for the same entity — say an SDK detects host while an infra agent emits host entity state events for the same machine — is consumer-side reconciliation guidance planned? Resource-borne entities carry no lifecycle/timestamps, while events do; and the entity-events examples hint at an "authoritative source" notion (e.g. the SDK for service.instance) without generalizing it.

Our working assumption while building a consumer (a temporal entity graph — still in development, not yet field-validated) is: events are authoritative for state/lifecycle; Resource-borne entities associate telemetry to an identity and may bootstrap presence. If that matches the intent, one sentence to that effect (here or in entity-events.md) would help consumer implementers; if it doesn't, even better to learn now.

Also +1 on env being a named detector so ENV-based propagation can be disabled by configuration — from a consumer standpoint, detector provenance on entities is genuinely useful.

@jsuereth

jsuereth commented Jun 10, 2026

Copy link
Copy Markdown
Contributor Author

One consumer-side question this amplifies (happy to move it to an issue if out of scope for this PR): with entity-aware detection, the same entity can now reach a consumer through two channels — attached to a Resource on ordinary telemetry, and as entity state events (entity-events.md describes the two as complementary). For cases where both exist for the same entity — say an SDK detects host while an infra agent emits host entity state events for the same machine — is consumer-side reconciliation guidance planned? Resource-borne entities carry no lifecycle/timestamps, while events do; and the entity-events examples hint at an "authoritative source" notion (e.g. the SDK for service.instance) without generalizing it.

Yes - the entity merge algorithm is defined in the data model and should allow merging entities appropriately. Effectively we only expect the description to change, and we have a merge algorithm to help you determine what labels to use.

We may want to update this to always prefer the entity-relationship event channel for descriptive attributes.

Our working assumption while building a consumer (a temporal entity graph — still in development, not yet field-validated) is: events are authoritative for state/lifecycle; Resource-borne entities associate telemetry to an identity and may bootstrap presence. If that matches the intent, one sentence to that effect (here or in entity-events.md) would help consumer implementers; if it doesn't, even better to learn now.

This is accurate, but I'd say it's more accurate to say Resource-borne entities are mostly about identity and descriptive attributes are an "opt-out" for storage optimisations in systems which do not engage with the Entity relationship signal or where joins are inefficient.

@MatthieuNoirbusson

Copy link
Copy Markdown

Thanks, that settles it for us: merge by exact identity is what we implement, so the two channels converge cleanly.

From a consumer standpoint, the "always prefer the event channel for descriptive attributes" update would be welcome: events carry timestamps and lifecycle, entities carried on the Resource do not, so the precedence falls out naturally. There may be entity types where the Resource side is the authoritative source, for example service.instance coming from its own SDK, so a notion of authoritative source per entity type, which the examples in entity-events.md already hint at, might be the generalizable form.

We'll track this PR.

Comment thread specification/resource/sdk.md
@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown

This PR was marked stale. It will be closed in 14 days without additional activity.

@linux-foundation-easycla

linux-foundation-easycla Bot commented Jul 27, 2026

Copy link
Copy Markdown

CLA Signed
The committers listed above are authorized under a signed CLA.

@jsuereth
jsuereth force-pushed the wip-entity-detectors branch from aaf738e to 3849b96 Compare July 27, 2026 15:55
| Name | Description | Default | Type | Notes |
|------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| OTEL_SDK_DISABLED | Disable the SDK for all signals | false | [Boolean][] | If "true", a no-op SDK implementation will be used for all telemetry signals. Any other value or absence of the variable will have no effect and the SDK will remain enabled. This setting has no effect on propagators configured through the OTEL_PROPAGATORS variable. |
| OTEL_EXPERIMENTAL_ENTITIES_ENABLED | Enables experimental entity support in the SDK. **Status**: [Development](../document-status.md) | false | [Boolean][] | If "true", preserve entity information on detected resources, else entity information is erased. See [Entity-aware resource detectors](../resource/sdk.md#entity-aware-resource-detectors) for more details. |

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We talked about this in the spec meeting today and in the entities meeting yesterday. I think this is not required.

  • Protocol is backwards compatible and consumers are safe to either ignore it or consume it.
  • There is no break to consumers who implement support for the experimental entity information. If consumers consume the field and we change it, the field number will be incremented.
  • There is no break to SDKs to emit entity information if it is provided by the detector
  • There is no break to resource-only consumers if entity information is included, provided the attributes in the entity-aware detector are backwards compatible with the attributes in the old detector

Given the above, I'm not sure why we would strip the information by default. In practice this will just slow adoption of entities and could falsely signal users to think it is a breaking change when it is not.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Going to update this proposal -

  1. We'll modify this enabled flag to be default true - i.e. we want this to be an opt-out, as @dyladan points out.
  2. I'll update the Java PR to showcase this behavior in the SDK to show it working.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update, noted on the opt-out.

While you're in this table: this PR drops OTEL_ENTITIES from it, but entity-propagation.md still requires the SDK to provide an EnvEntityDetector reading OTEL_ENTITIES, and the PR adds env to the reserved detector names pointing to that same document. Is the variable meant to stay, and simply be documented there rather than here?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes the variable stays - but it is NOT configuration, it's a propagation mechanism, similar to the w3c trace context propagation environment variables.

I.e. we'd want this ENV variable automatically provided by environments, e.g. k8s.

@opentelemetry-pr-dashboard

opentelemetry-pr-dashboard Bot commented Aug 11, 2026

Copy link
Copy Markdown

Pull request dashboard status

Waiting on the author · refreshed 2026-09-01 21:41 UTC

Respond to 3 review items (e.g. link a commit, explain why not, ask a follow-up):

  • Inline threads: 1, 2, 3
Status above doesn't look right?
  • Just replied or pushed? Anything around or after the refresh time above may not be picked up yet — give it a few minutes.
  • Should this be with reviewers? Comment /dashboard route:reviewers to route it to them.
  • Anything wrong — including the routing? Report it with what you expected; it helps us improve the dashboard.

* `service`: Populates `service` and `service.instance` entities described
[here](https://opentelemetry.io/docs/specs/semconv/registry/entities/service/),
or all relevant attributes of the entities, if entity support is not included
yet.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This effectively drops the connection of the OTEL_SERVICE_NAME environment variable handling to the "service" detector name. Is that intentional?

environment variable; populates `service.instance.id`
as [defined here](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/registry/attributes/service.md#service-attributes).
Populates [process](https://opentelemetry.io/docs/specs/semconv/registry/entities/process/)
entities, or all relevant attributes of the entities, if entity support is

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: process populates a single entity:

Suggested change
entities, or all relevant attributes of the entities, if entity support is
entity, or all relevant attributes of the entity, if entity support is

| Name | Description | Default | Type | Notes |
|------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| OTEL_SDK_DISABLED | Disable the SDK for all signals | false | [Boolean][] | If "true", a no-op SDK implementation will be used for all telemetry signals. Any other value or absence of the variable will have no effect and the SDK will remain enabled. This setting has no effect on propagators configured through the OTEL_PROPAGATORS variable. |
| OTEL_EXPERIMENTAL_ENTITIES_ENABLED | Enables experimental entity support in the SDK. **Status**: [Development](../document-status.md) | false | [Boolean][] | If "true", preserve entity information on detected resources, else entity information is erased. See [Entity-aware resource detectors](../resource/sdk.md#entity-aware-resource-detectors) for more details. |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It feels a bit strange to have what is basically a feature flag specified in the SDK environment variables spec. Not a huge deal, but I assume we could also omit this and let SDKs use their existing opt-in mechanisms to enable features. I guess it is nice to use the same env var across languages.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants