OpenTelemetry

Push Spice Cloud metrics to any OpenTelemetry-compatible backend over OTLP

Spice Cloud can push metrics to any OpenTelemetry-compatible backend over OTLP, as an alternative to having a monitoring agent scrape the metrics endpoint.

Pushing is configured in the app's spicepod under runtime.telemetry.otel_exporter. No collector or agent is required — the runtime exports directly to the endpoint.

runtime:
  telemetry:
    otel_exporter:
      endpoint: https://otlp.example-org.com/v1/metrics
      headers:
        api-key: ${secrets:OTLP_API_KEY}

Push or scrape

Both paths expose the same runtime metrics. They differ in which side initiates the connection:

	Scrape	Push (OTLP)
Initiated by	The monitoring agent	The Spice runtime
Configured in	The agent	The app spicepod
Reachability	Agent must reach the app endpoint	Runtime must reach the backend
Temporality	Always cumulative	delta by default, configurable

Push suits backends that offer a hosted OTLP intake, and avoids running an agent that has network access to the app. Scrape suits an existing Prometheus-based stack. See Grafana & Prometheus and Datadog for the scrape path.

Configuration reference

runtime.telemetry

Field	Type	Default	Description
enabled	boolean	true	Whether runtime telemetry is collected
properties	map	{}	Custom key/value pairs attached to every metric as OpenTelemetry resource attributes
metric_prefix	string	none	Prefix prepended to every exported metric name
otel_exporter	object	none	OTLP push configuration (below)

runtime.telemetry.otel_exporter

Field	Type	Default	Description
endpoint	string	—	Required. For HTTP, the full URL. For gRPC, a hostname with optional port
enabled	boolean	true	Whether the exporter is active
push_interval	string	60s	How often metrics are pushed
metrics	list	all	Whitelist of metric names to export
headers	map	none	Headers sent with each export. For gRPC these become metadata entries and keys must be lowercase
temporality	string	delta	How counters and histograms are encoded — see below

Temporality

temporality controls how counter and histogram values are encoded on the wire:

Value	Behavior	When to use
delta	Each export carries the change since the previous export	Default. Required by Datadog's OTLP intake and recommended by New Relic, AWS CloudWatch, and most hosted OTLP backends
cumulative	Each export carries the running total since process start	Collectors that feed Prometheus or other cumulative-native backends
low_memory	Counters cumulative, histograms delta	Histogram-heavy workloads, to reduce in-process state

temporality affects only the OTLP push exporter. The metrics endpoint always exposes cumulative metrics regardless of this setting.

Namespacing metrics

metric_prefix prepends a string to every exported metric name, which avoids collisions when several services report into one backend:

runtime:
  telemetry:
    metric_prefix: 'spiceai.'

The runtime metric query_duration_ms is then exported as spiceai.query_duration_ms.

The metrics whitelist is applied after the prefix. When metric_prefix is set, entries must include it — query_duration_ms will not match once the prefix is spiceai., so use spiceai.query_duration_ms.

Custom attributes

properties attaches key/value pairs to every metric as OpenTelemetry resource attributes, which backends surface as queryable dimensions:

runtime:
  telemetry:
    properties:
      environment: prod
      region: us-west-2
      team: data-platform

Full example

runtime:
  telemetry:
    metric_prefix: 'spiceai.'
    properties:
      environment: prod
      region: us-west-2
    otel_exporter:
      endpoint: https://otlp.example-org.com/v1/metrics
      push_interval: '30s'
      temporality: delta
      headers:
        api-key: ${secrets:OTLP_API_KEY}

Store credentials as secrets and reference them with ${secrets:NAME} rather than writing them into the spicepod.

Related

• New Relic — a worked OTLP example

• Metrics API — endpoint reference and the full metric list

• Grafana & Prometheus and Datadog — the scrape path