New Relic

Monitor Spice Cloud with New Relic

Spice Cloud can be monitored with New Relic two ways: pushing metrics directly to New Relic's hosted OTLP intake, or scraping the metrics endpoint with the New Relic infrastructure agent.

The OTLP path is covered first because it needs no agent, and the New Relic license key can be held as an app secret rather than configured on a host.

OpenTelemetry OTLP export

Add an otel_exporter block to the app's spicepod:

runtime:
  telemetry:
    otel_exporter:
      endpoint: https://otlp.nr-data.net/v1/metrics
      headers:
        api-key: ${secrets:NEW_RELIC_LICENSE_KEY}

Pick the endpoint matching the account's region:

Region	OTLP/HTTP endpoint
US (default)	https://otlp.nr-data.net/v1/metrics
EU	https://otlp.eu01.nr-data.net/v1/metrics
FedRAMP	https://gov-otlp.nr-data.net/v1/metrics

The header name is api-key, lowercase. Use a New Relic license key — either the account ingest license key or an ingest-specific key. Store it as an app secret and reference it with ${secrets:NAME}.

Metrics appear in New Relic's Metrics Explorer within a minute or two of the first push.

Spice exports with delta temporality by default, which is what New Relic expects. See OpenTelemetry if a different setting is needed.

Namespacing and attributes

metric_prefix avoids collisions with other services reporting into the same New Relic account, and properties become resource attributes that New Relic exposes as queryable dimensions:

runtime:
  telemetry:
    metric_prefix: 'spiceai.'
    properties:
      environment: prod
      region: us-west-2
      team: data-platform
    otel_exporter:
      endpoint: https://otlp.nr-data.net/v1/metrics
      push_interval: '30s'
      headers:
        api-key: ${secrets:NEW_RELIC_LICENSE_KEY}

Those attributes are then available in NRQL through WHERE and FACET:

SELECT average(spiceai.query_duration_ms) FROM Metric WHERE environment = 'prod' FACET region SINCE 1 hour ago

When metric_prefix is combined with the metrics whitelist, the whitelist entries must include the prefix — the filter runs after the prefix is applied.

Prometheus scrape

The New Relic infrastructure agent can scrape the metrics endpoint instead, which suits an estate already collecting Prometheus targets through New Relic.

curl https://<app-cname>.spiceai.io/v1/metrics \
  -H "X-API-Key: <API_KEY>"

Point a Prometheus scrape target at that URL, passing the app API key as the X-API-Key header. See New Relic's Prometheus integrations overview for agent configuration.

Metrics collected this way are cumulative, since the scrape endpoint always exposes cumulative values.

Related

• OpenTelemetry — full runtime.telemetry reference

• Metrics API — endpoint reference and the full metric list

• Grafana & Prometheus and Datadog — other monitoring integrations