OTLP receiver

Takes one signal OpenTelemetry exporters send over OTLP to its port, logs, metrics or traces: over HTTP, as protobuf or JSON, and over gRPC. It answers each export once every destination has its records.

Receivers of different signals may share one port. Each signal is records of its own fields, which its own pipelines read.

OpenTelemetry SDKs, Collectors and other OTLP clients send to it. Every worker in the group listens on its port, and the sender, or a balancer in front of the workers, picks a worker for each export. Add workers to take more.

Example

Takes OTLP logs on port 4318, over TLS with the identity receiver-tls. Senders present the token held in the worker-host variable OTEL_TOKEN.

{
  "id": "otel-logs",
  "type": "otlpReceiver",
  "port": 4318,
  "tokens": {
    "collectors": "OTEL_TOKEN"
  },
  "identity": "receiver-tls"
}

The OTLP receiver form on the Sources page: id otel-logs, port 4318, signal logs, no tokens yet. — The same source in the Sources page's add dialog, before you add its token and identity.

Takes traces on the same port, from the same senders:

{
  "id": "otel-traces",
  "type": "otlpReceiver",
  "port": 4318,
  "signal": "traces",
  "tokens": {
    "collectors": "OTEL_TOKEN"
  },
  "identity": "receiver-tls"
}

A sender posts an export of logs like this:

curl -H "Authorization: Bearer $OTEL_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"resourceLogs":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"demo"}}]},
       "scopeLogs":[{"logRecords":[{"severityText":"INFO","body":{"stringValue":"hello"}}]}]}]}' \
  https://workers.example.com:4318/v1/logs

Before you start

Pick a port no other receiver in the group uses. The one port takes both HTTP and gRPC. Point each sender at the port you pick.
Decide how senders prove who they are: a token, a client certificate, or both. See Tokens and certificates.
Put each token in a credential.
Open the port on every worker, and give senders one address that reaches them all. See Open its port.

Settings

It also takes the settings every source takes.

port integer required

The port it listens on. Every worker of the group binds it, so no two receivers share one, but OTLP receivers of different signals may, with the same identity certificate.

Examples

431814318

signal string default logs

The signal it takes, each a record of its own fields.

logs: Gives one record per log record, from /v1/logs and the logs gRPC service.
metrics: Gives one record per metric data point, from /v1/metrics and the metrics gRPC service.
traces: Gives one record per span, from /v1/traces and the trace gRPC service.

tokens map default none

Each token's name to the credential holding it. An exporter presents one as Authorization: Bearer <token>, and any other request is refused 401. A receiver with no token needs an identity with authorities.

Examples

collectors = OTEL_TOKEN{"apps": {"secret": "otlp-apps","version": 1}}

Advanced

identity identity default plain HTTP

The connection identity it serves TLS under: its certificate and key. With authorities, a sender must present a certificate that chains to one of them.

Example

receiver-tls

maxRequestBytes integer default 10485760

The largest request body it reads, between 1024 and 67108864. A larger one is refused 413. A body sent as gzip is held to it as it inflates.

Examples

104857667108864

Paths it serves

POST /v1/logs, /v1/metrics, /v1/traces: OTLP over HTTP: the body is an Export*ServiceRequest of its signal, protobuf with Content-Type: application/x-protobuf, or OTLP's JSON with Content-Type: application/json. The receiver answers in the encoding the export was sent in, and inflates a body sent with Content-Encoding: gzip.
POST /opentelemetry.proto.collector.logs.v1.LogsService/Export, and the MetricsService and TraceService methods: OTLP over gRPC, over HTTP/2 in the clear or through TLS: one message per call, plain or gzipped as grpc-encoding names it.
GET /health: Says whether the port's receivers take exports. See Health check.

An export of a signal no receiver on the port takes is refused 404, or UNIMPLEMENTED over gRPC.

Records it gives

Each log record, metric data point or span of an export becomes one record, its fields typed as OTLP types them. A field it leaves unset is null. Times are datetimes, to the microsecond; IDs lowercase hex, and an ID that is all zero, or not 16 and 8 bytes long, is null; an enum is its number. attributes and the other variants keep each value's own type: a string, an int, a double, a bool, bytes, a list or a map. Read an attribute as text with attributes["http.method"] as str.

Logs

time, observedTime: When it happened, and when it was seen.
severityNumber, severityText: The severity as OTLP's number, 1 to 24, and as the sender's text.
body, attributes: Variants.
traceId, spanId, flags: The span it was written in.
eventName, droppedAttributesCount: As the log record gives them.

Metrics

One record per data point, each with its metric's fields.

name, description, unit, metadata: Its metric's.
type: gauge, sum, histogram, exponentialHistogram or summary.
aggregationTemporality, isMonotonic: 1 for delta and 2 for cumulative, where its type has one; whether a sum only grows.
startTime, time, attributes, flags: The point's.
value: A gauge's or sum's number, an int or a double as it was sent.
count, sum, min, max: A histogram's, exponential histogram's or summary's. Counts are doubles.
bucketCounts, explicitBounds: A histogram's buckets: each one's count and each one's upper bound, both arrays of doubles.
scale, zeroCount, zeroThreshold, positive, negative: An exponential histogram's; positive and negative each hold offset and an array of doubles, bucketCounts.
quantileValues: A summary's, an array of quantile and value, both doubles.
exemplars: A variant list of the point's exemplars.

Traces

traceId, spanId, parentSpanId, traceState, flags: Where the span sits in its trace.
name, kind: Its name, and OTLP's kind: 1 internal, 2 server, 3 client, 4 producer, 5 consumer.
startTime, endTime: When it started and ended.
attributes, events, links: Its attributes, and lists of its events, each with its time, name and attributes, and of the spans it links to.
statusCode, statusMessage: 1 ok, 2 error, and why.
droppedAttributesCount, droppedEventsCount, droppedLinksCount: As the span gives them.

Fields it gives

The resource and instrumentation scope a record came under are __source fields, which every record of that scope shares. __source.remote holds the IP address the export came from: the balancer's, when one is in front of the workers.

Field	Type	Holds
`__source.name`	`str`	The name of the source that read the record.
`__source.type`	`str`	The kind of source that read it, its `type`, such as `kafka`, `s3List`, `httpApi` or `hecReceiver`.
`__source.remote`	`str`	The address of the sender that delivered the record to a receiver.
`__source.resource`	`variant`	The resource an OTLP log record came under: its attributes, each keeping its own type, or null where the export named none. Read one as text with `__source.resource["service.name"] as str`.
`__source.scope.name`	`str`	The name of the instrumentation scope an OTLP log record came under, the library that wrote it, or null where the export named none.
`__source.scope.version`	`str`	The version of that instrumentation scope, or null where the export named none.
`__source.scope.attributes`	`variant`	That instrumentation scope's attributes, each keeping its own type, or null where the export named none.

Tokens and certificates

Anyone a receiver admits can put records into the group's pipelines, so a receiver needs tokens, an identity with authorities, or both. The leader refuses a receiver with neither.

Tokens: a sender presents one as Authorization: Bearer <token>, set as a header on its exporter. The receiver refuses any other request before reading its body. Without an identity, the token crosses the network in the clear, so use a token alone only behind a balancer that terminates TLS.
Client certificates: with an identity, the receiver serves TLS with the identity's certificate and key. If the identity has authorities, every connection, health checks included, must present a certificate that chains to one of them.

With both, a sender presents a certificate and a token.

What it answers

The receiver answers success only once every destination has the export's records, so an exporter that retries an unanswered export loses nothing. Each refusal below carries an OTLP Status whose message says why. The receiver takes an export whole or refuses it whole. Over gRPC, the same answers are gRPC status codes.

200: Every destination has the export's records. The body is an empty Export*ServiceResponse, {} in JSON.
503, with Retry-After: The receiver is full, blocked or stopping, or the sender took more than 30 seconds to send the whole body. It refuses a new export at once, and a stopping receiver answers every export it holds. An exporter may send it again.
401: The request has no token, gives one other than as Bearer <token>, or gives one the receiver does not hold.
400: The body is not an OTLP export of the receiver's signal. An exporter may drop it.
413: The body, or what it inflates to, is larger than maxRequestBytes, or its records hold more decoded than a receiver holds at once. Send smaller exports.
404: No receiver on the port takes the export's signal. An exporter may drop it.
415: The body's Content-Type is neither application/x-protobuf nor application/json, or its Content-Encoding is other than gzip or identity.

Over gRPC, an export is answered OK once every destination has it, with an empty Export*ServiceResponse. A refusal is UNAVAILABLE where HTTP says 503, which an exporter may send again; UNAUTHENTICATED where it says 401; INVALID_ARGUMENT where it says 400, 413 or 415, which an exporter may drop; and UNIMPLEMENTED where it says 404, and for a grpc-encoding other than gzip or identity. Its grpc-message says why. A request whose content-type is not application/grpc is a plain 415.

Health check

GET /health on the receiver's port answers 200 while every receiver on it takes exports and 503 while one is full, blocked or stopping. It needs no token. Point a balancer's health check or a readiness probe at it, so a busy worker gets no exports until it can take them. Under an identity with authorities it needs a client certificate like any other request.

Senders it takes

Any sender of OTLP works: over HTTP, as protobuf or JSON, or over gRPC, plain or gzipped. Over gRPC, a sender presents its token as the authorization metadata, Bearer <token>.

The OpenTelemetry Collector's otlphttp and otlp exporters.
OpenTelemetry SDKs, with their exporters on any OTLP protocol.
Fluent Bit's opentelemetry output, and Vector's opentelemetry sink.

This OpenTelemetry Collector configuration sends logs and traces to the port of the example's two receivers, with OTEL_TOKEN set in the collector's environment:

receivers:
  otlp:
    protocols:
      grpc:
      http:

exporters:
  otlphttp:
    endpoint: https://workers.example.com:4318
    headers:
      Authorization: Bearer ${env:OTEL_TOKEN}

service:
  pipelines:
    logs:
      receivers: [otlp]
      exporters: [otlphttp]
    traces:
      receivers: [otlp]
      exporters: [otlphttp]

An SDK reads the same from its environment:

export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
export OTEL_EXPORTER_OTLP_ENDPOINT=https://workers.example.com:4318
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer%20$OTEL_TOKEN"

Open its port

Every worker in the group listens on the receiver's port. Open that port to senders on each worker, and give senders one address that reaches them all, such as a balancer's. On a Unix host, an export sent while a worker deploys a new configuration waits, and the new configuration answers it.

On Kubernetes, add the port to the worker pods and put a Service in front of them: Expose a receiver.
On a host or in a container, open the host's firewall or publish the container's port: Enroll a worker on a host.

Verify

POST an export you can recognize, as in the example. The receiver answers 200 with {} once every destination has it. Then find it at the destination.

Troubleshooting

The sender cannot connect: Check that the port is open on every worker, that the sender's address reaches them, and that its endpoint names the receiver's port. An identity with authorities refuses a connection without a certificate that chains to one of them.
Every export gets 401: Check that the sender presents Authorization: Bearer <token>, with a token one of the receiver's credentials holds.
Every export gets 404: No receiver on the port takes the signal it sends: add one for it, or stop sending it. Or the sender posts to a path other than its signal's, such as /v1/logs.
Every export gets 503: Check the source's status. A blocked receiver takes nothing until its destinations take records again. A full one takes more with more workers.
The same records arrive twice: An exporter that hears no answer may send the export again. Give the exporter a timeout longer than its destinations take to deliver.

OTLP receiver

Example

Before you start

Settings

Advanced

Paths it serves

Records it gives

Logs

Metrics

Traces

Fields it gives

Tokens and certificates

What it answers

Health check

Senders it takes

Open its port

Verify

Troubleshooting

See also