Tracing

In order to uniquely identify requests or response messages, a tracing ID may be added as HTTP header. This tracing id can be used to be able to trace the flow of calls across APIs and organizations.

The trace ID could be used for example:

  • for operational purposes, to enable lookup related technical logs

  • for audit logging, to be able to trace the origin of each request. It may be a legal requirement to have such audit logging when sensitive data is exchanged.

Rule: HTTP headers for tracing [trc-header]

APIs exposed to third parties SHOULD support the tracing HTTP header BelGov-Trace-Id on both requests and responses. Generated trace IDs MUST be unique, random and no longer than 36 characters, for instance using UUIDv4.

The trace id on a response MUST be generated by the API provider and is not copied from the request. Additionally, the BelGov-Related-Trace-Id HTTP header MAY be present on a response. Its value MUST be the same as the BelGov-Trace-Id on the request.

The BelGov-Trace-Id value on every outgoing call SHOULD be unique, even when they are made within a single context e.g. as part of handling a single incoming request. However for security reasons, an API provider SHOULD NOT depend on an external client to generate a correct unique value. When auditing functionality is needed, it SHOULD be possible to correlate the various tracing IDs used in the context of a request.

Rule: W3C headers for tracing and context [trace-w3c]

Within an organization’s trust boundary (i.e. internal calls), alternative HTTP headers MAY be used to trace calls.

The standardized W3C Trace Context and baggage headers are RECOMMENDED, which are compatible with the OpenTelemetry specification.

OpenTelemetry, also known as OTel, is a vendor-neutral open source observability framework for instrumenting, generating, collecting, and exporting telemetry data such as traces, metrics, and logs across heterogeneous platforms.

A trace context allows tracing tools to follow, analyze and debug a transaction across multiple software components. Baggage can be used, among other things, to annotate telemetry, adding contextual information to metrics, traces, and logs.

APIs MAY support both W3C and BelGov(-Related)-Trace-Id headers dynamically depending on the client being located within the trust boundary or not.

For calls across trust boundaries (e.g. by a 3rd party client), the W3C headers are not recommended because they are less designed for this use case:

  • gathered trace information wouldn’t be complete, missing trace parts at external parties

  • measures have to be taken to address security concerns (Denial of Service, information exposure) like verifying and filtering input values, restarting the trace, filtering sensitive information form outgoing headers (e.g. in baggage or tracestate)

  • W3C tracing and baggage headers are hard to implement (trace-id, parent-id, flags, tracestate) by parties that aren’t using an OpenTelemetry implementation

When crossing a trust boundary, a mapping can be made, for instance:

  • the client’s BelGov-Trace-Id can be added to the baggage of internal calls

  • for outgoing messages, the BelGov-Trace-Id header could be set to OpenTelemetry’s span-id