Observability

This page describes the approach towards observability in the Apeiro Reference Architecture. Observability is the practice of monitoring and troubleshooting infrastructure components and workloads by analyzing their telemetry data. Telemetry data encompasses logs, metrics, and traces.

ApeiroRA uses OpenTelemetry, also known as OTel

OpenTelemetry, also known as OTel, is a vendor-neutral open source Observability framework for instrumenting, generating, collecting, and exporting telemetry data such as traces, metrics, and logs.

, for telemetry. OTel provides a comprehensive open-source observability framework for producing and collecting telemetry data, including logs, metrics, and traces. It is compatible with various storage backends to persist telemetry data (e.g. OpenSearch, S3-compatible storages) and can be integrated with many observability tools for analysis (e.g. Prometheus, Zipkin, several commercial tools).

More information about OTel can be found in the official OpenTelemetry documentation.

Functional Overview

All components of the Apeiro Reference Architecture strive to produce telemetry data such that it is compatible with OTel.

ApeiroRA does not impose a specific sink for telemetry data. Instead, any supported storage or tool can be configured. Note that for audit logs, ApeiroRA recommends a durable write-once/read-many (WORM) storage such as an S3-compatible storage.

Some components of ApeiroRA may be deployed with an OTel Collector by default to streamline collection of telemetry data, however, any deployment pattern for the OTel Collector may be used.

Overview of OpenTelemetry in the Apeiro Reference Architecture

Audit Logging

Audit logging describes the capability of capturing audit-trail relevant events of a system to meet compliance requirements. Such events may originate from infrastructure components up to the workloads that run on top. It is a capability that is particularly relevant for providers of enterprise software.

Unlike regular application logs, audit logs are usually subject to long retention periods and software providers must guarantee their completeness (i.e. guarantee of delivery).

Examples of audit logs include:

• failed login attempts
• permission changes (e.g. of a service account or application user)
• accessing sensitive information
• modification of data

ApeiroRA aims to bring audit logging to OpenTelemetry. While many capabilities required to process audit logs already exist in OTel, standardized semantic conventions for audit logs and delivery guarantees are missing.

Therefore, ApeiroRA is working together with the OpenTelemetry community on the following deliverables:

• semantic convention for audit logs
• extension of OTel APIs/SDKs for audit logging purposes
• extension of OTel Collector for audit logging purposes