Observability Working Group

To evolve the tools in the ecosystem related to observability, such as metrics, distributed tracing and logging, with a strong focus on interoperability between BEAM languages.

Mission Statement

Evolve the tools in the ecosystem related to observability, such as metrics, distributed tracing and logging, with a strong focus on interoperability between BEAM languages.

Main Objectives

Improved runtime observability through integration with modern offerings – Zipkin, Jaeger, DataDog, Prometheus, Honeycomb, LightStep, Stackdriver and many more – without vendor lock in.
Improve state of whitebox monitoring of BEAM applications
Review possibilities to improve blackbox monitoring of BEAM (dtrace, eBPF)
Provide common interfaces to viewing and gathering VM and application statistics and traces
Cooperation with major OAM providers like New Relic to provide proper commercial support for Erlang monitoring

Benefits to the community

Provide tooling that will allow more integrated and application-wide monitoring, logging, and tracing in BEAM
BEAM integrating smoothly with other languages and services
Consolidate efforts across languages and combine tools
Encourage and assist library developers to make things observable
Encourage and make it easier for application developers to make their systems observable
Make it easier for users of the virtual machine to integrate it in polyglot environments where existing monitoring systems are already in place, and where lack of support is a blocker to adoption

Short term deliverables

Living document for current best practices for observability with a blog post on a prominent Erlang site like Erlang Solutions or the Foundation website to go along with it
Standard Logger/logger infrastructure to collect structured logs (logs, error reports and stacktraces) from both Erlang and Elixir.
Support for metrics and traces of popular Erlang and Elixir libraries (Hackney, Ecto, Phoenix, etc) through Telemetry and OpenCensus.
Support for HTTP and binary context propagation for distributed tracing in popular Erlang and Elixir libraries.

Long term deliverables

Helping library authors add standard instrumentation hooks (e.g. Telemetry or potentially even OpenCensus as it becomes more of a standard)

Why does this group require the Foundation

Promotion: Being a central group for user’s to find information on instrumenting and observing BEAM systems and promoting solutions to the community through the Foundation.
Coordination of work across languages and libraries. Bringing together the work as a central place to bring suggestions and PRs to the OTP team.