Observability basics for backend developers

Let's say it is 2 AM and your phone buzzes. The alert says: "Response time above 3 seconds.(above SLA)"

Good, you know something is wrong. But where? Which service? Which query? Since when? The alert does not tell you any of this.

This is the gap between monitoring and observability. Let's understand it, and then see how OpenTelemetry helps us close it.

Monitoring vs Observability

Monitoring is collecting and analyzing predefined metrics. For example, CPU usage > 90% or response time > 3s. You decide the checks in advance, and the system tells you when a check fails.

Observability is the ability to know the internal state of a system by analyzing the telemetry signals it produces.

The easiest way to see the difference is by using analogy of human body and health:

Stage	Human body	Your system
Healthy	Body temp 97-99°F	Latency under 3s
Monitoring (tells something is wrong)	Body temp > 100°F, fever	Latency above 3s
Observability (tells why it is wrong)	Bacterial infection	DB queries are the bottleneck
Fix	Antibiotic	Add an index

Monitoring tells you something is wrong. Observability tells you why.

Three pillars (signals) of observability

To understand the internal state, a system has to emit signals. There are three of them.

1. Logs

• Timestamped record of events that happened inside the system
• If a user reports an issue, logs give the exact sequence of events that led to the problem

2. Metrics

• Numeric measurements collected at regular intervals
• Used to track trends
• Examples: application throughput, response time, CPU and memory utilization, and application specific numbers like number of orders placed
• Used for monitoring and alerting

3. Traces

• Show the complete journey of a request, how it travels through different parts of the system
• We get to know at which part it failed, how much time it took, and other important details like bottlenecks

Problem: Observability was fragmented

A distributed system is complex, with many services involved. And for observability we have many tools available: Prometheus, InfluxDB, Jaeger, Datadog, New Relic, ELK and so on.

The issue is:

• There is no standard format for telemetry data
• Vendor lock-in, because your code is tied to one tool's way of doing things
• Observability became fragmented, vendor specific and costly to maintain
• Switching tools was not easy

OpenTelemetry (OTel)

OpenTelemetry is an open source, vendor neutral observability framework. It provides standard APIs, SDKs and tools to collect, generate and export telemetry data (logs, metrics and traces).

It is similar to JPA in this sense. JPA gives a standard contract for talking to databases. OTel gives a standard contract for collecting telemetry, and decouples your application from the observability backend or tool. Tomorrow if you want to move from one tool to another, your application code stays the same.

Micrometer vs OpenTelemetry

If you are on Spring Boot, you already have Micrometer, so why OTel? Micrometer is a vendor neutral facade for metrics on the JVM, like SLF4J is for logging. Since Spring Boot 3, Micrometer Tracing also handles distributed tracing, as the replacement of Spring Cloud Sleuth. So Micrometer can do metrics and traces.

The difference is scope. Micrometer is a JVM library. OTel is a cross language standard (Java, Go, Python, .NET, Node.js and more) that covers logs, metrics and traces. It also defines the wire format (OTLP), semantic conventions and context propagation between services. In a polyglot system, this is what lets one trace flow through services written in different languages.

They are not either or. In Spring Boot you can keep Micrometer APIs in your code and export the data through OTel. For a Java only team that needs just metrics and traces, Micrometer is enough. When you have multiple languages, or want one standard across all three signals, go with OTel.

Instrumentation

Instrumentation means adding the necessary code so that our application emits telemetry signals (logs, metrics, traces).

A simple way to think about it:

• Business feature -> development
• Observability feature -> instrumentation

There are two ways to do it.

1. Automatic / zero code

• An agent runs alongside your application and emits telemetry signals for you
• Good for quick setup and instant visibility, especially for new teams

2. Manual / code based

• You add code yourself to emit telemetry signals
• You can capture business specific signals, like available inventory or number of orders placed
• Fine grained control over what is observed

Quick recap

• Monitoring tells you something is wrong. Observability tells you why.
• Three pillars: logs (what happened), metrics (trends), traces (journey of a request).
• Too many tools with no standard format led to vendor lock-in.
• OTel is the vendor neutral standard, like JPA is for databases.
• Instrumentation can be automatic for quick start, or manual for business specific signals.