Logging, Tracing, and Metrics
Logs, traces, and metrics as complementary signals for system observability.
Logging, tracing, and metrics are often called the three pillars of observability because they answer different classes of operational question. They overlap, but they are not interchangeable. A team that understands the distinct strengths of each one can investigate incidents much faster than a team that treats them as three copies of the same signal.
Logs are records of discrete events. They are rich and flexible, which is why engineers reach for them first. A log line can include request IDs, payload summaries, feature flags, database errors, or business context. That detail is useful during debugging, but it also creates volume. Logs are expensive to store, easy to overproduce, and hard to query when formats drift between teams. Structured logging helps by making fields explicit and searchable instead of burying everything in free text.
Metrics trade detail for speed. They aggregate behaviour into numbers over time: request rate, error rate, queue depth, CPU usage, tail latency, cache hit ratio. Because metrics are compact and already aggregated, they are excellent for dashboards, alerting, and trend analysis. They tell you that a problem exists and how broad it is. They do not usually tell you which exact request or payload caused it. Metrics also come with a design trap: high-cardinality labels can explode storage costs and query performance if every user ID or request ID becomes a label dimension.
Tracing follows an individual request or workflow across service boundaries. A distributed trace shows where time was spent, which services were involved, and where errors surfaced along the path. This is especially valuable in microservice systems where no single log stream contains the whole story. Tracing answers questions metrics cannot, such as whether latency lives in the API gateway, a downstream payment call, or a slow database query. The challenge is propagation. If correlation headers are missing or one service is not instrumented, the trace breaks.
The three signals work best together. Metrics can trigger the alert by showing rising latency or error rate. Tracing can reveal that one downstream dependency is responsible for most of the delay. Logs from that service can then explain the concrete failure, such as timeouts, malformed input, or a schema mismatch. OpenTelemetry is popular precisely because it standardises how these signals are produced and linked.
Retention and cost decisions differ too. Metrics are cheap enough to keep longer. Traces are often sampled because full-fidelity collection can be expensive. Logs usually require tiered storage and clear retention policies. None of these are purely technical details. They shape what questions you can answer after an outage or security incident.
A useful observability stack is not one with the most tools. It is one where the signals are consistent, correlated, and deliberately designed. If every request has an identifier, every service emits stable metrics, and key paths are traced end to end, the system becomes much less mysterious when it misbehaves.