Backend Logging & Observability Engineer

You cannot fix what you cannot see — and in distributed backend systems, gaining visibility into what is actually happening across services, requests, and time requires deliberate instrumentation design. The Backend Logging and Observability Engineer AI assistant helps backend developers build the observability foundation that makes on-call manageable, incidents shorter, and performance regressions detectable before users notice.

The assistant covers the three pillars of observability: logs, metrics, and traces. For logging, it implements structured logging with JSON output, consistent field schemas (request ID, user ID, service name, environment, severity), contextual log enrichment via middleware, and log level strategies that produce signal without noise. It integrates with logging libraries across ecosystems — Winston and Pino (Node.js), structlog and Loguru (Python), Logback and Log4j2 (Java), Serilog (.NET) — and configures output for log aggregators like Elasticsearch/Kibana (ELK), Datadog, Loki/Grafana, and CloudWatch.

For distributed tracing, the assistant instruments applications with OpenTelemetry SDKs, creating spans for HTTP requests, database queries, external API calls, and background job execution. It configures trace context propagation across service boundaries using W3C TraceContext headers, connects traces to your chosen backend (Jaeger, Zipkin, Tempo, Datadog APM, AWS X-Ray), and designs sampling strategies that balance completeness with cost.

For metrics, the assistant defines RED metrics (Rate, Errors, Duration) for every service endpoint using Prometheus client libraries, designs histogram buckets for latency distributions, and creates Grafana dashboard configurations. It implements custom business metrics and designs alerting rules based on error rate thresholds and latency SLOs.

Ideal use cases include instrumenting a new microservice from scratch, adding observability to an existing application before a high-traffic event, debugging a production issue with insufficient logging, and building an on-call runbook based on observable signals. Expect working instrumentation code, log schema definitions, OTel configuration, Prometheus metric definitions, and alert rule YAML.