Troubleshooting Missing Telemetry Data
This guide provides a systematic approach to debug telemetry pipeline issues by starting with Grafana dashboards, then moving to the OpenTelemetry Collector, and finally to your application.
Step 1: Check OTel Collector Data Flow Dashboard
Start by checking the OTel Collector Data Flow dashboard in Grafana.
If No Data in Dashboard
There's an issue in the pipeline. Check the collector logs.
Viewing Collector Logs
Docker:
docker logs otel-collector --tail=100
Kubernetes:
kubectl logs <otel-collector-pod-name> -n <namespace> --tail=100
Linux Service:
sudo journalctl -u otel-collector -n 100
What to Look For in Logs
Successful startup:
Everything is ready. Begin running and processing data.
Configuration errors:
error decoding config
invalid configuration
Authentication errors:
rpc error: code = Unauthenticated
401 Unauthorized
Connection errors:
connection refused
failed to connect
If logs show errors: Validate your collector configuration using:
otelcol validate --config=/path/to/otel-collector-config.yaml
Or use otelbin.io.
If no errors in logs and application is not throwing errors: Verify the receiver is configured in the pipeline:
service:
pipelines:
metrics:
receivers: [otlp] # Ensure your receiver is listed
exporters: [otlp]
If Data Appears in Dashboard
Proceed to Step 2.
Step 2: Check Metrics Collected Dashboard
Go to the Metrics Collected dashboard and select your environment.
If You See Your Service and Data
Your telemetry is flowing correctly. Use the correct filters and dashboard options to view your data.
If No Data for Your Service
Check your application logs to verify it's successfully exporting telemetry.
Look for:
- OpenTelemetry SDK initialization messages
- Successful telemetry export confirmations
- Any SDK errors or warnings
If application shows no errors: Verify the receiver is used in the collector pipeline (see Step 1).
Related Guides
- Collector Configuration Detailed collector configuration guide
- Auto Instrumentation Automatic application instrumentation
- Custom Instrumentation Manual application instrumentation