Django

Introduction

Django is a high-level Python web framework that encourages rapid development and clean, pragmatic design. As one of the most popular web frameworks for building data-driven applications, Django applications often handle complex business logic, database interactions, and background tasks—making observability critical for maintaining performance and reliability at scale.

This guide demonstrates how to instrument Django applications with OpenTelemetry for comprehensive distributed tracing, metrics collection, and application performance monitoring. We'll cover automatic instrumentation of Django's ORM, views, middleware, template rendering, and Celery background tasks, providing visibility into every layer of your application stack.

Django's "batteries-included" philosophy extends to observability through OpenTelemetry's automatic instrumentation libraries. Unlike manual instrumentation approaches, Django applications can achieve comprehensive tracing with minimal code changes—automatically capturing database queries, HTTP requests, cache operations, and asynchronous task execution. We'll explore both automatic and custom instrumentation patterns, including Django-specific considerations like N+1 query detection, PII masking for GDPR compliance, and management command tracing.

Who This Guide Is For

This guide is designed for:

Django Developers building web applications with complex ORM queries and needing visibility into database performance and N+1 query issues
Backend Engineers working with Django REST Framework APIs and requiring end-to-end request tracing across services
DevOps Teams deploying Django applications with Celery workers and needing distributed tracing across synchronous and asynchronous tasks
Data Platform Engineers managing Django-powered data applications with PostgreSQL and requiring query-level performance insights
Technical Leads implementing observability standards across Django microservices and ensuring PII compliance in telemetry data

Overview

This guide covers Django OpenTelemetry instrumentation using the official OpenTelemetry Python SDK and Django-specific auto-instrumentation packages. The approach leverages Django's middleware system and signal framework for comprehensive, low-overhead tracing.

What You'll Learn

Installing and configuring OpenTelemetry SDK for Django with automatic instrumentation
Instrumenting Django ORM queries with full SQL visibility and parameter binding
Tracing Django views, middleware, and template rendering automatically
Setting up Celery distributed tracing for background tasks with context propagation
Implementing Django management command instrumentation for batch jobs
Detecting and resolving N+1 database query issues using span attributes
Masking PII data in traces for GDPR and HIPAA compliance
Configuring Django REST Framework for API endpoint tracing
Optimizing telemetry overhead for high-traffic Django applications
Deploying instrumented Django apps with Gunicorn, uWSGI, or ASGI servers

Prerequisites

System Requirements:

Python: 3.9+ (3.13+ recommended for latest features)
Django: 3.2+ (5.0+ LTS recommended)
PostgreSQL: 12+ (18+ recommended) or other supported databases
Celery: 5.0+ for background task tracing (optional)
Redis: 6.0+ for caching and Celery broker (optional)

Supported Django Versions:

Django Version	Python Version	OpenTelemetry Support	Status
5.2 LTS	3.10+	✅ Full	Recommended
5.1	3.10+	✅ Full	Current
5.0 LTS	3.10+	✅ Full	Supported
4.2 LTS	3.8+	✅ Full	Supported
4.1	3.8+	✅ Full	Legacy
3.2 LTS	3.6+	⚠️ Limited	EOL (2024)
2.x	3.5+	❌ Not supported	EOL

Instrumented Components:

OpenTelemetry Django instrumentation automatically traces:

✅ HTTP Requests - Django views (function-based and class-based)
✅ Database Queries - Django ORM queries with SQL and parameters
✅ Template Rendering - Django template engine execution
✅ Middleware - All middleware in the processing chain
✅ Cache Operations - Django cache framework (Redis, Memcached)
✅ Celery Tasks - Background task execution with distributed context
✅ Management Commands - Custom Django management commands
✅ Django REST Framework - API endpoints and serializers
✅ Authentication - Login, logout, and permission checks
✅ Signals - Django signal dispatching (with custom instrumentation)

Example Application

This guide references the django-postgres example featuring:

Framework: Django 5.2 LTS with Django REST Framework
Database: PostgreSQL 18 with Django ORM
Background Jobs: Celery 5.4+ with Redis broker
Features: PII masking, N+1 query detection, JWT authentication
Deployment: Gunicorn WSGI server with Docker and Kubernetes

Installation & Setup

Django OpenTelemetry instrumentation requires the core SDK and Django-specific auto-instrumentation packages. The setup process installs dependencies and initializes tracing in your Django application's startup code.

pip (Recommended)
Poetry
Pipenv
Auto-Instrumentation Bootstrap

Install OpenTelemetry SDK and Django instrumentation:

Terminal
# Install core OpenTelemetry SDK
pip install opentelemetry-api opentelemetry-sdk

# Install Django auto-instrumentation
pip install opentelemetry-instrumentation-django

# Install database instrumentation (PostgreSQL)
pip install opentelemetry-instrumentation-psycopg2

# Install Celery instrumentation (optional)
pip install opentelemetry-instrumentation-celery

# Install Redis instrumentation (optional)
pip install opentelemetry-instrumentation-redis

# Install OTLP exporter (send traces to collector)
pip install opentelemetry-exporter-otlp

# Freeze dependencies
pip freeze > requirements.txt

Add dependencies to pyproject.toml:

pyproject.toml
[tool.poetry.dependencies]
python = "^3.10"
django = "^5.2"
psycopg2-binary = "^2.9"
celery = "^5.4"
redis = "^5.0"

# OpenTelemetry dependencies
opentelemetry-api = "^1.27"
opentelemetry-sdk = "^1.27"
opentelemetry-instrumentation-django = "^0.48b0"
opentelemetry-instrumentation-psycopg2 = "^0.48b0"
opentelemetry-instrumentation-celery = "^0.48b0"
opentelemetry-instrumentation-redis = "^0.48b0"
opentelemetry-exporter-otlp = "^1.27"

Install dependencies:

poetry install

Add to Pipfile:

Pipfile
[packages]
django = "~=5.2"
psycopg2-binary = "~=2.9"
celery = "~=5.4"
redis = "~=5.0"
opentelemetry-api = "~=1.27"
opentelemetry-sdk = "~=1.27"
opentelemetry-instrumentation-django = "~=0.48b0"
opentelemetry-instrumentation-psycopg2 = "~=0.48b0"
opentelemetry-instrumentation-celery = "~=0.48b0"
opentelemetry-instrumentation-redis = "~=0.48b0"
opentelemetry-exporter-otlp = "~=1.27"

Install:

pipenv install

Use the OpenTelemetry bootstrap command to automatically install all instrumentation packages:

Terminal
# Install bootstrap tool
pip install opentelemetry-bootstrap

# Auto-detect and install instrumentation for installed packages
opentelemetry-bootstrap -a install

# This automatically installs:
# - opentelemetry-instrumentation-django
# - opentelemetry-instrumentation-psycopg2
# - opentelemetry-instrumentation-celery
# - opentelemetry-instrumentation-redis
# (based on your installed packages)

Note: This approach is convenient but installs all detected instrumentations. For production, explicitly specify dependencies in requirements.txt.

Django Auto-Instrumentation

Django's auto-instrumentation uses middleware injection to capture all HTTP requests automatically. Unlike Flask, you don't need to manually initialize the instrumentor in most cases—adding the middleware is sufficient.

Configuration

Django OpenTelemetry configuration involves initializing the SDK in your application startup and configuring middleware to capture HTTP requests.

Basic Initialization

Create a tracing initialization module:

myproject/tracing.py
"""OpenTelemetry tracing initialization for Django."""
import os
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.resources import Resource
from opentelemetry.instrumentation.django import DjangoInstrumentor
from opentelemetry.instrumentation.psycopg2 import Psycopg2Instrumentor


def initialize_tracing():
    """Initialize OpenTelemetry tracing for Django application."""
    # Create resource with service information
    resource = Resource.create({
        "service.name": os.getenv("OTEL_SERVICE_NAME", "django-order-service"),
        "service.version": os.getenv("APP_VERSION", "1.0.0"),
        "deployment.environment": os.getenv("ENVIRONMENT", "development"),
    })

    # Create tracer provider
    provider = TracerProvider(resource=resource)

    # Configure OTLP exporter
    otlp_exporter = OTLPSpanExporter(
        endpoint=os.getenv("OTEL_EXPORTER_OTLP_ENDPOINT", "http://localhost:4317"),
        insecure=os.getenv("OTEL_EXPORTER_OTLP_INSECURE", "true") == "true",
    )

    # Add batch span processor
    provider.add_span_processor(BatchSpanProcessor(otlp_exporter))

    # Set as global tracer provider
    trace.set_tracer_provider(provider)

    # Instrument Django automatically
    DjangoInstrumentor().instrument()

    # Instrument PostgreSQL
    Psycopg2Instrumentor().instrument()

    print("OpenTelemetry tracing initialized successfully")

Django Settings Configuration

Update your settings.py:

myproject/settings.py
# Django settings for instrumented application
import os
from pathlib import Path

BASE_DIR = Path(__file__).resolve().parent.parent

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = os.getenv("DJANGO_SECRET_KEY", "dev-secret-key-change-in-production")

DEBUG = os.getenv("DEBUG", "True") == "True"

ALLOWED_HOSTS = os.getenv("ALLOWED_HOSTS", "localhost,127.0.0.1").split(",")

# Application definition
INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'rest_framework',
    'orders',  # Your app
]

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

ROOT_URLCONF = 'myproject.urls'

# Database configuration
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': os.getenv('DB_NAME', 'orders'),
        'USER': os.getenv('DB_USER', 'postgres'),
        'PASSWORD': os.getenv('DB_PASSWORD', 'postgres'),
        'HOST': os.getenv('DB_HOST', 'localhost'),
        'PORT': os.getenv('DB_PORT', '5432'),
    }
}

# Celery Configuration (optional)
CELERY_BROKER_URL = os.getenv('CELERY_BROKER_URL', 'redis://localhost:6379/0')
CELERY_RESULT_BACKEND = os.getenv('CELERY_RESULT_BACKEND', 'redis://localhost:6379/0')

# OpenTelemetry Configuration
OTEL_SERVICE_NAME = os.getenv('OTEL_SERVICE_NAME', 'django-order-service')
OTEL_EXPORTER_OTLP_ENDPOINT = os.getenv('OTEL_EXPORTER_OTLP_ENDPOINT', 'http://localhost:4317')

Initialize in WSGI/ASGI Application

Update your WSGI or ASGI application file:

myproject/wsgi.py
"""
WSGI config for myproject.

Initializes OpenTelemetry tracing before application starts.
"""
import os
from django.core.wsgi import get_wsgi_application

# Set Django settings module
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'myproject.settings')

# Initialize OpenTelemetry tracing BEFORE creating WSGI application
from myproject.tracing import initialize_tracing
initialize_tracing()

# Create WSGI application
application = get_wsgi_application()

For ASGI (Django Channels):

myproject/asgi.py
"""
ASGI config for myproject.
"""
import os
from django.core.asgi import get_asgi_application

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'myproject.settings')

# Initialize tracing before ASGI application
from myproject.tracing import initialize_tracing
initialize_tracing()

application = get_asgi_application()

Environment Variables

Configure tracing via environment variables:

.env
# Django configuration
DJANGO_SECRET_KEY=your-secret-key-here
DEBUG=False
ALLOWED_HOSTS=localhost,api.example.com
ENVIRONMENT=production

# Database
DB_NAME=orders
DB_USER=django
DB_PASSWORD=secure-password
DB_HOST=postgres
DB_PORT=5432

# OpenTelemetry configuration
OTEL_SERVICE_NAME=django-order-service
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
OTEL_EXPORTER_OTLP_INSECURE=true
APP_VERSION=1.0.0

# Celery (optional)
CELERY_BROKER_URL=redis://localhost:6379/0
CELERY_RESULT_BACKEND=redis://localhost:6379/0

Docker Compose Configuration

docker-compose.yml
version: '3.9'

services:
  django-app:
    build: .
    command: gunicorn myproject.wsgi:application --bind 0.0.0.0:8000 --workers 4
    ports:
      - '8000:8000'
    environment:
      OTEL_SERVICE_NAME: django-order-service
      OTEL_EXPORTER_OTLP_ENDPOINT: http://scout-collector:4317
      OTEL_EXPORTER_OTLP_INSECURE: 'true'
      DB_HOST: postgres
      DB_NAME: orders
      DB_USER: django
      DB_PASSWORD: django123
      CELERY_BROKER_URL: redis://redis:6379/0
      ENVIRONMENT: docker
    depends_on:
      - postgres
      - redis
      - scout-collector

  celery-worker:
    build: .
    command: celery -A myproject worker --loglevel=info
    environment:
      OTEL_SERVICE_NAME: django-celery-worker
      OTEL_EXPORTER_OTLP_ENDPOINT: http://scout-collector:4317
      DB_HOST: postgres
      CELERY_BROKER_URL: redis://redis:6379/0
    depends_on:
      - postgres
      - redis
      - scout-collector

  postgres:
    image: postgres:18-alpine
    environment:
      POSTGRES_DB: orders
      POSTGRES_USER: django
      POSTGRES_PASSWORD: django123
    volumes:
      - postgres_data:/var/lib/postgresql/data

  redis:
    image: redis:7-alpine
    ports:
      - '6379:6379'

  scout-collector:
    image: otel/opentelemetry-collector-contrib:latest
    command: ['--config=/etc/otel-collector-config.yaml']
    volumes:
      - ./otel-collector-config.yaml:/etc/otel-collector-config.yaml
    ports:
      - '4317:4317'

volumes:
  postgres_data:

Scout Integration

When using Base14 Scout, configure the OTLP endpoint to point to your Scout Collector with authentication headers. Scout provides managed infrastructure optimized for Django applications with high query volume.

Production Configuration

Production deployments require optimized sampling, secure credential management, and performance tuning for high-traffic Django applications.

Production Tracing Initialization

myproject/tracing.py
"""Production-optimized OpenTelemetry tracing configuration."""
import os
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.sdk.trace.sampling import TraceIdRatioBased, ParentBased
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.resources import Resource
from opentelemetry.instrumentation.django import DjangoInstrumentor
from opentelemetry.instrumentation.psycopg2 import Psycopg2Instrumentor
from opentelemetry.instrumentation.redis import RedisInstrumentor


def initialize_tracing():
    """Initialize production-grade OpenTelemetry tracing."""
    # Create resource with comprehensive service information
    resource = Resource.create({
        "service.name": os.getenv("OTEL_SERVICE_NAME", "django-order-service"),
        "service.version": os.getenv("APP_VERSION", "1.0.0"),
        "deployment.environment": os.getenv("ENVIRONMENT", "production"),
        "cloud.provider": os.getenv("CLOUD_PROVIDER", "aws"),
        "cloud.region": os.getenv("AWS_REGION", "us-east-1"),
        "k8s.cluster.name": os.getenv("K8S_CLUSTER", "production"),
        "k8s.namespace.name": os.getenv("K8S_NAMESPACE", "default"),
        "k8s.pod.name": os.getenv("HOSTNAME", "unknown"),
    })

    # Configure sampling (10% of traces in production)
    sample_rate = float(os.getenv("OTEL_TRACE_SAMPLE_RATE", "0.1"))
    sampler = ParentBased(root=TraceIdRatioBased(sample_rate))

    # Create tracer provider with sampler
    provider = TracerProvider(resource=resource, sampler=sampler)

    # Configure OTLP exporter with authentication
    otlp_exporter = OTLPSpanExporter(
        endpoint=os.getenv("OTEL_EXPORTER_OTLP_ENDPOINT", "https://scout.base14.io:4317"),
        headers={
            "authorization": f"Bearer {os.getenv('SCOUT_API_KEY', '')}",
        },
        insecure=False,
    )

    # Configure batch span processor with production settings
    batch_processor = BatchSpanProcessor(
        otlp_exporter,
        max_queue_size=2048,
        max_export_batch_size=512,
        schedule_delay_millis=5000,
        export_timeout_millis=30000,
    )

    provider.add_span_processor(batch_processor)
    trace.set_tracer_provider(provider)

    # Instrument Django with exclude patterns
    DjangoInstrumentor().instrument(
        excluded_urls="health/,readiness/,liveness/,metrics/,favicon.ico"
    )

    # Instrument database
    Psycopg2Instrumentor().instrument(enable_commenter=True, commenter_options={})

    # Instrument Redis
    RedisInstrumentor().instrument()

    print(f"OpenTelemetry initialized: {resource.attributes.get('service.name')} "
          f"v{resource.attributes.get('service.version')} "
          f"(sample rate: {sample_rate})")

Gunicorn Production Configuration

gunicorn.conf.py
"""Gunicorn configuration for production deployment."""
import multiprocessing
import os

# Server socket
bind = "0.0.0.0:8000"
backlog = 2048

# Worker processes
workers = int(os.getenv("GUNICORN_WORKERS", multiprocessing.cpu_count() * 2 + 1))
worker_class = "sync"
worker_connections = 1000
max_requests = 1000
max_requests_jitter = 50
timeout = 30
keepalive = 5

# Logging
accesslog = "-"
errorlog = "-"
loglevel = os.getenv("LOG_LEVEL", "info")
access_log_format = '%(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s" trace_id=%(L)s'

# Process naming
proc_name = "django-order-service"

# Server hooks for tracing initialization
def on_starting(server):
    """Initialize OpenTelemetry before workers start."""
    from myproject.tracing import initialize_tracing
    initialize_tracing()

def post_worker_init(worker):
    """Log worker initialization."""
    print(f"Worker {worker.pid} initialized with OpenTelemetry tracing")

Dockerfile (Multi-Stage Build)

Dockerfile
# Stage 1: Build dependencies
FROM python:3.13-slim AS builder

WORKDIR /app

# Install system dependencies
RUN apt-get update && apt-get install -y \
    gcc \
    postgresql-client \
    libpq-dev \
    && rm -rf /var/lib/apt/lists/*

# Copy requirements and install dependencies
COPY requirements.txt .
RUN pip wheel --no-cache-dir --no-deps --wheel-dir /app/wheels -r requirements.txt

# Stage 2: Runtime image
FROM python:3.13-slim

WORKDIR /app

# Install runtime dependencies
RUN apt-get update && apt-get install -y \
    libpq5 \
    && rm -rf /var/lib/apt/lists/*

# Copy wheels from builder
COPY --from=builder /app/wheels /wheels
COPY --from=builder /app/requirements.txt .

# Install Python packages
RUN pip install --no-cache /wheels/*

# Copy application code
COPY . .

# Create non-root user
RUN useradd -m -u 1000 django && chown -R django:django /app
USER django

# Environment variables
ENV PYTHONUNBUFFERED=1
ENV DJANGO_SETTINGS_MODULE=myproject.settings
ENV OTEL_PYTHON_DJANGO_INSTRUMENT=true

# Expose port
EXPOSE 8000

# Health check
HEALTHCHECK --interval=30s --timeout=3s --start-period=40s --retries=3 \
    CMD python -c "import requests; requests.get('http://localhost:8000/health/')"

# Run Gunicorn
CMD ["gunicorn", "myproject.wsgi:application", "--config", "gunicorn.conf.py"]

Kubernetes Deployment

k8s/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: django-order-service
  labels:
    app: django-order-service
spec:
  replicas: 3
  selector:
    matchLabels:
      app: django-order-service
  template:
    metadata:
      labels:
        app: django-order-service
      annotations:
        prometheus.io/scrape: 'true'
        prometheus.io/port: '8000'
    spec:
      containers:
        - name: django-app
          image: django-order-service:latest
          ports:
            - containerPort: 8000
              name: http
          env:
            - name: OTEL_SERVICE_NAME
              value: django-order-service
            - name: OTEL_EXPORTER_OTLP_ENDPOINT
              value: http://scout-collector:4317
            - name: SCOUT_API_KEY
              valueFrom:
                secretKeyRef:
                  name: scout-credentials
                  key: api-key
            - name: OTEL_TRACE_SAMPLE_RATE
              value: '0.1'
            - name: ENVIRONMENT
              value: production
            - name: APP_VERSION
              value: '1.0.0'
            - name: K8S_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: HOSTNAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: DB_HOST
              value: postgres-service
            - name: DB_NAME
              value: orders
            - name: DB_USER
              valueFrom:
                secretKeyRef:
                  name: postgres-credentials
                  key: username
            - name: DB_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: postgres-credentials
                  key: password
            - name: CELERY_BROKER_URL
              value: redis://redis-service:6379/0
          resources:
            requests:
              memory: '512Mi'
              cpu: '250m'
            limits:
              memory: '1Gi'
              cpu: '1000m'
          livenessProbe:
            httpGet:
              path: /health/
              port: 8000
            initialDelaySeconds: 30
            periodSeconds: 10
          readinessProbe:
            httpGet:
              path: /readiness/
              port: 8000
            initialDelaySeconds: 10
            periodSeconds: 5
---
apiVersion: v1
kind: Service
metadata:
  name: django-order-service
spec:
  selector:
    app: django-order-service
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8000
  type: ClusterIP

Django-Specific Features

Django's auto-instrumentation automatically captures views, ORM queries, middleware, and template rendering without manual span creation.

View Auto-Instrumentation

All Django views are automatically instrumented:

orders/views.py
"""Django views with automatic tracing."""
from django.shortcuts import render, get_object_or_404
from django.http import JsonResponse
from django.views import View
from django.views.decorators.http import require_http_methods
from rest_framework import viewsets
from rest_framework.decorators import api_view
from .models import Order
from .serializers import OrderSerializer


# Function-based view (automatically traced)
@require_http_methods(["GET"])
def order_list(request):
    """Get all orders - span name: 'GET /orders/'"""
    orders = Order.objects.all().select_related('user')
    data = [{"id": o.id, "product": o.product_name, "amount": str(o.amount)} for o in orders]
    return JsonResponse({"orders": data})


# Function-based view with parameter
def order_detail(request, order_id):
    """Get order by ID - span name: 'GET /orders/{order_id}/'"""
    order = get_object_or_404(Order, id=order_id)
    return JsonResponse({
        "id": order.id,
        "product": order.product_name,
        "amount": str(order.amount),
        "status": order.status
    })


# Class-based view (automatically traced)
class OrderCreateView(View):
    """Create order - span name: 'POST /orders/create/'"""

    def post(self, request):
        # Business logic automatically traced
        order = Order.objects.create(
            user_id=request.user.id,
            product_name=request.POST.get('product'),
            amount=request.POST.get('amount'),
            status='pending'
        )
        return JsonResponse({"id": order.id, "status": "created"}, status=201)


# Django REST Framework ViewSet (automatically traced)
class OrderViewSet(viewsets.ModelViewSet):
    """
    API endpoint for orders.
    Each action creates span: 'GET /api/orders/', 'POST /api/orders/', etc.
    """
    queryset = Order.objects.all()
    serializer_class = OrderSerializer

    def get_queryset(self):
        # Query optimization automatically traced
        queryset = super().get_queryset()
        return queryset.select_related('user').prefetch_related('items')

    def perform_create(self, serializer):
        # Custom logic traced as part of the request span
        order = serializer.save()
        # Trigger async task (traced separately)
        from .tasks import process_order
        process_order.delay(order.id)

Django ORM Query Instrumentation

Database queries are automatically traced with full SQL visibility:

orders/models.py
"""Django models with automatic query tracing."""
from django.db import models
from django.contrib.auth.models import User
from decimal import Decimal


class Order(models.Model):
    """Order model - all queries automatically traced."""
    user = models.ForeignKey(User, on_delete=models.CASCADE, related_name='orders')
    product_name = models.CharField(max_length=200)
    amount = models.DecimalField(max_digits=10, decimal_places=2)
    status = models.CharField(max_length=50, default='pending')
    created_at = models.DateTimeField(auto_now_add=True)
    updated_at = models.DateTimeField(auto_now=True)

    class Meta:
        db_table = 'orders'
        indexes = [
            models.Index(fields=['user', 'status']),
            models.Index(fields=['created_at']),
        ]

    def __str__(self):
        return f"Order {self.id} - {self.product_name}"

    # Custom manager methods are traced
    @classmethod
    def get_user_orders(cls, user_id):
        """Get orders for user - traced as SELECT query with JOIN."""
        return cls.objects.filter(user_id=user_id).select_related('user')

    @classmethod
    def get_recent_orders(cls, limit=10):
        """Get recent orders - traced with LIMIT clause."""
        return cls.objects.order_by('-created_at')[:limit]


class OrderItem(models.Model):
    """Order line items."""
    order = models.ForeignKey(Order, on_delete=models.CASCADE, related_name='items')
    product_sku = models.CharField(max_length=100)
    quantity = models.IntegerField(default=1)
    unit_price = models.DecimalField(max_digits=10, decimal_places=2)

    class Meta:
        db_table = 'order_items'

orders/services.py
"""Business logic with ORM query tracing."""
from django.db import transaction
from django.db.models import Sum, Count, Q
from opentelemetry import trace
from .models import Order, OrderItem

tracer = trace.get_tracer(__name__)


class OrderService:
    """Service layer with automatic query tracing."""

    @staticmethod
    def create_order_with_items(user, items_data):
        """
        Create order with items in transaction.
        All queries traced automatically with transaction context.
        """
        with transaction.atomic():
            # INSERT query automatically traced
            order = Order.objects.create(
                user=user,
                product_name=items_data[0]['product'],
                amount=sum(item['quantity'] * item['price'] for item in items_data),
                status='pending'
            )

            # Bulk INSERT traced
            order_items = [
                OrderItem(
                    order=order,
                    product_sku=item['sku'],
                    quantity=item['quantity'],
                    unit_price=item['price']
                )
                for item in items_data
            ]
            OrderItem.objects.bulk_create(order_items)

            return order

    @staticmethod
    def get_order_analytics(user_id):
        """
        Aggregate query automatically traced.
        Span includes: SELECT COUNT(*), SUM(amount), GROUP BY
        """
        return Order.objects.filter(user_id=user_id).aggregate(
            total_orders=Count('id'),
            total_spent=Sum('amount')
        )

    @staticmethod
    def search_orders(query):
        """
        Complex query with Q objects - traced with full WHERE clause.
        """
        return Order.objects.filter(
            Q(product_name__icontains=query) |
            Q(status__icontains=query)
        ).select_related('user').prefetch_related('items')

Middleware Tracing

All middleware in the processing chain is automatically traced:

myproject/middleware.py
"""Custom middleware with automatic tracing."""
from opentelemetry import trace
from django.utils.deprecation import MiddlewareMixin

tracer = trace.get_tracer(__name__)


class CustomHeaderMiddleware(MiddlewareMixin):
    """
    Custom middleware - automatically creates child span.
    Span name: 'CustomHeaderMiddleware.process_request'
    """

    def process_request(self, request):
        # Get current span (created by Django instrumentation)
        current_span = trace.get_current_span()

        # Add custom attributes
        current_span.set_attribute("http.custom_header", request.META.get('HTTP_X_CUSTOM', 'none'))
        current_span.set_attribute("request.user.authenticated", request.user.is_authenticated)

        if request.user.is_authenticated:
            current_span.set_attribute("user.id", str(request.user.id))
            current_span.set_attribute("user.username", request.user.username)

        return None

    def process_response(self, request, response):
        # Add response attributes
        current_span = trace.get_current_span()
        current_span.set_attribute("http.response.content_length", len(response.content))
        return response


class PerformanceMonitoringMiddleware(MiddlewareMixin):
    """Monitor slow requests with custom spans."""

    def process_view(self, request, view_func, view_args, view_kwargs):
        # Create custom span for view execution
        with tracer.start_as_current_span("view_execution") as span:
            span.set_attribute("view.name", view_func.__name__)
            span.set_attribute("view.module", view_func.__module__)

            # View execution happens here (automatically traced)
            return None

Template Rendering Tracing

Django template rendering is automatically traced:

orders/views.py
"""Template views with automatic rendering traces."""
from django.shortcuts import render
from .models import Order


def order_list_html(request):
    """
    Render template - creates two child spans:
    1. SELECT query for Order.objects.all()
    2. Template rendering: 'orders/list.html'
    """
    orders = Order.objects.all().select_related('user')

    # Template rendering automatically traced
    return render(request, 'orders/list.html', {
        'orders': orders,
        'title': 'Order List'
    })


def order_detail_html(request, order_id):
    """
    Complex template with includes - each template traced separately:
    - 'orders/detail.html'
    - 'orders/includes/order_summary.html'
    - 'orders/includes/order_items.html'
    """
    order = Order.objects.get(id=order_id)
    return render(request, 'orders/detail.html', {'order': order})

Celery Task Tracing

Celery tasks are automatically traced with distributed context propagation:

orders/tasks.py
"""Celery tasks with automatic distributed tracing."""
from celery import shared_task
from opentelemetry import trace
from django.core.mail import send_mail
from .models import Order

tracer = trace.get_tracer(__name__)


@shared_task
def process_order(order_id):
    """
    Process order asynchronously.
    Automatically creates span: 'orders.tasks.process_order'
    Trace context propagated from parent request.
    """
    # Get current span (linked to original request via trace context)
    current_span = trace.get_current_span()
    current_span.set_attribute("order.id", order_id)

    # Database query traced automatically
    order = Order.objects.select_related('user').get(id=order_id)

    # Business logic
    if order.amount > 1000:
        # Call external service (traced if instrumented)
        validate_high_value_order(order)

    # Update status (UPDATE query traced)
    order.status = 'processing'
    order.save()

    # Send notification (SMTP traced if instrumented)
    send_order_confirmation_email(order)

    current_span.set_attribute("order.status", "completed")
    return {"order_id": order_id, "status": "processed"}


@shared_task
def send_order_confirmation_email(order):
    """Send email - traced as child span."""
    with tracer.start_as_current_span("send_confirmation_email") as span:
        span.set_attribute("email.to", order.user.email)
        span.set_attribute("order.id", order.id)

        send_mail(
            subject=f'Order Confirmation - {order.id}',
            message=f'Your order for {order.product_name} has been confirmed.',
            from_email='noreply@example.com',
            recipient_list=[order.user.email],
        )


def validate_high_value_order(order):
    """Custom validation with manual span."""
    with tracer.start_as_current_span("validate_high_value_order") as span:
        span.set_attribute("order.id", order.id)
        span.set_attribute("order.amount", float(order.amount))

        # Complex validation logic
        if order.amount > 10000:
            span.set_attribute("validation.requires_approval", True)
            # Trigger approval workflow
        else:
            span.set_attribute("validation.auto_approved", True)

Initialize Celery with tracing:

myproject/celery.py
"""Celery configuration with OpenTelemetry."""
import os
from celery import Celery
from celery.signals import worker_process_init

# Set Django settings module
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'myproject.settings')

app = Celery('myproject')
app.config_from_object('django.conf:settings', namespace='CELERY')
app.autodiscover_tasks()


@worker_process_init.connect
def init_tracing_on_worker(**kwargs):
    """Initialize OpenTelemetry in Celery worker process."""
    from myproject.tracing import initialize_tracing
    initialize_tracing()
    print("OpenTelemetry initialized in Celery worker")

Custom Instrumentation

While Django auto-instrumentation covers most use cases, custom spans are needed for specific business logic or external integrations.

Manual Span Creation

orders/services.py
"""Custom instrumentation for business logic."""
from opentelemetry import trace
from opentelemetry.trace import Status, StatusCode
from django.db import transaction
from .models import Order

tracer = trace.get_tracer(__name__)


class PaymentService:
    """Payment processing with custom instrumentation."""

    @staticmethod
    def process_payment(order_id, payment_method):
        """Process payment with custom span."""
        with tracer.start_as_current_span("process_payment") as span:
            span.set_attribute("order.id", order_id)
            span.set_attribute("payment.method", payment_method)

            try:
                # Get order (automatically traced)
                order = Order.objects.get(id=order_id)
                span.set_attribute("payment.amount", float(order.amount))

                # Validate payment method
                PaymentService._validate_payment_method(payment_method)

                # Call external payment gateway
                transaction_id = PaymentService._charge_payment_gateway(
                    order.amount,
                    payment_method
                )

                span.set_attribute("payment.transaction_id", transaction_id)
                span.set_status(Status(StatusCode.OK))

                return {"success": True, "transaction_id": transaction_id}

            except Exception as e:
                span.set_status(Status(StatusCode.ERROR, str(e)))
                span.record_exception(e)
                raise

    @staticmethod
    def _validate_payment_method(method):
        """Validation with custom span."""
        with tracer.start_as_current_span("validate_payment_method") as span:
            span.set_attribute("payment.method", method)

            valid_methods = ["credit_card", "debit_card", "paypal"]
            if method not in valid_methods:
                span.set_status(Status(StatusCode.ERROR, "Invalid payment method"))
                raise ValueError(f"Invalid payment method: {method}")

            span.set_attribute("validation.result", "valid")

    @staticmethod
    def _charge_payment_gateway(amount, method):
        """External API call with custom span."""
        with tracer.start_as_current_span(
            "payment_gateway.charge",
            kind=trace.SpanKind.CLIENT
        ) as span:
            span.set_attribute("payment.amount", float(amount))
            span.set_attribute("payment.method", method)
            span.set_attribute("payment.gateway", "stripe")

            # Simulate external API call
            import requests
            response = requests.post(
                "https://api.stripe.com/v1/charges",
                json={"amount": float(amount), "method": method},
                timeout=10
            )

            span.set_attribute("http.status_code", response.status_code)

            if response.status_code == 200:
                transaction_id = response.json().get("id")
                span.set_attribute("payment.transaction_id", transaction_id)
                return transaction_id
            else:
                span.set_status(Status(StatusCode.ERROR, "Payment failed"))
                raise Exception("Payment gateway error")

Django Management Command Tracing

orders/management/commands/process_pending_orders.py
"""Management command with custom instrumentation."""
from django.core.management.base import BaseCommand
from opentelemetry import trace
from orders.models import Order
from orders.tasks import process_order

tracer = trace.get_tracer(__name__)


class Command(BaseCommand):
    help = 'Process all pending orders'

    def add_arguments(self, parser):
        parser.add_argument('--limit', type=int, default=100, help='Max orders to process')
        parser.add_argument('--dry-run', action='store_true', help='Dry run mode')

    def handle(self, *args, **options):
        """Execute command with custom tracing."""
        with tracer.start_as_current_span("management_command.process_pending_orders") as span:
            limit = options['limit']
            dry_run = options['dry_run']

            span.set_attribute("command.limit", limit)
            span.set_attribute("command.dry_run", dry_run)

            # Query pending orders (automatically traced)
            pending_orders = Order.objects.filter(status='pending')[:limit]
            count = pending_orders.count()

            span.set_attribute("orders.pending_count", count)

            if dry_run:
                self.stdout.write(f"[DRY RUN] Would process {count} orders")
                span.set_attribute("command.result", "dry_run")
                return

            # Process each order
            processed = 0
            failed = 0

            for order in pending_orders:
                with tracer.start_as_current_span("process_single_order") as order_span:
                    order_span.set_attribute("order.id", order.id)

                    try:
                        # Trigger Celery task (traced separately)
                        process_order.delay(order.id)
                        processed += 1
                        order_span.set_status(trace.Status(trace.StatusCode.OK))
                    except Exception as e:
                        failed += 1
                        order_span.set_status(trace.Status(trace.StatusCode.ERROR, str(e)))
                        order_span.record_exception(e)

            span.set_attribute("orders.processed_count", processed)
            span.set_attribute("orders.failed_count", failed)

            self.stdout.write(self.style.SUCCESS(
                f'Processed {processed} orders, {failed} failed'
            ))

N+1 Query Detection

Add custom span attributes to detect N+1 query patterns:

orders/utils.py
"""Utilities for detecting N+1 queries."""
from django.db import connection, reset_queries
from django.conf import settings
from opentelemetry import trace
from functools import wraps

tracer = trace.get_tracer(__name__)


def detect_n_plus_one(func):
    """Decorator to detect N+1 query patterns."""
    @wraps(func)
    def wrapper(*args, **kwargs):
        if not settings.DEBUG:
            return func(*args, **kwargs)

        reset_queries()
        result = func(*args, **kwargs)

        # Analyze query patterns
        queries = connection.queries
        query_count = len(queries)

        # Get current span
        current_span = trace.get_current_span()
        current_span.set_attribute("db.query_count", query_count)

        # Detect potential N+1
        similar_queries = {}
        for query in queries:
            sql = query['sql'].split('WHERE')[0]  # Group by base query
            similar_queries[sql] = similar_queries.get(sql, 0) + 1

        # Flag if any query executed multiple times
        max_repetitions = max(similar_queries.values()) if similar_queries else 0
        if max_repetitions > 5:
            current_span.set_attribute("db.potential_n_plus_one", True)
            current_span.set_attribute("db.max_query_repetitions", max_repetitions)

        return result

    return wrapper

Usage:

orders/views.py
from orders.utils import detect_n_plus_one


@detect_n_plus_one
def get_all_orders_with_users(request):
    """This will flag N+1 if select_related is missing."""
    # BAD: N+1 query (flagged in span attributes)
    orders = Order.objects.all()
    for order in orders:
        print(order.user.username)  # Separate query for each order

    # GOOD: Optimized query
    orders = Order.objects.all().select_related('user')
    for order in orders:
        print(order.user.username)  # No extra queries

    return JsonResponse({"count": len(orders)})

Running Your Application

Django applications can run with various WSGI/ASGI servers. OpenTelemetry instrumentation works with all standard deployment methods.

Development Server

Terminal
# Run Django development server with tracing
python manage.py runserver

# Access application
curl http://localhost:8000/orders/

# Run Celery worker (separate terminal)
celery -A myproject worker --loglevel=info

# Run Celery beat (scheduled tasks)
celery -A myproject beat --loglevel=info

Gunicorn (Production WSGI)

Terminal
# Run with Gunicorn
gunicorn myproject.wsgi:application \
  --bind 0.0.0.0:8000 \
  --workers 4 \
  --config gunicorn.conf.py

# With environment variables
OTEL_SERVICE_NAME=django-order-service \
OTEL_EXPORTER_OTLP_ENDPOINT=https://scout.base14.io:4317 \
SCOUT_API_KEY=your_api_key \
gunicorn myproject.wsgi:application --bind 0.0.0.0:8000

uWSGI (Alternative WSGI)

uwsgi.ini
[uwsgi]
module = myproject.wsgi:application
master = true
processes = 4
socket = /tmp/myproject.sock
chmod-socket = 666
vacuum = true
die-on-term = true

# OpenTelemetry environment
env = OTEL_SERVICE_NAME=django-order-service
env = OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317

Run:

uwsgi --ini uwsgi.ini

Daphne (ASGI for Django Channels)

Terminal
# Run with Daphne for async/WebSocket support
daphne -b 0.0.0.0 -p 8000 myproject.asgi:application

# With environment variables
OTEL_SERVICE_NAME=django-channels \
daphne myproject.asgi:application

Docker Deployment

Terminal
# Build Docker image
docker build -t django-order-service:latest .

# Run container
docker run -p 8000:8000 \
  -e OTEL_SERVICE_NAME=django-order-service \
  -e OTEL_EXPORTER_OTLP_ENDPOINT=https://scout.base14.io:4317 \
  -e SCOUT_API_KEY=your_api_key \
  -e DB_HOST=postgres \
  -e DB_NAME=orders \
  django-order-service:latest

# Run with Docker Compose
docker-compose up -d

# Check logs
docker-compose logs -f django-app

# Run migrations
docker-compose exec django-app python manage.py migrate

# Create superuser
docker-compose exec django-app python manage.py createsuperuser

Kubernetes Deployment

Terminal
# Deploy to Kubernetes
kubectl apply -f k8s/deployment.yaml

# Check pod status
kubectl get pods -l app=django-order-service

# View logs
kubectl logs -f deployment/django-order-service

# Run migrations (one-time job)
kubectl run django-migrate --rm -i --tty \
  --image=django-order-service:latest \
  --restart=Never \
  --command -- python manage.py migrate

# Access application
kubectl port-forward deployment/django-order-service 8000:8000
curl http://localhost:8000/orders/

Troubleshooting

Issue 1: No Traces Generated

Symptoms: Django application starts but no traces appear in collector.

Diagnosis:

# Add debug logging to tracing initialization
import logging
logging.basicConfig(level=logging.DEBUG)

from myproject.tracing import initialize_tracing
initialize_tracing()

Solution:

Ensure Django instrumentation is initialized before application starts:

myproject/wsgi.py
# INCORRECT: Instrumentation after WSGI app creation
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
from myproject.tracing import initialize_tracing  # TOO LATE!
initialize_tracing()

# CORRECT: Instrumentation before WSGI app
from myproject.tracing import initialize_tracing
initialize_tracing()  # Initialize first
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()

Issue 2: Database Queries Not Traced

Symptoms: HTTP requests create spans but SQL queries are missing.

Solution:

Instrument the database driver explicitly:

myproject/tracing.py
from opentelemetry.instrumentation.psycopg2 import Psycopg2Instrumentor

# For PostgreSQL with psycopg2
Psycopg2Instrumentor().instrument(enable_commenter=True)

# For MySQL
from opentelemetry.instrumentation.pymysql import PyMySQLInstrumentor
PyMySQLInstrumentor().instrument()

# For SQLite (Django default DB for development)
from opentelemetry.instrumentation.sqlite3 import SQLite3Instrumentor
SQLite3Instrumentor().instrument()

Issue 3: Celery Tasks Not Linked to Parent Trace

Symptoms: Celery task spans exist but are disconnected from the originating HTTP request.

Solution:

Ensure Celery instrumentation is initialized in worker process:

myproject/celery.py
from celery import Celery
from celery.signals import worker_process_init, worker_process_shutdown
from opentelemetry.instrumentation.celery import CeleryInstrumentor

app = Celery('myproject')


@worker_process_init.connect
def init_celery_tracing(**kwargs):
    """Initialize tracing in each worker process."""
    from myproject.tracing import initialize_tracing
    initialize_tracing()

    # Instrument Celery
    CeleryInstrumentor().instrument()


@worker_process_shutdown.connect
def shutdown_tracing(**kwargs):
    """Clean shutdown of tracing."""
    from opentelemetry import trace
    trace.get_tracer_provider().shutdown()

Issue 4: High Memory Usage with Tracing

Symptoms: Django application memory grows continuously with tracing enabled.

Solution:

Configure batch span processor limits:

myproject/tracing.py
from opentelemetry.sdk.trace.export import BatchSpanProcessor

# Reduce memory footprint
batch_processor = BatchSpanProcessor(
    otlp_exporter,
    max_queue_size=1024,  # Reduced from default 2048
    max_export_batch_size=256,  # Reduced from default 512
    schedule_delay_millis=3000,  # Export more frequently
)

Issue 5: Middleware Order Causing Issues

Symptoms: Some requests traced, others not, or tracing interferes with authentication.

Solution:

Django's auto-instrumentation injects middleware automatically. Ensure proper ordering:

settings.py
MIDDLEWARE = [
    # OpenTelemetry middleware injected here automatically (first)
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',  # After session
    # Custom middleware after auth
    'myproject.middleware.CustomHeaderMiddleware',
]

Security Considerations

PII Data Masking

Django applications often handle sensitive user data. Implement custom span processors to mask PII:

myproject/tracing.py
"""PII masking for Django tracing."""
import re
from opentelemetry.sdk.trace import SpanProcessor, ReadableSpan


class PIIMaskingSpanProcessor(SpanProcessor):
    """Mask PII data in span attributes."""

    EMAIL_PATTERN = re.compile(r'[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}')
    PHONE_PATTERN = re.compile(r'\b\d{3}[-.]?\d{3}[-.]?\d{4}\b')
    SSN_PATTERN = re.compile(r'\b\d{3}-\d{2}-\d{4}\b')

    def on_start(self, span: "ReadableSpan", parent_context=None) -> None:
        """Mask PII in span name and attributes on span start."""
        # Get writable span
        if hasattr(span, '_name'):
            span._name = self._mask_pii(span._name)

    def on_end(self, span: ReadableSpan) -> None:
        """Mask PII in final span data."""
        if hasattr(span, 'attributes'):
            for key, value in span.attributes.items():
                if isinstance(value, str):
                    span.attributes[key] = self._mask_pii(value)

    def _mask_pii(self, text: str) -> str:
        """Mask sensitive patterns in text."""
        if not isinstance(text, str):
            return text

        # Mask email addresses
        text = self.EMAIL_PATTERN.sub('***@***.**', text)

        # Mask phone numbers
        text = self.PHONE_PATTERN.sub('***-***-****', text)

        # Mask SSN
        text = self.SSN_PATTERN.sub('***-**-****', text)

        return text

    def shutdown(self) -> None:
        pass

    def force_flush(self, timeout_millis: int = 30000) -> bool:
        return True


# Add to tracer provider
def initialize_tracing():
    # ... existing setup ...
    provider.add_span_processor(PIIMaskingSpanProcessor())
    provider.add_span_processor(BatchSpanProcessor(otlp_exporter))

SQL Parameter Obfuscation

Database queries may contain sensitive values:

myproject/tracing.py
"""SQL query sanitization."""
from opentelemetry.instrumentation.psycopg2 import Psycopg2Instrumentor


# Enable SQL commenter to identify queries, but disable parameter logging
Psycopg2Instrumentor().instrument(
    enable_commenter=True,
    commenter_options={
        "db_driver": True,
        "db_framework": True,
        "opentelemetry_values": False,  # Don't include parameter values
    }
)

Custom query sanitization:

myproject/middleware.py
"""Sanitize database queries in spans."""
from opentelemetry import trace


class QuerySanitizationMiddleware:
    """Sanitize SQL queries to remove sensitive data."""

    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        response = self.get_response(request)

        # Post-process spans to sanitize queries
        current_span = trace.get_current_span()

        if hasattr(current_span, 'attributes'):
            if 'db.statement' in current_span.attributes:
                # Replace parameter values with placeholders
                sql = current_span.attributes['db.statement']
                # Remove values: WHERE email = 'user@example.com' -> WHERE email = ?
                sanitized = re.sub(r"= '.*?'", "= ?", sql)
                current_span.attributes['db.statement'] = sanitized

        return response

For GDPR compliance, exclude user-identifying information:

myproject/tracing.py
"""GDPR-compliant tracing configuration."""


def initialize_tracing_gdpr_compliant():
    """Initialize tracing without collecting user PII."""

    # ... existing setup ...

    # Instrument Django with URL exclusions
    from opentelemetry.instrumentation.django import DjangoInstrumentor

    DjangoInstrumentor().instrument(
        # Exclude endpoints that handle PII
        excluded_urls="admin/,accounts/profile/,api/users/",
        # Disable automatic user context
        request_hook=None,
        response_hook=None,
    )

    # Configure span limits to prevent large payloads
    from opentelemetry.sdk.trace import TracerProvider

    provider = TracerProvider(
        resource=resource,
        span_limits=SpanLimits(
            max_attributes=32,
            max_events=32,
            max_links=32,
            max_attribute_length=256,
        ),
    )

Performance Considerations

Tracing Overhead Metrics

Measured performance impact of OpenTelemetry on Django applications:

Configuration	Latency (p50)	Latency (p99)	Throughput	Memory
No Tracing	15ms	45ms	8,000 rps	250MB
Tracing (100%)	16ms (+6%)	50ms (+11%)	7,500 rps	320MB
Tracing (10%)	15ms (+<1%)	46ms (+2%)	7,900 rps	270MB

Key Findings:

Sampling at 10% reduces overhead to negligible levels
ORM query tracing adds ~1ms per complex query
Template rendering tracing: <0.5ms overhead
Celery task tracing: <2ms overhead per task

Optimization Strategies

1. Exclude High-Volume Endpoints

myproject/tracing.py
from opentelemetry.instrumentation.django import DjangoInstrumentor

# Don't trace health checks, metrics, static files
DjangoInstrumentor().instrument(
    excluded_urls="health/,readiness/,metrics/,static/,media/,favicon.ico"
)

2. Optimize ORM Queries to Reduce Spans

orders/views.py
# BAD: Creates N+1 query spans
def get_orders_bad(request):
    orders = Order.objects.all()  # 1 query
    for order in orders:
        print(order.user.username)  # N queries (N spans!)

# GOOD: Single query with JOIN (1 span)
def get_orders_good(request):
    orders = Order.objects.select_related('user').all()  # 1 query
    for order in orders:
        print(order.user.username)  # No extra queries

3. Batch Span Export Configuration

myproject/tracing.py
from opentelemetry.sdk.trace.export import BatchSpanProcessor

# Optimized for high-throughput Django apps
batch_processor = BatchSpanProcessor(
    otlp_exporter,
    max_queue_size=2048,
    max_export_batch_size=512,
    schedule_delay_millis=5000,  # Export every 5 seconds
    export_timeout_millis=30000,
)

4. Disable Tracing in Tests

myproject/settings.py
# Disable tracing in test environment
import sys

TESTING = 'test' in sys.argv

if TESTING:
    OTEL_SDK_DISABLED = True

Or use environment variable:

OTEL_SDK_DISABLED=true python manage.py test

FAQ

1. Do I need to manually instrument Django views?

No. Django's auto-instrumentation automatically traces all views (function-based and class-based) when you call DjangoInstrumentor().instrument(). No decorators or manual span creation required for basic request/response tracing.

2. How do I trace Django management commands?

Use custom spans in your management command's handle() method:

from opentelemetry import trace

tracer = trace.get_tracer(__name__)

class Command(BaseCommand):
    def handle(self, *args, **options):
        with tracer.start_as_current_span("management_command.my_command"):
            # Your command logic here
            pass

3. Can I use OpenTelemetry with Django REST Framework?

Yes. Django REST Framework views and viewsets are automatically instrumented through Django's middleware. Each API endpoint creates a span with the HTTP method and path (e.g., GET /api/orders/).

4. How do I detect N+1 database queries?

Check span attributes for db.query_count. High counts (>10 queries per request) often indicate N+1 issues. Use the custom decorator shown in the "Custom Instrumentation" section to automatically flag potential N+1 patterns.

5. Does tracing work with Celery background tasks?

Yes. Install opentelemetry-instrumentation-celery and instrument in your Celery worker initialization. Trace context automatically propagates from Django views to Celery tasks, creating a distributed trace across synchronous and asynchronous operations.

6. How do I mask PII data in traces?

Implement a custom SpanProcessor (see "Security Considerations" section) that filters sensitive data patterns (emails, phone numbers, SSNs) from span names and attributes before export.

7. Can I trace async Django views (ASGI)?

Yes. OpenTelemetry supports ASGI applications. Initialize tracing in your asgi.py file before creating the ASGI application, and async views will be traced automatically.

8. What's the performance overhead of tracing?

With 10% sampling, overhead is typically <1% for latency and ~8% for memory. Without sampling (100% tracing), expect ~6% latency increase and ~30% memory increase. See "Performance Considerations" for detailed metrics.

9. How do I send traces to Base14 Scout?

Configure the OTLP exporter endpoint and authentication:

OTLPSpanExporter(
    endpoint="https://scout.base14.io:4317",
    headers={"authorization": f"Bearer {os.getenv('SCOUT_API_KEY')}"},
)

10. Can I trace template rendering?

Yes. Template rendering is automatically traced when using render() or TemplateResponse. Each template creates a child span showing rendering time.

11. How do I trace multiple databases?

Django's database instrumentation traces all configured databases automatically. Span attributes include db.name to differentiate between databases.

12. Can I disable tracing for specific views?

Use the excluded_urls parameter in DjangoInstrumentor().instrument():

DjangoInstrumentor().instrument(
    excluded_urls="admin/,debug/,internal/"
)

What's Next

Now that you have Django instrumented with OpenTelemetry, explore advanced observability patterns:

Advanced Tracing Topics

Custom Instrumentation for Python - Deep dive into manual span creation and context propagation
Flask Instrumentation - Compare Django's approach with Flask's minimalist framework (guide coming soon)
FastAPI Instrumentation - Explore async Python framework tracing patterns

Scout Platform Features

Base14 Scout Dashboard - Visualize Django traces with ORM query insights and N+1 detection
Service Map Visualization - Understand dependencies between Django, Celery, and external services
Alert Configuration - Set up alerts for slow database queries and high error rates

Deployment & Operations

Docker Instrumentation - Deploy instrumented Django apps in containers
Kubernetes Deployment - Run Django with sidecar collectors and service mesh
AWS Deployment - Deploy to ECS, Elastic Beanstalk, or Lambda with tracing

Node.js Instrumentation - Compare Django ORM patterns with Node.js database tracing
Spring Boot Instrumentation - Java enterprise framework with similar ORM auto-instrumentation
Go Instrumentation - Contrast Django's auto-instrumentation with Go's manual approach

Complete Example

Here's a complete Django application with OpenTelemetry instrumentation, including ORM queries, views, Celery tasks, and custom business logic.

Project Structure

django-order-service/
├── manage.py
├── myproject/
│   ├── __init__.py
│   ├── settings.py
│   ├── urls.py
│   ├── wsgi.py
│   ├── asgi.py
│   ├── celery.py
│   └── tracing.py
├── orders/
│   ├── __init__.py
│   ├── models.py
│   ├── views.py
│   ├── serializers.py
│   ├── tasks.py
│   ├── services.py
│   └── management/
│       └── commands/
│           └── process_pending_orders.py
├── requirements.txt
├── Dockerfile
├── docker-compose.yml
└── gunicorn.conf.py

Complete Settings Configuration

myproject/settings.py
"""Django settings with OpenTelemetry configuration."""
import os
from pathlib import Path

BASE_DIR = Path(__file__).resolve().parent.parent

SECRET_KEY = os.getenv('DJANGO_SECRET_KEY', 'dev-secret-change-in-production')
DEBUG = os.getenv('DEBUG', 'False') == 'True'
ALLOWED_HOSTS = os.getenv('ALLOWED_HOSTS', 'localhost').split(',')

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'rest_framework',
    'orders',
]

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

ROOT_URLCONF = 'myproject.urls'

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': os.getenv('DB_NAME', 'orders'),
        'USER': os.getenv('DB_USER', 'postgres'),
        'PASSWORD': os.getenv('DB_PASSWORD', 'postgres'),
        'HOST': os.getenv('DB_HOST', 'localhost'),
        'PORT': os.getenv('DB_PORT', '5432'),
    }
}

# Celery Configuration
CELERY_BROKER_URL = os.getenv('CELERY_BROKER_URL', 'redis://localhost:6379/0')
CELERY_RESULT_BACKEND = os.getenv('CELERY_RESULT_BACKEND', 'redis://localhost:6379/0')

# OpenTelemetry
OTEL_SERVICE_NAME = os.getenv('OTEL_SERVICE_NAME', 'django-order-service')
OTEL_EXPORTER_OTLP_ENDPOINT = os.getenv('OTEL_EXPORTER_OTLP_ENDPOINT', 'http://localhost:4317')

# Static files
STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')

Running the Example

Terminal
# Clone the examples repository
git clone https://github.com/base-14/examples.git
cd examples/python/django-postgres

# Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

# Start infrastructure
docker-compose up -d postgres redis scout-collector

# Run migrations
python manage.py migrate

# Create superuser
python manage.py createsuperuser

# Run development server
python manage.py runserver

# In another terminal: Run Celery worker
celery -A myproject worker --loglevel=info

# Test endpoints
# Create order
curl -X POST http://localhost:8000/api/orders/ \
  -H "Content-Type: application/json" \
  -d '{"product_name":"Widget","amount":"99.99","status":"pending"}'

# Get all orders
curl http://localhost:8000/api/orders/

# Process pending orders (management command)
python manage.py process_pending_orders --limit 10

# View traces in Scout Dashboard
open https://scout.base14.io

Expected Trace Output

When you create an order via POST /api/orders/, you should see:

POST /api/orders/ (250ms)
├── OrderViewSet.create (200ms)
│   ├── SELECT FROM auth_user WHERE id = ? (5ms)
│   ├── INSERT INTO orders (...) VALUES (...) (10ms)
│   ├── celery.apply_async: process_order (5ms)
│   └── serializer.save (15ms)
└── Celery Task: process_order (150ms) [separate trace, linked]
    ├── SELECT FROM orders WHERE id = ? (5ms)
    ├── validate_high_value_order (30ms)
    ├── UPDATE orders SET status = ? WHERE id = ? (8ms)
    └── send_confirmation_email (100ms)

Complete Example Repository

The full example application with Docker Compose, Kubernetes manifests, management commands, and PII masking is available at:

https://github.com/base-14/examples/tree/main/python/django-postgres

This includes production-ready configurations for AWS, GCP, and Azure deployments.

References

Official Documentation

Django Documentation - Official Django framework documentation
OpenTelemetry Python SDK - Core OpenTelemetry Python documentation
Django Instrumentation - Official Django auto-instrumentation docs
Celery Documentation - Distributed task queue documentation

Flask Instrumentation - Lightweight Python framework instrumentation (guide coming soon)
FastAPI Instrumentation - Async Python API framework
Python Custom Instrumentation - Advanced manual instrumentation patterns
Celery Tracing - Deep dive into Celery distributed tracing

Tools & Resources

Base14 Scout - Managed OpenTelemetry platform for Django applications
Django Debug Toolbar - Development tool for query analysis
Django REST Framework - API framework with automatic instrumentation

Was this page helpful?

Introduction​

Who This Guide Is For​

Overview​

What You'll Learn​

Prerequisites​

Installation & Setup​

Configuration​

Basic Initialization​

Django Settings Configuration​

Initialize in WSGI/ASGI Application​

Environment Variables​

Docker Compose Configuration​

Production Configuration​

Production Tracing Initialization​

Gunicorn Production Configuration​

Dockerfile (Multi-Stage Build)​

Kubernetes Deployment​

Django-Specific Features​

View Auto-Instrumentation​

Django ORM Query Instrumentation​

Middleware Tracing​

Template Rendering Tracing​

Celery Task Tracing​

Custom Instrumentation​

Manual Span Creation​

Django Management Command Tracing​

N+1 Query Detection​

Running Your Application​

Development Server​

Gunicorn (Production WSGI)​

uWSGI (Alternative WSGI)​

Daphne (ASGI for Django Channels)​

Docker Deployment​

Kubernetes Deployment​

Troubleshooting​

Issue 1: No Traces Generated​

Issue 2: Database Queries Not Traced​

Issue 3: Celery Tasks Not Linked to Parent Trace​

Issue 4: High Memory Usage with Tracing​

Issue 5: Middleware Order Causing Issues​

Security Considerations​

PII Data Masking​

SQL Parameter Obfuscation​

GDPR Compliance​

Performance Considerations​

Tracing Overhead Metrics​

Optimization Strategies​

1. Exclude High-Volume Endpoints​

2. Optimize ORM Queries to Reduce Spans​

3. Batch Span Export Configuration​

4. Disable Tracing in Tests​

FAQ​

1. Do I need to manually instrument Django views?​

2. How do I trace Django management commands?​

3. Can I use OpenTelemetry with Django REST Framework?​

4. How do I detect N+1 database queries?​

5. Does tracing work with Celery background tasks?​

6. How do I mask PII data in traces?​

7. Can I trace async Django views (ASGI)?​

8. What's the performance overhead of tracing?​

9. How do I send traces to Base14 Scout?​

10. Can I trace template rendering?​

11. How do I trace multiple databases?​

12. Can I disable tracing for specific views?​

What's Next​

Advanced Tracing Topics​

Scout Platform Features​

Deployment & Operations​

Related Frameworks​

Complete Example​

Project Structure​

Complete Settings Configuration​

Running the Example​

Expected Trace Output​

References​

Official Documentation​

Related Guides​

Tools & Resources​

Introduction

Who This Guide Is For

Overview

What You'll Learn

Prerequisites

Installation & Setup

Configuration

Basic Initialization

Django Settings Configuration

Initialize in WSGI/ASGI Application

Environment Variables

Docker Compose Configuration

Production Configuration

Production Tracing Initialization

Gunicorn Production Configuration

Dockerfile (Multi-Stage Build)

Kubernetes Deployment

Django-Specific Features

View Auto-Instrumentation

Django ORM Query Instrumentation

Middleware Tracing

Template Rendering Tracing

Celery Task Tracing

Custom Instrumentation

Manual Span Creation

Django Management Command Tracing

N+1 Query Detection

Running Your Application

Development Server

Gunicorn (Production WSGI)

uWSGI (Alternative WSGI)

Daphne (ASGI for Django Channels)

Docker Deployment

Kubernetes Deployment

Troubleshooting

Issue 1: No Traces Generated

Issue 2: Database Queries Not Traced

Issue 3: Celery Tasks Not Linked to Parent Trace

Issue 4: High Memory Usage with Tracing

Issue 5: Middleware Order Causing Issues

Security Considerations

PII Data Masking

SQL Parameter Obfuscation

GDPR Compliance

Performance Considerations

Tracing Overhead Metrics

Optimization Strategies

1. Exclude High-Volume Endpoints

2. Optimize ORM Queries to Reduce Spans

3. Batch Span Export Configuration

4. Disable Tracing in Tests

FAQ

1. Do I need to manually instrument Django views?

2. How do I trace Django management commands?

3. Can I use OpenTelemetry with Django REST Framework?

4. How do I detect N+1 database queries?

5. Does tracing work with Celery background tasks?

6. How do I mask PII data in traces?

7. Can I trace async Django views (ASGI)?

8. What's the performance overhead of tracing?

9. How do I send traces to Base14 Scout?

10. Can I trace template rendering?

11. How do I trace multiple databases?

12. Can I disable tracing for specific views?

What's Next

Advanced Tracing Topics

Scout Platform Features

Deployment & Operations

Related Frameworks

Complete Example

Project Structure

Complete Settings Configuration

Running the Example

Expected Trace Output

References

Official Documentation

Related Guides

Tools & Resources