Skip to main content

Kubernetes using Helm

Deploy and configure the Scout Collector on Kubernetes using Helm.

Overview​

This guide covers how to collect telemetry data (logs, metrics, and traces) from your Kubernetes environment and send it to base14 Scout.

  • Install base14 Scout's Scout Collector using Helm
  • Configure telemetry collection for Kubernetes pods
  • Set up multi-namespace monitoring
  • Configure custom metrics endpoints
  • Implement trace collection

Prerequisites​

  • A Kubernetes cluster (EKS, GKE, AKS, or other distributions)
  • Helm 3.x installed
  • kubectl configured with cluster access
  • Scout account credentials
    • Endpoint URL
    • Token URL
    • Client ID and Client Secret
    • Application Name

Quick Start Guide​

Deploy Scout Collector in minutes by following these steps:

helm repo add base14 https://charts.base14.io/
helm install scout base14/scout-collector --version 0.6.1 \
--namespace scout --create-namespace -f values.yaml

Windows Nodes​

note

Windows node support is experimental. It is disabled by default, so enabling it is an explicit opt-in.

If your cluster has Windows node groups, the collector needs a second DaemonSet to run on them. A single DaemonSet cannot serve both operating systems: the collector image is published separately for Linux and Windows, and the Linux daemon mounts Linux host paths that Windows pods do not support.

Enabling it​

Set this in your values.yaml, then install as shown above:

values.yaml
scout:
  windowsDaemon:
    enabled: true

That is all you need if you let the chart generate your collector configuration.

If you supply your own collector configuration​

If you set scout.agent.config or scout.daemon.config — as the otelcol style configuration below does — you must supply scout.windowsDaemon.config too. The chart cannot generate one for you: your configuration carries its own credentials, under value names you chose, and a generated configuration would use different ones. Rather than authenticate the Windows daemon differently from every other collector, the chart refuses to render and tells you so.

Use the same shape as your agent configuration, with Windows receivers:

values.yaml
scout:
  windowsDaemon:
    enabled: true
    config: |
      extensions:
        oauth2client:
          client_id: {{ .Values.scout.clientId }}
          client_secret: {{ .Values.scout.clientSecret }}
          token_url: {{ .Values.scout.tokenUrl }}
      receivers:
        filelog:
          include:
            - 'C:\var\log\pods\*\*\*.log'
          start_at: end
          operators:
            - type: container
              id: container-parser
      # ...exporters, processors and pipelines as in your agent configuration

extraEnvs behaves the same way: it is per-collector, and Helm replaces lists rather than merging them. If you supply the Scout secret through an environment variable, add the same entry under windowsDaemon.extraEnvs as well as agent-collector.extraEnvs.

What it collects​

Container logs from C:\var\log\pods and kubelet stats from the Windows kubelet.

Host metrics, Windows Event Log and performance counters are not collected yet.

Telemetry from applications running on Windows nodes does not need any of this — point your SDKs at the agent as usual. The service is named after your Helm release, so with the helm install scout ... above it is:

scout-agent-collector.scout.svc:4318

Windows Server version​

The default image targets Windows Server 2022. A Windows container image must match the host build, so on Server 2019 set the tag explicitly:

values.yaml
scout:
  windowsDaemon:
    enabled: true
    image:
      tag: "0.130.1-windows-2019-amd64"

A cluster with a mix of 2019 and 2022 nodes needs one DaemonSet per build, since a single pod template carries a single image. Contact support and we will help you configure it.

Other options​

values.yaml
scout:
  windowsDaemon:
    enabled: true
    # Windows node pools are often tainted; tolerate yours here.
    tolerations: []
    # Where the kubelet writes container logs, if your nodes differ.
    hostLogPath: 'C:\var\log\pods'
    # The collector runs as ContainerUser, which is not an administrator. If log
    # collection fails with permission errors, set ContainerAdministrator.
    runAsUserName: ""
    resources:
      requests:
        memory: 128Mi
        cpu: 100m
      limits:
        memory: 512Mi
        cpu: 400m

Configuration Guide​

Using Otelcol style configuration​

Following is an example of a values.yaml file that can be used to configure scout collector using otelcol style configuration. Here the configuration follows the same semantics as the Scout Collector config. This gives a greater flexibility in terms of what you can configure to be scraped, collected etc. Reference the otel-collector-config for more details.

values.yaml
scout:
  endpoint: __YOUR_ENDPOINT__
  tokenUrl: __YOUR_TOKEN_URL__
  appName: __YOUR_APP_NAME__
  clientId: __YOUR_CLIENT_ID__
  clientSecret: __YOUR_CLIENT_SECRET__
  environment: <environment>
  clusterName: <cluster-name>
  distribution: microk8s


  daemon:
    enabled: false
  agent:
    enabled: true
    config: |
      receivers:
        otlp:
          protocols:
            grpc:
              endpoint: ${env:MY_POD_IP}:4317
            http:
              endpoint: ${env:MY_POD_IP}:4318

        k8s_cluster:
          auth_type: serviceAccount
          collection_interval: 60s
          node_conditions_to_report: [
            ready,
            memorypressure,
            diskpressure,
            pidpressure,
            networkunavailable ]
          resource_attributes:
            k8s.container.status.last_terminated_reason:
              enabled: true
          metrics:
            k8s.pod.status_reason:
              enabled: true
            k8s.node.condition:
              enabled: true
            # Requested and limit values, so used-vs-requested comparisons
            # work. Pinned because a receiver default is not a contract.
            k8s.container.cpu_request:
              enabled: true
            k8s.container.memory_request:
              enabled: true
            k8s.container.cpu_limit:
              enabled: true
            k8s.container.memory_limit:
              enabled: true
          # `pods` is the capacity denominator for per-node pod counts.
          allocatable_types_to_report: [
            cpu,
            memory,
            ephemeral-storage,
            storage,
            pods ]
        k8sobjects:
          objects:
            # Events use `watch`, which emits each event once as it happens.
            # `pull` re-ships every event every interval and multiplies rows
            # downstream without adding information.
            - name: events
              mode: watch
              exclude_watch_type: [DELETED]
              group: events.k8s.io
            # Everything below uses `mode: pull`: a watch only fires when an
            # object changes, so an object that is stable would be collected
            # once at startup and then never again.
            #
            # ResourceQuotas and PersistentVolumeClaims are core resources and
            # take no `group:`. Workload kinds live in the `apps` group.
            - name: resourcequotas
              mode: pull
              interval: 60s
            - name: deployments
              mode: pull
              interval: 60s
              group: apps
            # The objects below back the k8X app's node, workload and storage
            # views. They are the bulk of the object-snapshot volume, so drop
            # any you do not need if you are not using k8X.
            - name: pods
              mode: pull
              interval: 60s
            - name: nodes
              mode: pull
              interval: 60s
            - name: persistentvolumeclaims
              mode: pull
              interval: 60s
            - name: replicasets
              mode: pull
              interval: 60s
              group: apps
            - name: daemonsets
              mode: pull
              interval: 60s
              group: apps
            - name: statefulsets
              mode: pull
              interval: 60s
              group: apps

      processors:
        batch:
          timeout: 2s
          send_batch_size: 8192
          send_batch_max_size: 10000
        memory_limiter:
          check_interval: 5s
          limit_percentage: 80
          spike_limit_percentage: 30
        resource:
          attributes:
            - key: service.name
              value: {{ .Values.scout.appName }}
              action: upsert
        resource/k8s:
          attributes:
            - key: service.name
              value: k8s
              action: upsert
        resource/k8s-events:
          attributes:
            - key: service.name
              value: k8s-events
              action: upsert
        resource/env:
          attributes:
            - key: environment
              value: {{ .Values.scout.environment }}
              action: upsert
            - key: k8s.cluster.name
              value: {{ .Values.scout.clusterName }}
              action: upsert
        resourcedetection/eks:
          detectors: [env, eks]
          override: true
          eks:
            resource_attributes:
              k8s.cluster.name:
                enabled: true
        k8sattributes:
          auth_type: 'serviceAccount'
          extract:
            metadata:
              - k8s.namespace.name
              - k8s.pod.name
              - k8s.pod.hostname
              - k8s.pod.ip
              - k8s.pod.start_time
              - k8s.pod.uid
              - k8s.replicaset.uid
              - k8s.replicaset.name
              - k8s.deployment.uid
              - k8s.deployment.name
              - k8s.daemonset.uid
              - k8s.daemonset.name
              - k8s.statefulset.uid
              - k8s.statefulset.name
              - k8s.cronjob.name
              - k8s.job.uid
              - k8s.job.name
              - k8s.node.name
              - k8s.cluster.uid
              - container.image.name
              - container.image.tag
              - container.id
            annotations:
              - tag_name: service.name
                key: resource.opentelemetry.io/service.name
                from: pod
              - tag_name: service.namespace
                key: resource.opentelemetry.io/service.namespace
                from: pod
              - tag_name: service.version
                key: resource.opentelemetry.io/service.version
                from: pod
              - tag_name: service.instance.id
                key: resource.opentelemetry.io/service.instance.id
                from: pod
            labels:
              - tag_name: kube_app_name
                key: app.kubernetes.io/name
                from: pod
              - tag_name: kube_app_instance
                key: app.kubernetes.io/instance
                from: pod
              - tag_name: kube_app_version
                key: app.kubernetes.io/version
                from: pod
              - tag_name: kube_app_component
                key: app.kubernetes.io/component
                from: pod
              - tag_name: kube_app_part_of
                key: app.kubernetes.io/part-of
                from: pod
              - tag_name: kube_app_managed_by
                key: app.kubernetes.io/managed-by
                from: pod
          pod_association:
            - sources:
                - from: resource_attribute
                  name: k8s.pod.ip
            - sources:
                - from: resource_attribute
                  name: k8s.pod.uid
            - sources:
                - from: connection


      exporters:
        otlphttp/base14:
          endpoint: {{ .Values.scout.endpoint }}
          auth:
            authenticator: oauth2client
          tls:
            insecure_skip_verify: true

      extensions:
        health_check:
        pprof:
        zpages:
        oauth2client:
          client_id: {{ .Values.scout.clientId }}
          client_secret: {{ .Values.scout.clientSecret }}
          endpoint_params:
            audience: b14collector
          token_url: {{ .Values.scout.tokenUrl }}
          tls:
            insecure_skip_verify: true

      service:
        extensions: [health_check, pprof, zpages, oauth2client]
        pipelines:
          traces:
            receivers: [otlp]
            processors: [memory_limiter, batch]
            exporters: [otlphttp/base14]
          logs:
            receivers: [otlp]
            processors: [memory_limiter, batch]
            exporters: [otlphttp/base14]
          logs/k8s-events:
            receivers: [ k8sobjects]
            processors: [
              memory_limiter,
              resource/k8s-events,
              resourcedetection/eks,
              resource/env,
              batch ]
            exporters: [ otlphttp/base14 ]
          logs/k8s-cluster:
            receivers: [ k8s_cluster ]
            processors: [
              memory_limiter,
              resource/k8s,
              resourcedetection/eks,
              resource/env,
              batch ]
            exporters: [ otlphttp/base14 ]
          metrics:
            receivers: [ otlp ]
            processors: [ memory_limiter, resource/env, batch ]
            exporters: [ otlphttp/base14 ]
          metrics/k8s:
            receivers: [ k8s_cluster ]
            processors: [
              memory_limiter,
              resource/k8s,
              resourcedetection/eks,
              resource/env,
              k8sattributes,
              batch ]
            exporters: [ otlphttp/base14 ]
        telemetry:
          logs:
            level: warn
            encoding: json

Scout helm chart uses the above configuration to configure the Scout Collector​

  1. Collects logs for the current cluster(for Managed nodes only).
  2. Sends k8s events data.
  3. Sends node and pods metrics data.
  4. Sends apps metrics data for the configured app endpoints.
  5. Sets up a local otlp endpoint for apps to send traces which are then forwarded to Scout.

Once deployed, your cluster telemetry flows into base14 Scout - unified logs, metrics, and traces from every node and pod in a single platform.

Learn More​

Was this page helpful?