How to Write a Kubernetes HPA for a Custom Metric

Why custom metrics instead of CPU

autoscaling/v1 only knew about CPU. autoscaling/v2 (the current API version) adds three metric source types: Resource (CPU/memory), Pods (per-pod metric averaged across replicas), and External (cluster-external metrics like SQS depth).

For a web service, scaling on http_requests_per_second is almost always more accurate than CPU - request rate is what matters to users, and it responds to traffic faster than CPU does.

The autoscaling/v2 HPA manifest

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web
  minReplicas: 2
  maxReplicas: 10
  metrics:
    - type: Pods
      pods:
        metric:
          name: http_requests_per_second
        target:
          type: AverageValue
          averageValue: "100"

A few key points:

• type: Pods averages the metric value across all pods matched by the HPA's scaleTargetRef. The HPA scales up when the per-pod average exceeds averageValue.
• averageValue: "100" means "keep each pod handling no more than 100 requests per second on average." Above that, the HPA adds pods; below it, the HPA removes them (down to minReplicas).
• minReplicas: 2 is a production baseline - never drop to a single replica, so a restart doesn't mean downtime.

How the metric gets into Kubernetes

The HPA reads metrics from the Kubernetes metrics API. For custom metrics like http_requests_per_second, the pipeline is:

1. Your app exposes a Prometheus-format /metrics endpoint.
2. Prometheus scrapes it.
3. The Prometheus adapter (k8s-prometheus-adapter) translates Prometheus queries into the custom.metrics.k8s.io API that the HPA reads.

The HPA manifest is only half the story - the adapter needs a rule that maps the metric name to a PromQL query:

# prometheus-adapter ConfigMap (rules section)
rules:
  - seriesQuery: 'http_requests_total{namespace!="",pod!=""}'
    resources:
      overrides:
        namespace: {resource: "namespace"}
        pod: {resource: "pod"}
    name:
      matches: "^http_requests_total$"
      as: "http_requests_per_second"
    metricsQuery: 'rate(http_requests_total{<<.LabelMatchers>>}[2m])'

Once the adapter is running, kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1 lists the available custom metrics and you can verify the HPA picks them up.

Apply and inspect

kubectl apply -f hpa.yaml
kubectl get hpa web-hpa
kubectl describe hpa web-hpa

kubectl describe hpa shows the current metric value, the target, and why the HPA is or is not scaling. TARGETS will show <unknown>/100 until the adapter is running and the metric is flowing.

KEDA: the modern alternative

KEDA (Kubernetes Event-Driven Autoscaling) is the preferred autoscaler for external metrics in newer clusters - it can scale on SQS depth, Kafka lag, Datadog dashboards, or any Prometheus query without a separate adapter. The mental model is the same: declarative manifest, named metric, target value. Worth knowing both.