Horizontal Pod Autoscaling (HPA)

You can bring your own Horizontal Pod Autoscaler (HPA) plug-in to kgateway. This way, you can automatically scale gateway proxy pods up and down based on certain thresholds, like memory and CPU consumption.

To allow integration with HPA plug-ins, do not specify any custom replicas in the GatewayParameters resource. This way, your HPA plug-in can scale the proxy based on your autoscaling strategy.

Before you begin

1. Follow the Get started guide to install kgateway.

2. Follow the Sample app guide to create a gateway proxy with an HTTP listener and deploy the httpbin sample app.

3. Get the external address of the gateway and save it in an environment variable.

   Cloud Provider LoadBalancerPort-forward for local testing

   export INGRESS_GW_ADDRESS=$(kubectl get svc -n kgateway-system http -o jsonpath="{.status.loadBalancer.ingress[0]['hostname','ip']}")
   echo $INGRESS_GW_ADDRESS  

   kubectl port-forward deployment/http -n kgateway-system 8080:8080

Set up your own HPA plug-in

1. Deploy the Kubernetes metrics-server in your cluster. The metrics-server retrieves metrics, such as CPU and memory consumption for your workloads. These metrics can be used by the HPA plug-in to determine if the pod must be scaled up or down.

   kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
   kubectl -n kube-system patch deployment metrics-server \
    --type=json \
    -p='[{"op":"add","path":"/spec/template/spec/containers/0/args/-","value":"--kubelet-insecure-tls"}]'

   Example output:

   serviceaccount/metrics-server created
   clusterrole.rbac.authorization.k8s.io/system:aggregated-metrics-reader created
   clusterrole.rbac.authorization.k8s.io/system:metrics-server created
   rolebinding.rbac.authorization.k8s.io/metrics-server-auth-reader created
   clusterrolebinding.rbac.authorization.k8s.io/metrics-server:system:auth-delegator created
   clusterrolebinding.rbac.authorization.k8s.io/system:metrics-server created
   service/metrics-server created
   deployment.apps/metrics-server configured
   apiservice.apiregistration.k8s.io/v1beta1.metrics.k8s.io created
   

2. Review the metrics for the http Gateway.

   kubectl top pod -n kgateway-system  | grep http

   Example output:

   http-594455765c-5szpq        3m           19Mi  

3. Create a HorizontalPodAutoscaler resource that scales your gateway proxy up to 10 replicas if the memory consumption exceeds 10Mi. You can adjust this value depending on the memory value that you retrieved in the previous step.

   kubectl apply -f- <<EOF
   apiVersion: autoscaling/v2
   kind: HorizontalPodAutoscaler
   metadata:
     name: hpa
     namespace: kgateway-system
   spec:
     scaleTargetRef:
       apiVersion: apps/v1
       kind: Deployment
       name: http
     minReplicas: 1
     maxReplicas: 10
     metrics:
       - type: Resource
         resource:
           name: memory
           target:
             type: AverageValue
             averageValue: 10Mi
   EOF

4. Wait a few minutes for Kubernetes to scale up your gateway proxies. Then, check the number of http pods that were created. Because every pod exceeds the memory threshold that you defined in the HPA policy, Kubernetes scales up the pods to the 10 maximum replicas.

   kubectl top pod -n kgateway-system  | grep http

   Example output:

   http-594455765c-5szpq                   3m           19Mi            
   http-594455765c-gvf9g                   3m           19Mi            
   http-594455765c-h2zb9                   3m           19Mi            
   http-594455765c-mghsf                   3m           19Mi            
   http-594455765c-n6prj                   3m           19Mi            
   http-594455765c-nvs6c                   3m           19Mi            
   http-594455765c-srfcz                   3m           19Mi            
   http-594455765c-ssvjx                   3m           19Mi            
   http-594455765c-tx2z6                   3m           19Mi            
   http-594455765c-vxjbp                   3m           19Mi 

Cleanup

kubectl delete hpa hpa -n kgateway-system
kubectl delete -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml