TLS termination for TCPRoutes

Set up a TLS listener on the Gateway that terminates incoming TLS traffic and routes it using a TCPRoute. Unlike TLS termination for TLSRoutes, which uses SNI-based hostname matching, the TCPRoute does not perform hostname matching and routes traffic based on the listener port only.

Before you begin

  1. Set up kgateway by following the Quick start or Installation guides.

  2. Deploy the httpbin sample app.

  3. Make sure that you have the OpenSSL version of openssl, not LibreSSL. The openssl version must be at least 1.1.

    1. Check the openssl version that is installed. If you see LibreSSL in the output, continue to the next step.

      openssl version

    2. Install the OpenSSL version (not LibreSSL). For example, you might use Homebrew.

      brew install openssl

    3. Review the output of the OpenSSL installation for the path of the binary file. You can choose to export the binary to your path, or call the entire path whenever the following steps use an openssl command.

      • For example, openssl might be installed along the following path: /usr/local/opt/openssl@3/bin/
      • To run commands, you can append the path so that your terminal uses this installed version of OpenSSL, and not the default LibreSSL. /usr/local/opt/openssl@3/bin/openssl req -new -newkey rsa:4096 -x509 -sha256 -days 3650...
  4. Decide whether to set up a listener inline on the Gateway resource or as a separate ListenerSet resource. For more information, see the Listener overview.

Install the experimental channel of the Kubernetes Gateway API so that you can use TCPRoutes.

kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.5.1/experimental-install.yaml

Create a TLS certificate

  1. Create a directory to store your TLS credentials in.

    mkdir example_certs

  2. Create a self-signed root certificate.

    openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 \
  -subj '/O=any domain/CN=*' \
  -keyout example_certs/root.key -out example_certs/root.crt

  3. Create an OpenSSL configuration for the app.example.com hostname.

    cat <<'EOF' > example_certs/gateway.cnf
[ req ]
default_bits = 2048
prompt = no
default_md = sha256
distinguished_name = dn
req_extensions = req_ext
[ dn ]
CN = *.example.com
O = any domain
[ req_ext ]
subjectAltName = @alt_names
[ alt_names ]
DNS.1 = *.example.com
DNS.2 = example.com
EOF

  4. Create and sign the gateway certificate.

    openssl req -new -nodes \
  -keyout example_certs/gateway.key \
  -out example_certs/gateway.csr \
  -config example_certs/gateway.cnf
openssl x509 -req -sha256 -days 365 \
  -CA example_certs/root.crt -CAkey example_certs/root.key -set_serial 0 \
  -in example_certs/gateway.csr -out example_certs/gateway.crt \
  -extfile example_certs/gateway.cnf -extensions req_ext

  5. Create a Kubernetes secret to store the gateway TLS certificate.

    kubectl create secret tls tls-terminate-tcp \
  -n kgateway-system \
  --key example_certs/gateway.key \
  --cert example_certs/gateway.crt

Set up TLS termination

Set up a TLS listener on the Gateway with tls.mode: Terminate. The Gateway decrypts incoming TLS traffic using the certificate you created and forwards the plain traffic to the backend service. Unlike TLS termination with TLSRoutes, no hostname is required on the listener because TCPRoutes route based on port only.

  1. Create a Gateway with a TLS termination listener. To use the default Envoy-based kgateway proxy, set the gatewayClassName to kgateway. To use agentgateway, set the gatewayClassName to agentgateway.

    kubectl apply -f- <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: tls-terminate-tcp
  namespace: kgateway-system
  labels:
    example: tls-terminate-tcp
spec:
  gatewayClassName: kgateway
  listeners:
  - name: tls
    protocol: TLS
    port: 8443
    tls:
      mode: Terminate
      certificateRefs:
        - name: tls-terminate-tcp
          kind: Secret
    allowedRoutes:
      kinds:
        - kind: TCPRoute
      namespaces:
        from: All
EOF

    Review the following table to understand this configuration.

    Setting Description
    spec.gatewayClassName The name of the Kubernetes GatewayClass that you want to use to configure the Gateway. When you set up kgateway, a default GatewayClass is set up for you. To use the default Envoy-based kgateway proxy, set the gatewayClassName to kgateway. To use agentgateway, set the gatewayClassName to agentgateway.
    spec.listeners Configure the listeners for this Gateway. In this example, you configure a TLS listener that terminates incoming TLS traffic on port 8443. No hostname is set because TCPRoutes do not perform SNI-based hostname matching.
    spec.listeners.tls.mode The TLS mode for incoming requests. In this example, TLS requests are terminated at the Gateway and the decrypted traffic is forwarded to the backend service.
    spec.listeners.tls.certificateRefs The Kubernetes secret that holds the TLS certificate and key. The Gateway uses these to terminate the TLS connection.
    spec.listeners.allowedRoutes.kinds Restricts the listener to accept only TCPRoute resources.

    1. Create a Gateway that enables the attachment of ListenerSets.

      kubectl apply -f- <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: tls-terminate-tcp
  namespace: kgateway-system
  labels:
    example: tls-terminate-tcp
spec:
  gatewayClassName: kgateway
  allowedListeners:
    namespaces:
      from: All
  listeners:
  - protocol: TLS
    port: 80
    name: generic-tls
    tls:
      mode: Terminate
      certificateRefs:
        - name: tls-terminate-tcp
          kind: Secret
    allowedRoutes:
      kinds:
        - kind: TCPRoute
      namespaces:
        from: All
EOF

      Review the following table to understand this configuration.

      Setting Description
      spec.gatewayClassName The name of the Kubernetes GatewayClass that you want to use to configure the Gateway. When you set up kgateway, a default GatewayClass is set up for you. To use the default Envoy-based kgateway proxy, set the gatewayClassName to kgateway. To use agentgateway, set the gatewayClassName to agentgateway.
      spec.allowedListeners Enable the attachment of ListenerSets to this Gateway. The example allows listeners from any namespace.
      spec.listeners Optionally, you can configure a listener that is specific to the Gateway. Note that due to a Gateway API limitation, you must configure at least one listener on the Gateway resource, even if the listener is not used and is a generic, “dummy” listener. This generic listener cannot conflict with the listener that you configure in the ListenerSet, such as using the same port or name. In this example, the generic listener is configured on port 80, which differs from port 8443 in the ListenerSet that you create later.

    2. Create a ListenerSet that configures a TLS termination listener for the Gateway.

      kubectl apply -f- <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: ListenerSet
metadata:
  name: my-tls-terminate-tcp-listenerset
  namespace: kgateway-system
  labels:
    example: tls-terminate-tcp
spec:
  parentRef:
    name: tls-terminate-tcp
    namespace: kgateway-system
    kind: Gateway
    group: gateway.networking.k8s.io
  listeners:
  - protocol: TLS
    port: 8443
    name: tls-listener-set
    tls:
      mode: Terminate
      certificateRefs:
        - name: tls-terminate-tcp
          kind: Secret
    allowedRoutes:
      kinds:
        - kind: TCPRoute
      namespaces:
        from: All
EOF

      Review the following table to understand this configuration.

      Setting Description
      spec.parentRef The name of the Gateway to attach the ListenerSet to.
      spec.listeners Configure the listeners for this ListenerSet. In this example, you configure a TLS termination listener on port 8443. No hostname is set because TCPRoutes do not perform SNI-based hostname matching.
      spec.listeners.tls.mode The TLS mode for incoming requests. TLS requests are terminated at the Gateway and the decrypted traffic is forwarded to the backend service.
      spec.listeners.tls.certificateRefs The Kubernetes secret that holds the TLS certificate and key for TLS termination.
      spec.listeners.allowedRoutes.kinds Restricts the listener to accept only TCPRoute resources.

  2. Check the status of the Gateway to make sure that your configuration is accepted.

    kubectl get gateway tls-terminate-tcp -n kgateway-system -o yaml

Create a TCPRoute

Create a TCPRoute that routes traffic on port 8443 to the httpbin app.

kubectl apply -f- <<EOF
apiVersion: gateway.networking.k8s.io/v1alpha2
kind: TCPRoute
metadata:
  name: tls-terminate-tcp-route
  namespace: httpbin
  labels:
    example: tls-terminate-tcp
spec:
  parentRefs:
    - name: tls-terminate-tcp
      namespace: kgateway-system
      sectionName: tls
  rules:
    - backendRefs:
        - name: httpbin
          port: 8000
EOF
kubectl apply -f- <<EOF
apiVersion: gateway.networking.k8s.io/v1alpha2
kind: TCPRoute
metadata:
  name: tls-terminate-tcp-route
  namespace: httpbin
  labels:
    example: tls-terminate-tcp
spec:
  parentRefs:
    - name: my-tls-terminate-tcp-listenerset
      namespace: kgateway-system
      kind: ListenerSet
      group: gateway.networking.k8s.io
      sectionName: tls-listener-set
  rules:
    - backendRefs:
        - name: httpbin
          port: 8000
EOF

Verify TLS termination traffic

  1. Get the external address of the gateway and save it in an environment variable.

    export INGRESS_GW_ADDRESS=$(kubectl get svc -n kgateway-system tls-terminate-tcp -o jsonpath="{.status.loadBalancer.ingress[0]['hostname','ip']}")
echo $INGRESS_GW_ADDRESS

  2. Send a request to the app.example.com domain and verify that you get back a 200 HTTP response code. The TLS connection is terminated at the Gateway and the plain traffic is forwarded to the httpbin app.

    • Load balancer IP:

      curl -vik --resolve "app.example.com:8443:${INGRESS_GW_ADDRESS}" \
  --cacert example_certs/root.crt \
  https://app.example.com:8443/status/200

    • Load balancer hostname:

      curl -vik --resolve "app.example.com:8443:$(dig +short $INGRESS_GW_ADDRESS | head -n1)" \
  --cacert example_certs/root.crt \
  https://app.example.com:8443/status/200

    Example output:

    * Request completely sent off
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
...

  1. Port-forward the gateway service to your local machine.

    kubectl port-forward svc/tls-terminate-tcp -n kgateway-system 8443:8443

  2. Send a request to the app.example.com domain and verify that you get back a 200 HTTP response code.

    curl -vik --connect-to app.example.com:8443:localhost:8443 \
  --cacert example_certs/root.crt \
  https://app.example.com:8443/status/200

    Example output:

    * Request completely sent off
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
...

Cleanup

You can remove the resources that you created in this guide. 
kubectl delete -A gateways,tcproutes,secret -l example=tls-terminate-tcp
rm -rf example_certs
kubectl delete -A gateways,tcproutes,listenersets,secret -l example=tls-terminate-tcp
rm -rf example_certs
TLS passthroughTLS termination for TLSRoutes