Kubernetes ingress-nginx Migration to Gateway API: Step-by-Step Guide and Rollback Plan

Kubernetes networking has a vocabulary problem. I once heard “move to Gateway API” and assumed it referred to an external API service. It actually means replacing Kubernetes Ingress resources and their controller-specific behavior with Gateway API resources managed by a Gateway controller.

Gateway API is maintained by the Kubernetes SIG Network community. A GatewayClass selects a controller, a Gateway defines listeners and TLS, and an HTTPRoute defines host and path routing. Controller implementations determine support for optional features and policies.

The steps use namespace apps, Service web, and hostname canary.example.com; substitute your own values.

1. Inventory the current Ingress behavior

Start by locating all Kubernetes Ingress objects and noting their class, hosts, paths, TLS secrets, and annotations:

kubectl get ingress -A
kubectl get ingress -A -o yaml > ingress-inventory.yaml

Keep source manifests or Helm values in version control for restoration. Inspect the ingress-nginx ConfigMap and controller arguments too, since global settings may affect traffic without an Ingress annotation.

Map each behavior before translating it:

ingress-nginx setting Gateway API direction Verify during migration
ingressClassName GatewayClass and HTTPRoute.parentRefs Controller ownership and route attachment
spec.tls HTTPS listener and certificateRefs on Gateway Secret namespace and certificate renewal
rewrite-target URLRewrite filter on HTTPRoute Prefix, full path, and trailing slash behavior
ssl-redirect RequestRedirect filter or controller policy Status code and TLS termination location
proxy-read-timeout Route timeout field or implementation policy Exact timeout semantics and feature support
proxy-body-size Controller-specific policy Maximum request size and rejection behavior
whitelist-source-range Controller authorization or network policy Client IP preservation through proxies
auth-url Controller-specific external authorization policy Headers, failure mode, and policy scope

Check the chosen controller's conformance and feature documentation for every row. Annotation names can look similar while their behavior differs. Record unsupported features and decide how to handle them before moving production traffic.

2. Install and verify Gateway API

Choose a Gateway controller that supports your Kubernetes version, infrastructure, and required features. CRDs define Gateway API resources; the controller configures the data plane. Follow its install guide and pin the controller version and configuration.

Some controller installation methods install the CRDs automatically. If yours requires separate installation, use a Gateway API release supported by that controller. For example, set the version to the documented compatible release:

export GATEWAY_API_VERSION=v1.x.y
kubectl apply -f \
  "https://github.com/kubernetes-sigs/gateway-api/releases/download/${GATEWAY_API_VERSION}/standard-install.yaml"

Replace v1.x.y with the actual release before running the command. Then confirm the API types are registered:

kubectl get crd gateways.gateway.networking.k8s.io
kubectl get crd httproutes.gateway.networking.k8s.io
kubectl get gatewayclass

The controller's install guide supplies its class name and controllerName. Define that class using those exact values:

apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: your-controller-class
spec:
  controllerName: controller.example.net/gateway-controller

Replace both example identifiers with the values for your installed controller, then apply the class:

kubectl apply -f gateway-class.yaml

3. Create a Gateway for a canary hostname

Keep ingress-nginx running. Create the new Gateway on a separate address or hostname first. This example assumes the TLS Secret canary-example-com already exists in the apps namespace:

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: public-gateway
  namespace: apps
spec:
  gatewayClassName: your-controller-class
  listeners:
    - name: https
      hostname: canary.example.com
      port: 443
      protocol: HTTPS
      tls:
        mode: Terminate
        certificateRefs:
          - kind: Secret
            name: canary-example-com

Replace your-controller-class with the class provided by your controller. Listener and certificate reference support can have implementation-specific requirements; check its documentation if the Gateway reports a rejected listener.

4. Attach an HTTPRoute to the application Service

The route below sends requests for the canary host to port 8080 of the web Service. Adjust the Service name and port to match the application:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: web-canary
  namespace: apps
spec:
  parentRefs:
    - name: public-gateway
      sectionName: https
  hostnames:
    - canary.example.com
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /
      backendRefs:
        - name: web
          port: 8080

Apply both manifests and inspect their conditions:

kubectl apply -f gateway.yaml
kubectl apply -f web-canary-route.yaml
kubectl describe gateway public-gateway -n apps
kubectl describe httproute web-canary -n apps

Look for Accepted=True and Programmed=True on the Gateway, and Accepted=True plus ResolvedRefs=True on the route's parent status. Resolve the Gateway's address with kubectl get gateway public-gateway -n apps -o yaml; the controller or your infrastructure may take time to assign it.

5. Test the new path, then shift traffic

Before changing DNS, test TLS and routing directly against the Gateway address:

curl -i --resolve canary.example.com:443:203.0.113.20 \
  https://canary.example.com/health

Replace the reserved example IP with the Gateway address. Exercise important paths, redirects, authentication, large uploads, and unhealthy-backend behavior. Compare application logs, response codes, latency, and controller metrics with the ingress-nginx path.

Choose one low-risk service for the first real cutover, such as an internal status page. Send it to the Gateway using a canary hostname, a small DNS change, or a load balancer's supported traffic-shifting feature. Keep other services on ingress-nginx. Once the canary is stable across representative traffic and normal operating conditions, move services in small groups and repeat the same checks.

6. Keep rollback ready

Before each traffic change, record the old DNS or load-balancer target, know who can restore it, and define rollback signals. Examples include a rise in 5xx responses, TLS errors, broken redirects, unexpected backend selection, or authorization failures.

When a signal fires, restore traffic to the recorded ingress-nginx target. Keep the old controller, Ingress resources, certificates, and configuration available while you verify recovery. DNS changes can remain cached until their TTL expires, so monitor both paths during that window. Remove the old ingress-nginx setup only after every service has passed its observation period and the rollback route is no longer needed.

The first cutover should teach you how your chosen controller behaves in your cluster. Use that evidence to refine the mapping and rollout steps before migrating the next Kubernetes service.

Comments