GKE Autopilot clusters are fully managed Kubernetes environments where Google controls the node infrastructure, enforces security best practices, and optimizes resource allocation. One consequence of this managed model is that Autopilot clusters use private control planes by default, restricted by masterAuthorizedNetworksConfig to a specific set of CIDR ranges. This creates a challenge for CI/CD pipelines running on external platforms like GitHub Actions: the pipeline’s ephemeral runner IP addresses are not in the authorized CIDR list and change with every run, making direct kubectl access to the cluster impossible.
Google Cloud Connect Gateway solves this by providing an authenticated, Google-managed proxy to the Kubernetes API server. Instead of connecting directly to the cluster’s private endpoint, the CI/CD pipeline authenticates with a GCP service account, sends its Kubernetes API requests through the Connect Gateway, and the gateway forwards them to the cluster’s control plane via Google’s internal network. The pipeline never needs direct network access to the control plane, and the authorized CIDR restrictions are bypassed because the traffic originates from Google’s infrastructure.
In the deploy platform’s infrastructure-as-code project, Pulumi provisions the full Connect Gateway stack: enabling the required GCP APIs (connectgateway, gkeconnect, gkehub), registering the Autopilot cluster as a Fleet membership, and granting the GitHub Actions deploy service account the roles/gkehub.gatewayEditor role. This topic covers the architecture, the Pulumi implementation, and the patterns for integrating Connect Gateway into CI/CD pipelines.
A Fleet is a Google Cloud concept for grouping Kubernetes clusters under a single management plane. Registering a cluster as a Fleet membership enables fleet-level features like Connect Gateway, Config Sync, and Policy Controller. The membership creates a bidirectional connection between the cluster and Google’s fleet management infrastructure, allowing Google services to interact with the cluster’s API server without direct network access. Each membership is identified by a membership name and associated with a specific GKE cluster in a specific project and location.
Connect Gateway is a Google-managed reverse proxy that authenticates requests using GCP IAM, translates them into Kubernetes API calls, and forwards them to the registered cluster through the Fleet membership’s agent. The gateway endpoint follows the pattern https://connectgateway.googleapis.com/v1/projects/{project}/locations/{location}/memberships/{name}. Clients use this as the Kubernetes API server URL in their kubeconfig, combined with a GCP auth plugin that obtains access tokens from the service account.
Access to the Connect Gateway is controlled by GCP IAM roles, not Kubernetes RBAC (though both apply). roles/gkehub.gatewayEditor allows the principal to send any Kubernetes API request through the gateway (read and write). roles/gkehub.gatewayReader restricts to read-only Kubernetes API calls. The CI/CD service account needs gatewayEditor because deployments require creating, updating, and deleting Kubernetes resources. These IAM roles control gateway access; Kubernetes RBAC within the cluster provides the second layer of authorization.
Rather than distributing a long-lived GCP service account key to GitHub Actions, the deploy platform uses Workload Identity Federation (WIF). GitHub’s OIDC provider is registered as an identity provider in GCP, and a service account pool maps GitHub repository identities to a GCP service account. When a GitHub Actions workflow runs, it obtains a short-lived OIDC token from GitHub, exchanges it for a GCP access token via the Security Token Service, and uses that token to authenticate with the Connect Gateway. No static credentials are stored in GitHub secrets.
GKE’s master authorized networks feature restricts which IP addresses can reach the cluster’s Kubernetes API server. For Autopilot clusters, this defaults to allowing only the cluster’s own VPC and a set of Google-managed CIDRs. External IPs (like GitHub Actions runners) are blocked. Connect Gateway sidesteps this restriction entirely because the gateway-to-cluster traffic traverses Google’s internal network and arrives from an authorized source, regardless of where the original client request originated.
The Connect Gateway setup involves three layers: API enablement (turning on the required GCP services), Fleet registration (creating the membership that links the cluster to the gateway), and IAM configuration (granting the CI/CD service account access to the gateway). The deploy platform provisions all three layers using Pulumi, ensuring the entire stack is reproducible and version-controlled.
Three APIs must be enabled before Connect Gateway can function: connectgateway.googleapis.com for the gateway itself, gkeconnect.googleapis.com for the connect agent that runs in the cluster, and gkehub.googleapis.com for Fleet management.
// infra/connect-gateway.ts
import * as gcp from "@pulumi/gcp";
const requiredAPIs = [
"connectgateway.googleapis.com",
"gkeconnect.googleapis.com",
"gkehub.googleapis.com",
];
const apiServices = requiredAPIs.map(api =>
new gcp.projects.Service(`api-${api}`, {
project: projectId,
service: api,
disableOnDestroy: false,
})
);
// All subsequent resources depend on API enablement.
const apiDependency = apiServices.map(s => s.id);The Fleet membership links the Autopilot cluster to the Connect Gateway. The membership references the cluster by its resource link and specifies the location. Once registered, the connect agent in the cluster establishes a persistent connection to Google’s fleet infrastructure.
// infra/connect-gateway.ts (continued)
const membership = new gcp.gkehub.Membership("autopilot-membership", {
project: projectId,
location: region,
membershipId: `${clusterName}-membership`,
endpoint: {
gkeCluster: {
resourceLink: pulumi.interpolate`//container.googleapis.com/projects/${projectId}/locations/${region}/clusters/${clusterName}`,
},
},
}, { dependsOn: apiServices });
// Export the membership name for kubeconfig generation.
export const membershipName = membership.membershipId;The GitHub Actions deploy service account needs the roles/gkehub.gatewayEditor role at the project level to send Kubernetes API requests through the gateway. This IAM binding is scoped to the specific service account, following the principle of least privilege.
// infra/connect-gateway.ts (continued)
const deployServiceAccount = new gcp.serviceaccount.Account("gh-deploy-sa", {
project: projectId,
accountId: "gh-deploy",
displayName: "GitHub Actions Deploy Service Account",
});
// Grant Connect Gateway editor access.
const gatewayBinding = new gcp.projects.IAMMember("gateway-editor", {
project: projectId,
role: "roles/gkehub.gatewayEditor",
member: pulumi.interpolate`serviceAccount:${deployServiceAccount.email}`,
}, { dependsOn: [membership] });
// Grant container.developer for Kubernetes RBAC passthrough.
const k8sBinding = new gcp.projects.IAMMember("k8s-developer", {
project: projectId,
role: "roles/container.developer",
member: pulumi.interpolate`serviceAccount:${deployServiceAccount.email}`,
});The CI/CD pipeline authenticates with Workload Identity Federation, generates a kubeconfig pointing at the Connect Gateway endpoint, and runs standard kubectl commands to deploy to the Autopilot cluster.
# .github/workflows/deploy.yml
name: Deploy to GKE Autopilot
on:
push:
branches: [main]
permissions:
id-token: write # Required for Workload Identity Federation
contents: read
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- id: auth
uses: google-github-actions/auth@v2
with:
workload_identity_provider: projects/${{ secrets.PROJECT_NUMBER }}/locations/global/workloadIdentityPools/github-pool/providers/github-provider
service_account: gh-deploy@${{ secrets.PROJECT_ID }}.iam.gserviceaccount.com
- uses: google-github-actions/setup-gcloud@v2
# Generate kubeconfig using Connect Gateway.
- name: Configure kubectl via Connect Gateway
run: |
gcloud container fleet memberships get-credentials \
${{ env.CLUSTER_NAME }}-membership \
--project=${{ secrets.PROJECT_ID }} \
--location=${{ env.REGION }}
# Standard kubectl commands work through the gateway.
- name: Deploy application
run: |
kubectl apply -f k8s/
kubectl rollout status deployment/app -n production --timeout=300sThe deploy platform uses Connect Gateway as the sole mechanism for CI/CD access to production Autopilot clusters. This eliminates the need to maintain a list of GitHub Actions runner IP ranges in masterAuthorizedNetworksConfig — a list that changes frequently as GitHub updates its infrastructure. Before Connect Gateway, the team had to either use a self-hosted runner inside the VPC (adding operational burden) or whitelist GitHub’s entire IP range (widening the attack surface).
The Connect Gateway approach also simplifies multi-cluster deployments. The deploy platform manages clusters in multiple GCP regions, each registered as a Fleet membership. The deployment pipeline iterates over memberships and deploys to each cluster using the same kubeconfig generation command, just with different membership names. There is no need to manage per-cluster VPN tunnels, bastion hosts, or IP whitelists.
For developer access, the same pattern applies. Developers authenticate with their Google Cloud identity (which is already managed through the organization’s Google Workspace), and the Connect Gateway provides kubectl access without VPN. This is particularly valuable for incident response, where engineers need immediate cluster access from any network without waiting for VPN provisioning.
gcloud container fleet memberships get-credentials for ad-hoc cluster access without VPN.The Connect Gateway configuration lives in the infrastructure Pulumi stack, but the CI/CD pipeline needs the membership name and service account email. The deploy platform uses Pulumi stack outputs to bridge this gap: the infrastructure stack exports membershipName and deployServiceAccountEmail, and the CI/CD configuration references these outputs via GitHub repository variables that are set during infrastructure provisioning.
Connect Gateway enforces two authorization checks. First, GCP IAM verifies the caller has roles/gkehub.gatewayEditor (or gatewayReader). Second, the Kubernetes API server applies standard RBAC rules to the impersonated identity. The CI/CD service account needs both: the IAM role to pass through the gateway, and a Kubernetes ClusterRoleBinding (typically cluster-admin or a custom deploy role) to perform the actual operations. If either check fails, the request is denied.
# k8s/deploy-rbac.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: gh-deploy-binding
subjects:
- kind: User
name: gh-deploy@my-project.iam.gserviceaccount.com
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: cluster-admin
apiGroup: rbac.authorization.k8s.ioPulumi’s disableOnDestroy: false prevents API disablement when the stack is torn down, avoiding cascading failures if other resources in the project depend on the same APIs. This is a defensive pattern: enabling an API is safe and idempotent, but disabling it can break unrelated workloads.
The roles/gkehub.gatewayEditor role grants access to the Connect Gateway, but the service account also needs Kubernetes-level permissions. If the account doesn’t have roles/container.developer (or a custom role with equivalent permissions) and no in-cluster RBAC binding exists, requests through the gateway will be authenticated but unauthorized. The error message (“forbidden: User … cannot …”) can be confusing because it appears to be a Kubernetes RBAC issue when the root cause is a missing IAM role.
The Fleet membership’s location must match the cluster’s region (or be global for multi-region memberships). A mismatch causes the membership to be created but the connect agent to fail registration, resulting in a gateway that accepts connections but can’t reach the cluster. The error manifests as timeouts on kubectl commands, which is difficult to diagnose without checking the connect agent’s logs.
GCP API enablement is asynchronous — the gcp.projects.Service resource may return before the API is fully activated. Creating a Fleet membership immediately after enabling gkehub.googleapis.com can fail with a “service not enabled” error. Pulumi’s dependency system handles this via dependsOn, but if dependencies are missing, the race condition causes intermittent failures on first deployment.
Granting roles/gkehub.gatewayEditor at the project level gives the service account gateway access to every Fleet membership in the project. For production environments with multiple clusters at different sensitivity levels, use IAM conditions or membership-level bindings to restrict access to specific clusters. A CI/CD service account for the staging cluster should not have gateway access to the production cluster.
Distributing a JSON key file for the deploy service account and storing it in GitHub secrets is the legacy approach. Key files are long-lived, cannot be automatically rotated, and create a secret sprawl problem. Workload Identity Federation eliminates this by using short-lived OIDC tokens that are generated on-the-fly during each workflow run and expire within minutes. Always prefer WIF over static keys for CI/CD authentication.