Skip to main content

Cluster access and interconnection

To reach the Kubernetes API and your resources, Bunker provides a managed, ready-to-use access: the Kubernetes access (kube-access). If your needs go further, interconnection is possible.

What the managed Kubernetes access provides​

The Kubernetes access is included: a private Headscale network dedicated to your organization, run entirely by Bunker.

  • API access through Capsule Proxy: this network connects you to Capsule Proxy, the multi-tenant entry point of the Kubernetes API. Your kubeconfig targets it at kube.france-nuage.fr:31443 (or kube-dc05.france-nuage.fr:31443), reachable once your workstation is connected with the standard Tailscale client.
  • A view limited to your hosted-* namespaces: through Capsule Proxy, you only see your hosted-* namespaces and what they contain, never the rest of the cluster or other customers. You work with your resources there (logs, exec, port-forward to your databases and services) within your account's permissions.
  • Operations: the Headscale server, routers, access rules, updates, monitoring and backups are on us.

Its limits are deliberate: Bunker sets its access rules, and it only publishes the API. It does not connect your own machines and does not carry other networks.

When your needs go further​

The building blocks available to you:

  • A dedicated Headscale: your own private network, whose access rules you edit in Headplane. On request, Bunker publishes access to a managed service on it, a database for instance.
  • A router of your own in a hosted-* namespace: a subnet router pod that publishes on your network what your namespaces can reach, including the Kubernetes API. The example at the end of this page details it.

Who does what​

ItemBunkerYou
Managed Kubernetes accessProvides, runs, sets the access rulesUse it
Dedicated Headscale serverRuns, updates, backs upEnroll your machines
Access rules of the dedicated HeadscaleProvides the editor (Headplane), sets the rules for the routes we publishWrite and maintain the rest
Router in a hosted-* namespace—Deploy, monitor, update
Outbound flows from your namespacesOpens the Kubernetes API (port 31443), opens other flows on requestAsk for what you need
Permissions on the clusterDefines them, nothing bypasses them—

Example: publishing the Kubernetes API on your dedicated Headscale​

1. Create the user and its key​

In Headplane:

  1. Users: create a kube-router user.
  2. Settings, Pre-Auth Keys: create a key for kube-router, with Reusable and Ephemeral checked and a long expiration. Copy it: it is shown only once.

2. Allow and approve the routes​

The API answers on the three nodes of its control plane: dig +short kube.france-nuage.fr returns their addresses, currently 10.0.1.11, 10.1.1.11 and 10.2.1.11. In Headplane, under Access Control, Edit file tab, add this rule and these autoApprovers lines, keeping your existing rules:

{
  "acls": [
    { "action": "accept", "src": ["*"], "dst": ["10.0.1.11:31443", "10.1.1.11:31443", "10.2.1.11:31443"] }
  ],
  "autoApprovers": {
    "routes": {
      "10.0.1.11/32": ["kube-router@"],
      "10.1.1.11/32": ["kube-router@"],
      "10.2.1.11/32": ["kube-router@"]
    }
  }
}

A rule allows a flow, it does not create a path: the router is needed too.

3. Store the key in your namespace​

Ideally in a hosted-* namespace dedicated to the router:

kubectl -n <NAMESPACE> create secret generic kube-router-authkey \
  --from-literal=TS_AUTHKEY='<the key created in step 1>'

4. Deploy the router​

Replace <NAMESPACE> and the address of your Headscale in TS_EXTRA_ARGS, then kubectl apply -f kube-router.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: kube-router
  namespace: <NAMESPACE>
spec:
  replicas: 2
  selector:
    matchLabels:
      app: kube-router
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 0
      maxUnavailable: 1
  template:
    metadata:
      labels:
        app: kube-router
    spec:
      automountServiceAccountToken: false
      enableServiceLinks: false
      securityContext:
        runAsNonRoot: true
        runAsUser: 1000
        runAsGroup: 1000
        seccompProfile:
          type: RuntimeDefault
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 100
              podAffinityTerm:
                labelSelector:
                  matchLabels:
                    app: kube-router
                topologyKey: topology.kubernetes.io/zone
      containers:
        - name: tailscale
          image: tailscale/tailscale:v1.104.1
          env:
            - name: TS_AUTHKEY
              valueFrom:
                secretKeyRef:
                  name: kube-router-authkey
                  key: TS_AUTHKEY
            - name: TS_EXTRA_ARGS
              value: --login-server=https://managed-yourslug-headscale-yourproject.apps.france-nuage.fr
            - name: TS_ROUTES
              value: 10.0.1.11/32,10.1.1.11/32,10.2.1.11/32
            - name: TS_USERSPACE
              value: "true"
            - name: TS_KUBE_SECRET
              value: ""
            - name: TS_ACCEPT_DNS
              value: "false"
            - name: HOME
              value: /tmp
          livenessProbe:
            exec:
              command: ["sh", "-c", "tailscale --socket=/tmp/tailscaled.sock status --peers=false --json | grep -qE '\"Online\":[[:space:]]*true'"]
            initialDelaySeconds: 60
            periodSeconds: 30
            timeoutSeconds: 10
            failureThreshold: 6
          readinessProbe:
            exec:
              command: ["sh", "-c", "tailscale --socket=/tmp/tailscaled.sock status --peers=false --json | grep -qE '\"Online\":[[:space:]]*true'"]
            initialDelaySeconds: 10
            periodSeconds: 15
            timeoutSeconds: 10
            failureThreshold: 4
          resources:
            requests: { cpu: 50m, memory: 64Mi, ephemeral-storage: 16Mi }
            limits: { cpu: 200m, memory: 256Mi, ephemeral-storage: 64Mi }
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities:
              drop: ["ALL"]
          volumeMounts:
            - name: tmp
              mountPath: /tmp
      volumes:
        - name: tmp
          emptyDir:
            sizeLimit: 64Mi

The two pods spread over two zones: one serves the routes, the other takes over.

5. Check​

In Headplane, under Machines, two kube-router-* machines show up online, with their three routes approved. On your workstation, enrolled on your dedicated Headscale with --accept-routes, your usual kubeconfig works as is:

kubectl -n <one of your namespaces> get pods

If it does not work​

  • dial tcp 10.x.x.x:31443: i/o timeout: no route reaches your workstation. Check that it is enrolled on your dedicated Headscale with --accept-routes, that the routes of the kube-router-* machines are approved, and that the rule targets port 31443.
  • Route available but not approved: the name in autoApprovers does not match the user of the key (kube-router@ expects the user kube-router).
  • Pods that do not start: kubectl -n <NAMESPACE> describe pod -l app=kube-router. An exceeded quota, a missing secret or an expired key are the usual causes.

References​