#Creating Clusters on Huawei DCS

This document provides instructions for creating Kubernetes clusters on the Huawei DCS platform. YAML-based cluster creation is available through manifests. If Fleet Essentials is installed and Alauda Container Platform DCS Infrastructure Provider is 1.0.13 or later, you can also create clusters through the web UI. If the workflow relies on pool-managed persistent disks, use DCS provider v1.0.16 or later. In v1.0.16, the persistentDisk declaration on DCSIpHostnamePool is available through YAML only and is not exposed in the web UI. In ACP v4.4 or later, YAML-based cluster creation can use Self-built VIP for the control plane endpoint.

#Prerequisites

Before creating clusters, ensure all of the following prerequisites are met:

#1. Infrastructure Resources

Configure the following infrastructure resources before creating a cluster:

• Cloud Credential - DCS platform access information
• IP Pool - Network configuration for cluster nodes, additional NICs, and any IP-slot persistent disks such as /var/cpaas
• Machine Template - VM specifications for control plane and worker nodes, excluding pool-managed persistent disks

See Infrastructure Resources for Huawei DCS for detailed configuration instructions.

#2. Required Plugin Installation

Install the following plugins on the global cluster:

• Alauda Container Platform Kubeadm Provider
• Alauda Container Platform DCS Infrastructure Provider

For detailed installation instructions, refer to the Installation Guide.

#3. Virtual Machine Template Preparation

For Kubernetes installation, you must:

• Upload the Alauda OS image to the DCS platform
• Create a virtual machine template based on this image
• Ensure the template includes all necessary Kubernetes components
• Use DCS VM templates 4.2.1 or later if you plan to use persistent disks, because safe shutdown and disk detach depend on guest tools
• Use one-by-one replacement for any cluster that will rely on pool-managed persistent disks. Keep maxSurge: 0 on the control plane and on worker node pools.

For details on the Kubernetes components included in each VM image, see OS Support Matrix.

#4. Network Connectivity

Global cluster nodes must be able to reach the DCS platform at two distinct destinations:

File upload is a two-step flow: the provider calls applyUpload on the VRM virtual IP (TCP/7443); the DCS platform responds with a URL that points to a specific physical host's management IP and port; the provider then streams the file to that URL. Both destinations must be reachable end-to-end before cluster creation.

If global cluster nodes use a multi-NIC layout (for example, one NIC on the ACP cluster network and another NIC on the customer's management network where DCS is deployed), ensure that both destinations — the VRM virtual IP and every DCS physical host MGMT IP — are routable from the appropriate NIC.

Requirement: Connectivity to both destinations is mandatory for cluster creation and management.

#5. LoadBalancer Configuration

Configure a stable Kubernetes API endpoint before creating the cluster. This endpoint distributes or directs API server traffic for control plane access.

Use one of these modes:

For a single-control-plane deployment that has no load balancer in front of the API server, see Single-Control-Plane (No External LB) Layout.

#6. Public Registry Configuration

Configure the public registry credentials. This includes:

• Registry repository address configuration
• Proper authentication credentials setup

#Using the Web UI

Version requirement: This workflow requires Fleet Essentials and Alauda Container Platform DCS Infrastructure Provider 1.0.13 or later. If the provider version is earlier than 1.0.13, use YAML manifests. If you use pool-managed persistent disks, use DCS provider v1.0.16 or later. In v1.0.16, configure DCSIpHostnamePool.spec.pool[].persistentDisk through YAML because the web UI does not expose that field.

If the new cluster will rely on pool-managed persistent disks, create or update the backing DCSIpHostnamePool with YAML and then use the web UI for the rest of the cluster workflow.

#Creation Workflow

The cluster creation follows a 5-step wizard:

Step 1: Basic Info
  ↓
Step 2: Control Plane Node Pool
  ↓
Step 3: Worker Node Pools
  ↓
Step 4: Networking
  ↓
Step 5: Review

Navigation: Clusters → Clusters → Create Cluster → Select Huawei DCS

#Step 1: Basic Info

Prerequisites Check:

Before creating a cluster, ensure:

• DCS VM Templates exist in the DCS platform, and the Alauda OS version matches the Kubernetes version
• The Kubernetes API endpoint is ready. For an external LoadBalancer, create it before cluster creation. For Self-built VIP, reserve the VIP and use the YAML path.

Version Constraint: Only the latest Kubernetes version supported by the platform can be created.

#Step 2: Control Plane Node Pool

The control plane node pool is fixed at 3 replicas for high availability.

Validation: The associated IP Pool must have sufficient available IP addresses (≥ 3).

#Step 3: Worker Node Pools

You can add multiple worker node pools. Each pool has the following configuration:

Validation Rules:

• Pool names must be unique within the cluster
• IP Pool must have sufficient available IP addresses (≥ Replicas)
• maxSurge and maxUnavailable must satisfy the constraint: if maxSurge = 0, then maxUnavailable > 0
• If the cluster will rely on pool-managed persistent disks, keep maxSurge = 0 so nodes are replaced one by one during future upgrades

Tip: Prefix the pool name with the cluster name followed by a hyphen (e.g., mycluster-worker-1) to avoid naming conflicts across different clusters.

#Step 4: Networking

Validation: Pods CIDR and Services CIDR must not overlap.

#Step 5: Review

Review all configuration settings before creating the cluster:

Basic Info:

• Name, Display Name, Infrastructure Credential
• Distribution Version, Kubernetes Version
• Cluster API Address

Control Plane Node Pool:

• Machine Template with VM Template Name, OS Version, Kubernetes Version
• CPU, Memory, Replicas, SSH Keys

Worker Node Pools (list view):

• Pool Name, Machine Template, Replicas
• Max Surge, Max Unavailable, SSH Keys

If the cluster will rely on pool-managed persistent disks, keep Max Surge set to 0 for worker node pools.

Networking:

• Pods CIDR, Services CIDR, Join CIDR

Click Create to start the cluster creation process.

#Using YAML

#Cluster Creation Workflow

When using YAML, you create Cluster API resources in the global cluster to provision infrastructure and bootstrap a functional Kubernetes cluster.

#Configuration Workflow

Follow these steps in order to provision a functional cluster (control plane and worker nodes):

1. Configure KubeadmControlPlane (control-plane spec and kubeadm bootstrap).
2. Configure DCSCluster (infrastructure binding and load balancer reference).
3. Create the Cluster resource (top-level CAPI object that links the two above).
4. Configure worker resources: KubeadmConfigTemplate (worker bootstrap), worker DCSMachineTemplate, worker DCSIpHostnamePool, and MachineDeployment. The control plane alone is not a usable cluster. See Managing Nodes on Huawei DCS for the four worker YAML resources.

Note: Infrastructure resources (Secret, control-plane DCSIpHostnamePool, control-plane DCSMachineTemplate) should be configured separately. See Infrastructure Resources for Huawei DCS for instructions.

If you need any disk to survive rolling replacement, declare it in the matching DCSIpHostnamePool.spec.pool[].persistentDisk entry. This includes the platform-required /var/cpaas disk. The provider creates new persistent volumes as independent persistent normal volumes. When the DCS environment requires an explicit thin-provisioning setting, set persistentDisk[].isThin; if you omit it, the provider does not send isThin and DCS uses the platform default.

If the cluster needs additional NICs, declare them in DCSIpHostnamePool.spec.pool[].additionNic[] before the corresponding Machines are created. The provider applies additional NICs only during new VM creation. Existing VMs do not receive hot-added NICs when the Pool is edited later.

#Resolving Placeholder Values

The example manifests below use <placeholder> syntax for values that are environment-specific. Several of these have a canonical source of truth that you should query rather than hand-pick:

#Magic-Token Placeholders

A few values in the example manifests look like placeholders but are actually literal tokens substituted by the DCS Provider at machine-join time. Leave them exactly as written:

Replacing or quoting these tokens manually breaks the join flow and produces nodes that never register with the control plane.

#Network Planning and Load Balancer

Before creating control plane resources, plan the network architecture and choose the control plane endpoint mode.

Requirements:

• Network segmentation: Plan IP address ranges for control plane nodes
• Additional NICs: If nodes require storage, management, or isolated application networks, plan DCSIpHostnamePool.spec.pool[].additionNic[] values for each IP slot, including DVS and Port Group names
• API endpoint mode: Use an external LoadBalancer, or use Self-built VIP on ACP v4.4 or later
• API server address: Prepare a stable VIP or load balancer address for the Kubernetes API Server
• Connectivity: Ensure network connectivity between all components

For Self-built VIP, validate that the target DCS port group allows VIP ownership, GARP or ARP updates, and VRRP traffic between the control plane VMs. The VIP must be in the same Layer 2 network as the NIC that will hold it.

#Configure KubeadmControlPlane

The KubeadmControlPlane resource defines the control plane configuration including Kubernetes version, node specifications, and bootstrap settings.

apiVersion: controlplane.cluster.x-k8s.io/v1beta1
kind: KubeadmControlPlane
metadata:
  name: <cluster-name>-kcp
  namespace: cpaas-system
  annotations:
    controlplane.cluster.x-k8s.io/skip-coredns: ""
    controlplane.cluster.x-k8s.io/skip-kube-proxy: ""
spec:
  rolloutStrategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 0 # Required when the cluster relies on pool-managed persistent disks
  kubeadmConfigSpec:
    users:
    - name: boot
      sshAuthorizedKeys:
      - "<ssh-authorized-keys>"
    format: ignition
    files:
    - path: /etc/kubernetes/admission/psa-config.yaml
      owner: "root:root"
      permissions: "0644"
      content: |
        # ... (Admission Configuration Content) ...
    - path: /etc/kubernetes/patches/kubeletconfiguration0+strategic.json
      owner: "root:root"
      permissions: "0644"
      content: |
        {
          "apiVersion": "kubelet.config.k8s.io/v1beta1",
          "kind": "KubeletConfiguration",
          "_comment": "... (Kubelet Configuration Content) ..."
        }
    # ... (other files) ...
    clusterConfiguration:
      imageRepository: cloud.alauda.io/alauda
      dns:
        imageTag: <dns-image-tag>
      etcd:
        local:
          imageTag: <etcd-image-tag>
      # ... (apiServer, controllerManager, scheduler) ...
    initConfiguration:
      patches:
        directory: /etc/kubernetes/patches
      nodeRegistration:
        kubeletExtraArgs:
          node-labels: "kube-ovn/role=master"
          provider-id: PROVIDER_ID
          volume-plugin-dir: "/opt/libexec/kubernetes/kubelet-plugins/volume/exec/"
          protect-kernel-defaults: "true"
    joinConfiguration:
      patches:
        directory: /etc/kubernetes/patches
      nodeRegistration:
        kubeletExtraArgs:
          node-ip: NODE_IP
          node-labels: "kube-ovn/role=master"
          provider-id: PROVIDER_ID
          volume-plugin-dir: "/opt/libexec/kubernetes/kubelet-plugins/volume/exec/"
          protect-kernel-defaults: "true"
  machineTemplate:
    nodeDrainTimeout: 1m
    nodeDeletionTimeout: 5m
    infrastructureRef:
      apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
      kind: DCSMachineTemplate
      name: <cp-dcs-machine-template-name>
  replicas: 3
  version: <control-plane-kubernetes-version>

Parameter Descriptions:

For component versions (e.g., <dns-image-tag>, <etcd-image-tag>), refer to OS Support Matrix.

#Configure DCSCluster

DCSCluster is the infrastructure cluster declaration that references the load balancer and DCS platform credentials.

Use type: external when you manage the load balancer outside the cluster. This is the compatible mode for all supported versions.

apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: DCSCluster
metadata:
  name: "<cluster-name>"
  namespace: cpaas-system
spec:
  controlPlaneLoadBalancer:
    host: <load-balancer-ip-or-domain-name>
    port: 6443
    type: external
  credentialSecretRef:
    name: <auth-secret-name>
  controlPlaneEndpoint:
    host: <load-balancer-ip-or-domain-name>
    port: 6443
  # Optional. Enable only when the target DCS compute cluster has DRS enabled.
  controlPlaneHA:
    enabled: true
  networkType: kube-ovn
  site: <site>

For ACP v4.4 or later, you can use type: internal to enable Self-built VIP. In this mode, the DCS provider injects a temporary bootstrap VIP on the first control plane node, creates or updates the provider-managed alive ModuleInfo, and waits until the alive runtime exposes the Kubernetes API through the VIP.

apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: DCSCluster
metadata:
  name: "<cluster-name>"
  namespace: cpaas-system
spec:
  controlPlaneLoadBalancer:
    host: <self-built-vip>
    port: 6443
    type: internal
    vrid: 42
    interface: eth0
  credentialSecretRef:
    name: <auth-secret-name>
  controlPlaneEndpoint:
    host: <self-built-vip>
    port: 6443
  controlPlaneHA:
    enabled: true
  networkType: kube-ovn
  site: <site>

Parameter Descriptions:

controlPlaneHA is optional. When enabled, the provider creates and maintains one DRS ruleType=2 mutual-exclusion rule for the current control-plane VMs in this workload cluster. DCS performs the actual placement and runtime migration. The provider does not run DRS or apply DRS recommendations. If the rule is maintained but placement has not converged, wait for the DCS scheduling mechanism or trigger/apply DRS from the DCS platform side. For infrastructure prerequisites, see Cross-Host High Availability for Control Plane.

#Self-built VIP behavior and limitations

Self-built VIP is supported only on ACP v4.4 or later. Plan this mode before cluster creation.

Recommendation: Use type: internal only for new highly available clusters with three control plane replicas. For existing clusters that use type: external, keep the external load balancer unless you have a tested migration plan.

The DCS provider handles Self-built VIP in two phases:

1. During bootstrap, the provider injects dcs-bootstrap-vip.service only on the first kubeadm init control plane node. This service adds <self-built-vip>/32 before kubeadm starts.
2. After the control plane nodes register, the provider creates or updates the provider-managed alive ModuleInfo. The platform plugin flow renders the workload AppRelease, and alive installs keepalived and IPVS runtime on the control plane nodes.

Important constraints:

• type: internal requires an IPv4 VIP. Domain names are not supported for Self-built VIP.
• The VIP must be excluded from DCSIpHostnamePool primary IPs and additional NIC IPs.
• The target DCS port group must allow VIP ownership, GARP or ARP updates, and VRRP traffic.
• interface must be a Linux interface name of 15 characters or fewer. Use it when the VIP must be bound to a specific NIC, such as eth0 or eth1.
• type, host, port, vrid, and interface are not intended to be changed after the cluster control plane is initialized.
• The DCS provider does not directly create the workload AppRelease. It creates or patches the alive ModuleInfo, and the platform plugin flow renders the workload resources.
• The DCS provider does not inject or persist IPVS kernel modules or sysctl parameters. Use the DCS VM template and plugin package delivered with ACP v4.4 or later.

#Single-Control-Plane (No External LB) Layout

For development, PoC, or any deployment where the control plane has only one replica (KubeadmControlPlane.spec.replicas: 1), you do not have a real load balancer in front of the API server. Two fields nevertheless still require a value:

• .spec.controlPlaneLoadBalancer.host and .spec.controlPlaneEndpoint.host — set both to the IP of the sole control plane node (the same IP allocated to that node in the control-plane DCSIpHostnamePool).
• .spec.controlPlaneLoadBalancer.type — keep as external. Do not use Self-built VIP for a single-control-plane layout.

Concretely:

spec:
  controlPlaneLoadBalancer:
    host: 10.226.82.150     # same IP as the control plane node from the IP pool
    port: 6443
    type: external
  controlPlaneEndpoint:
    host: 10.226.82.150     # same as above
    port: 6443

This layout has no HA — losing the single control plane node makes the cluster API unreachable until the node is recovered. For production, use replicas: 3 with an external LoadBalancer or Self-built VIP.

#Configure Cluster

The Cluster resource declares the cluster and references the control plane and infrastructure resources.

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  annotations:
    capi.cpaas.io/resource-group-version: infrastructure.cluster.x-k8s.io/v1beta1
    capi.cpaas.io/resource-kind: DCSCluster
    cpaas.io/kube-ovn-join-cidr: <kube-ovn-join-cidr>
  labels:
    cluster-type: DCS
  name: <cluster-name>
  namespace: cpaas-system
spec:
  clusterNetwork:
    pods:
      cidrBlocks:
      - <pods-cidr>
    services:
      cidrBlocks:
      - <services-cidr>
  controlPlaneRef:
    apiVersion: controlplane.cluster.x-k8s.io/v1beta1
    kind: KubeadmControlPlane
    name: <cluster-name>-kcp
  infrastructureRef:
    apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
    kind: DCSCluster
    name: <cluster-name>

Parameter Descriptions:

Cluster Annotations:

The example above shows three annotations, but a complete Cluster resource carries a few more. The table below lists the annotations the operator authors (some others are written by ACP controllers and you should not pre-set them):

ACP controllers may write additional read-only annotations (cpaas.io/cpu-cores-number, cpaas.io/memories, cpaas.io/nodes-number, and so on) after the cluster is up — these are computed and must not be pre-set in the YAML you apply.

#Deploying Nodes

Refer to Managing Nodes on Huawei DCS for instructions on deploying worker nodes.

#Cluster Verification

After deploying all cluster resources, verify that the cluster has been created successfully and is operational.

#Using the Console

1. Navigate to Clusters → Clusters
2. Locate your newly created cluster in the cluster list
3. Verify that the cluster status shows as Running
4. Check that all control plane and worker nodes are Ready

#Using kubectl

Alternatively, verify the cluster using kubectl commands:

# Check cluster status
kubectl get cluster -n cpaas-system <cluster-name>

# Verify control plane
kubectl get kubeadmcontrolplane -n cpaas-system <cluster-name>-kcp

# Check machine status
kubectl get machines -n cpaas-system

# Verify cluster deployment
kubectl get clustermodule <cluster-name> -o jsonpath='{.status.base.deployStatus}'

For clusters that use additional NICs, also verify that the provider recorded them on the matching DCSMachine objects:

kubectl -n cpaas-system get dcsmachine -l cluster.x-k8s.io/cluster-name=<cluster-name> \
  -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.status.networkConfig.ip}{"\t"}{.status.additionalNic}{"\n"}{end}'

#Verify Self-built VIP

If you set DCSCluster.spec.controlPlaneLoadBalancer.type to internal, verify the provider-managed VIP path after the cluster is created.

Check that DCSCluster and the top-level CAPI Cluster point to the VIP:

kubectl -n cpaas-system get dcscluster <cluster-name> \
  -o jsonpath='{.spec.controlPlaneLoadBalancer.type}{" "}{.spec.controlPlaneEndpoint.host}{":"}{.spec.controlPlaneEndpoint.port}{"\n"}'

kubectl -n cpaas-system get cluster <cluster-name> \
  -o jsonpath='{.spec.controlPlaneEndpoint.host}{":"}{.spec.controlPlaneEndpoint.port}{"\n"}'

Check that the provider-created alive ModuleInfo exists:

kubectl -n cpaas-system get moduleinfo \
  -l cpaas.io/cluster-name=<cluster-name>,cpaas.io/module-name=alive

Check the workload cluster runtime:

kubectl --kubeconfig <workload-kubeconfig> -n cpaas-system get apprelease alive
kubectl --kubeconfig <workload-kubeconfig> -n kube-system get pods -l app=alive -o wide
kubectl --kubeconfig <vip-kubeconfig> get --raw=/version

Expected results:

• The endpoint host is the configured <self-built-vip>.
• The provider-created alive ModuleInfo exists for the cluster.
• The workload AppRelease for alive is synced and healthy.
• Each control plane node has a Ready alive pod.
• kubectl --kubeconfig <vip-kubeconfig> get --raw=/version returns the Kubernetes version through the VIP.

#Verify Control Plane HA

If you enabled DCSCluster.spec.controlPlaneHA.enabled, inspect the DCSCluster condition first:

kubectl get dcscluster <cluster-name> -n cpaas-system \
  -o jsonpath='{range .status.conditions[?(@.type=="ControlPlaneHAReady")]}{.status}{" "}{.reason}{" "}{.message}{"\n"}{end}'

Interpret the condition as follows:

Inspect the observed rule and membership snapshot:

kubectl get dcscluster <cluster-name> -n cpaas-system \
  -o jsonpath='rule={.status.controlPlaneHA.ruleName}{" index="}{.status.controlPlaneHA.ruleIndex}{" cluster="}{.status.controlPlaneHA.clusterUrn}{"\n"}{range .status.controlPlaneHA.members[*]}{.machineName}{" "}{.vmUrn}{"\n"}{end}'

The members[] list contains the CAPI Machine name and DCS VM URN for each control-plane VM in the rule. The current DCS status snapshot does not include the DCS host name. If you need to confirm the exact host placement, query the VM details from the DCS platform or DCS API and compare the VM hostName or hostUrn values.

#Expected Results

A successfully created cluster should show:

• Cluster status: Running or Provisioned
• All control plane machines: Running
• All worker nodes (if deployed): Running
• Kubernetes nodes: Ready
• Cluster Module Status: Completed
• For multi-NIC clusters, each created VM has the expected additional NICs in DCSMachine.status.additionalNic, and the guest OS shows the corresponding ethN interfaces.

#Appendix

#Complete KubeadmControlPlane Configuration

Below is the complete KubeadmControlPlane configuration, including all default audit policies, admission controls, and file contents.

apiVersion: controlplane.cluster.x-k8s.io/v1beta1
kind: KubeadmControlPlane
metadata:
  name: <cluster-name>-kcp
  namespace: cpaas-system
  annotations:
    controlplane.cluster.x-k8s.io/skip-coredns: ""
    controlplane.cluster.x-k8s.io/skip-kube-proxy: ""
spec:
  rolloutStrategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 0 # Required when the cluster relies on pool-managed persistent disks
  kubeadmConfigSpec:
    users:
    - name: boot
      sshAuthorizedKeys:
      - "<ssh-authorized-keys>"
    format: ignition
    files:
    - path: /etc/kubernetes/admission/psa-config.yaml
      owner: "root:root"
      permissions: "0644"
      content: |
        apiVersion: apiserver.config.k8s.io/v1
        kind: AdmissionConfiguration
        plugins:
        - name: PodSecurity
          configuration:
            apiVersion: pod-security.admission.config.k8s.io/v1
            kind: PodSecurityConfiguration
            defaults:
              enforce: "privileged"
              enforce-version: "latest"
              audit: "baseline"
              audit-version: "latest"
              warn: "baseline"
              warn-version: "latest"
            exemptions:
              usernames: []
              runtimeClasses: []
              namespaces:
              - kube-system
              - cpaas-system
    - path: /etc/kubernetes/patches/kubeletconfiguration0+strategic.json
      owner: "root:root"
      permissions: "0644"
      content: |
        {
          "apiVersion": "kubelet.config.k8s.io/v1beta1",
          "kind": "KubeletConfiguration",
          "protectKernelDefaults": true,
          "tlsCertFile": "/etc/kubernetes/pki/kubelet.crt",
          "tlsPrivateKeyFile": "/etc/kubernetes/pki/kubelet.key",
          "streamingConnectionIdleTimeout": "5m",
          "clientCAFile": "/etc/kubernetes/pki/ca.crt"
        }
    - path: /etc/kubernetes/encryption-provider.conf
      owner: "root:root"
      append: false
      permissions: "0644"
      content: |
        apiVersion: apiserver.config.k8s.io/v1
        kind: EncryptionConfiguration
        resources:
        - resources:
          - secrets
          providers:
          - aescbc:
              keys:
              - name: key1
                secret: <base64-encoded-secret>
    - path: /etc/kubernetes/audit/policy.yaml
      owner: "root:root"
      append: false
      permissions: "0644"
      content: |
        apiVersion: audit.k8s.io/v1
        kind: Policy
        omitStages:
        - "RequestReceived"
        rules:
        - level: None
          users:
          - system:kube-controller-manager
          - system:kube-scheduler
          - system:serviceaccount:kube-system:endpoint-controller
          verbs: ["get", "update"]
          namespaces: ["kube-system"]
          resources:
          - group: ""
            resources: ["endpoints"]
        - level: None
          nonResourceURLs:
          - /healthz*
          - /version
          - /swagger*
        - level: None
          resources:
          - group: ""
            resources: ["events"]
        - level: None
          resources:
          - group: "devops.alauda.io"
        - level: None
          verbs: ["get", "list", "watch"]
        - level: None
          resources:
          - group: "coordination.k8s.io"
            resources: ["leases"]
        - level: None
          resources:
          - group: "authorization.k8s.io"
            resources: ["subjectaccessreviews", "selfsubjectaccessreviews"]
          - group: "authentication.k8s.io"
            resources: ["tokenreviews"]
        - level: None
          resources:
          - group: "app.alauda.io"
            resources: ["imagewhitelists"]
          - group: "k8s.io"
            resources: ["namespaceoverviews"]
        - level: Metadata
          resources:
          - group: ""
            resources: ["secrets", "configmaps"]
        - level: Metadata
          resources:
          - group: "operator.connectors.alauda.io"
            resources: ["installmanifests"]
          - group: "operators.katanomi.dev"
            resources: ["katanomis"]
        - level: RequestResponse
          resources:
          - group: ""
          - group: "aiops.alauda.io"
          - group: "apps"
          - group: "app.k8s.io"
          - group: "authentication.istio.io"
          - group: "auth.alauda.io"
          - group: "autoscaling"
          - group: "asm.alauda.io"
          - group: "clusterregistry.k8s.io"
          - group: "crd.alauda.io"
          - group: "infrastructure.alauda.io"
          - group: "monitoring.coreos.com"
          - group: "operators.coreos.com"
          - group: "networking.istio.io"
          - group: "extensions.istio.io"
          - group: "install.istio.io"
          - group: "security.istio.io"
          - group: "telemetry.istio.io"
          - group: "opentelemetry.io"
          - group: "networking.k8s.io"
          - group: "portal.alauda.io"
          - group: "rbac.authorization.k8s.io"
          - group: "storage.k8s.io"
          - group: "tke.cloud.tencent.com"
          - group: "devopsx.alauda.io"
          - group: "core.katanomi.dev"
          - group: "deliveries.katanomi.dev"
          - group: "integrations.katanomi.dev"
          - group: "artifacts.katanomi.dev"
          - group: "builds.katanomi.dev"
          - group: "versioning.katanomi.dev"
          - group: "sources.katanomi.dev"
          - group: "tekton.dev"
          - group: "operator.tekton.dev"
          - group: "eventing.knative.dev"
          - group: "flows.knative.dev"
          - group: "messaging.knative.dev"
          - group: "operator.knative.dev"
          - group: "sources.knative.dev"
          - group: "operator.devops.alauda.io"
          - group: "flagger.app"
          - group: "jaegertracing.io"
          - group: "velero.io"
            resources: ["deletebackuprequests"]
          - group: "connectors.alauda.io"
          - group: "operator.connectors.alauda.io"
            resources: ["connectorscores", "connectorsgits", "connectorsocis"]
        - level: Metadata
    preKubeadmCommands:
    - while ! ip route | grep -q "default via"; do sleep 1; done; echo "NetworkManager started"
    - mkdir -p /run/cluster-api && restorecon -Rv /run/cluster-api
    - if [ -f /etc/disk-setup.sh ]; then bash /etc/disk-setup.sh; fi
    postKubeadmCommands:
    - chmod 600 /var/lib/kubelet/config.yaml
    clusterConfiguration:
      imageRepository: cloud.alauda.io/alauda
      dns:
        imageTag: <dns-image-tag>
      etcd:
        local:
          imageTag: <etcd-image-tag>
      apiServer:
        extraArgs:
          audit-log-format: json
          audit-log-maxage: "30"
          audit-log-maxbackup: "10"
          audit-log-maxsize: "200"
          profiling: "false"
          audit-log-mode: batch
          audit-log-path: /etc/kubernetes/audit/audit.log
          audit-policy-file: /etc/kubernetes/audit/policy.yaml
          tls-cipher-suites: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384"
          encryption-provider-config: /etc/kubernetes/encryption-provider.conf
          admission-control-config-file: /etc/kubernetes/admission/psa-config.yaml
          tls-min-version: VersionTLS12
          kubelet-certificate-authority: /etc/kubernetes/pki/ca.crt
        extraVolumes:
        - name: vol-dir-0
          hostPath: /etc/kubernetes
          mountPath: /etc/kubernetes
          pathType: Directory
      controllerManager:
        extraArgs:
          bind-address: "::"
          profiling: "false"
          tls-min-version: VersionTLS12
          flex-volume-plugin-dir: "/opt/libexec/kubernetes/kubelet-plugins/volume/exec/"
      scheduler:
        extraArgs:
          bind-address: "::"
          tls-min-version: VersionTLS12
          profiling: "false"
    initConfiguration:
      patches:
        directory: /etc/kubernetes/patches
      nodeRegistration:
        kubeletExtraArgs:
          node-labels: "kube-ovn/role=master"
          provider-id: PROVIDER_ID
          volume-plugin-dir: "/opt/libexec/kubernetes/kubelet-plugins/volume/exec/"
          protect-kernel-defaults: "true"
    joinConfiguration:
      patches:
        directory: /etc/kubernetes/patches
      nodeRegistration:
        kubeletExtraArgs:
          node-ip: NODE_IP
          node-labels: "kube-ovn/role=master"
          provider-id: PROVIDER_ID
          volume-plugin-dir: "/opt/libexec/kubernetes/kubelet-plugins/volume/exec/"
          protect-kernel-defaults: "true"
  machineTemplate:
    nodeDrainTimeout: 1m
    nodeDeletionTimeout: 5m
    infrastructureRef:
      apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
      kind: DCSMachineTemplate
      name: <cp-dcs-machine-template-name>
  replicas: 3
  version: <control-plane-kubernetes-version>

files:
- contentFrom:
    secret:
      key: psa-config.yaml
      name: dcs-kubernetes-1.33-files
  owner: "root:root"
  path: /etc/kubernetes/admission/psa-config.yaml
  permissions: "0644"
- contentFrom:
    secret:
      key: control-plane-kubelet-patch.json
      name: dcs-kubernetes-1.33-files
  owner: "root:root"
  path: /etc/kubernetes/patches/kubeletconfiguration0+strategic.json
  permissions: "0644"
- contentFrom:
    secret:
      key: audit-policy.yaml
      name: dcs-kubernetes-1.33-files
  owner: "root:root"
  path: /etc/kubernetes/audit/policy.yaml
  permissions: "0644"
- path: /etc/kubernetes/encryption-provider.conf
  owner: "root:root"
  append: false
  permissions: "0644"
  content: |
    apiVersion: apiserver.config.k8s.io/v1
    kind: EncryptionConfiguration
    resources:
    - resources:
      - secrets
      providers:
      - aescbc:
          keys:
          - name: key1
            secret: <base64-encoded-secret>

# Replace <kubernetes-major-minor> with the value matching this cluster
# (for example, 1.33 for Kubernetes v1.33.x).
K8S_MM=<kubernetes-major-minor>
kubectl -n cpaas-system get secret "dcs-kubernetes-${K8S_MM}-files" >/dev/null 2>&1 \
  && echo "Secret present — contentFrom.secret form is supported" \
  || echo "Secret missing — use inline content form"

#Next Steps

After creating a cluster:

• Manage Nodes
• Upgrade Cluster

#Troubleshooting

If the cluster reaches Provisioned but never becomes Ready — for example, workload nodes stay NotReady because the CNI is not deployed — see Troubleshoot a Workload Cluster Stuck in Provisioned.