通过RBAC权限最小化创建云效AppStack流水线用户

云效官方文档对于 Pipeline 的权限要求

权限要求

权限类型	非分批发布	分批发布
Namespace	读权限	读权限
ControllerRevision (apps/v1/ControllerRevision)	全部权限	全部权限
待部署的 Kubernetes 对象	全部权限	全部权限
Rollout (standard.oam.dev/v1alpha1)	-	全部权限
待部署的工作负载对象	全部权限	全部权限
CRD (apiextensions.k8s.io/v1/CustomResourceDefinition)	-	全部权限（用于安装 Rollout CRD）
ServiceAccount (core/v1/ServiceAccount)	-	全部权限（用于维持 Rollout 控制器的运行）
ClusterRole (rbac.authorization.k8s.io/v1/ClusterRole)	-	全部权限（用于维持 Rollout 控制器的运行）
ClusterRoleBinding (rbac.authorization.k8s.io/v1/ClusterRoleBinding)	-	全部权限（用于维持 Rollout 控制器的运行）
Role (rbac.authorization.k8s.io/v1/Role)	-	全部权限（用于维持 Rollout 控制器的运行）
RoleBinding (rbac.authorization.k8s.io/v1/RoleBinding)	-	全部权限（用于维持 Rollout 控制器的运行）
Deployment (apps/v1/Deployment)	-	全部权限（用于维持 Rollout 控制器的运行）
Pod (core/v1/Pod)	-	全部权限（用于 Rollout 控制器的安装后 E2E 测试）

注："-" 表示在该场景下不需要相应的权限。

“待部署的工作负载对象"是指在 Kubernetes 集群中创建和管理的工作负载资源，例如 Deployment / StatefulSet / DaemonSet 等。这些工作负载对象定义了应用程序的运行方式，包括副本数、更新策略、容器规范等。

在文档中，“待部署的 Kubernetes 对象"和"待部署的工作负载对象"可能指的是同一类资源。无论是分批发布还是非分批发布，部署应用程序时都需要对这些对象拥有全部权限，以便创建、更新和删除这些资源。

在分批发布的情况下，除了对待部署的工作负载对象拥有全部权限外，还需要对 Rollout 对象以及与 Rollout 控制器相关的其他资源（如 CRD / ServiceAccount / ClusterRole 等）拥有全部权限，以确保分批发布功能的正常运行。

RABC规则创建

非分批发布的情况

apiVersion: v1
kind: ServiceAccount
metadata:
  name: appstack-non-batch-release

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: appstack-non-batch-release
rules:
- apiGroups: [""]
  resources: ["namespaces"]
  verbs: ["get", "list", "watch"]
- apiGroups: ["apps"]
  resources: ["controllerrevisions"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: ["apps"]
  resources: ["deployments", "statefulsets"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: appstack-non-batch-release
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: appstack-non-batch-release
subjects:
- kind: ServiceAccount
  name: appstack-non-batch-release
  namespace: appstack-release

分批发布的情况

apiVersion: v1
kind: ServiceAccount
metadata:
  name: appstack-batch-release

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: appstack-batch-release
rules:
- apiGroups: [""]
  resources: ["namespaces"]
  verbs: ["get", "list", "watch"]
- apiGroups: ["apps"]
  resources: ["controllerrevisions"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: ["apps"]
  resources: ["deployments", "statefulsets"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: ["standard.oam.dev"]
  resources: ["rollouts"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: ["apiextensions.k8s.io"]
  resources: ["customresourcedefinitions"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: [""]
  resources: ["serviceaccounts"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: ["rbac.authorization.k8s.io"]
  resources: ["clusterroles", "clusterrolebindings", "roles", "rolebindings"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: appstack-batch-release
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: appstack-batch-release
subjects:
- kind: ServiceAccount
  name: appstack-batch-release
  namespace: appstack-release

在这里为两种情况分别创建了 ServiceAccount / ClusterRole / ClusterRoleBinding，下一步直接将 ServiceAccount 绑定到具体用户并且导出 KubeConfig 即可

使用 Token 创建 Kubeconfig

使用以下命令获取每个 ServiceAccount 的 Token，但是注意在 1.24 版本后，集群已经不会再自动创建 Token 了

如果是以前的版本，可以直接跳过第一条 Secret 的创建，同时注意命名空间的问题

apiVersion: v1
kind: Secret
metadata:
  name: appstack-non-batch-release-token
  annotations:
    kubernetes.io/service-account.name: "appstack-non-batch-release"
type: kubernetes.io/service-account-token
---
apiVersion: v1
kind: Secret
metadata:
  name: appstack-batch-release-token
  annotations:
    kubernetes.io/service-account.name: "appstack-batch-release"
type: kubernetes.io/service-account-token

$ kubectl get secret -n appstack-release
NAME                               TYPE                                  DATA   AGE
appstack-batch-release-token       kubernetes.io/service-account-token   3      12s
appstack-non-batch-release-token   kubernetes.io/service-account-token   3      12s
cr-cr-basic-idc                    kubernetes.io/dockerconfigjson        1      35m

系统会自动将 Token 注入到对应的 Secret 中，接下来可以进行提取

kubectl get secret -n appstack-release appstack-batch-release-token -o jsonpath='{.data.token}' | base64 --decode
kubectl get secret -n appstack-release appstack-non-batch-release-token -o jsonpath='{.data.token}' | base64 --decode

如果还是老的机制可以使用下面方式进行读取

kubectl get secret $(kubectl get serviceaccount appstack-non-batch-release -o jsonpath='{.secrets[0].name}') -o jsonpath='{.data.token}' | base64 --decode
kubectl get secret $(kubectl get serviceaccount appstack-batch-release -o jsonpath='{.secrets[0].name}') -o jsonpath='{.data.token}' | base64 --decode

将输出的 Token 值复制并保存，稍后将使用它们创建 kubeconfig

API_SERVER_URL=$(kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}')

对于每个 ServiceAccount，创建一个新的 kubeconfig 文件，它不会对集群产生影响，只是修改本地内容

kubectl config set-cluster idc-cluster --server=$API_SERVER_URL --kubeconfig=non-batch-release.kubeconfig
kubectl config set-credentials appstack-non-batch-release --token=$NON_BATCH_RELEASE_TOKEN --kubeconfig=non-batch-release.kubeconfig
kubectl config set-context appstack-non-batch-release --cluster=idc-cluster --user=appstack-non-batch-release --kubeconfig=non-batch-release.kubeconfig
kubectl config use-context appstack-non-batch-release --kubeconfig=non-batch-release.kubeconfig

注意这里可以从老的 Kubeconfig 中将 CA 证书内容复制进来，否则需要使用 --insecure-skip-tls-verify=true ，不然会提示报错 x509 异常

验证

$ kubectl.exe apply -f .\batch-release.yml -n appstack-release
Warning: resource serviceaccounts/appstack-batch-release is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
serviceaccount/appstack-batch-release configured
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "rbac.authorization.k8s.io/v1, Resource=clusterroles", GroupVersionKind: "rbac.authorization.k8s.io/v1, Kind=ClusterRole"
Name: "appstack-batch-release", Namespace: ""
from server for: ".\\batch-release.yml": clusterroles.rbac.authorization.k8s.io "appstack-batch-release" is forbidden: User "system:serviceaccount:appstack-release:appstack-non-batch-release" cannot get resource "clusterroles" in API group "rbac.authorization.k8s.io" at the cluster scope
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "rbac.authorization.k8s.io/v1, Resource=clusterrolebindings", GroupVersionKind: "rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding"
Name: "appstack-batch-release", Namespace: ""
from server for: ".\\batch-release.yml": clusterrolebindings.rbac.authorization.k8s.io "appstack-batch-release" is forbidden: User "system:serviceaccount:appstack-release:appstack-non-batch-release" cannot get resource "clusterrolebindings" in API group "rbac.authorization.k8s.io" at the cluster scope

如果不属于划定范围内的权限，则直接报错 403

测试脚本

Windows 平台下的测试脚本

param(
    [string]$Mode,
    [string]$Namespace = "default"
)

$batchPermissions = @{
    "namespaces" = "get list watch"
    "deployments.apps" = "get list watch create update patch delete"
    "controllerrevisions.apps" = "get list watch create update patch delete"
    "statefulsets.apps" = "get list watch create update patch delete"
    "serviceaccounts" = "get list watch create"
    "pods" = "get list watch create update patch delete"
}

$nonBatchPermissions = @{
    "namespaces" = "get list watch"
    "controllerrevisions.apps" = "get list watch create update patch delete"
    "deployments.apps" = "get list watch"
    "statefulsets.apps" = "get list watch"
    "serviceaccounts" = "get list watch"
}

# 注意这里可以调整 kubeconfig 的位置
if ($Mode -eq "batch") {
    $kubeconfigName = "batch-release.kubeconfig"
    $permissions = $batchPermissions
}
else {
    $kubeconfigName = "non-batch-release.kubeconfig"
    $permissions = $nonBatchPermissions
}

if (-Not (Test-Path $kubeconfigName)) {
    Write-Host "Error: kubeconfig file '$kubeconfigName' does not exist."
    exit 1
}

foreach ($resource in $permissions.Keys) {
    foreach ($verb in $permissions[$resource].Split()) {
        $result = kubectl --kubeconfig $kubeconfigName auth can-i $verb $resource -n $Namespace
        Write-Host "Permission to $verb $resource in namespace ${Namespace}: $result"
    }
}

Linux 平台下的测试脚本

#!/bin/bash

MODE=$1
NAMESPACE=${2:-"default"}

declare -A batch_permissions=(
    ["namespaces"]="get list watch"
    ["deployments.apps"]="get list watch create update patch delete"
    ["statefulsets.apps"]="get list watch create update patch delete"
    ["serviceaccounts"]="get list watch create"
    ["pods"]="get list watch create update patch delete"
)

declare -A non_batch_permissions=(
    ["namespaces"]="get list watch"
    ["controllerrevisions.apps"]="get list watch create update patch delete"
    ["deployments.apps"]="get list watch"
    ["statefulsets.apps"]="get list watch"
    ["serviceaccounts"]="get list watch"
)

# 注意这里可以调整 kubeconfig 的位置
if [[ $MODE == "batch" ]]; then
    KUBECONFIG_NAME="batch-release.kubeconfig"
    declare -n permissions=batch_permissions
else
    KUBECONFIG_NAME="non-batch-release.kubeconfig"
    declare -n permissions=non_batch_permissions
fi

if [ ! -f "$KUBECONFIG_NAME" ]; then
    echo "Error: kubeconfig file '$KUBECONFIG_NAME' does not exist."
    exit 1
fi

KUBECTL_CMD="kubectl --kubeconfig $KUBECONFIG_NAME auth can-i"

for resource in "${!permissions[@]}"; do
    for verb in ${permissions[$resource]}; do
        result=$($KUBECTL_CMD $verb $resource -n $NAMESPACE)
        echo "Permission to $verb $resource in namespace $NAMESPACE: $result"
    done
done

直接调用 script.sh -Mode [batch|non-batch] 即可

权限要求#

RABC规则创建#

使用 Token 创建 Kubeconfig#

验证#

测试脚本#

权限要求

RABC规则创建

使用 Token 创建 Kubeconfig

验证

测试脚本