云效官方文档对于 Pipeline 的权限要求

权限要求

权限类型非分批发布分批发布
Namespace读权限读权限
ControllerRevision (apps/v1/ControllerRevision)全部权限全部权限
待部署的 Kubernetes 对象全部权限全部权限
Rollout (standard.oam.dev/v1alpha1)-全部权限
待部署的工作负载对象全部权限全部权限
CRD (apiextensions.k8s.io/v1/CustomResourceDefinition)-全部权限(用于安装 Rollout CRD)
ServiceAccount (core/v1/ServiceAccount)-全部权限(用于维持 Rollout 控制器的运行)
ClusterRole (rbac.authorization.k8s.io/v1/ClusterRole)-全部权限(用于维持 Rollout 控制器的运行)
ClusterRoleBinding (rbac.authorization.k8s.io/v1/ClusterRoleBinding)-全部权限(用于维持 Rollout 控制器的运行)
Role (rbac.authorization.k8s.io/v1/Role)-全部权限(用于维持 Rollout 控制器的运行)
RoleBinding (rbac.authorization.k8s.io/v1/RoleBinding)-全部权限(用于维持 Rollout 控制器的运行)
Deployment (apps/v1/Deployment)-全部权限(用于维持 Rollout 控制器的运行)
Pod (core/v1/Pod)-全部权限(用于 Rollout 控制器的安装后 E2E 测试)

注:"-" 表示在该场景下不需要相应的权限。

“待部署的工作负载对象"是指在 Kubernetes 集群中创建和管理的工作负载资源,例如 Deployment / StatefulSet / DaemonSet 等。这些工作负载对象定义了应用程序的运行方式,包括副本数、更新策略、容器规范等。

在文档中,“待部署的 Kubernetes 对象"和"待部署的工作负载对象"可能指的是同一类资源。无论是分批发布还是非分批发布,部署应用程序时都需要对这些对象拥有全部权限,以便创建、更新和删除这些资源。

在分批发布的情况下,除了对待部署的工作负载对象拥有全部权限外,还需要对 Rollout 对象以及与 Rollout 控制器相关的其他资源(如 CRD / ServiceAccount / ClusterRole 等)拥有全部权限,以确保分批发布功能的正常运行。

RABC规则创建

非分批发布的情况

apiVersion: v1
kind: ServiceAccount
metadata:
  name: appstack-non-batch-release

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: appstack-non-batch-release
rules:
- apiGroups: [""]
  resources: ["namespaces"]
  verbs: ["get", "list", "watch"]
- apiGroups: ["apps"]
  resources: ["controllerrevisions"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: ["apps"]
  resources: ["deployments", "statefulsets"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: appstack-non-batch-release
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: appstack-non-batch-release
subjects:
- kind: ServiceAccount
  name: appstack-non-batch-release
  namespace: appstack-release

分批发布的情况

apiVersion: v1
kind: ServiceAccount
metadata:
  name: appstack-batch-release

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: appstack-batch-release
rules:
- apiGroups: [""]
  resources: ["namespaces"]
  verbs: ["get", "list", "watch"]
- apiGroups: ["apps"]
  resources: ["controllerrevisions"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: ["apps"]
  resources: ["deployments", "statefulsets"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: ["standard.oam.dev"]
  resources: ["rollouts"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: ["apiextensions.k8s.io"]
  resources: ["customresourcedefinitions"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: [""]
  resources: ["serviceaccounts"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: ["rbac.authorization.k8s.io"]
  resources: ["clusterroles", "clusterrolebindings", "roles", "rolebindings"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: appstack-batch-release
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: appstack-batch-release
subjects:
- kind: ServiceAccount
  name: appstack-batch-release
  namespace: appstack-release

在这里为两种情况分别创建了 ServiceAccount / ClusterRole / ClusterRoleBinding,下一步直接将 ServiceAccount 绑定到具体用户并且导出 KubeConfig 即可

使用 Token 创建 Kubeconfig

使用以下命令获取每个 ServiceAccount 的 Token,但是注意在 1.24 版本后,集群已经不会再自动创建 Token 了

如果是以前的版本,可以直接跳过第一条 Secret 的创建,同时注意命名空间的问题

apiVersion: v1
kind: Secret
metadata:
  name: appstack-non-batch-release-token
  annotations:
    kubernetes.io/service-account.name: "appstack-non-batch-release"
type: kubernetes.io/service-account-token
---
apiVersion: v1
kind: Secret
metadata:
  name: appstack-batch-release-token
  annotations:
    kubernetes.io/service-account.name: "appstack-batch-release"
type: kubernetes.io/service-account-token
$ kubectl get secret -n appstack-release
NAME                               TYPE                                  DATA   AGE
appstack-batch-release-token       kubernetes.io/service-account-token   3      12s
appstack-non-batch-release-token   kubernetes.io/service-account-token   3      12s
cr-cr-basic-idc                    kubernetes.io/dockerconfigjson        1      35m

系统会自动将 Token 注入到对应的 Secret 中,接下来可以进行提取

kubectl get secret -n appstack-release appstack-batch-release-token -o jsonpath='{.data.token}' | base64 --decode
kubectl get secret -n appstack-release appstack-non-batch-release-token -o jsonpath='{.data.token}' | base64 --decode

如果还是老的机制可以使用下面方式进行读取

kubectl get secret $(kubectl get serviceaccount appstack-non-batch-release -o jsonpath='{.secrets[0].name}') -o jsonpath='{.data.token}' | base64 --decode
kubectl get secret $(kubectl get serviceaccount appstack-batch-release -o jsonpath='{.secrets[0].name}') -o jsonpath='{.data.token}' | base64 --decode

将输出的 Token 值复制并保存,稍后将使用它们创建 kubeconfig

API_SERVER_URL=$(kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}')

对于每个 ServiceAccount,创建一个新的 kubeconfig 文件,它不会对集群产生影响,只是修改本地内容

kubectl config set-cluster idc-cluster --server=$API_SERVER_URL --kubeconfig=non-batch-release.kubeconfig
kubectl config set-credentials appstack-non-batch-release --token=$NON_BATCH_RELEASE_TOKEN --kubeconfig=non-batch-release.kubeconfig
kubectl config set-context appstack-non-batch-release --cluster=idc-cluster --user=appstack-non-batch-release --kubeconfig=non-batch-release.kubeconfig
kubectl config use-context appstack-non-batch-release --kubeconfig=non-batch-release.kubeconfig

注意这里可以从老的 Kubeconfig 中将 CA 证书内容复制进来,否则需要使用 --insecure-skip-tls-verify=true ,不然会提示报错 x509 异常

验证

$ kubectl.exe apply -f .\batch-release.yml -n appstack-release
Warning: resource serviceaccounts/appstack-batch-release is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
serviceaccount/appstack-batch-release configured
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "rbac.authorization.k8s.io/v1, Resource=clusterroles", GroupVersionKind: "rbac.authorization.k8s.io/v1, Kind=ClusterRole"
Name: "appstack-batch-release", Namespace: ""
from server for: ".\\batch-release.yml": clusterroles.rbac.authorization.k8s.io "appstack-batch-release" is forbidden: User "system:serviceaccount:appstack-release:appstack-non-batch-release" cannot get resource "clusterroles" in API group "rbac.authorization.k8s.io" at the cluster scope
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "rbac.authorization.k8s.io/v1, Resource=clusterrolebindings", GroupVersionKind: "rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding"
Name: "appstack-batch-release", Namespace: ""
from server for: ".\\batch-release.yml": clusterrolebindings.rbac.authorization.k8s.io "appstack-batch-release" is forbidden: User "system:serviceaccount:appstack-release:appstack-non-batch-release" cannot get resource "clusterrolebindings" in API group "rbac.authorization.k8s.io" at the cluster scope

如果不属于划定范围内的权限,则直接报错 403