逃脱只会部署集群系列 —— k8s集群认证、授权、安全控制
我们可以利用调试，获取请求时读取的文件（/root/.kube/）以及请求地址（:6443即的地址）
[root@k8s-master ~]# kubectl get nodes -v=7
I1120 12:31:58.065224 43018 loader.go:375] Config loaded from file: /root/.kube/config
I1120 12:31:58.068895 43018 round_trippers.go:420] GET https://192.168.0.121:6443/api?timeout=32s
I1120 12:31:58.068964 43018 round_trippers.go:427] Request Headers:
I1120 12:31:58.068974 43018 round_trippers.go:431] Accept: application/json, */*2、怎么默认授权
init启动完节点后，会默认输出类似下面的提示内容：
... ...Your Kubernetes master has initialized successfully!To start using your cluster, you need to run the following as a regular user:mkdir -p $HOME/.kubesudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/configsudo chown $(id -u):$(id -g) $HOME/.kube/config... ...这些信息是在告知我们如何配置文件。按照上述命令配置后，节点上的就可以直接使用$HOME/.kube/的信息访问k8s 了。 并且，通过这种配置方式，也拥有了整个集群的管理员(root)权限。当然，如果是二进制部署该步骤也是手动创建好admin.conf文件然后拷贝的。
现在讲解下的kube-是如何对来自的访问进行认证()和授权()，以及具备哪些授权。
3、的认证阶段
通过上述得知，利用文件向发起请求，前面提到过的支持通过tls 、basic auth、token等方式对客户端发起的请求进行身份校验， 从信息来看，显然在请求中使用了tls 的方式，即客户端的证书。
我们获取--data的内容进行证书解码：
echo 'LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM4akNDQWRxZ0F3SUJBZ0lJYi8xa3dCNzZ4TVF3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TVRFeE1Ea3hNelV6TUROYUZ3MHlNakV4TURreE16VXpNRGhhTURReApGekFWQmdOVkJBb1REbk41YzNSbGJUcHRZWE4wWlhKek1Sa3dGd1lEVlFRREV4QnJkV0psY201bGRHVnpMV0ZrCmJXbHVNSUlCSWpBTkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4QU1JSUJDZ0tDQVFFQXcvY0IyK0crRURaWDhveDYKTjMvUk8rQmxFNExUUlMrSzZJZzJrTVJ5WVcxbkZEMHVobDloaTd1OXhzVEt3eVlrdE1OQWFMeFUzbFpLcDAzMApDTnRPdVg4aXFRYVlZOGg0NEc5V3Q1YTNjM00rYitRNnFQZWhwYmhTanVCZnNFU3JOUG5ZTnlnTnZLMFM5T0RPCjdHLzBrVHdJajA3OW9sWS96eEkxQXJBaWVnb2dBbnJ0OUZ0NWZ2eUpuZjYzTW5FdXgyVG5oc2h1YitTc2VSYkgKNU5HWFMzVHNPRXlmQUZqUW95SkRsbkdIU0xLVzJTdVVkODhlVDhCL0x5b0NHN0pNZllmKzVEMjBHN3ROZktKRQpYT2xiZFp0Sldub3pjSTlCMEFjdHlLNWo0NndTOVcyU2V4em9hQVV3VEhDM0Y4Qy9IQlROQ3JrZXpRSU1FQ0hlCnNCYy9Vd0lEQVFBQm95Y3dKVEFPQmdOVkhROEJBZjhFQkFNQ0JhQXdFd1lEVlIwbEJBd3dDZ1lJS3dZQkJRVUgKQXdJd0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFLT2ZCdFlmWTltcFg3TU9FaEdtditCQk1IeTRleVd3eUFzUwpHSUgxSUNmNnd6aXZhVTJjRnRzektON2xkMy9nNXZwVHJSMUV5akJYK1B6SjFPVmdubFU4c0p0L0ZDdHBPWnVkCmhRbU9tejRqeTBRLzErMnZ4TGhqTXlXbmxLYmtBOFhMRDlqVkpIM21iRS9FOXNIVU03WHNCT2Ixbk14TU5MUTMKSXRQSWRQVWd3UXVOS2ErY0diaXdqMmVJMlgwSTE2aDRvLzJhQ2Z6R0VtMHV0bFRIdmczdkQydlZNTlpYV3lTQQowVUxaZVkyeThyQVViN3FpbHp4YmdIMStCUndiOStvSEovdW80UlNURjlXaVFmcFFiN3RCQU40TGlxOFJRTjY5CmU3MEoxVit4cThkSk9PZFBGVUFUWWsxeUdkbi81bDY1dlAzYnNneitaMjNLTGw3ZmxNcz0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=' | base64 -d > 123.crt在认证阶段，会首先使用---ca-file配置的CA证书去验证提供的证书的有效性,我们可以手动认证下证书是否有效 ，到这里为止，认证通过，代表这个请求可以进来了，大门开放
[root@k8s-master ~]# openssl verify -CAfile /etc/kubernetes/pki/ca.crt 123.crt
kubectl.crt: OK除了认证身份，还会取出必要的信息供授权阶段使用，文本形式查看证书内容，我们取出请求对应的用户组或者用户发给授权机制：
[root@k8s-master ~]# openssl x509 -in 123.crt -text
Certificate:Data:Version: 3 (0x2)Serial Number: 8069716883634046148 (0x6ffd64c01efac4c4)Signature Algorithm: sha256WithRSAEncryptionIssuer: CN=kubernetesValidityNot Before: Nov 9 13:53:03 2021 GMTNot After : Nov 9 13:53:08 2022 GMT# 重点是这里，我们可以获取到该请求对应的用户组：system:masters，用户：kubernetes-adminSubject: O=system:masters, CN=kubernetes-adminSubject Public Key Info:Public Key Algorithm: rsaEncryptionPublic-Key: (2048 bit)Modulus:00:c3:f7:01:db:e1:be:10:36:57:f2:8c:7a:37:7f:d1:3b:e0:65:13:82:d3:45:2f:8a:e8:88:36:90:c4:4、的授权阶段
在init初始引导集群启动过程中，创建了许多默认的RBAC规则， 在k8s有关RBAC的官方文档中(使用 RBAC 鉴权 | )，我们看到下面一些 列表:
其中第一个-admin这个 role 绑定了: group，这和环节传递过来的身份信息不谋而合。此处涉及到RBAC的用户组和角色绑定，不做具体阐述了Using RBAC | 。
我们可以看出：请求授权传递的用户组是:，该用户组绑定了-admin的 role角色，并且具有超级用户在平台上的任何资源上执行所有操作。
[root@k8s-master ~]# kubectl describe clusterrolebinding cluster-admin
Name: cluster-admin
Labels: kubernetes.io/bootstrapping=rbac-defaults
Annotations: rbac.authorization.kubernetes.io/autoupdate: true
Role:Kind: ClusterRoleName: cluster-admin
Subjects:Kind Name Namespace---- ---- ---------Group system:masters [root@k8s-master ~]# kubectl describe clusterrole cluster-admin
Name: cluster-admin
Labels: kubernetes.io/bootstrapping=rbac-defaults
Annotations: rbac.authorization.kubernetes.io/autoupdate: true
PolicyRule:Resources Non-Resource URLs Resource Names Verbs--------- ----------------- -------------- -----*.* [] [] [*][*] [] [*]
整个请求过程如图：
三、的认证授权 1、的认证阶段
老规矩，通过v=7查看请求用到的配置文件，从而获取对应的证书等内容，进程通过查看启动参数即可。
继续，查看启动参数的配置文件，获取证书内容，base64解密证书把请求的用户或者用户组先拎出来，然后看看集群有没有利用binding赋予相应的权限。
# 1、读取启动参数配置文件
[root@k8s-master ~]# # vim /etc/kubernetes/kubelet.conf # 2、解密配置文件中的证书内容
[root@k8s-master ~]# echo 'LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM5akNDQWQ2Z0F3SUJBZ0lJWlNLMS8vdCtvM1V3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TVRFeE1Ea3hNelV6TUROYUZ3MHlNakV4TURreE16VXpNRGhhTURneApGVEFUQmdOVkJBb1RESE41YzNSbGJUcHViMlJsY3pFZk1CMEdBMVVFQXhNV2MzbHpkR1Z0T201dlpHVTZhemh6CkxXMWhjM1JsY2pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTUJsblJzMWkxS0UKTWw3ejgvMWN5VTlwczlIZVVsMllOenh0bEl3ZzFDOUtGUXVuMHV0aEMrNk0xNG9lcFNRVkF4cUZjK2dnQXlsMgpXTEFoZ0U5YVdyU0FWZWNjaHdKd0dianFIYmZBd2UvR3JZMGZrUEtMQlZ0UVJ1Znl2TEovQUluNUdqVkZ0Z2UxClVWa1JtZXBkWGxlUTc4bG5rZExvbGRQMThFYU5OcEdqcjY5Q1VSbjI1Zm8xTmpXSjhOZ3JDOHJxY2YxSHZUaUgKWVFWenRwaHh1MEo5b1Z1K2Ftc3h3RU5pRTBOOHdsS29lNG1NRTUvaTJhRHZ4VU9GNjlRMmE1c0QwQzBXVVJMYwppWWd6bWxsZjRuMldHMkxCRnRpRlpYWmdYTkNyMm91MkcvMjlZN0pIR0V4VjdOOFN0S0tvSEFjcm9FWGFhVTQwCmFHS04vRmVsSUxjQ0F3RUFBYU1uTUNVd0RnWURWUjBQQVFIL0JBUURBZ1dnTUJNR0ExVWRKUVFNTUFvR0NDc0cKQVFVRkJ3TUNNQTBHQ1NxR1NJYjNEUUVCQ3dVQUE0SUJBUUJmZnR4RlB6ODl6TURJVGNOVUpTTURxL2c4NzhsVQpKMmphdjZMRzVZNTlWeWQwMzMzd2ZidG5hY0VtZnY1L0paVDFPZXREdHJVMmJnOTA2VllFR3RDR3laM2d6dDY5Ckc2VFZBY3pBS3VZMGlIcVl6UTRIdFVKSEZyZXY1aDduaXRaNmhiTEVJU3l5VUpvMFlIc1FxRmUzYkduOGg2UVYKMzRoalQ5NXEzRUp3SWV2VGM3UWh2dlBoaEpEMjU2elFTYkR5QzN6VWdMamxDcmNONERkWmw1VjRVU1FUWVpKUApuT01WbGhoTkRBOVMrWEJZN09MMDkxQzJaR0dDRHVrSnpMOVgwaUQ1VHlNazBZbnZhb01yQy9Cblg0U2FKSDFUCm5XV0Vud0lwQTlXNzVvZlJ6ZTRCSndkNmk1eExnQ0MrVkpOK0pIaVRkK2VnQ3ZiN3l2cTI5eTR1Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K' | base64 -d > kubelet.crt# 3、验证证书有效性，OK说明认证阶段通过
[root@k8s-master ~]# openssl verify -CAfile /etc/kubernetes/pki/ca.crt kubelet.crt
kubelet.crt: OK# 4、或者kubelet请求的用户组和用户
[root@k8s-master ~]# openssl x509 -in kubelet.crt -text
Certificate:Data:Version: 3 (0x2)Serial Number: 7287587258079552373 (0x6522b5fffb7ea375)Signature Algorithm: sha256WithRSAEncryptionIssuer: CN=kubernetesValidityNot Before: Nov 9 13:53:03 2021 GMTNot After : Nov 9 13:53:08 2022 GMTSubject: O=system:nodes, CN=system:node:k8s-master
2、的授权阶段
K8s会把O作为Group来进行请求，因此如果有权限绑定给这个组，肯定在的定义中可以找得到。因此尝试去找一下绑定了:nodes组的
[root@k8s-master ~]# kubectl get clusterrolebinding|awk 'NR>1{print $1}'|xargs kubectl get clusterrolebinding -oyaml|grep -n10 system:nodes
76- resourceVersion: "172"
77- selfLink: /apis/rbac.authorization.k8s.io/v1/clusterrolebindings/kubeadm%3Anode-autoapprove-certificate-rotation
78- uid: 5d227fdf-57e9-4003-b410-55acf820b765
79- roleRef:
80- apiGroup: rbac.authorization.k8s.io
81- kind: ClusterRole
82- name: system:certificates.k8s.io:certificatesigningrequests:selfnodeclient
83- subjects:
84- - apiGroup: rbac.authorization.k8s.io
85- kind: Group
86: name: system:nodes
87-- apiVersion: rbac.authorization.k8s.io/v1
88- kind: ClusterRoleBinding
89- metadata:
90- creationTimestamp: "2021-11-09T13:53:47Z"
91- name: kubeadm:node-proxier
92- resourceVersion: "200"
93- selfLink: /apis/rbac.authorization.k8s.io/v1/clusterrolebindings/kubeadm%3Anode-proxier
94- uid: 97c4ec4b-f745-4060-977f-77ed94321100
95- roleRef:
96- apiGroup: rbac.authorization.k8s.io[root@k8s-master ~]# kubectl describe clusterrole system:certificates.k8s.io:certificatesigningrequests:selfnodeclient
Name: system:certificates.k8s.io:certificatesigningrequests:selfnodeclient
Labels: kubernetes.io/bootstrapping=rbac-defaults
Annotations: rbac.authorization.kubernetes.io/autoupdate: true
PolicyRule:Resources Non-Resource URLs Resource Names Verbs--------- ----------------- -------------- -----certificatesigningrequests.certificates.k8s.io/selfnodeclient [] [] [create]结局有点意外，除了:.k8s.io::外，没有找到相关的，显然和我们的理解不一样。 尝试去找Using RBAC | 发现了这么一段 :
:kube-
:kube- user
to the by the .
:-
:kube- user
to the by the kube- .
:kube--
:kube-- user
to the by the . The by are in the roles.
:node
None
to by the , read to all , and write to all pod . You use the Node and of the :node role, and allow API to based on the Pods to run on them. The :node role only for with from prior to v1.8.
:node-
:kube-proxy user
to the by the kube-.
大致意思是说：之前会定义:node这个角色，目的是为了可以访问到必要的资源，包括所有的读权限及更新pod状态的写权限。如果1.8版本后，是建议使用 Node and 来代替这个角色的。
我们目前使用1.16，查看一下授权策略：
$ ps axu|grep apiserver kube-apiserver --authorization-mode=Node,RBAC --enable-admission-plugins=NodeRestriction
查看一下官网对Node 的介绍（Using Node | ）：
Node is a - mode that API made by .
In , the node may add or to have the set of to .
In order to be by the Node , must use a that them as being in the :nodes group, with a of :node:
3、授权记住特殊性
如果你看的有点懵，那么记住结论就好了：（1.8版本后）利用证书模式仅作认证，利用NODE模式进行授权，Node 授权器允许 执行 API 操作。而且目前看来，只有利用node授权，其他还是都用RBAC授权。
四、 及K8S Api调用
前面说，认证可以通过证书，也可以通过使用（服务账户）的方式来做认证。大多数时候，我们在基于k8s做二次开发时都是选择通过 + RBAC 的方式。我们之前访问的时候，是如何做的？
目的：创建一个，创建或者使用一个现有的角色与sa绑定，当sa请求时不通过携带证书，而是通过token的方式成功获取集群内容
1、创建并绑定角色
新建一个名为admin的，并且把名为-admin的这个集群角色的权限授予新建的
[root@k8s-master ~]# cat ca-api.yaml
apiVersion: v1
kind: ServiceAccount
metadata:name: adminnamespace: default
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:name: adminannotations:rbac.authorization.kubernetes.io/autoupdate: "true"
roleRef:kind: ClusterRolename: cluster-adminapiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccountname: adminnamespace: default
[root@k8s-master ~]# kubectl create -f ca-api.yaml
serviceaccount/admin created
clusterrolebinding.rbac.authorization.k8s.io/admin created2、查看创建sa的token
默认创建sa都会同时创建一个对应的，我们可以直接查看内容获取token
# 查看创建的sa
[root@k8s-master ~]# kubectl get sa admin -o yaml
apiVersion: v1
kind: ServiceAccount
metadata:creationTimestamp: "2021-11-20T09:24:15Z"name: adminnamespace: defaultresourceVersion: "60267"selfLink: /api/v1/namespaces/default/serviceaccounts/adminuid: 7a2db3d5-84eb-4339-a8cd-4b95115f4688
secrets:
- name: admin-token-bs4sg# 查看创建ca对应的secret，获取token
[root@k8s-master ~]# kubectl get secret
NAME TYPE DATA AGE
admin-token-bs4sg kubernetes.io/service-account-token 3 116s
default-token-pshvd kubernetes.io/service-account-token 3 10d
[root@k8s-master ~]# kubectl describe secret admin-token-bs4sg
Name: admin-token-bs4sg
Namespace: default
Labels:
Annotations: kubernetes.io/service-account.name: adminkubernetes.io/service-account.uid: 7a2db3d5-84eb-4339-a8cd-4b95115f4688Type: kubernetes.io/service-account-tokenData
====
ca.crt: 1025 bytes
namespace: 7 bytes
token: eyJhbGciOiJSUzI1NiIsImtpZCI6IlJYY0FNa3lvdjVtNERqbzJzVzlJM2RTa2dZQ3pxSnI5NDFxUFZHRk9zbGsifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZWZhdWx0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6ImFkbWluLXRva2VuLWJzNHNnIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiN2EyZGIzZDUtODRlYi00MzM5LWE4Y2QtNGI5NTExNWY0Njg4Iiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50OmRlZmF1bHQ6YWRtaW4ifQ.xTyQ9ZJFgQkbQx0TLpuueku2DkQc49S3DP8E_3fDqHES615cBaGof-HU77daywO7YW4vRxTk0MGIvPDarnh04gYH37ZSqUGWAF4uJNjrXsKVjly7sq7o-iS_nJef-1zWxr7FBKYjYMbRobMYHPwMdguz4GBK2S9C9IYCzpifo1ERm36WbA7gA6b6ylET8x_5zbORBhNm7FUDBRbpXLGXCyHlA2o6batMc2cAwHog0hIWYZ0HPvZkSjl7E2uIKiG8QefApdgDgck2jWztSyvBB6QGA8buGcIevws4I78GfGA9hgq0tzPuDTHNdFSAkCY4J3lirdQwKT84yWsgqQg4Rw 3、调用
认证即进入大门，授权即确认可以对资源进行哪些操作，所以API调用需要两点：1、调用资源时需要携带证书或者token等鉴权信息；2、注明请求的API接口地址
token我们已经拿到了，API接口以获取的pod信息为例，利用v=7查看
[root@k8s-master ~]# kubectl get po -v=7
I1120 17:34:32.083451 89976 loader.go:375] Config loaded from file: /root/.kube/config
I1120 17:34:32.088355 89976 round_trippers.go:420] GET https://192.168.0.121:6443/api/v1/namespaces/default/pods?limit=500
I1120 17:34:32.088372 89976 round_trippers.go:427] Request Headers:
I1120 17:34:32.088377 89976 round_trippers.go:431] Accept: application/json;as=Table;v=v1beta1;g=meta.k8s.io, application/json
I1120 17:34:32.088386 89976 round_trippers.go:431] User-Agent: kubectl/v1.16.1 (linux/amd64) kubernetes/d647ddb
I1120 17:34:32.099754 89976 round_trippers.go:446] Response Status: 200 OK in 11 milliseconds
NAME READY STATUS RESTARTS AGE
busybox-5d7b4b65d6-mk6c6 1/1 Running 1 42hhttps://192.168.0.121:6443/api/v1/namespaces/default/pods?limit=500
就是需要请求的API接口
curl: (35) Peer reports incompatible or unsupported protocol version.curl 不兼容或不支持的协议版本centos系统解决方法：yum update -y nss curl libcurl
利用：curl -k -H ": +空格+token+空格+API请求地址"调用成功
[root@k8s-master ~]# curl -k -H "Authorization: Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6IlJYY0FNa3lvdjVtNERqbzJzVzlJM2RTa2dZQ3pxSnI5NDFxUFZHRk9zbGsifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZWZhdWx0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6ImFkbWluLXRva2VuLWJzNHNnIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiN2EyZGIzZDUtODRlYi00MzM5LWE4Y2QtNGI5NTExNWY0Njg4Iiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50OmRlZmF1bHQ6YWRtaW4ifQ.xTyQ9ZJFgQkbQx0TLpuueku2DkQc49S3DP8E_3fDqHES615cBaGof-HU77daywO7YW4vRxTk0MGIvPDarnh04gYH37ZSqUGWAF4uJNjrXsKVjly7sq7o-iS_nJef-1zWxr7FBKYjYMbRobMYHPwMdguz4GBK2S9C9IYCzpifo1ERm36WbA7gA6b6ylET8x_5zbORBhNm7FUDBRbpXLGXCyHlA2o6batMc2cAwHog0hIWYZ0HPvZkSjl7E2uIKiG8QefApdgDgck2jWztSyvBB6QGA8buGcIevws4I78GfGA9hgq0tzPuDTHNdFSAkCY4J3lirdQwKT84yWsgqQg4Rw" https://192.168.0.121:6443/api/v1/namespaces/default/pods?limit=500
{"kind": "PodList","apiVersion": "v1","metadata": {"selfLink": "/api/v1/namespaces/default/pods","resourceVersion": "61327"},"items": [{"metadata": {"name": "busybox-5d7b4b65d6-mk6c6","generateName": "busybox-5d7b4b65d6-","namespace": "default","selfLink": "/api/v1/namespaces/default/pods/busybox-5d7b4b65d6-mk6c6","uid": "7a06c495-bd70-4816-9845-3bd2db69ec7f","resourceVersion": "43135","creationTimestamp": "2021-11-18T14:38:38Z","labels": {"app": "busybox","pod-template-hash": "5d7b4b65d6"},"ownerReferences": [{"apiVersion": "apps/v1","kind": "ReplicaSet","name": "busybox-5d7b4b65d6","uid": "8f36b9d4-9fb2-4642-97da-1dbad70809c3","controller": true,"blockOwnerDeletion": true}]},。。。。。
