Kubernetes Advanced RBAC Authorization Security Framework

K8S Security Mechanism

  1. Security Framework for Kubernetes
  2. Transport security, authentication, authorization, access control
  3. With RBAC authorization, how do we as a user authorize different colleagues to access the cluster, such as development colleagues, which resources and namespaces to access, test which colleagues can access, and how do we restrict that?

I. K8S Security Framework

* Access to K8S cluster resources requires three levels: authentication, authentication, access control
* Ordinary users often need a certificate, Token, or username + password to access cluster API Server securely; Pod access, such as Deshboard of the Ingres controller Ui, requires a ServiceAccount, primarily to allow this container to access the API, that is, all interactions are through the API, which may go through one personInteracting through kubectl may also allow your program to call APIs, but these are authorized

* The K8S security control framework is mainly controlled by the following three phases, each of which supports plug-in mode and enables plug-ins through API Server configuration.

  1. Authentication Certification
  2. Authorization Authentication
  3. Admission Control Access Control

In the previous words, that is, every stage is a plug-in design, you can develop plug-ins by yourself and integrate them into the steps to achieve related access control, so you do not need to modify the original code to add, so many of the k8s design principles are designed with extensibility.It's possible to let users customize something and integrate it.

Next, let's look at a diagram, which is some of the stages that we've gone through in accessing the API

From the above kubectl, API, UI, access is the k8s API, k8s API provides many interfaces

These are all provided by Apiserver and support different functions to complete related processing, related certification, and then down there are three layers of authorization built into the API. The first layer is authentication, the second layer is authentication, and the third layer is access control. After passing, you can access related resources. These resourcesSources are called from ETCD, some information about storage status

Transmission security:
Now k8s has been changed to https for access, that is, whether you are a kubeadm deployment or a binary deployment, it is recommended that you use https for full stack communication, say goodbye to 8080, use 6443

When the API receives a request from a user, it will authenticate first, and there are three ways to authenticate it
Three types of client authentication:

* HTTPS Certificate Authentication: Digital Certificate Authentication based on CA Certificate Signature, i.e. k8s, a CA-checked certificate can be used as the certificate that your client access carries, it will help you authenticate, which is a way to extract from this certificate whether you have permission to access.
* HTTP Token authentication: Users are identified by a Token.
* HTTP Base authentication: User name + password authentication, which is based on http's own authentication, but this is rarely used because of the low security factor.

The first level is to identify which certificate you came in with, or token, to see if I am trusted here, to see if I have this token I have the relevant authorization here, that is, if I have this token here, if it is not allowed to pass, then it will not pass below, if it passesIf, for example, this token has already been created locally, let you go and make the next judgment, that is, the second level authorization.

To grant authorization:
RBAC (Role-Based Access Control): Responsible for completing Authorization.That is, you will see that your access character does not meet your permissions, so it will judge you here. If you come here with this identity, you will not be allowed to access this resource without this permission.

Authorized resources have many types of support

Access Control:
Simply speaking, it is to develop some advanced functions that are directly plugged-in to design, that is, access controller is a collection of plug-ins. There are some advanced features in the collection that are implemented by plug-ins. If you do not enable these plug-ins, you will not be able to use this function, which is the third level.That is, your request will go through your plug-in access control. Access control will validate the way the request is implemented. Whether this plug-in is enabled or not, but most default plug-ins are enabled. Requesting related resources after startup is allowed because it is enabledPlug-in will not pass if it is not started

Adminssion Control is actually a list of access controller plug-ins in which requests sent to API Server go through each access controller. If the plug-in checks fail, the request is rejected.

Recommended plug-ins for versions 1.11 and above:
--enable-admission-plugins= \

Use RBAC Authorization
RBAC (Role-Based Access Control), which allows dynamic configuration of policies through the Kubernetes API.That is, even if it is configured, it takes effect immediately and does not require restarting the service

Role: Authorize access to a specific namespace
* ClusterRole: Grant access to all namespaces

Role Binding
* RoleBinding: Bind roles to principals (that is, subject)
* ClusterRoleBinding: Bind cluster roles to principals

* User: User
* Group: User groups
* ServiceAccout: Service Account
To make a permission management system, there are two parts, the first one is who the object is, the user created, and the second one is the permission group, such as creating a development group, what permissions the development group has, which permissions can access the system. This is also to facilitate the management of these permissions, to divide this permission group, each user gives itCreating a permission is cumbersome. If you have this group, just join the user into the group. For example, open it up and set permissions for him. He is a group of guest users. This group can only see some things, but he is a development group and has some permissions for some projects.Administrator goes in and adds a development group to this administration page so that he has the rights to the development group

K8s and just mentioned are the same. The user is the principal, who accesses it, and then the permission group is the role, which defines a set of permissions. If the user wants to make an additional set of users and permissions, he will add a permission group to the development, which is called role binding. The roles are divided into corners.Color and cluster role, which authorizes the cluster namespace.
That is, k8s has a namespace, so it is divided into a single namespace and all namespaces. The setting of this permission set, ClusterRole, authorizes all namespaces. In other words, adding a user to this permission role means that it can access all namespaces, related to othersJurisdiction.

Example: Authorize default namespace Pod read permissions for the zhaocheng user
For example, you can only view some resources running in the default space of the pod, such as svc, the log is not authorized to view, when your trial period has expired, add some privileges to you, and then do some related operations
There are three steps to achieve this goal

  1. Issue Client Certificate with K8S CA
  2. Generate kubeconfig authorization file
  3. Create RBAC Permission Policy
    Ca.crt and Ca.key are the two certificates that need to be issued.

    [root@k8s-master ~]# ls /etc/kubernetes/pki/
    apiserver.crt              apiserver.key                 ca.crt  front-proxy-ca.crt      front-proxy-client.key
    apiserver-etcd-client.crt  apiserver-kubelet-client.crt  ca.key  front-proxy-ca.key      sa.key
    apiserver-etcd-client.key  apiserver-kubelet-client.key  etcd    front-proxy-client.crt  sa.pub
  4. Issue Client Certificate with K8S CA
    [root@k8s-master ~]# cd demo/
    [root@k8s-master demo]# mkdir rbac
    [root@k8s-master demo]# cd rbac/
    [root@k8s-master rbac]# rz -E
    rz waiting to receive.
    [root@k8s-master rbac]# ls
    cert.sh  cfssl.sh  config.sh  rbac.yaml
    [root@k8s-master rbac]# cat cfssl.sh 
    wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64
    wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64
    wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64
    chmod +x cfssl*
    mv cfssl_linux-amd64 /usr/bin/cfssl
    mv cfssljson_linux-amd64 /usr/bin/cfssljson
    mv cfssl-certinfo_linux-amd64 /usr/bin/cfssl-certinfo
    [root@k8s-master rbac]# sh cfssl.sh 

Then here we take the script of our certificate issuance. Note here that the CNs of the issuing user refers to the user name of the issuing user, that is, k8s takes this CA to authenticate. It not only validates the certificate issued by me, but also the user name inside, that is, the CNs field is authorized, which is equivalent.In the authentication section, O is the user group, or you can do this permission restriction based on this user group.

[root@k8s-master rbac]# chmod +x cert.sh 
[root@k8s-master rbac]# vim cert.sh 
cat > ca-config.json <<EOF
  "signing": {
    "default": {
      "expiry": "87600h"
    "profiles": {
      "kubernetes": {
        "usages": [
            "client auth"
        "expiry": "87600h"

cat > zhaocheng-csr.json <<EOF
  "CN": "zhaocheng",
  "hosts": [],
  "key": {
    "algo": "rsa",
    "size": 2048
  "names": [
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "k8s",
      "OU": "System"

cfssl gencert -ca=/etc/kubernetes/pki/ca.crt -ca-key=/etc/kubernetes/pki/ca.key -config=ca-config.json -profile=kubernetes zhaocheng-csr.json | cfssljson -bare zhaocheng

[root@k8s-master rbac]# bash cert.sh 

The primary generation is zhaocheng-key.pem zhaocheng.pem

[root@k8s-master rbac]# ls
ca-config.json  cert.sh  cfssl.sh  config.sh  rbac.yaml  zhaocheng.csr  zhaocheng-csr.json  zhaocheng-key.pem  zhaocheng.pem

You need these two certificates to request authentication from the client, write them into the client authorization file, and use this authorization file to request this k8s cluster
For example, if your company has multiple clusters, then this development needs to log in to this cluster, need this config certificate, and then you can switch back and forth to log in to different clusters

[root@k8s-master rbac]# vim config.sh 
kubectl config set-cluster kubernetes \
  --certificate-authority=/etc/kubernetes/pki/ca.crt \
  --embed-certs=true \
  --server= \

//Set up client authentication
kubectl config set-credentials zhaocheng \
  --client-key=zhaocheng-key.pem \
  --client-certificate=zhaocheng.pem \
  --embed-certs=true \

//Set Default Context
kubectl config set-context kubernetes \
  --cluster=kubernetes \
  --user=zhaocheng \

//Set current usage configuration
kubectl config use-context kubernetes --kubeconfig=zhaocheng.kubeconfig

[root@k8s-master rbac]# bash config.sh 
[root@k8s-master rbac]# cat zhaocheng.kubeconfig 

Now you can access the cluster with this kubeconfig
This is actually a call back to the authentication place. node does not have access rights, it has been recognized as a zhaocheng user, but authentication does not have relevant authorization, so now it is necessary to authorize this user

[root@k8s-master rbac]# kubectl get --kubeconfig=./zhaocheng.kubeconfig node
Error from server (Forbidden): nodes is forbidden: User "zhaocheng" cannot list resource "nodes" in API group "" at the cluster scope

If you have a group namespace with more permissions, you have access to all namespaces. For example, Operations, there are many operations, and you can create a cluster role so that it is the role that controls all namespaces. This role is specific to a specific namespace, such as development, testing, and it onlyA project that has access to a namespace, the project it is responsible for, and can then use the specified namespace, then a binding to the cluster, and then to the user zhaocheng, where I define a user or a group.

pod is also inaccessible

[root@k8s-master rbac]# kubectl get --kubeconfig=./zhaocheng.kubeconfig pod
Error from server (Forbidden): pods is forbidden: User "zhaocheng" cannot list resource "pods" in API group "" in the namespace "default"
[root@k8s-master rbac]# vim rbac.yaml 
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
  namespace: default
  name: pod-reader
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "watch", "list"]


kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
  name: read-pods
  namespace: default
- kind: User
  name: zhaocheng
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io

[root@k8s-master rbac]# kubectl create -f rbac.yaml 

Then you can access it now

[root@k8s-master rbac]# kubectl get --kubeconfig=./zhaocheng.kubeconfig pod
NAME                                     READY   STATUS             RESTARTS   AGE
my-pod                                   1/1     Running            0          10h
nfs-744d977b46-dh9xj                     1/1     Running            0          29h
nfs-744d977b46-kcx6h                     1/1     Running            0          29h
nfs-744d977b46-wqhc6                     1/1     Running            0          29h
nfs-client-provisioner-fbc77b9d4-kkkll   1/1     Running            0          11h
nginx-797db8dc57-tdd5s                   1/1     Running            0          8h
nginx-a1-6d5fd7b8dd-w647x                1/1     Running            0          4h55m
nginx-statefulset-0                      1/1     Running            0          7h7m
nginx-statefulset-1                      1/1     Running            0          7h6m
nginx-statefulset-2                      1/1     Running            0          7h6m
web-0                                    1/1     Running            0          4h55m
web-1                                    1/1     Running            0          4h53m
web-2                                    1/1     Running            0          4h52m

But svc is undefined and cannot be accessed yet

[root@k8s-master rbac]# kubectl get --kubeconfig=./zhaocheng.kubeconfig svc
Error from server (Forbidden): services is forbidden: User "zhaocheng" cannot list resource "services" in API group "" in the namespace "default"

For example, let's add another access right, such as service,deployment
Add these permissions to our rbac policy just now

- apiGroups: [""]
  resources: ["pods","services"]
  verbs: ["get", "watch", "list"]

Test again and you will be able to access svc normally

[root@k8s-master rbac]# kubectl get --kubeconfig=./zhaocheng.kubeconfig service
NAME         TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)        AGE
kubernetes   ClusterIP      <none>        443/TCP        30h
my-service   ClusterIP   None          <none>        80/TCP         7h22m
nginx        ClusterIP   None          <none>        80/TCP         4h59m
service      NodePort   <none>        80:30963/TCP   8h
zhao         ClusterIP   <none>        80/TCP         7h31m
zhaocheng    ClusterIP   <none>        80/TCP         7h33m

Like other authorizations, you can view official offers

Tags: Linux Kubernetes JSON Nginx vim

Posted on Sat, 07 Sep 2019 17:16:19 -0700 by Orpheus13