Kubernetes

---

Timeline

Timeline

2025-10-14

init

This article introduces the basic concepts, core features, and cluster architecture components of Kubernetes, summarizing the functions of Master and Node nodes as well as core concepts such as Pod, Controller, and Service. Additionally, it discusses methods for building clusters using kubeadm and binary packages, along with the kubectl command-line tool, YAML file configuration standards, and Pod implementation mechanisms and resource management strategies.

---

Kubernetes Overview and Features

K8s Overview

• K8s is a containerized cluster management system open-sourced by Google in 2014
• Deploy containerized applications using K8s
• Using K8s facilitates application scaling
• The goal of K8s is to make deploying containerized applications simpler and more efficient

K8s Features

1. Automatic Bin Packing
2. Self-Healing
3. Horizontal Scaling
4. Service Discovery
5. Rolling Updates
6. Version Rollback
7. Secret and Configuration Management (Similar to Hot Deployment)
8. Storage Orchestration
9. Batch Processing

K8s Cluster Architecture Components

Master (Control Node) and Node (Worker Node)

(1) Master Components

• apiserver

Unified cluster entry point, stored in etcd via RESTful API

• scheduler

Node scheduling, selecting node for application deployment

• controller-manager

Handles routine background tasks in the cluster, one resource corresponds to one controller

• etcd

Storage system for saving cluster-related data

(2) Node Components

• kubelet

Representative dispatched from master to node, manages local containers

• kube-proxy

Provides network proxy, implements load balancing and other operations

K8s Core Concepts

1. Pod

• Minimum deployment unit
• A collection of containers
• Shared network
• Lifecycle is ephemeral

1. Controller

• Ensure the desired number of pod replicas
• Stateless application deployment
• Stateful application deployment
• Ensure all nodes run the same pod
• One-time tasks and scheduled tasks

1. Service

• Define access rules for a set of pods

• **Master (control plane node)😗*The machine that controls Kubernetes nodes and is where job tasks are created.

• **Node:**These machines execute assigned tasks under the control of the Kubernetes master node.

• **Pod：**A set of one or more containers deployed as a single unit to a single node. Containers in the same pod share IP addresses, inter-process communication (IPC), hostname, and other resources. Pods abstract the underlying container network and storage, making container migration within the cluster more convenient.

• **Replication controller:**Controls the number of instances of a pod running on the cluster.

• **Service:**Separates service content from specific pods. The Kubernetes service proxy automatically distributes service requests to the correct pod, regardless of where the pod moves in the cluster, even if it is replaced.

• **Kubelet：**This daemon runs on each worker node, responsible for obtaining the container list and ensuring that the declared containers are started and running normally.

• **kubectl：**This is the command-line configuration tool for Kubernetes.

Building a K8s cluster

1. Environment platform planning

• Single-master cluster

Single-master cluster

Disadvantage: If the master goes down, it’s game over.

• Multi-master cluster

Multi-master cluster

High-availability cluster

1. Server hardware configuration requirements

• Test environment:

master: 2 cores 4G 20G

node: 4 cores 8G 40G

• Production environment:

Higher requirements

1. Methods for building k8s cluster deployment

• kubeadm

A k8s deployment tool that provides kubeadm init and kubeadm join for quickly deploying Kubernetes clusters

• Binary packages

Download the release binary packages from GitHub, each component must be deployed separately

Building with kubeadm

https://kubernetes.io/zh-cn/docs/setup/production-environment/tools/kubeadm/install-kubeadm/

# Set server hostname
$ hostnamectl set-hostname <hostname>
$ yum install -y kubelet-1.18.0 kubeadm-1.18.0 kubectl-1.18.0
# Enable startup on boot
$ systemctl enable kubelet
# Deploy Kubernetes Master, execute on the master node
# Since the default image pull address k8s.gcr.io is inaccessible in China, specify the Alibaba Cloud image registry address here
$ kubeadm init \
--apiserver-advertise-address=192.168.31.61 \
--image-repository registry.aliyuncs.com/google_containers \
--kubernetes-version v1.17.0 \
--service-cidr=10.96.0.0/12 \
--pod-network-cidr=10.244.0.0/16
# Use the kubectl tool
$ mkdir -p $HOME/.kube
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config
# View current nodes
$ kubectl get nodes
# Join a Kubernetes Node
# Execute on 192.168.31.62/63 (Node nodes, not the master node)
# Add a new node to the cluster by executing the kubeadm join command output from kubeadm init
$ kubeadm join 192.168.31.61:6443 --token esce21.q6hetwm8si29qxwn \
--discovery-token-ca-cert-hash
sha256:00603a05805807501d7181c3d60b478788408cfe6cedefedb1f97569708be9c5
# Install Pod network plugin
$ kubectl apply –f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
# Test Kubernetes cluster
# Create a pod in the Kubernetes cluster to verify it is running correctly:
$ kubectl create deployment nginx --image=nginx
$ kubectl expose deployment nginx --port=80 --type=NodePort
$ kubectl get pod,svc

1. Install 3 virtual machines and install the operating system
2. Initialize the three installed operating systems
3. Install Docker, kubelet, kubeadm, and kubectl on the three nodes, and change the Docker source
4. Execute the kubeadm init command on the master node to initialize
5. Execute the kubeadm join command on the node nodes to add them to the current cluster

Binary package setup

Binary package setup

1. Create multiple virtual machines and install the Linux operating system
2. Operating system initialization
3. Self-signed certificates for etcd and apiserver
4. Deploy etcd cluster
5. Deploy master components

kube-apiserver,kube-controller-manager,kube-scheduler,etcd

1. Deploy node components

kubelet,kube-proxy,docker,etcd

1. Deploy cluster network

Kubernetes cluster command-line tool kubectl

kubectl is the command-line tool for Kubernetes clusters, allowing you to manage the cluster itself and install and deploy containerized applications on the cluster

$ kubectl [command] [type] [name] [flags]
$ kubectl --help

YAML file description

File writing format

• Indentation indicates hierarchical relationships

• Do not use the Tab key for indentation

• Add a space after the colon

• Generally indent two spaces at the beginning

• Indent one space after a character

• — indicates the start of a new YAML file

• Use # to represent comments

YAML file example

YAML file example

Field	Description
apiVersion	API version
kind	Resource type
metadata	Resource metadata
spec	Resource specification
replicas	Number of replicas
selector	Label selector
template	Pod template
metadata	Pod metadata
spec	Pod specification
container	Container Configuration

How to Quickly Write a YAML File

1. Usekubectl createCommand to Generate a YAML File

$ kubectl create deployment web --image=nginx -o yaml --dry-run > web1.yaml

Kubernetes Core Technology

Pod

Pod Overview

1. A Pod is the smallest unit that can be created (deployed) and managed in the k8s system

2. k8s does not directly handle containers, but rather Pods; a Pod can contain multiple containers (a collection of containers)

3. A Pod shares the network namespace

4. Each Pod has a special Pause container called the “root container.” The image corresponding to the Pause container is part of the k8s platform. In addition to the Pause container, each Pod also contains one or more closely related user business containers.

5. Pods are ephemeral

The Purpose of Pods

1. Creating containers uses Docker; one Docker corresponds to one container, one container has one process, and one container runs one application
2. Pods are designed for multi-process, allowing multiple applications to run; a Pod has multiple containers, and each container runs one application
3. Pods exist for intimate interaction:
   • Interaction between two applications
   • Calling between networks
   • Two applications need frequent calls

Pod implementation mechanism

1. Shared network

Containers themselves are isolated from each other. Kubernetes uses the Pod’s Pause container (info container) to add other business containers into the Pause container, so that all business containers are in the same namespace, enabling network sharing.

2. Shared storage

Pod persistent data: log data, business data

Use Volume for persistent storage

volumes

Pod image pull policy

imagePullPolicy

imagePullPolicy

IfNotPresent: default value, pull the image only if it does not exist on the host

Always: pull the image every time a Pod is created

Never: Pod will never actively pull this image; it needs to be pulled manually

Pod resource limits

Pod resource limits

resources

1c = 1000m (1 core CPU)

Pod Restart Mechanism

restartPolicy

restartPolicy

Always: Always restart the container after it terminates; default policy

OnFailure: Restart the container only when it exits abnormally (exit code non-zero)

Never: Never restart the container when it terminates

Pod Health Check

Container Check:

Cannot detect Java heap memory overflow (status remains running)

Application-Level Health Check:

Application-Level Health Check

echo $? indicates whether a command executed successfully on Linux

Pod Creation Process

Pod Creation Process

• Master Node

createpod – apiserver – etcd

Scheduler -> API Server -> etcd -> Scheduling algorithm schedules the pod to a specific node

• node

kubelet --apiserver --reads etcd to get pods assigned to the current node --docker creates containers

attributes affecting scheduling

1. pod resource limits: resources

2. node selector labels affect Pod scheduling

nodeSelector

nodeSelector

need to label the node first

$ kubectl label node k8snode1 env_role=prod
$ kubectl get nodes k8snode1 --show-labels

3. node affinity affects Pod scheduling

node affinity

node affinitynodeAffinity is basically the same as the previous nodeSelector, it determines which nodes Pods are scheduled to based on node label constraints

(1) hard affinity (requireDuringSchedulingIgnoreDuringExecution)

constraints must be satisfied

(2) soft affinity (preferredDuringSchedulingIgnoredDuringExecution)

attempt to satisfy, not guaranteed

commonly used operators:

In NotIn Exists Gt Lt DoesNotExists

Anti-affinity: Using NotIn and DoesNotExist

4. Taints and Tolerations

• Basic Introduction:

nodeSelector and nodeAffinity: Pods are scheduled to certain nodes, Pod attributes, implemented during scheduling

Taint: Nodes are not used for normal scheduling, it is a node attribute

• Scenarios

Dedicated Nodes

Configuring Nodes with Special Hardware

Eviction Based on Taints

# View the taints of the current node
$ kubectl describe node k8smaster | grep Taint

There are three taint effects:

NoSchedule: Will not be scheduled

PreferNoSchedule: Try not to be scheduled

NoExecute: Will not be scheduled, and will also evict existing Pods on the Node

Add taint to node

kubectl taint node [node] key=value:Three taint values

$ kubectl get pods
$ kubectl create deployment web --image=nginx
$ kubectl get pods -o wide
# Create 4 more
$ kubectl scale deployment web --replicas=5
$ kubectl taint node [node] key=value
# Delete pod
$ kubectl delete deployment web
$ kubectl get pods
No resources found in default namespace
$ kubectl taint node k8snode1 env_role=yes:NoSchedule
$ kubectl describe node k8snode1 | grep Taint

Delete taint

$ kubectl taint node k8snode1 env_role:NoSchedule-
$ kubectl describe node k8snode1 | grep Taint

Taint toleration:

tolerations

Controller

What is a Controller

An object that manages and runs containers on the cluster

Relationship between Pod and Controller

• Pod achieves application operations such as scaling and rolling updates through Controller

  • Relationship between Pod and Controller is established through labels

  

  Pod and Controller establish relationship through labels

Relationship between Pod and Controller

Deployment use cases

• Deploy stateless applications
• Manage Pods and ReplicaSets
• Deployment, rolling update, and other features

Use cases: web services, microservices

Deploy applications using Deployment (YAML)

# Export YAML file
$ kubectl create deployment web --image=nginx --dry-run -o yaml > web.yaml
# Deploy using YAML
$ kubectl apply -f web.yaml
$ kubectl get nodes
# Publish externally, expose ports
$ kubectl expose deployment web --port=80 --type=NodePort --target-port=80 --name=web1 -o yaml > web1.yaml
$ kubectl apply -f web1.yaml
$ kubectl get nodes -o wide

Application upgrade, rollback, and auto-scaling

# Application upgrade
$ kubectl set image deployment web nginx=nginx:1.15
# Check upgrade status
$ kubectl rollout status deployment web
# View historical versions
$ kubectl rollout history deployment web
# Rollback to the previous version
$ kubectl rollout undo deployment web
# Auto-scaling
$ kubectl scale deployment web --replicas=10

Difference between stateless and stateful

1. Stateless
   • Consider all Pods to be identical (replicas are identical)
   • No ordering requirements
   • No need to consider which node to run on
   • Freely scale and expand
2. Stateful
   • All the above factors need to be considered
   • Make each pod independent, maintain pod startup order and uniqueness (unique network identifier, persistent storage, ordered, such as MySQL master-slave)

Deploy stateful applications

• Headless service:
  • ClusterIP: node

StatefulSetDeploy stateful applications

StatefulSet deploys stateful applications

StatefulSet deploys stateful applications

After execution, check the pods; there are 3 Pods, each with a unique name

Check svc, ClusterIP is None

Difference between deployment and statefulset: statefulset has identity (unique identifier)

• Generate domain name based on hostname + certain rules
• Unique domain name

Format: hostname.service name.namespace.svc.cluster.local

example: nginx-statefulset-0.nginx.default.svc

Deploy DaemonSet

• Run one pod on each node, newly added nodes also run a pod
• Example: Install data collection tool on each node

Deploy DaemonSet

$ kubectl delete statefulset --all
$ kubectl delete svc nginx
$ kubectl delete svc web
$ kubectl apply -f ds.yaml
$ kubectl exec -it ds-test-cbk6cv bash

Job (one-time task) and CronJob (scheduled task)

job

$ kubectl create -f job.yaml
$ kubectl get pods -o wide
$ kubectl get jobs
pi-qpqff Completed
$ kubectl logs pi-qpqff
# Delete
$ kubectl delete -f job.yaml

Scheduled task:

CronJob

$ kubectl apply -f cronjob.yaml
$ kubectl get pods
$ kubectl get cronjobs
$ kubectl logs hello-1599100140-wkn79

Service

What is a Service

Define access rules for a group of Pods

Purpose of Service

• Prevent Pod disconnection (service discovery)

  

  Service discovery

• Define a set of Pod access policies (load balancing)

Load balancing

Relationship between Pod and Service

Establish relationship based on label tags

Pod and Service establish relationship through labels

Common Service types

1. ClusterIP: used within the cluster
2. NodePort: used for external access to applications
3. LoadBalancer: used for external access to applications, public cloud

$ kubectl get svc
$ kubectl expose deployment web --port=80 --target-port=80 --dry-run -o yaml > service1.yaml
$ kubectl apply -f service1.yaml
$ kubectl get svc

Applications deployed on nodes in the internal network are generally not accessible from the external network:

Use a machine with external network access, install nginx, reverse proxy

Manually add accessible nodes to nginx

LoadBalancer: public cloud, load balancer, controller

Configuration management

Secret

Function: Encrypt data stored in etcd, allowing Pod containers to access it by mounting a Volume

Scenario: Credentials

Base64 encoding

$ echo -n "admin" | base64

1. Create secret encrypted data

Create secret encrypted data

$ kubectl create -f secret.yaml
$ kubectl get secret

2. Mount into the pod container as environment variables

valueFrom

valueFrom

$ kubectl apply -f secret-val.yaml
$ kubectl get pods
$ kubectl exec -it mypod bash
$ echo $SECRET_USERNAME

3. Mount into a volume

Mount into a volume

Mount into a volume

$ kubectl delete -f secret-val.yaml
$ kubectl apply -f secret-vol.yaml
$ kubectl get pods
$ kubectl exec -it mypod bash
$ ls /etc/foo
$ cat /etc/foo/username

ConfigMap

Purpose: Store unencrypted data in etcd, allowing Pods to mount it into containers as environment variables or volumes

Scenario: Configuration files

1. Create a configuration file

$ kubectl delete secret --all
$ kubectl delete Pod --all
$ vim redis.properties

2. Create a ConfigMap

$ kubectl create configmap redis-config --from-file=redis.properties
# View
$ kubectl get cm
$ kubectl describe cm redis-config

3. Mount into the Pod container as a volume

Mount into the Pod container as a volume

$ kubectl apply -f cm.yaml
# View logs
$ kubectl logs mypod

4. Mount as variables

(1) Create YAML, declare variable information, create ConfigMap

Declare variables

(2) Mount as variables

$ kubectl apply -f myconfig.yaml
$ kubectl get cm

Mount as variables

$ kubectl apply -f config-var.yaml
$ kubectl get pods
$ kubectl get cm
$ kubectl logs mypod

K8s cluster installation mechanism

Overview

1. Accessing a K8s cluster requires three steps to complete specific operations

(1) Authentication

(2) Authorization

(3) Admission control

2. When accessing, the process must go through the apiserver, which acts as a unified coordinator, like a gatekeeper. During access, certificates, tokens, or username+password are required. If accessing a pod, a serviceAccount is needed.

First step: Authentication

• Transport security: Port 8080 is not exposed externally, only accessible internally; port 6443 is used externally.
• Authentication: Common methods for client identity authentication:
  • HTTPS certificate authentication, based on CA certificate
  • HTTP token authentication, identifying users via token
  • HTTP basic authentication, username + password authentication

Step 2: Authorization

• Perform authorization operations based on RBAC
• Role-based access control

Step 3: Admission Control

• It is the list of admission controllers; if the list has content, it passes

RBAC: Role-Based Access Control

Role Based Access Control

1. Role

• Role: Access permissions for a specific namespace

• ClusterRole: Access permissions for all namespaces

# View namespaces
$ kubectl get ns
# Create namespace
$ kubectl create ns roletest

2. Role Binding

• roleBinding: Bind a role to a subject
• ClusterRoleBinding: Bind a cluster role to a subject

3. Subject

• user: User
• group: Group
• serviceAccount: Service Account

RBAC Implements Authorization

# 1. Create a Namespace
$ kubectl create ns roledemo
# 2. Create a Pod in the New Namespace
$ kubectl run nginx --image=nginx -n roledemo

RBAC Implements Authorization

# 3. Create a Role
$ vim rbac-role.yaml
$ kubectl apply -f rbac-role.yaml
# View Roles in the Namespace
$ kubectl get role -n roledemo

RBAC Implements Authorization

# 4. Create a Role Binding
$ vim rbac-rolebing.yaml
$ kubectl apply -f rbac-rolebinding.yaml
$ kubectl get role,rolebinding -n roledemo

5. Use Certificate

Use Certificate

$ vim rbac-user.sh
$ cp /root/TLS/k8s/ca* ./
$ bash rbac.sh
$ kubectl get pods -n roledemo

Ingress

Overview

1. Expose the port number externally, access via IP + port number, implemented using NodePort in Service
2. NodePort Limitations

• A port is opened on each node; access is achieved through any node via the node’s IP and the exposed port
• This means each port can only be used once, with one port corresponding to one application
• In practice, domain names are used for access, redirecting to different port services based on different domain names

Relationship between Ingress and Pod

• Pods and Ingress are associated through Service
• Ingress acts as a unified entry point, with Service linking a group of Pods

Ingress

Use Ingress

Expose Applications Externally Using Ingress

1. Create an nginx application and expose the port externally

$ kubectl create deployment web --image=nginx
$ kubectl get pods
$ kubectl get deploy
$ kubectl expose deployment web --port=80 --target-port=80 --type=NodePort
$ kubectl get svc

2. Deploy Ingress Controller

Deploy ingress controller

$ kubectl apply -f ingress-con.yaml
# Check ingress controller status
$ kubectl get pods -n ingress-nginx
# Create ingress rules

Create ingress rules

$ vim ingress-h.yaml
$ kubectl apply -f ingress-h.yaml
$ kubectl get pods -n ingress-nginx -o wide
$ kubectl get ing

Helm

Overview

Basic process of deploying applications previously:

Write yaml files: deployment, Service, Ingress

It is suitable to deploy a single application or a few services using the previous method

If deploying a microservices project with dozens of services, each service has its own set of yaml files, requiring maintenance of a large number of yaml files, making version management very inconvenient

What problems can using Helm solve?

1. Using Helm, these yaml files can be managed as a whole
2. Achieve efficient reuse of yaml
3. Use Helm for application-level version management

Helm introduction

Helm is a package management tool for Kubernetes, similar to package managers like yum/apt on Linux, making it easy to deploy pre-packaged yaml files to Kubernetes

Three important concepts

• helm

  • Is a command-line client tool

• Chart

  • Is packaging yaml, a collection of yaml

• Release

  • Deploy entities based on charts, application-level version management

Helm v3 architecture

https://helm.sh/docs/intro/quickstart/

Install

https://helm.sh/docs/intro/install/

Add repository

$ helm repo add brigade https://brigadecore.github.io/charts
"brigade" has been added to your repositories
$ helm search repo brigade

Use Helm to quickly deploy applications

1. Search for applications using commands

$ helm search repo 名称

2. Select and install based on search results

$ helm install 安装之后的名称 搜索之后的名称

3. Check the status after installation

$ helm list
$ helm status 安装后的名称

example

$ helm search repo weave
$ helm install ui stable/weave-scope
$ kubectl get pods
$ kubectl get svc
# Found that the port is not exposed, need to change to nodePort
$ kubectl edit svc ui-weave-scope

How to create your own Chart

1. Create a chart using commands

$ helm create mychart
$ cd mychart

• Chart.yaml: Configuration information of the current chart
• templates: Write yaml files and place them in this directory
• values.yaml: Global variables that can be used by yaml files

2. Create two yaml files under the templates folder

• deployment.yaml
• service.yaml

$ kubectl create deployment web1 --images=nginx --dry-run -o yaml > deployment.yaml
$ kubectl create deployment web1 --image=nginx
$ kubectl expose deployment web1 --port=80 --target-port=80 --type=NodePort --dry-run -o yaml > service.yaml
$ kubectl delete deployment web1

3. Install mychart

$ helm install web1 mychart/
$ kubectl get pods
$ kubectl get svc

4. Application Upgrade

$ helm upgrade chart名称 chart文件夹
# example
$ helm upgrade web1 mychart/

Achieve efficient reuse of yaml

Dynamically render templates by passing parameters, generating yaml content with dynamically passed parameters

1. Define variables and values in values.yaml
2. Retrieve defined variables and values in specific yaml files

• Yaml files generally differ in these few places
  • image
  • tag
  • label
  • port
  • replicas

1. Define variables and values in values.yaml

replicas: 1
image: nginx
tag: 1.16
label: nginx
port: 80

2. Use variables defined in values.yaml in the yaml files under templates

• Use global variables in expression form

{{.Values.变量名称}}

For example

{{ .Release.Name}}-deploy

{{.Values.image}}

$ helm install --dry-run web2 mychart/

Persistent storage

The data volume emptydir is local storage. When a pod restarts, the data no longer exists, so persistent storage is needed.

NFS network storage

When a pod restarts, the data still exists.

Step 1: Find a server as the NFS server.

(1) Install NFS

$ yum install -y nfs-utils

(2) Set the mount path

$ vim /etc/exports
/data/nfs *(rw,no_root_squash)

(3) The external mount path needs to be created first.

$ mkdir /data/nfs

Step 2: Install NFS on the node nodes of the Kubernetes cluster.

$ yum install -y nfs-utils

Step 3: Start the NFS service on the NFS server.

$ systemctl start nfs
$ ps -elf | grep nfs

Step 4: Deploy applications in the Kubernetes cluster using NFS persistent network storage.

$ mkdir pv
$ cd pv
$ vim nfs-nginx.yaml

Using NFS persistent network storage

$ kubectl describe pod nginx-dep1-79x79jg79-9sn8gx

PV and PVC

1. PV: Persistent storage, which abstracts storage resources and provides externally callable interfaces (producer).
2. PVC: Users call it without needing to care about internal implementation details (consumer).
3. Implementation process:

PV and PVC

$ vim pvc.yaml

pvc.yaml

pvc.yaml

$ kubectl apply -f pvc.yaml

$ vim pv.yaml

pv.yaml

$ kubectl apply -f pv.yaml
$ kubectl get pv,pvc
$ kubectl get pods
$ kubectl exec -it nginx-dep1-79u99x9g68s bash
$ ls /usr/share/nginx/html

K8s cluster resource monitoring

Monitoring metrics

1. Cluster monitoring
   • Node resource utilization
   • Number of nodes
   • Running pods
2. Pod monitoring
   • Container metrics
   • Application

Monitoring platform

prometheus+Grafana

(1) prometheus

• Open source
• Monitoring, alerting, database
• Periodically fetch the status of monitored components via HTTP protocol
• No complex inheritance process required, just use the HTTP interface to connect

(2) Grafana

• Open-source data analysis and visualization tool
• Supports multiple data sources

Deployment:

https://developer.aliyun.com/article/836300

Building a high-availability cluster

High-availability cluster

High-availability cluster technology

High-availability cluster technology

Deployment

Deployment

https://kubernetes.io/zh-cn/docs/setup/production-environment/tools/kubeadm/high-availability/

Deploying a Java project

Container delivery process

Container delivery process

Container delivery process