打造你的CI/CD管道:在Kubernetes上快速建立Jenkins

2024/01/29閱讀時間約 21 分鐘

將基礎的Kubernetes cluster建置之後,最多的用法就是建置CI/CD管道(pipeline),完整的將開發與部署的整個流程串連起來。

raw-image

本篇將說明如何建立一個基本的Jenkins,並且在架構上,將Jenkins Agent以動態的方式在有需要的方式進行產生,執行結束後又會自動將Agent Pod刪除,不會造成系統內有一大堆沒有作用的Pod耗用有限的資源。

最後也在Jenkins Web UI上建立一個最基本的Pipeline,簡單驗證本篇建立出來的Jenkins engine是可以正常運行。

照慣例,本篇章節如下:

  1. 架構說明
  2. 部署
  3. 安裝Plugins
  4. Jenkins Master設定
  5. 測試:執行單一Job
  6. 測試:執行Pipeline
  7. 結論

1.架構說明

傳統的Jenkins 一對多的架構會有以下問題:

  1. 如果Master出現問題,整個流程就無法運作
  2. 每個Slave的環境設定不同,當要完全不同語言的打包動作時,不同的環境配置對管理就相對不方便
  3. 資源分配不均,有些Slave JOB必須要排隊等待,但有的是空閒狀態,最後導致資源浪費

所以透過運行Jenkins master/slave在k8s cluster之內,可以很好的處理上述問題。

做法:Master 將設定資料存儲到volume內,Slave運行在各個節點上,且不是一直運行,是有需要才動態產生,結束時自動刪除。

這種方式解決了:

  1. 如果Jenkins master故障時,由k8s自動再重新建立一個Jenkins master,並且將volume分配給新建的master,資料不會丟失。
  2. 每運行一個Job才會建立一個slave,完成後就會自動刪除容器,自動釋放資源。同時由K8S自動分配資源到適合空閒的節點,避免過度集中在某些節上運行的排隊情況。
  3. 如果整個資源都不夠了,K8S直接再加入一個節點就可以解決。

2.部署

實際部署的方式有二種:

  • 全部都在K8S內運行 (本篇使用)
  • Master在K8S外部運行,所有執行的Slave在K8S內產生
#---------------------------------------------------
# S2-1. create namespace
#---------------------------------------------------
[master]# kubectl create ns jenkinspoc
#---------------------------------------------------
# S2-2. create service account
#---------------------------------------------------
[master]# vim jenkinspoc-sa.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: jenkins-admin
namespace: jenkinspoc
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: jenkins
namespace: jenkinspoc
labels:
"app.kubernetes.io/name": 'jenkins'
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
resources: ["pods/exec"]
verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
resources: ["pods/log"]
verbs: ["get","list","watch"]
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: jenkins-role-binding
namespace: jenkinspoc
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: jenkins
subjects:
- kind: ServiceAccount
name: jenkins-admin
namespace: jenkinspoc

[master]# kubectl create -f jennkinspoc-sa.yaml -n jenkinspoc
serviceaccount/jenkins-admin created
role.rbac.authorization.k8s.io/jenkins created
rolebinding.rbac.authorization.k8s.io/jenkins-role-binding created

[master]# kubectl get sa -n jenkinspoc
#---------------------------------------------------
# S2-3. 建立給master存放資料的volume
#---------------------------------------------------
[master]# vim jenkinspoc-pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: jenkins-pv-claim
spec:
storageClassName: managed-nfs-storage
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi

[master]# kubectl create -f jenkinspoc-pvc.yaml -n jenkinspoc
[master]# kubectl get pvc -n jenkinspoc
raw-image
#---------------------------------------------------
# S2-4. 部署Jenkins master
#---------------------------------------------------
[master]# vim jenkinspoc-deploy.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: jenkins-deployment
spec:
replicas: 1
selector:
matchLabels:
app: jenkins
template:
metadata:
labels:
app: jenkins
spec:
serviceAccountName: jenkins-admin
securityContext:
fsGroup: 1000
runAsUser: 1000
containers:
- name: jenkins
image: jenkins/jenkins:lts
resources:
limits:
memory: "2Gi"
cpu: "1000m"
requests:
memory: "500Mi"
cpu: "500m"
ports:
- name: httpport
containerPort: 8080
- name: jnlpport
containerPort: 50000
livenessProbe:
httpGet:
path: "/login"
port: 8080
initialDelaySeconds: 90
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 5
readinessProbe:
httpGet:
path: "/login"
port: 8080
initialDelaySeconds: 60
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
volumeMounts:
- name: jenkins-data
mountPath: /var/jenkins_home
volumes:
- name: jenkins-data
persistentVolumeClaim:
claimName: jenkins-pv-claim

[master]# kubectl create -f jenkinspoc-deploy.yaml -n jenkinspoc
[master]# kubectl get pod -n jenkinspoc
#---------------------------------------------------
# S2-5. create svc
#---------------------------------------------------
[master]# vim jenkinspoc-svc.yaml
apiVersion: v1
kind: Service
metadata:
name: jenkins-service
annotations:
prometheus.io/scrape: 'true'
prometheus.io/path: /
prometheus.io/port: '8080'
spec:
selector:
app: jenkins
type: NodePort
ports:
- name: httpport
port: 8080
targetPort: 8080
nodePort: 32003
- name: jnlpport
port: 50000
targetPort: 50000

[master]# kubectl create -f jenkinspoc-svc.yaml -n jenkinspoc
[master]# kubectl get all
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/jenkins-service NodePort 10.109.168.87 <none> 8080:32003/TCP,50000:30779/TCP 3sf
#---------------------------------------------------
# S2-6. Obtain admin passowrd
#---------------------------------------------------
[master]# kubectl get pod -n jenkinspoc
[master]# kubectl exec jenkins-deployment-bdf778654-br6mv -n jenkinspoc -- cat /var/jenkins_home/secrets/initialAdminPassword
b1a8117e9a364353855ebfe03b3308be

http://worker01.test.example.poc:32003
admin/P@ssw0rd

3.安裝Plugins

#---------------------------------------------
# S3-1. Install Suggestion plugin
#---------------------------------------------
raw-image
#---------------------------------------------
# S3-2. Install kubernetes plugin
#---------------------------------------------
Manage Jenkins > Plugins > Available plugins > "Kubernetes" > Install
raw-image

4.Jenkins Master設定

#---------------------------------------------
# S4-1. create cloud
#---------------------------------------------
Manage Jenkins > Clouds > New cloud > Name: K8s-pipeline (勾選Kubernetes) > Create
#---------------------------------------------
# S4-2. create credentials
# 因為全部都部署在同一座k8s,透過ServiceAccount的授權
#---------------------------------------------
Kubernetes URL : 免填
Certification key: 免填
Kubernetes namespace: jenkinspoc
=> test connection
raw-image
#---------------------------------------------
# S4-3. Jenkins URL
#---------------------------------------------
[syntax] http://<service-name>.<namespace>.svc.cluster.local:8080
Jenkins URL: http://jenkins-service.jenknspoc.svc.cluster.local:8080

=> SAVE
#---------------------------------------------
# S4-4. Pod template
#---------------------------------------------
Name: jenkins-agent
namespace: jenkinspoc
Labels: jenkinsagent <== Job會套用這個Label來找到要用那個Pod配置來建立
Containers:
(如果可以連到DockerHub,可以使用預設的jenkins/inbound-agent)
(以下使用自訂的jnlp,並且覆蓋預設值)
(1) Name: jnlp
(2) Image: jenkins/inbound-agent:latest
(3) 移除"Sleep" , "99999" <== 不然會改寫預設的Entrypoint
(4) Service Account : jenkins-admin

5.測試:執行單一Job

#---------------------------------------------
# S5-1. 建立Job
#---------------------------------------------
Dashboard > New item > name: jenkins-job-1 (free style project) > OK
Label express: jenkinsagent
Build Step > Excute Shell > echo "Test Jenkins JOB Successfully." > SAVE

> Build Now
raw-image
raw-image
raw-image
raw-image

6.測試:執行Pipeline

#---------------------------------------------
# S6-1. 建立Pipeline
#---------------------------------------------
Dashboard > New item > pipeline (name: pipeline-test) > OK
raw-image
#---------------------------------------------
# S6-2. Pipeline script
#---------------------------------------------
node('jenkinsagent') {
stage('Clone') {
echo "1.Clone Stage"
}
stage('Test') {
echo "2.Test Stage"
}
stage('Build') {
echo "3.Build Stage"
}
stage('Deploy') {
echo "4. Deploy Stage"
}
}

> SAVE > Build Now
raw-image
raw-image
raw-image

7.結論

基本上許多使用K8S架構的組織,都會配合CI/CD的各種工具來實際運作,本篇所說的Jenkins便是其中一個廣泛使用的工具之一,並且許多工作的技能需求也要求K8S工程師也要負責維護CI/CD Pipeline的順利運作,所以建議學習K8S平台的各位,至少也要掌握一個Pipeline的工具建置、操作、維護的技能。

本篇只演示了CI/CD中的其中一個階段,有興趣的話,可以先了解整個CI/CD完整的循環有那些階段與相關的工具。例如先前提到的Harbor registry就是循環內的另一個階段。

那今天就先到此為止,下篇再見囉 !!


References:

8會員
39內容數
記錄IT社畜的自我學習筆記,如同專題名稱,主要是怕自已忘記自已做過什麼、學到什麼。索性就分享我自已在學習Kubernetes這條路上的各種測試、學習心得。
留言0
查看全部
發表第一個留言支持創作者!