上傳到 GitLab 的程式有辦法自行進行測試嗎? 可以透過 CI/CD 的功能.
測試環境
- GitLab – Ubuntu 22.04 Desktop ( 虛擬機 IP:192.168.31.138 )
- Linux SSH Server – CentOS 8 ( 虛擬機 IP:192.168.31.128 )
安裝 GitLab ( 虛擬機 Ubuntu 22.04 Desktop )
關於 GitLab 的安裝請參考 – https://benjr.tw/105652 , 並建立新使用者 ben .
安裝 GitLab CI Runner ( 虛擬機 Ubuntu 22.04 Desktop )
首先須建立 Runner (也就是上傳後的程式可以在哪邊環境來進行測試)
- 安裝 GitLab CI Runner 套件
root@ben-virtual-machine:~# curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh | sudo bash root@ben-virtual-machine:~# sudo apt-get install gitlab-runner
- 註冊 runner
要註冊一個新的 Runner ,須回到 Gitlab 網頁的管理介面 Admin Area / CI/CD / Runners (使用 root Administrator 身分) ,這邊主要是可以產生註冊 (Register) runner 語法
我這邊要產生的 Runner (程式測試環境) 使用 另外一台 CentOS 8 其 Executor 為 SSH .
點選 New Instance Runner 產生 Runner 的語法.
- 首先需要選擇測試環境為何, OS 選擇 Linux
- 接下來是 Tags
之後 .gitlab-ci.yml 設定檔要對應到相對應的 Tags 來決定 Runner / Executor 是哪一個.
- 執行 Register Runner 的語法
剛剛選擇完畢就會出現下面的語法(產生新的 Runner)
主要是 Step 1 即可.
root@ben-virtual-machine:~# gitlab-runner register --url http://192.168.31.138 --token glrt-sx6hmA4FfgUmPn-Kx4Yb Runtime platform arch=amd64 os=linux pid=22890 revision=865283c5 version=16.1.0 Running in system-mode. Enter the GitLab instance URL (for example, https://gitlab.com/): [http://192.168.31.138]: Verifying runner... is valid runner=sx6hmA4Ff Enter a name for the runner. This is stored only in the local config.toml file: [ben-virtual-machine]: VM CentOS8 SSH Enter an executor: ssh, docker-autoscaler, docker+machine, instance, kubernetes, parallels, shell, docker-windows, virtualbox, custom, docker: ssh Enter the SSH server address (for example, my.server.com): 192.168.31.128 Enter the SSH server port (for example, 22): 22 Enter the SSH user (for example, root): root Enter the SSH password (for example, docker.io): 111111 Enter the path to the SSH identity file (for example, /home/user/.ssh/id_rsa): /root/.ssh/id_rsa Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded! Configuration (with the authentication token) was saved in "/etc/gitlab-runner/config.toml"
- 設定皆使用預設值即可.
- 關於 executor 有很多種類,我選擇的是 ssh ( 指定 Runner 透過 ssh 的方式去遠端 server 去執行 CI Job, 須確保環境要包含 如 make , pip , python 等相關套件) , 其他 executor 請參考 – https://chengweichen.com/2021/03/gitlab-ci-executor.html#%E7%9B%AE%E5%89%8D%E5%8F%AF%E9%81%B8%E7%94%A8%E7%9A%84-executor 說明.
- 其他 ssh 設定
須依據遠端可以執行 ssh 的 Linux 環境來設定, 我這邊會使用遠端的 SSH Server 的 root 去執行,其 SSH identity file 為 /root/.ssh/id_rsa (SSH 公私鑰後面說明)
剛剛設定內容皆會寫到 config.toml 檔案去.
root@ben-virtual-machine:~# cat /etc/gitlab-runner/config.toml concurrent = 1 check_interval = 0 shutdown_timeout = 0 [session_server] session_timeout = 1800 [[runners]] name = "VM CentOS8 SSH" url = "http://192.168.31.138" id = 1 token = "glrt-sx6hmA4FfgUmPn-Kx4Yb" token_obtained_at = 2023-07-18T06:05:56Z token_expires_at = 0001-01-01T00:00:00Z executor = "ssh" [runners.cache] MaxUploadedArchiveSize = 0 [runners.ssh] user = "root" password = "111111" host = "192.168.31.128" port = "22" identity_file = "/root/.ssh/id_rsa"
這樣 Runner 就產生好了,其 Tags 為 VM-CentOS8
SSH 設定( 虛擬機 Ubuntu 22.04 Desktop )
GitLab Runner 會透過 SSH 的方式連線到遠端 SSH Server 來進行測試,這邊需先在 GitLab 端先設定好 SSH .關於 SSH 的 公/私金鑰 請參考 – https://benjr.tw/98344
步驟參考 – https://docs.gitlab.com/ee/ci/ssh_keys/
- 透過 ssh-keygen 指令來產生 SSH 的公私鑰.
這邊的 root 是指 SSH Server (我使用 CentOS8 VM) ,並不使用 passhrase (需要一串密碼來解開私鑰) 保護私鑰 .root@ben-virtual-machine:~# ssh-keygen -t rsa -C root Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): Created directory '/root/.ssh'. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_rsa Your public key has been saved in /root/.ssh/id_rsa.pub The key fingerprint is: SHA256:1cEJjl1edVL8BQwe5yxk+81PYB9O9VA6jlvBI3PNSrU root The key's randomart image is: +---[RSA 3072]----+ | ygfghj *B| | kj BoX.BB| | hgy . +k &E*| | . oytu | | kjj S . ==| | ,m. .| | uyi ..| | ghj | | | +----[SHA256]-----+
- 設定 CI 時指定 SSH private key 檔案位置.
前面設定 CI 時有指定 SSH private key 檔案的位置.identity_file = "/root/.ssh/id_rsa" - Run the ssh-agent during job to load the private key.
這應該是 Docker 才需要? - 匯入 public key 到 SSH Server
並在 .gitlab-ci.yml 加上以下兩行,後面會在 GitLab Client 看到.before_script: - echo "$SSH_KNOWN_HOSTS" >> ~/.ssh/known_hosts - chmod 644 ~/.ssh/known_hosts
登入 SSH Server (讓 GitLab 這邊端先產生 Key 檔案 )
root@ben-virtual-machine:~# ssh root@192.168.31.128 The authenticity of host '192.168.31.128 (192.168.31.128)' can't be established. ED25519 key fingerprint is SHA256:gVQMC2PcfkTPQruPWXi+2MVa3WbsfCPOti7NuCNSsSw. This key is not known by any other names Are you sure you want to continue connecting (yes/no/[fingerprint])? yes Warning: Permanently added '192.168.31.128' (ED25519) to the list of known hosts. root@192.168.31.128's password: Activate the web console with: systemctl enable --now cockpit.socket Last login: Mon Jul 17 22:52:24 2023 from 192.168.31.1 [root@localhost ~]# exit logout Connection to 192.168.31.128 closed.
Linux SSH Server ( 虛擬機 CentOS 8 )
在 這邊端需要安裝 git 指令, python (執行程式碼環境)
安裝 git
[root@localhost ~]# yum install git
[root@localhost ~]# unset SSH_ASKPASS
安裝 python
[root@localhost ~]# yum install python39
因為 GitLab 會透過 SSH 進行連線,並匯入 public Key 到 .ssh/known_hosts (預設檔案不存在)
[root@localhost ~]# mkdir .ssh [root@localhost ~]# touch .ssh/known_hosts
GitLab Client
我們可以開始從 Client 端,將程式上傳至 GitLab , 需有以下步驟.
初始化(只需要做一次)
- git config
設定使用者帳號與 Mail .root@ben-virtual-machine:~# git config --global user.name "Ben" root@ben-virtual-machine:~# git config --global user.email "ben@gmail.com"
- git init
設定哪一個目錄是需要透過 git 來保管.root@ben-virtual-machine:~# mkdir runner_code root@ben-virtual-machine:~# cd runner_code
並做初始化設定,會產生 .git 目錄來記錄相關設定.
root@ben-virtual-machine:~/runner_code# git init hint: Using 'master' as the name for the initial branch. This default branch name hint: is subject to change. To configure the initial branch name to use in all hint: of your new repositories, which will suppress this warning, call: hint: hint: git config --global init.defaultBranch <name> hint: hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and hint: 'development'. The just-created branch can be renamed via this command: hint: hint: git branch -m <name> Initialized empty Git repository in /root/test_code/.git/
- 程式碼
建立一個程式碼.root@ben-virtual-machine:~/runner_code# vi hello.py import sys print(f'Hello {sys.argv[1]}')需要透過 runner 執行程式需有 .gitlab-ci.yml 這個檔案.
root@ben-virtual-machine:~/runner_code# vi .gitlab-ci.yml before_script: - echo "$SSH_KNOWN_HOSTS" >> ~/.ssh/known_hosts - chmod 644 ~/.ssh/known_hosts stages: - test Run-Test: stage: test tags: - VM-CentOS8 script: - python3 hello.py Ben說明:
在 before_script 匯入 public key 到 SSH Server
before_script: - echo "$SSH_KNOWN_HOSTS" >> ~/.ssh/known_hosts - chmod 644 ~/.ssh/known_hosts
要執行那些 stage
stages: - test
區塊名稱
Run-Test:
Stage 的名稱
stage: test
透過哪一個 runner 來執行.
tags: - VM-CentOS8執行指令
script: - python3 hello.py Ben有新增或是修改程式與資料後都需要 add , commit 與 push
- git add 與 commit
並透過 git add 與 commit 將程式交給 git 保管. git add 的 “.” 代表現有目錄所有檔案,也可以直接指定檔案名稱.root@ben-virtual-machine:~/runner_code# git add .
root@ben-virtual-machine:~/test_code# git commit -m "First Code" [master (root-commit) 938dfba] First Code 1 file changed, 27 insertions(+) create mode 100644 test.cpp
須注意這邊程式交給 Local 端而已,需要下面步驟從能上傳到 GitLab. 這邊使用 ben 這個使用者 (需事先至 GitLab 網頁建立) 來建立專案( Repository 預設權限為 Private,需到 網頁變更 project visibility 為 public).
- git remote 與 push
同一個目錄的 Project 只須設定一次 git remote 即可,之後可以透過 git remote show origin 檢視其設定.root@ben-virtual-machine:~/runner_code# git remote add origin http://192.168.31.138/ben/runner_test.git
root@ben-virtual-machine:~/runner_code# git push -u origin master Username for 'http://192.168.31.138': ben Password for 'http://ben@192.168.31.138': Enumerating objects: 4, done. Counting objects: 100% (4/4), done. Delta compression using up to 4 threads Compressing objects: 100% (3/3), done. Writing objects: 100% (4/4), 379 bytes | 379.00 KiB/s, done. Total 4 (delta 0), reused 0 (delta 0), pack-reused 0 remote: remote: remote: The private project ben/runner_test was successfully created. remote: remote: To configure the remote, run: remote: git remote add origin http://192.168.31.138/ben/runner_test.git remote: remote: To view the project, visit: remote: http://192.168.31.138/ben/runner_test remote: remote: remote: To http://192.168.31.138/ben/runner_test.git * [new branch] master -> master Branch 'master' set up to track remote branch 'master' from 'origin'.
程式與資料已上傳完畢.
使用 ben 身分登入 GitLab 網頁,並檢視 runner_test 專案的 CI/CD Pipiline 或是 Jobs 看程式是否有自行執行.
沒有解決問題,試試搜尋本站其他內容