1,457 瀏覽數

CoreOS 新增/移除 etcd2 Cluster member

新增/移除 etcd2 Cluster member 參考文章 – https://coreos.com/etcd/docs/latest/etcd-live-cluster-reconfiguration.html#change-etcd-cluster-size
ETCD2 Cluster members

  1. CoreOS1 (Node1) , IP: 172.16.15.21 (Leader)
  2. CoreOS2 (Node2) , IP: 172.16.15.22
  3. CoreOS3 (Node3) , IP: 172.16.15.23

  1. CoreOS1 (Node1) , IP: 172.16.15.21

    前面 http://benjr.tw/96404 已經裝好單一台的 CoreOS .設定檔如下

    #cloud-config
    hostname: coreos1
    ssh_authorized_keys:
      - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5wZYPD/mBs+9O9CrUxdg9kpOus24VrMuNncdt4BRc4iF5npV90HYe5j/y3IG6+2MRbAb2edyf/FUcaJHN/V+i123456yuqyAT2rv9T0eB2+wpmYCUQzqZscJP2uLK8jMhezKWS0l7X5CgJf+d17VooS6CADR9MyTbku3upKp5yEnsCfB+pBLGdrqCUTnGHPfJcLTBIvuMriz/kae0azxcderfbw7YWR8oKdWjKYKlznnBmH6VYFcgv/jSXbRbdZjKNSXIm2xIj6TIIJmo6sWhptcGohi467ODyrzCDioXD1MsYx6ImTMcY5mzL2RDePAW7CM4gWIMaIxDeL5e10SX ben@appledeAir
    
    coreos:
      units:
        - name: etcd2.service
          command: start
        - name: systemd-networkd.service
          command: stop
        - name: 00-eth0.network
          runtime: true
          content: |
            [Match]
            Name=ens32
    
            [Network]
            Address=172.16.15.21/24
            Gateway=172.16.15.2
            DNS=168.95.1.1
        - name: systemd-networkd.service
          command: start
      etcd2:
        name: "node01"
        discovery: https://discovery.etcd.io/9dd875ca6dd759d67445a681adde3875
        advertise-client-urls: http://172.16.15.21:2379
        initial-advertise-peer-urls: http://172.16.15.21:2380
        listen-client-urls: http://0.0.0.0:2379
        listen-peer-urls: http://172.16.15.21:2380
    
  2. CoreOS2 (Node2), IP: 172.16.15.22

    設定檔如下,與 CoreOS1 不同的是名稱跟 IP.

    #cloud-config
    hostname: coreos2
    ssh_authorized_keys:
      - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5wZYPD/mBs+9O9CrUxdg9kpOus24VrMuNncdt4BRc4iF5npV90HYe5j/y3IG6+2MRbAb2edyf/FUcaJHN/V+i123456yuqyAT2rv9T0eB2+wpmYCUQzqZscJP2uLK8jMhezKWS0l7X5CgJf+d17VooS6CADR9MyTbku3upKp5yEnsCfB+pBLGdrqCUTnGHPfJcLTBIvuMriz/kae0azxcderfbw7YWR8oKdWjKYKlznnBmH6VYFcgv/jSXbRbdZjKNSXIm2xIj6TIIJmo6sWhptcGohi467ODyrzCDioXD1MsYx6ImTMcY5mzL2RDePAW7CM4gWIMaIxDeL5e10SX ben@appledeAir
    
    coreos:
      units:
        - name: etcd2.service
          command: start
        - name: systemd-networkd.service
          command: stop
        - name: 00-eth0.network
          runtime: true
          content: |
            [Match]
            Name=ens32
    
            [Network]
            Address=172.16.15.22/24
            Gateway=172.16.15.2
            DNS=168.95.1.1
        - name: systemd-networkd.service
          command: start
      etcd2:
        name: "node02"
        discovery: https://discovery.etcd.io/9dd875ca6dd759d67445a681adde3875
        advertise-client-urls: http://172.16.15.22:2379
        initial-advertise-peer-urls: http://172.16.15.22:2380
        listen-client-urls: http://0.0.0.0:2379
        listen-peer-urls: http://172.16.15.22:2380
    

    第二台一樣透過設定檔來安裝 CoreOS ,後面會再用指令 #etcdctl member add 的方式去加入.

    core@localhost ~ $ # sudo coreos-install -d /dev/sda -C stable -c ~/cloud-config.yaml
    2016/12/21 09:41:12 Checking availability of "local-file"
    2016/12/21 09:41:12 Fetching user-data from datasource of type "local-file"
    Downloading the signature for https://stable.release.core-os.net/amd64-usr/1185.3.0/coreos_production_image.bin.bz2...
    2016-12-21 09:41:14 URL:https://stable.release.core-os.net/amd64-usr/1185.3.0/coreos_production_image.bin.bz2.sig [543/543] -> "/tmp/coreos-install.fmCj9mKD5k/coreos_production_image.bin.bz2.sig" [1]
    Downloading, writing and verifying coreos_production_image.bin.bz2...
    ...
    Success! CoreOS stable 1185.3.0 is installed on /dev/sda
    core@localhost ~ $ sudo reboot
    

    新增 Node2
    先確定 etcd2 的服務是否正確.

    core@coreos2 ~ $ sudo systemctl status etcd2
    ● etcd2.service - etcd2
       Loaded: loaded (/usr/lib/systemd/system/etcd2.service; disabled; vendor prese
      Drop-In: /run/systemd/system/etcd2.service.d
               └─20-cloudinit.conf
       Active: active (running) since Wed 2017-01-18 08:25:43 UTC; 1min 37s ago
     Main PID: 954 (etcd2)
        Tasks: 6
       Memory: 17.3M
          CPU: 334ms
       CGroup: /system.slice/etcd2.service
               └─954 /usr/bin/etcd2
    
    Jan 18 08:25:39 coreos2 etcd2[954]: setting maximum number of CPUs to 1, total n
    Jan 18 08:25:39 coreos2 etcd2[954]: listening for peers on http://172.16.15.22:2
    Jan 18 08:25:39 coreos2 etcd2[954]: listening for client requests on http://0.0.
    Jan 18 08:25:42 coreos2 etcd2[954]: stopping listening for client requests on ht
    Jan 18 08:25:42 coreos2 etcd2[954]: stopping listening for peers on http://172.1
    Jan 18 08:25:42 coreos2 etcd2[954]: discovery cluster full, falling back to prox
    Jan 18 08:25:43 coreos2 etcd2[954]: proxy: using peer urls [http://172.16.15.21:
    Jan 18 08:25:43 coreos2 systemd[1]: Started etcd2.
    Jan 18 08:25:43 coreos2 etcd2[954]: endpoints found ["http://172.16.15.21:2379"]
    Jan 18 08:25:43 coreos2 etcd2[954]: proxy: listening for client requests on http
    

    在 Node1 透過下面指令來觀察一下 Cluster 的狀態,可以看到目前 Cluster 只正常的,但是只有第一台存在.

    core@coreos1 ~ $ etcdctl cluster-health
    member 74a1e2b4597f8a15 is healthy: got healthy result from http://172.16.15.21:2379
    cluster is healthy
    core@coreos1 ~ $ etcdctl member list
    74a1e2b4597f8a15: name=node01 peerURLs=http://172.16.15.21:2380 clientURLs=http://172.16.15.21:2379 isLeader=true
    

    在 CoreOS1-Node1 透過指令把 CoreOS2-Node2 加入到 Cluster.

    core@coreos1 ~ $ etcdctl member add node02 http://172.16.15.22:2380
    Added member named node02 with ID d53fc9960b85d9cc to cluster
    
    ETCD_NAME="node02"
    ETCD_INITIAL_CLUSTER="node01=http://172.16.15.21:2380,node02=http://172.16.15.22:2380"
    ETCD_INITIAL_CLUSTER_STATE="existing"
    

    看一下 node2 的狀態,怎麼狀態是 unhealthy , unstarted

    core@coreos1 ~ $ etcdctl cluster-health
    member 74a1e2b4597f8a15 is unhealthy: got unhealthy result from http://172.16.15.21:2379
    member d53fc9960b85d9cc is unreachable: no available published client urls
    cluster is unhealthy
    core@coreos1 ~ $ etcdctl member list
    74a1e2b4597f8a15: name=node01 peerURLs=http://172.16.15.21:2380 clientURLs=http://172.16.15.21:2379 isLeader=true
    d53fc9960b85d9cc[unstarted]: peerURLs=http://172.16.15.22:2380
    

    雖然已經加入到 Cluster 但需要重新載入 Node2 設定值,加入 Cluster 的工作才算結束.

    core@coreos2 ~ $ sudo vi /run/systemd/system/etcd2.service.d/99-restore.conf
    [Service]
    ExecStartPre=/usr/bin/rm -rf /var/lib/etcd2/proxy
    Environment="ETCD_DISCOVERY="
    Environment="ETCD_NAME=node02"
    Environment="ETCD_INITIAL_CLUSTER=node01=http://172.16.15.21:2380,node02=http://172.16.15.22:2380"
    Environment="ETCD_INITIAL_CLUSTER_STATE=existing"
    

    服務重啟

         
    core@coreos2 ~ $ sudo systemctl daemon-reload
    core@coreos2 ~ $ sudo systemctl restart etcd2
    

    檢查一下,可以看到 Cluster 狀態是正常的.

    core@coreos2 ~ $ etcdctl member list
    74a1e2b4597f8a15: name=node01 peerURLs=http://172.16.15.21:2380 clientURLs=http://172.16.15.21:2379 isLeader=true
    d53fc9960b85d9cc: name=node02 peerURLs=http://172.16.15.22:2380 clientURLs=http://172.16.15.22:2379 isLeader=false
    core@coreos2 ~ $ etcdctl cluster-health
    member 74a1e2b4597f8a15 is healthy: got healthy result from http://172.16.15.21:2379
    member d53fc9960b85d9cc is healthy: got healthy result from http://172.16.15.22:2379
    cluster is healthy
    
  3. CoreOS3 (Node3), IP: 172.16.15.23

    設定檔如下,與其他 Node 不同的是名稱跟 IP.

    #cloud-config
    hostname: coreos3
    ssh_authorized_keys:
      - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5wZYPD/mBs+9O9CrUxdg9kpOus24VrMuNncdt4BRc4iF5npV90HYe5j/y3IG6+2MRbAb2edyf/FUcaJHN/V+i123456yuqyAT2rv9T0eB2+wpmYCUQzqZscJP2uLK8jMhezKWS0l7X5CgJf+d17VooS6CADR9MyTbku3upKp5yEnsCfB+pBLGdrqCUTnGHPfJcLTBIvuMriz/kae0azxcderfbw7YWR8oKdWjKYKlznnBmH6VYFcgv/jSXbRbdZjKNSXIm2xIj6TIIJmo6sWhptcGohi467ODyrzCDioXD1MsYx6ImTMcY5mzL2RDePAW7CM4gWIMaIxDeL5e10SX ben@appledeAir
    
    coreos:
      units:
        - name: etcd2.service
          command: start
        - name: systemd-networkd.service
          command: stop
        - name: 00-eth0.network
          runtime: true
          content: |
            [Match]
            Name=ens32
    
            [Network]
            Address=172.16.15.23/24
            Gateway=172.16.15.2
            DNS=168.95.1.1
        - name: systemd-networkd.service
          command: start
      etcd2:
        name: "node03"
        discovery: https://discovery.etcd.io/9dd875ca6dd759d67445a681adde3875
        advertise-client-urls: http://172.16.15.23:2379
        initial-advertise-peer-urls: http://172.16.15.23:2380
        listen-client-urls: http://0.0.0.0:2379
        listen-peer-urls: http://172.16.15.23:2380
    
  4. 加入方式如同 Node2
    在 CoreOS1-Node1 透過指令把 CoreOS3-Node3 加入到 Cluster.

    core@coreos1 ~ $ etcdctl member add node03 http://172.16.15.23:2380
    Added member named node03 with ID 48852466e20da1b8 to cluster
    
    ETCD_NAME="node03"
    ETCD_INITIAL_CLUSTER="node01=http://172.16.15.21:2380,node03=http://172.16.15.23:2380,node02=http://172.16.15.22:2380"
    ETCD_INITIAL_CLUSTER_STATE="existing"
    

    雖然已經加入到 Cluster 但需要重新載入 node3 設定值,加入 Cluster 的工作才算結束.

    core@coreos3 ~ $ sudo vi /run/systemd/system/etcd2.service.d/99-restore.conf
    [Service]
    ExecStartPre=/usr/bin/rm -rf /var/lib/etcd2/proxy
    Environment="ETCD_DISCOVERY="
    Environment="ETCD_NAME=node03"
    Environment="ETCD_INITIAL_CLUSTER=node01=http://172.16.15.21:2380,node02=http://172.16.15.22:2380,node03=http://172.16.15.23:2380"
    Environment="ETCD_INITIAL_CLUSTER_STATE=existing"
    

    服務重啟

         
    core@coreos3 ~ $ sudo systemctl daemon-reload
    core@coreos3 ~ $ sudo systemctl restart etcd2
    

    檢查一下,可以看到 Cluster 狀態是正常的.

    core@coreos3 ~ $ etcdctl member list
    etcdctl member list
    74a1e2b4597f8a15: name=node01 peerURLs=http://172.16.15.21:2380 clientURLs=http://172.16.15.21:2379 isLeader=true
    48852466e20da1b8: name=node03 peerURLs=http://172.16.15.23:2380 clientURLs=http://172.16.15.23:2379 isLeader=false
    d53fc9960b85d9cc: name=node02 peerURLs=http://172.16.15.22:2380 clientURLs=http://172.16.15.22:2379 isLeader=false
    core@coreos3 ~ $ etcdctl cluster-health
    member 74a1e2b4597f8a15 is healthy: got healthy result from http://172.16.15.21:2379
    member 48852466e20da1b8 is healthy: got healthy result from http://172.16.15.23:2379
    member d53fc9960b85d9cc is healthy: got healthy result from http://172.16.15.22:2379
    cluster is healthy
    
  • 測試資料庫

    在 Node1 新增一筆 資料

    core@coreos1 ~ $ etcdctl set /test "CoreOS testing"
    CoreOS testing
    

    的確在 Node2 有看到這一筆資料了.

    core@coreos2 ~ $ etcdctl get /test 
    CoreOS testing
    
    
  • 移除 Node

    那要移除 node 很簡單在 CoreOS1 – Node 執行

    core@coreos1 ~ $ etcdctl member remove 48852466e20da1b8
    

6 Replies to “CoreOS 新增/移除 etcd2 Cluster member”

  1. 自動參照通知: CoreOS – Flannel – Benjr.tw

  2. 自動參照通知: CoreOS etcd2 Cluster 的容錯能力 – Benjr.tw

  3. 自動參照通知: 安裝 CoreOS – 設定 etcd2 – Benjr.tw

  4. 自動參照通知: CoreOS – Fleet – Benjr.tw

  5. 自動參照通知: CoreOS 設定檔 – Benjr.tw

  6. 自動參照通知: CoreOS – etcd2 Cluster 的災難復原 – Benjr.tw

發表迴響