[Hyperviseur] Créer un cluster Proxmox - iZero

Article original Publié le : 29 juin 2022
Mise à jour le : –

Toujours curieux de savoir comment ca fonctionne, je test un cluster d’hyperviseur Proxmox sous VirtualBox

Les pré-requis pour un hyperviseur Proxmox a des fins de test sont :

Minimum Hardware (for testing only)

CPU: 64bit (Intel EMT64 or AMD64)
Intel VT/AMD-V capable CPU/Mainboard (for KVM Full Virtualization support)
Minimum 1 GB RAM
Hard drive
One NIC

Les pré-requis pour un cluster Proxmox sont :

Requirements

All nodes must be able to connect to each other via UDP ports 5404 and 5405 for corosync to work.
Date and time must be synchronized.
An SSH tunnel on TCP port 22 between nodes is required.
If you are interested in High Availability, you need to have at least three nodes for reliable quorum. All nodes should have the same version.
We recommend a dedicated NIC for the cluster traffic, especially if you use shared storage.
The root password of a cluster node is required for adding nodes.
Online migration of virtual machines is only supported when nodes have CPUs from the same vendor. It might work otherwise, but this is never guaranteed.

Labs

Les specs des vms sont :

1cpu
2GO de ram
20GO de disque (SSD)
Une interface réseau pour l’administration de Proxmox dans le subnet 192.168.3.0/24.
Une seconde interface réseau pour le trafic du cluster et du stockage partagé dans le subnet 192.168.100.0/24.

Configuration réseau pour les hyperviseurs Proxmox

Machine 1: prox1.local

La première interface sera liée au bridge vmbr0 avec l’adresse 192.168.3.60/24
La seconde interface sera liée au bridge vmbr1 avec l’adresse 192.168.100.60/24

Machine 2: prox2.local

La première interface sera liée au bridge vmbr0 avec l’adresse 192.168.3.61/24
La seconde interface sera liée au bridge vmbr1 avec l’adresse 192.168.100.61/24

Machine 3: prox3.local

La première interface sera liée au bridge vmbr0 avec l’adresse 192.168.3.62/24
La seconde interface sera liée au bridge vmbr1 avec l’adresse 192.168.100.62/24

Cela devrait ressembler a cette capture d’ecran

Création du cluster

Depuis la première machine, lancer la commande pvecm create avec un nom pour identifier le cluster

# pvecm create cluster-proxmox

1	# pvecm create cluster-proxmox

Corosync Cluster Engine Authentication key generator.
Gathering 2048 bits for key from /dev/urandom.
Writing corosync key to /etc/corosync/authkey.
Writing corosync config to /etc/pve/corosync.conf
Restart corosync and cluster filesystem

Corosync Cluster Engine Authentication key generator.

Gathering 2048 bits for key from /dev/urandom.

Writing corosync key to /etc/corosync/authkey.

Writing corosync config to /etc/pve/corosync.conf

Restart corosync and cluster filesystem

Puis vérifier son status

# pvecm status

1	# pvecm status

Cluster information
-------------------
Name:             cluster-proxmox
Config Version:   1
Transport:        knet
Secure auth:      on

Quorum information
------------------
Date:             Wed Jun 29 22:22:01 2022
Quorum provider:  corosync_votequorum
Nodes:            1
Node ID:          0x00000001
Ring ID:          1.5
Quorate:          Yes

Votequorum information
----------------------
Expected votes:   1
Highest expected: 1
Total votes:      1
Quorum:           1  
Flags:            Quorate 

Membership information
----------------------
    Nodeid      Votes Name
0x00000001          1 192.168.3.60 (local)

Cluster information

-------------------

Name: cluster-proxmox

Config Version: 1

Transport: knet

Secure auth: on

Quorum information

------------------

Date: Wed Jun 29 22:22:01 2022

Quorum provider: corosync_votequorum

Nodes: 1

Node ID: 0x00000001

Ring ID: 1.5

Quorate: Yes

Votequorum information

----------------------

Expected votes: 1

Highest expected: 1

Total votes: 1

Quorum: 1

Flags: Quorate

Membership information

----------------------

Nodeid Votes Name

0x00000001 1 192.168.3.60 (local)

Joindre ensuite la deuxième machine, il sera demander le mot de passe root du serveur qui a crée le cluster

# pvecm add 192.168.3.60

1	# pvecm add 192.168.3.60

Please enter superuser (root) password for '192.168.3.60': ******
Establishing API connection with host '192.168.3.60'
The authenticity of host '192.168.3.60' can't be established.
X509 SHA256 key fingerprint is 2A:12:F7:EA:C6:F3:68:50:EC:C1:3D:5F:04:96:B6:8C:AC:CB:B8:99:74:FA:07:23:9D:68:C4:CE:EE:2C:24:5A.
Are you sure you want to continue connecting (yes/no)? yes
Login succeeded.
check cluster join API version
No cluster network links passed explicitly, fallback to local node IP '192.168.3.61'
Request addition of this node
Join request OK, finishing setup locally
stopping pve-cluster service
backup old database to '/var/lib/pve-cluster/backup/config-1656534406.sql.gz'
waiting for quorum...OK
(re)generate node files
generate new node certificate
merge authorized SSH keys and known hosts
generated new node certificate, restart pveproxy and pvedaemon services
successfully added node 'prox2' to cluster.

Please enter superuser (root) password for '192.168.3.60': ******

Establishing API connection with host '192.168.3.60'

The authenticity of host '192.168.3.60' can't be established.

X509 SHA256 key fingerprint is 2A:12:F7:EA:C6:F3:68:50:EC:C1:3D:5F:04:96:B6:8C:AC:CB:B8:99:74:FA:07:23:9D:68:C4:CE:EE:2C:24:5A.

Are you sure you want to continue connecting (yes/no)? yes

check cluster join API version

No cluster network links passed explicitly, fallback to local node IP '192.168.3.61'

Request addition of this node

Join request OK, finishing setup locally

stopping pve-cluster service

backup old database to '/var/lib/pve-cluster/backup/config-1656534406.sql.gz'

waiting for quorum...OK

(re)generate node files

generate new node certificate

merge authorized SSH keys and known hosts

generated new node certificate, restart pveproxy and pvedaemon services

successfully added node 'prox2' to cluster.

Verifier son status

# pvecm status

1	# pvecm status

Cluster information
-------------------
Name:             cluster-proxmox
Config Version:   2
Transport:        knet
Secure auth:      on

Quorum information
------------------
Date:             Wed Jun 29 22:27:40 2022
Quorum provider:  corosync_votequorum
Nodes:            2
Node ID:          0x00000002
Ring ID:          1.9
Quorate:          Yes

Votequorum information
----------------------
Expected votes:   2
Highest expected: 2
Total votes:      2
Quorum:           2  
Flags:            Quorate 

Membership information
----------------------
    Nodeid      Votes Name
0x00000001          1 192.168.3.60
0x00000002          1 192.168.3.61 (local)

Cluster information

-------------------

Name: cluster-proxmox

Config Version: 2

Transport: knet

Secure auth: on

Quorum information

------------------

Date: Wed Jun 29 22:27:40 2022

Quorum provider: corosync_votequorum

Nodes: 2

Node ID: 0x00000002

Ring ID: 1.9

Quorate: Yes

Votequorum information

----------------------

Expected votes: 2

Highest expected: 2

Total votes: 2

Quorum: 2

Flags: Quorate

Membership information

----------------------

Nodeid Votes Name

0x00000001 1 192.168.3.60

0x00000002 1 192.168.3.61 (local)

Puis joindre la troisieme machine

# pvecm add 192.168.3.60

1	# pvecm add 192.168.3.60

Please enter superuser (root) password for '192.168.3.60': ******
Establishing API connection with host '192.168.3.60'
The authenticity of host '192.168.3.60' can't be established.
X509 SHA256 key fingerprint is 2A:12:F7:EA:C6:F3:68:50:EC:C1:3D:5F:04:96:B6:8C:AC:CB:B8:99:74:FA:07:23:9D:68:C4:CE:EE:2C:24:5A.
Are you sure you want to continue connecting (yes/no)? yes
Login succeeded.
check cluster join API version
No cluster network links passed explicitly, fallback to local node IP '192.168.3.62'
Request addition of this node
Join request OK, finishing setup locally
stopping pve-cluster service
backup old database to '/var/lib/pve-cluster/backup/config-1656534488.sql.gz'
waiting for quorum...OK
(re)generate node files
generate new node certificate
merge authorized SSH keys and known hosts
generated new node certificate, restart pveproxy and pvedaemon services
successfully added node 'prox3' to cluster.

Please enter superuser (root) password for '192.168.3.60': ******

Establishing API connection with host '192.168.3.60'

The authenticity of host '192.168.3.60' can't be established.

X509 SHA256 key fingerprint is 2A:12:F7:EA:C6:F3:68:50:EC:C1:3D:5F:04:96:B6:8C:AC:CB:B8:99:74:FA:07:23:9D:68:C4:CE:EE:2C:24:5A.

Are you sure you want to continue connecting (yes/no)? yes

check cluster join API version

No cluster network links passed explicitly, fallback to local node IP '192.168.3.62'

Request addition of this node

Join request OK, finishing setup locally

stopping pve-cluster service

backup old database to '/var/lib/pve-cluster/backup/config-1656534488.sql.gz'

waiting for quorum...OK

(re)generate node files

generate new node certificate

merge authorized SSH keys and known hosts

generated new node certificate, restart pveproxy and pvedaemon services

successfully added node 'prox3' to cluster.

Verifier son status

# pvecm status

1	# pvecm status

Cluster information
-------------------
Name:             cluster-proxmox
Config Version:   3
Transport:        knet
Secure auth:      on

Quorum information
------------------
Date:             Wed Jun 29 22:28:32 2022
Quorum provider:  corosync_votequorum
Nodes:            3
Node ID:          0x00000003
Ring ID:          1.d
Quorate:          Yes

Votequorum information
----------------------
Expected votes:   3
Highest expected: 3
Total votes:      3
Quorum:           2  
Flags:            Quorate 

Membership information
----------------------
    Nodeid      Votes Name
0x00000001          1 192.168.3.60
0x00000002          1 192.168.3.61
0x00000003          1 192.168.3.62 (local)

Cluster information

-------------------

Name: cluster-proxmox

Config Version: 3

Transport: knet

Secure auth: on

Quorum information

------------------

Date: Wed Jun 29 22:28:32 2022

Quorum provider: corosync_votequorum

Nodes: 3

Node ID: 0x00000003

Ring ID: 1.d

Quorate: Yes

Votequorum information

----------------------

Expected votes: 3

Highest expected: 3

Total votes: 3

Quorum: 2

Flags: Quorate

Membership information

----------------------

Nodeid Votes Name

0x00000001 1 192.168.3.60

0x00000002 1 192.168.3.61

0x00000003 1 192.168.3.62 (local)

On peut voir depuis le Webgui que les hyperviseurs sont bien reliés et que le quorum est ok