pve Transparent Failover
Ray Tracy
HA 只是高可用度,他沒有不停頓 (Non-stop) 等級的容錯(FT)能力,所以只能把死掉的 VM 重開機來復原,他做不到:接手死掉 VM 當時狀態,不停頓的執行下去....市面上可穩定商轉的 FT 軟體最便宜也要上百萬起跳....且對硬體條件有諸多限制...
還有,測試 HA 盡量採用拔電源線的方式,而不是只拔網路線,否則在複雜網路環境中,復原後可能整個 Cluster 會發生 #叢集腦裂(雖然機率很小,你還是可以賭看看)....
而且原本初階 HA 就是設計成:用來對抗 Host 全機故障,而不是對抗只有網路故障;所以即便將網路隔離,若 Host 沒掛掉的話,復原的時候都會有風險.....
完整的 HA 流程,在啟動備用 VM 之前,Cluster 會先自動下 ipmi 或 BMC 指令給隔離的 Host 強制它完全關機之後,才啟動備用 VM, 就是為了避免發生上述風險....(這個動作稱為 fence)
這裡大部分玩 cluster 的人都不知道要去設定 ipmi/bmc 做 fence; 如果有設定 fence 的話, 就可以用拔網路線來做測試了, 因為拔掉之後, fence 會自動去把那台隔離的 host 關機, 就不會發生復原之後的腦裂問題.....(當然, 你不能去拔 fence 用的那條 ipmi 網路線...🤣🤣🤣)
Jason Cheng
如果經費不多想做 FT,可以參考台灣團隊開發的 Cuju 專案,基於 kvm 實作,去年底 Proxmox 社團使用者年會該團隊講者有來分享https://github.com/Cuju-ft/Cuju
[轉]虛擬機跑起來!RouterOS CHR 軟路由效能輕鬆突破 1000M!
https://www.jkg.tw/?p=2531&fbclid=IwAR0TdjR76xT3k3w_bTNPfLZ8eW9Wuri9WBGJU7xoHsQo4PDUq2M5iIN4zfc
https://youtu.be/4PzJulQTSrY?t=1016
占美 i5 4278u
https://youtu.be/4PzJulQTSrY?t=1016
占美 i5 4278u
[轉]Proxmox VE - One Public IP
https://raymii.org/s/tutorials/Proxmox_VE_One_Public_IP.html
實例 https://www.facebook.com/groups/pve.tw/permalink/1225387670963125/
楊凱麟
以下也是我目前interfaces的設定
其中1.2.3.x為真實IP
1.2.3.Y為IDC提供的gateway
設置完後再將區網設定填上10.0.0.X (X=0~255)
即可上網,連外網的公共IP會是1.2.3.X
----network configuration----
auto lo
iface lo inet loopback
iface eno1 inet manual
auto vmbr0
iface vmbr0 inet static
address 1.2.3.X
netmask 255.255.255.248
network 1.2.3.0
broadcast 1.2.3.255
gateway 1.2.3.Y
bridge_ports eno1
bridge_stp off
bridge_fd 0
auto vmbr1
iface vmbr1 inet static
address 10.0.0.254
netmask 255.255.255.0
bridge_ports none
bridge_stp off
bridge_fd 0
post-up echo 1 > /proc/sys/net/ipv4/ip_forward
post-up iptables -t nat -A POSTROUTING -s '10.21.21.0/24' -o vmbr0 -j MASQUERADE
post-down iptables -t nat -D POSTROUTING -s '10.21.21.0/24' -o vmbr0 -j MASQUERADE
post-up iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 2222 -j DNAT --to 10.21.21.5:22
post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp --dport 2222 -j DNAT --to 10.21.21.5:22
https://forum.proxmox.com/threads/pfsense-proxmox-5-one-nic.41512/#post-208059
==============
#############################
https://download.configserver.com/csf/readme.txt11. Multiple Ethernet Devices
實例 https://www.facebook.com/groups/pve.tw/permalink/1225387670963125/
楊凱麟
以下也是我目前interfaces的設定
其中1.2.3.x為真實IP
1.2.3.Y為IDC提供的gateway
設置完後再將區網設定填上10.0.0.X (X=0~255)
即可上網,連外網的公共IP會是1.2.3.X
----network configuration----
auto lo
iface lo inet loopback
iface eno1 inet manual
auto vmbr0
iface vmbr0 inet static
address 1.2.3.X
netmask 255.255.255.248
network 1.2.3.0
broadcast 1.2.3.255
gateway 1.2.3.Y
bridge_ports eno1
bridge_stp off
bridge_fd 0
auto vmbr1
iface vmbr1 inet static
address 10.0.0.254
netmask 255.255.255.0
bridge_ports none
bridge_stp off
bridge_fd 0
post-up echo 1 > /proc/sys/net/ipv4/ip_forward
post-up iptables -t nat -A POSTROUTING -s '10.21.21.0/24' -o vmbr0 -j MASQUERADE
post-down iptables -t nat -D POSTROUTING -s '10.21.21.0/24' -o vmbr0 -j MASQUERADE
post-up iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 2222 -j DNAT --to 10.21.21.5:22
post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp --dport 2222 -j DNAT --to 10.21.21.5:22
https://forum.proxmox.com/threads/pfsense-proxmox-5-one-nic.41512/#post-208059
==============
- CSF firewall can focus vmbr2
#############################
https://download.configserver.com/csf/readme.txt11. Multiple Ethernet Devices
If you have multiple ethernet NICs that you want to apply all rules to, then you can set ETH_DEVICE to the interface name immediately followed by a plus sign. For example, eth+ will apply all iptables rules to eth0, eth1, etc.
That said, if you leave ETH_DEVICE blank all rules will be applied to all ethernet devices equally.
#############################
#############################
https://www.dmehosting.com/clients/knowledgebase.php?action=displayarticle&id=79&language=chinese
2. Edit /etc/csf/csf.conf, set the following parameters:
ETH_DEVICE = "venet0"
MONOLITHIC_KERNEL = "1"
VERBOSE = "0"
2. Edit /etc/csf/csf.conf, set the following parameters:
ETH_DEVICE = "venet0"
MONOLITHIC_KERNEL = "1"
VERBOSE = "0"
- CSF iptables nat
https://blackonsole.org/how-to-add-nat-iptables-rules-with-csf/
cpu scaling freq cpufreq powersave
apt install cpufrequtils
more /etc/init.d/cpufrequtils
cpufreq-set -g powersave
cpufreq-info
=====
lscpu
cat /proc/cpuinfo | grep -i 'mhz'
grep -E '^model name|^cpu MHz' /proc/cpuinfo
cat /sys/devices/system/cpu/cpufreq/policy7/scaling_governor
cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_max_freq
cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_min_freq
**performance
for CPUFREQ in /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor; do [ -f $CPUFREQ ] || continue; echo -n performance > $CPUFREQ; done
**powersave
for CPUFREQ in /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor; do [ -f $CPUFREQ ] || continue; echo -n powersave > $CPUFREQ; done
echo "powersave" > /sys/devices/system/cpu/cpufreq/policy0/scaling_governor
=====
more /etc/init.d/cpufrequtils
cpufreq-set -g powersave
cpufreq-info
=====
lscpu
cat /proc/cpuinfo | grep -i 'mhz'
grep -E '^model name|^cpu MHz' /proc/cpuinfo
cat /sys/devices/system/cpu/cpufreq/policy7/scaling_governor
cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_max_freq
cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_min_freq
**performance
for CPUFREQ in /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor; do [ -f $CPUFREQ ] || continue; echo -n performance > $CPUFREQ; done
**powersave
for CPUFREQ in /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor; do [ -f $CPUFREQ ] || continue; echo -n powersave > $CPUFREQ; done
echo "powersave" > /sys/devices/system/cpu/cpufreq/policy0/scaling_governor
=====
[轉]io-comparison-proxmox-kvm-raw-qcow2
http://sclabs.blogspot.tw/2016/07/io-comparison-proxmox-kvm-raw-qcow2_23.html
convert lvm to raw
http://pve.proxmox.com/wiki/Storage_Migration
==========
qemu-img convert -O qcow2 /dev/VMStor1/vm-100-disk-1 /mnt/VMStor2/images/100/vm-100-disk-1.qcow2
==========
qemu-img convert -O qcow2 /dev/VMStor1/vm-100-disk-1 /mnt/VMStor2/images/100/vm-100-disk-1.qcow2
convert raw to lvm
qemu-img convert -p -f vmdk "IE11 - Win7-disk1.vmdk" -O raw /dev/mapper/pve-vm--101--disk-1
qemu-img convert -p -f vmdk "IE11 - Win7-disk1.vmdk" -O raw "IE11 - Win7-disk1.raw"
dd if="IE11 - Win7-disk1.raw" bs=1M|pv| dd of=/dev/mapper/pve-vm--101--disk--1
https://forum.proxmox.com/threads/dd-raw-drives-to-lvm-is-slow-is-there-a-better-way.29785/
Remove Cluster configuration
http://pve.proxmox.com/wiki/Proxmox_Cluster_File_System_(pmxcfs)
stop the cluster file system in /etc/pve/
# systemctl stop pve-cluster
start it again but forcing local mode
# pmxcfs -l
remove the cluster configuration
# rm /etc/pve/cluster.conf
# rm /etc/cluster/cluster.conf
# rm /var/lib/pve-cluster/corosync.authkey
stop the cluster file system again
# systemctl stop pve-cluster
restart PVE services (or reboot)
# systemctl start pve-cluster
# systemctl restart pvedaemon
# systemctl restart pveproxy
# systemctl restart pvestatd
=======移除node==============
經PVE03 執行pvecm delnode pve07
刪除 已重灌的pve07 節點,corosync.conf 內的node 資料已刪除
但gui上仍出現pve07 的資料
需手動進pve03 的shell,進入/etc/pve/nodes,rm -rf pve07
手動刪除pve07 的資料夾,再重整網頁gui,pve07即消失
stop the cluster file system in /etc/pve/
# systemctl stop pve-cluster
start it again but forcing local mode
# pmxcfs -l
remove the cluster configuration
# rm /etc/pve/cluster.conf
# rm /etc/cluster/cluster.conf
# rm /var/lib/pve-cluster/corosync.authkey
stop the cluster file system again
# systemctl stop pve-cluster
restart PVE services (or reboot)
# systemctl start pve-cluster
# systemctl restart pvedaemon
# systemctl restart pveproxy
# systemctl restart pvestatd
=======移除node==============
經PVE03 執行pvecm delnode pve07
刪除 已重灌的pve07 節點,corosync.conf 內的node 資料已刪除
但gui上仍出現pve07 的資料
需手動進pve03 的shell,進入/etc/pve/nodes,rm -rf pve07
手動刪除pve07 的資料夾,再重整網頁gui,pve07即消失
proxmox cluster failed
pvecm expected 1
暫時把集叢的期待數量從2降低為1
service pve-cluster restart
==========
本機運作
systemctl stop pve-cluster
pmxcfs -l
==========
systemctl restart corosync.service
systemctl restart pve-cluster.service
systemctl restart pvedaemon.service
systemctl restart pveproxy.service
https://pve.proxmox.com/wiki/Separate_Cluster_Network
暫時把集叢的期待數量從2降低為1
service pve-cluster restart
==========
本機運作
systemctl stop pve-cluster
pmxcfs -l
==========
systemctl restart corosync.service
systemctl restart pve-cluster.service
systemctl restart pvedaemon.service
systemctl restart pveproxy.service
https://pve.proxmox.com/wiki/Separate_Cluster_Network
kvm win vm
http://itheiso.blogspot.tw/2013/01/kvmubuntu-1204-lts.html
Windows虛擬機器效能調整
如果想讓Windows虛擬機器在KVM中有比較好的效能,可以安裝virtio驅動程式來提昇虛擬機器中的硬碟及網路卡速度。請先到 http://alt.fedoraproject.org/pub/alt/virtio-win/latest/images/bin/ 下載最新版本的virtio(本範例版本為virtio-win-0.1-30.iso),接著先將Windows的虛擬機器關機,並從虛擬機器管理員編輯虛擬機器細節,將下載的virtio-win-0.1-30.iso掛載到IDE CDROM 1(先將原先Windows7的安裝光碟斷線再連線,並指定virtio-win-0.1-30.iso檔的位置),接著再加入硬體,新增一個Storage儲存裝置,並指定裝置類型為Virtio disk,然後將網路界面(NIC)的裝置類型也改為virtio,最後就可以將Windows虛擬機器再度開啟。
Virtio disk
Windows虛擬機器再度開啟之後,裝置管理員中會產生二個有驚嘆號的裝置,分別是Ethernet控制卡和SCSI控制卡,只要更新此二個裝置的驅動程式,並將驅動程式的位置指向剛剛掛載virtio的光碟機就可以正確安裝。
安裝完畢請再度將Windows虛擬機器關機,並再次從虛擬機器管理員編輯虛擬機器細節,移除剛剛新增的0.1G虛擬硬碟(請勿刪錯磁碟),再將原先安裝Windows系統的IDE Disk 1之虛擬磁碟型態改為Virtio,然後再重新開啟Windows虛擬機器。如此就可以提昇Windows的效能。
IDE to Virtio
Windows虛擬機器效能調整
如果想讓Windows虛擬機器在KVM中有比較好的效能,可以安裝virtio驅動程式來提昇虛擬機器中的硬碟及網路卡速度。請先到 http://alt.fedoraproject.org/pub/alt/virtio-win/latest/images/bin/ 下載最新版本的virtio(本範例版本為virtio-win-0.1-30.iso),接著先將Windows的虛擬機器關機,並從虛擬機器管理員編輯虛擬機器細節,將下載的virtio-win-0.1-30.iso掛載到IDE CDROM 1(先將原先Windows7的安裝光碟斷線再連線,並指定virtio-win-0.1-30.iso檔的位置),接著再加入硬體,新增一個Storage儲存裝置,並指定裝置類型為Virtio disk,然後將網路界面(NIC)的裝置類型也改為virtio,最後就可以將Windows虛擬機器再度開啟。
Virtio disk
Windows虛擬機器再度開啟之後,裝置管理員中會產生二個有驚嘆號的裝置,分別是Ethernet控制卡和SCSI控制卡,只要更新此二個裝置的驅動程式,並將驅動程式的位置指向剛剛掛載virtio的光碟機就可以正確安裝。
安裝完畢請再度將Windows虛擬機器關機,並再次從虛擬機器管理員編輯虛擬機器細節,移除剛剛新增的0.1G虛擬硬碟(請勿刪錯磁碟),再將原先安裝Windows系統的IDE Disk 1之虛擬磁碟型態改為Virtio,然後再重新開啟Windows虛擬機器。如此就可以提昇Windows的效能。
IDE to Virtio
proxmox mount extend usb hdd
1、
apt-get update
apt-get install ntfs-3g
2、fdisk -l 看有沒有找到usb hdd
blkid use uuid
3、根目錄底下有mnt
mount -t ntfs /dev/sdxxxx /mnt
apt-get update
apt-get install ntfs-3g
2、fdisk -l 看有沒有找到usb hdd
blkid use uuid
3、根目錄底下有mnt
mount -t ntfs /dev/sdxxxx /mnt
該死的proxmox usb install….....
Loading, please wait...
Volume group "pve" not found
Cannot process volume group pve
Unable to find LVM volume pve/root
like this picture

just reboot then
choese menu
Advanced options for Proxmox Virutal Enviroment
then press e
then add rootdelay=2 after pve-root ro quiet
before
after
then ctrl+x now can boot success!
!!! remeber change /boot/grub/grub.cfg
https://forum.proxmox.com/threads/solved-proxmox-4-pve-not-found-after-first-start.24702/
[轉]Proxmox VE Physical disk to kvm 實體硬碟給kvm用
http://blogs.yyes.chc.edu.tw/post/2/75
1、ls -l /dev/disk/by-id/ 找出硬碟
2、修改vm設定檔
nano /etc/pve/qemu-server/xxxxx(vmid).conf
Example:
virtio0: /dev/disk/by-id/ata-ST2000DL003-9VT166_5YD1ZYK5,size=2000G
===============
https://forum.proxmox.com/threads/proxmox-and-hdd-passthrough.17856/
1、ls -l /dev/disk/by-id/ 找出硬碟
2、修改vm設定檔
nano /etc/pve/qemu-server/xxxxx(vmid).conf
Example:
virtio0: /dev/disk/by-id/ata-ST2000DL003-9VT166_5YD1ZYK5,size=2000G
===============
https://forum.proxmox.com/threads/proxmox-and-hdd-passthrough.17856/
[轉]proxmox 增加其他硬碟
http://blog.pulipuli.info/2014/10/proxmox-ve-add-another-hard-disk-in.html
1、找硬碟
ls /dev/sd* (sata is sd, ide is hd)
2、碟磁分割
fdisk /dev/sd*
a. n (new)
b.p (primary)
c. enter
d. enter
e. enter
f. w (寫入)
c d e 操作是使用整個硬碟空間
3、格式化
mkfs -t ext3 /dev/sd*
4、掛截磁碟
a.建目錄 mkdir aaa
b.寫 /etc/fstab
/dev/sd* /aaa ext3 defaults 1 2
重開,用df查
5、用proxmox掛上去
記得 content 要全選
1、找硬碟
ls /dev/sd* (sata is sd, ide is hd)
2、碟磁分割
fdisk /dev/sd*
a. n (new)
b.p (primary)
c. enter
d. enter
e. enter
f. w (寫入)
c d e 操作是使用整個硬碟空間
3、格式化
mkfs -t ext3 /dev/sd*
4、掛截磁碟
a.建目錄 mkdir aaa
b.寫 /etc/fstab
/dev/sd* /aaa ext3 defaults 1 2
重開,用df查
5、用proxmox掛上去
記得 content 要全選
proxmox use vmdk
http://blog.pulipuli.info/2015/06/proxmox-vevmdk-use-vmdk-format-as.html
這是因為在KVM虛擬技術中,預設快取寫入硬碟的方式為「Default (No cache)」,但是vmdk必須使用「Write Through」設定,因此只要調整硬碟快取寫入方式就可以解決這個問題。
其實上面的Bus/Device改成VIRTIO的話,一般來說效能會更好,因為這是虛擬裝置原生的通道,而非模擬出來的IDE或SATA。
這是因為在KVM虛擬技術中,預設快取寫入硬碟的方式為「Default (No cache)」,但是vmdk必須使用「Write Through」設定,因此只要調整硬碟快取寫入方式就可以解決這個問題。
其實上面的Bus/Device改成VIRTIO的話,一般來說效能會更好,因為這是虛擬裝置原生的通道,而非模擬出來的IDE或SATA。
訂閱:
文章 (Atom)



