The following actions need to be carried out from the command line of each Proxmox VE node in your cluster
Perform the actions via console or ssh; preferably via console to avoid interrupted ssh connections. Do not carry out the upgrade when connected via the virtual console offered by the GUI; as this will get interrupted during the upgrade.
Remember to ensure that a valid backup of all VMs and CTs has been created before proceeding.
Continuously use the pve7to8 checklist script
A small checklist program named pve7to8 is included in the latest Proxmox VE 7.4 packages. The program will provide hints and warnings about potential issues before, during and after the upgrade process.
To run it with all checks enabled, execute:
pve7to8 --full
Make sure to run the full checks at least once before the upgrade.
Run pve7to8 --full
root@pve:~# pve7to8 --full
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LC_ADDRESS = "it_IT.UTF-8",
LC_NAME = "it_IT.UTF-8",
LC_MONETARY = "it_IT.UTF-8",
LC_PAPER = "it_IT.UTF-8",
LC_IDENTIFICATION = "it_IT.UTF-8",
LC_TELEPHONE = "it_IT.UTF-8",
LC_MEASUREMENT = "it_IT.UTF-8",
LC_TIME = "it_IT.UTF-8",
LC_NUMERIC = "it_IT.UTF-8",
LANG = "en_US.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to a fallback locale ("en_US.UTF-8").
= CHECKING VERSION INFORMATION FOR PVE PACKAGES =
Checking for package updates..
WARN: updates for the following packages are available:
bind9-host, libx11-xcb1, libmagic-mgc, libmagic1, file, libflac8, libxpm4, libx11-data, proxmox-backup-file-restore, libc6, locales, bind9-dnsutils, libwebp6, libjson-c5, proxmox-backup-client, libc-dev-bin, pve-manager, libc-l10n, bind9-libs, libc-bin, libc-devtools, pve-kernel-5.15, pve-kernel-5.15.116-1-pve, libc6-dev, pve-firewall, libx11-6, libgstreamer-plugins-base1.0-0, dnsutils, linux-libc-dev
Checking proxmox-ve package version..
PASS: proxmox-ve package has version >= 7.4-1
Checking running kernel version..
PASS: running kernel '5.15.107-2-pve' is considered suitable for upgrade.
= CHECKING CLUSTER HEALTH/SETTINGS =
SKIP: standalone node.
= CHECKING HYPER-CONVERGED CEPH STATUS =
SKIP: no hyper-converged ceph setup detected!
= CHECKING CONFIGURED STORAGES =
PASS: storage 'local' enabled and active.
PASS: storage 'local-lvm' enabled and active.
PASS: storage 'lvm-thin-1' enabled and active.
INFO: Checking storage content type configuration..
PASS: no storage content problems found
PASS: no storage re-uses a directory for multiple content types.
= MISCELLANEOUS CHECKS =
INFO: Checking common daemon services..
PASS: systemd unit 'pveproxy.service' is in state 'active'
PASS: systemd unit 'pvedaemon.service' is in state 'active'
PASS: systemd unit 'pvescheduler.service' is in state 'active'
PASS: systemd unit 'pvestatd.service' is in state 'active'
INFO: Checking for supported & active NTP service..
WARN: systemd-timesyncd is not the best choice for time-keeping on servers, due to only applying updates on boot.
While not necessary for the upgrade it's recommended to use one of:
* chrony (Default in new Proxmox VE installations)
* ntpsec
* openntpd
INFO: Checking for running guests..
WARN: 14 running guest(s) detected - consider migrating or stopping them.
INFO: Checking if the local node's hostname 'pve' is resolvable..
INFO: Checking if resolved IP is configured on local node..
PASS: Resolved node IP '<NODE_IP>' configured and active on single interface.
INFO: Check node certificate's RSA key size
PASS: Certificate 'pve-root-ca.pem' passed Debian Busters (and newer) security level for TLS connections (4096 >= 2048)
PASS: Certificate 'pve-ssl.pem' passed Debian Busters (and newer) security level for TLS connections (2048 >= 2048)
PASS: Certificate 'pveproxy-ssl.pem' passed Debian Busters (and newer) security level for TLS connections (4096 >= 2048)
INFO: Checking backup retention settings..
PASS: no backup retention problems found.
INFO: checking CIFS credential location..
PASS: no CIFS credentials at outdated location found.
INFO: Checking permission system changes..
INFO: Checking custom role IDs for clashes with new 'PVE' namespace..
PASS: no custom roles defined, so no clash with 'PVE' role ID namespace enforced in Proxmox VE 8
INFO: Checking if LXCFS is running with FUSE3 library, if already upgraded..
SKIP: not yet upgraded, no need to check the FUSE library version LXCFS uses
INFO: Checking node and guest description/note length..
PASS: All node config descriptions fit in the new limit of 64 KiB
PASS: All guest config descriptions fit in the new limit of 8 KiB
INFO: Checking container configs for deprecated lxc.cgroup entries
PASS: No legacy 'lxc.cgroup' keys found.
INFO: Checking if the suite for the Debian security repository is correct..
INFO: Checking for existence of NVIDIA vGPU Manager..
PASS: No NVIDIA vGPU Service found.
INFO: Checking bootloader configuration...
SKIP: not yet upgraded, no need to check the presence of systemd-boot
= SUMMARY =
TOTAL: 29
PASSED: 22
SKIPPED: 4
WARNINGS: 3
FAILURES: 0
ATTENTION: Please check the output for detailed information!
Update the configured APT repositories
First, make sure that the system is using the latest Proxmox VE 7.4 packages:
The last command should report at least 7.4-15 or newer.
Update Debian Base Repositories to Bookworm
Update all Debian and Proxmox VE repository entries to Bookworm.
sed -i 's/bullseye/bookworm/g' /etc/apt/sources.list
Ensure that there are no remaining Debian Bullseye specific repositories left, if you can use the # symbol at the start of the respective line to comment these repositories out. Check all files in the /etc/apt/sources.list.d/pve-enterprise.list and /etc/apt/sources.list and see Package_Repositories for the correct Proxmox VE 8 / Debian Bookworm repositories.
Add the Proxmox VE 8 Package Repository
# BEFORE
root@pve:~# cat /etc/apt/sources.list.d/pve-enterprise.list
deb https://enterprise.proxmox.com/debian/pve bullseye pve-enterprise
Upgrade the system to Debian Bookworm and Proxmox VE 8.0
âī¸Note that the time required for finishing this step heavily depends on the system's performance, especially the root filesystem's IOPS and bandwidth. A slow spinner can take up to 60 minutes or more, while for a high-performance server with SSD storage, the dist-upgrade can be finished in under 5 minutes.
Start with this step, to get the initial set of upgraded packages:
apt dist-upgrade
During the above step, you will be asked to approve changes to configuration files, where the default config has been updated by their respective package.
It's suggested to check the difference for each file in question and choose the answer accordingly to what's most appropriate for your setup.
đ Common configuration files with changes, and the recommended choices are:
/etc/issue
-> Proxmox VE will auto-generate this file on boot, and it has only cosmetic effects on the login console.
â Using the default "No" (keep your currently-installed version) is safe here.
/etc/lvm/lvm.conf
-> Changes relevant for Proxmox VE will be updated, and a newer config version might be useful.
â If you did not make extra changes yourself and are unsure it's suggested to choose "Yes" (install the package maintainer's version) here.
/etc/ssh/sshd_config
-> If you have not changed this file manually, the only differences should be a replacement of ChallengeResponseAuthentication no with KbdInteractiveAuthentication no and some irrelevant changes in comments (lines starting with #).
â If this is the case, both options are safe, though we would recommend installing the package maintainer's version in order to move away from the deprecated ChallengeResponseAuthentication option. If there are other changes, we suggest to inspect them closely and decide accordingly.
/etc/default/grub
-> Here you may want to take special care, as this is normally only asked for if you changed it manually, e.g., for adding some kernel command line option.
It's recommended to check the difference for any relevant change, note that changes in comments (lines starting with #) are not relevant. If unsure, we suggested to selected "No" (keep your currently-installed version)
Check Result & Reboot Into Updated Kernel
If the dist-upgrade command exits successfully, you can re-check the pve7to8 checker script and reboot the system in order to use the new Proxmox VE kernel.
pve7to8 --full
root@pve:~# pve7to8 --full
= CHECKING VERSION INFORMATION FOR PVE PACKAGES =
Checking for package updates..
PASS: all packages up-to-date
Checking proxmox-ve package version..
PASS: already upgraded to Proxmox VE 8
Checking running kernel version..
WARN: a suitable kernel (proxmox-kernel-6.2) is intalled, but an unsuitable (5.15.107-2-pve) is booted, missing reboot?!
= CHECKING CLUSTER HEALTH/SETTINGS =
SKIP: standalone node.
= CHECKING HYPER-CONVERGED CEPH STATUS =
SKIP: no hyper-converged ceph setup detected!
= CHECKING CONFIGURED STORAGES =
PASS: storage 'local' enabled and active.
PASS: storage 'local-lvm' enabled and active.
PASS: storage 'lvm-thin-1' enabled and active.
INFO: Checking storage content type configuration..
PASS: no storage content problems found
PASS: no storage re-uses a directory for multiple content types.
= MISCELLANEOUS CHECKS =
INFO: Checking common daemon services..
PASS: systemd unit 'pveproxy.service' is in state 'active'
PASS: systemd unit 'pvedaemon.service' is in state 'active'
PASS: systemd unit 'pvescheduler.service' is in state 'active'
PASS: systemd unit 'pvestatd.service' is in state 'active'
INFO: Checking for supported & active NTP service..
WARN: systemd-timesyncd is not the best choice for time-keeping on servers, due to only applying updates on boot.
While not necessary for the upgrade it's recommended to use one of:
* chrony (Default in new Proxmox VE installations)
* ntpsec
* openntpd
INFO: Checking for running guests..
PASS: no running guest detected.
INFO: Checking if the local node's hostname 'pve' is resolvable..
INFO: Checking if resolved IP is configured on local node..
PASS: Resolved node IP '<NODE_IP>' configured and active on single interface.
INFO: Check node certificate's RSA key size
PASS: Certificate 'pve-root-ca.pem' passed Debian Busters (and newer) security level for TLS connections (4096 >= 2048)
PASS: Certificate 'pve-ssl.pem' passed Debian Busters (and newer) security level for TLS connections (2048 >= 2048)
PASS: Certificate 'pveproxy-ssl.pem' passed Debian Busters (and newer) security level for TLS connections (4096 >= 2048)
INFO: Checking backup retention settings..
PASS: no backup retention problems found.
INFO: checking CIFS credential location..
PASS: no CIFS credentials at outdated location found.
INFO: Checking permission system changes..
INFO: Checking custom role IDs for clashes with new 'PVE' namespace..
PASS: no custom roles defined, so no clash with 'PVE' role ID namespace enforced in Proxmox VE 8
INFO: Checking if LXCFS is running with FUSE3 library, if already upgraded..
WARN: systems seems to be upgraded but LXCFS is still running with FUSE 2 library, not yet rebooted?
INFO: Checking node and guest description/note length..
PASS: All node config descriptions fit in the new limit of 64 KiB
PASS: All guest config descriptions fit in the new limit of 8 KiB
INFO: Checking container configs for deprecated lxc.cgroup entries
PASS: No legacy 'lxc.cgroup' keys found.
INFO: Checking if the suite for the Debian security repository is correct..
PASS: found no suite mismatch
INFO: Checking for existence of NVIDIA vGPU Manager..
PASS: No NVIDIA vGPU Service found.
INFO: Checking bootloader configuration...
SKIP: proxmox-boot-tool not used for bootloader configuration
= SUMMARY =
TOTAL: 30
PASSED: 24
SKIPPED: 3
WARNINGS: 3
FAILURES: 0
ATTENTION: Please check the output for detailed information!
Please note that you should reboot even if you already used the 6.2 kernel previously, through the opt-in package on Proxmox VE 7. This is required to guarantee the best compatibility with the rest of the system, as the updated kernel was (re-)build with the newer Proxmox VE 8 compiler and ABI versions.
Reboot node from GUI
Install chrony to fix the warning:
root@pve:~# apt install chrony
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following packages were automatically installed and are no longer required:
bsdmainutils libboost-coroutine1.74.0 libbpf0 libcbor0 libdns-export1110 libicu67 libisc-export1105 libleveldb1d libmpdec3 libopts25 libperl5.32 libprocps8 libprotobuf23 libpython3.11
libpython3.9 libpython3.9-minimal libpython3.9-stdlib libtiff5 liburing1 libwebp6 libzpool4linux perl-modules-5.32 python3-ldb python3-talloc python3.9 python3.9-minimal telnet
Use 'apt autoremove' to remove them.
Suggested packages:
networkd-dispatcher
The following packages will be REMOVED:
systemd-timesyncd
The following NEW packages will be installed:
chrony
0 upgraded, 1 newly installed, 1 to remove and 0 not upgraded.
Need to get 288 kB of archives.
After this operation, 504 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://ftp.it.debian.org/debian bookworm/main amd64 chrony amd64 4.3-2+deb12u1 [288 kB]
Fetched 288 kB in 0s (761 kB/s)
(Reading database ... 84561 files and directories currently installed.)
Removing systemd-timesyncd (252.12-pmx1) ...
Selecting previously unselected package chrony.
(Reading database ... 84546 files and directories currently installed.)
Preparing to unpack .../chrony_4.3-2+deb12u1_amd64.deb ...
Unpacking chrony (4.3-2+deb12u1) ...
Setting up chrony (4.3-2+deb12u1) ...
Creating config file /etc/chrony/chrony.conf with new version
Creating config file /etc/chrony/chrony.keys with new version
dpkg-statoverride: warning: --update given but /var/log/chrony does not exist
Created symlink /etc/systemd/system/chronyd.service â /lib/systemd/system/chrony.service.
Created symlink /etc/systemd/system/multi-user.target.wants/chrony.service â /lib/systemd/system/chrony.service.
Processing triggers for dbus (1.14.8-2~deb12u1) ...
Processing triggers for man-db (2.11.2-2) ...
root@pve:~# pve7to8 --full
= CHECKING VERSION INFORMATION FOR PVE PACKAGES =
Checking for package updates..
PASS: all packages up-to-date
Checking proxmox-ve package version..
PASS: already upgraded to Proxmox VE 8
Checking running kernel version..
PASS: running new kernel '6.2.16-12-pve' after upgrade.
INFO: Found outdated kernel meta-packages, taking up extra space on boot partitions.
After a successful upgrade, you can remove them using this command:
apt remove pve-kernel-5.4 pve-kernel-5.15
= CHECKING CLUSTER HEALTH/SETTINGS =
SKIP: standalone node.
= CHECKING HYPER-CONVERGED CEPH STATUS =
SKIP: no hyper-converged ceph setup detected!
= CHECKING CONFIGURED STORAGES =
PASS: storage 'local' enabled and active.
PASS: storage 'local-lvm' enabled and active.
PASS: storage 'lvm-thin-1' enabled and active.
INFO: Checking storage content type configuration..
PASS: no storage content problems found
PASS: no storage re-uses a directory for multiple content types.
= MISCELLANEOUS CHECKS =
INFO: Checking common daemon services..
PASS: systemd unit 'pveproxy.service' is in state 'active'
PASS: systemd unit 'pvedaemon.service' is in state 'active'
PASS: systemd unit 'pvescheduler.service' is in state 'active'
PASS: systemd unit 'pvestatd.service' is in state 'active'
INFO: Checking for supported & active NTP service..
PASS: Detected active time synchronisation unit 'chrony.service'
INFO: Checking for running guests..
WARN: 14 running guest(s) detected - consider migrating or stopping them.
INFO: Checking if the local node's hostname 'pve' is resolvable..
INFO: Checking if resolved IP is configured on local node..
PASS: Resolved node IP '<NODE_IP>' configured and active on single interface.
INFO: Check node certificate's RSA key size
PASS: Certificate 'pve-root-ca.pem' passed Debian Busters (and newer) security level for TLS connections (4096 >= 2048)
PASS: Certificate 'pve-ssl.pem' passed Debian Busters (and newer) security level for TLS connections (2048 >= 2048)
PASS: Certificate 'pveproxy-ssl.pem' passed Debian Busters (and newer) security level for TLS connections (4096 >= 2048)
INFO: Checking backup retention settings..
PASS: no backup retention problems found.
INFO: checking CIFS credential location..
PASS: no CIFS credentials at outdated location found.
INFO: Checking permission system changes..
INFO: Checking custom role IDs for clashes with new 'PVE' namespace..
PASS: no custom roles defined, so no clash with 'PVE' role ID namespace enforced in Proxmox VE 8
INFO: Checking if LXCFS is running with FUSE3 library, if already upgraded..
PASS: systems seems to be upgraded and LXCFS is running with FUSE 3 library
INFO: Checking node and guest description/note length..
PASS: All node config descriptions fit in the new limit of 64 KiB
PASS: All guest config descriptions fit in the new limit of 8 KiB
INFO: Checking container configs for deprecated lxc.cgroup entries
PASS: No legacy 'lxc.cgroup' keys found.
INFO: Checking if the suite for the Debian security repository is correct..
PASS: found no suite mismatch
INFO: Checking for existence of NVIDIA vGPU Manager..
PASS: No NVIDIA vGPU Service found.
INFO: Checking bootloader configuration...
SKIP: proxmox-boot-tool not used for bootloader configuration
= SUMMARY =
TOTAL: 30
PASSED: 26
SKIPPED: 3
WARNINGS: 1
FAILURES: 0
ATTENTION: Please check the output for detailed information!
After the Proxmox VE upgrade
Empty the browser cache and/or force-reload (CTRL + SHIFT + R, or for MacOS â + Alt + R) the Web UI.
For Clusters
Check that all nodes are up and running on the latest package versions.
Disable root user from Proxmox Datacenter Permissions/Users menu
