Experience with Proxmox Datacenter Manager

Currently in the middle of refactoring different Proxmox clusters, I had the opportunity to use Proxmox Datacenter Manager (PDM) to migrate VMs from one cluster to another.

Proxmox Datacenter Manager (PDM) is a tool that allows you to manage multiple Proxmox clusters from a single location. It can be compared (with a few functional differences) to VMware vCenter. In addition to having information about your clusters in a single interface, PDM allows you to perform VM migrations between PVE clusters in a simplified manner.

Migration context#

Here is the working environment:

• A source cluster (CV1) with two different subnets and the currently placed VMs;
• A destination cluster (CV2) to decouple the subnets and have one cluster per IS;
• All VMs are based on Debian.
• The VMs have backups (snapshots) performed by Proxmox Backup Server.

It was time to separate the ISs, and to migrate the VMs from one cluster to another. A few months ago, one of the solutions to migrate VMs between clusters was to export/import them via scp or rsync commands, which required stopping them.

With PDM, it is possible to do live migrations, which is much more convenient. However, there are a few subtleties to know.

I am voluntarily omitting the details of the PDM configuration steps, as these are well documented on the Proxmox website. Also, depending on your network configuration, some steps may vary, particularly the opening of ports and the configuration of firewalls in your environments.

VM specifics#

The virtual machines to be migrated are for the most part similar, notably with this type of configuration:

• Virtual processor of type ‘x86-64-v3’;
• One or more virtual disks on Ceph-type storage;
• Several network cards (virtio) per virtual machine;
• Some VMs are created via OpenTofu;
• Some VMs use cloud-init for initial configuration.

In themselves, the virtual machines are not as complex as they appear. However, it is important to understand their configuration well before launching a migration. I have virtual machines where the storage is a physical disk attached directly to the virtual machine, or even GPUs passed in direct I/O. These machines cannot be migrated as is, they will be the subject of a manual migration.

Preparation and implementation of the migration#

PDM connects to PVE clusters via the Proxmox API. You will need to create a user, an api-token on each cluster and assign the PVEAdmin role to the user and the api-token so that PDM can connect. Once the connections are established, you can start the migrations.

The cluster CV2 must be ready to receive the virtual machines and their workflow. This preparation includes the configuration of networks, storage, and security policies. In my case, I also had to configure storage to receive the .yml files used by cloud-init. This storage is, for my context, a storage of type cephfs (for the example, the storage will be called sto-cephfs-snippets) in the cluster CV2. From a node of the cluster CV1, I transferred the cloud-init .yml files to the sto-cephfs-snippets storage of CV2. The command was as follows:

rsync -av --progress /mnt/pve/sto-cephfs-snippets/snippets/* root@cv2-node1:/mnt/pve/sto-cephfs-snippets/snippets/

Launching a migration and functionality tests#

The cluster CV2 is ready to receive the virtual machines, it is time to launch a migration of a non-critical virtual machine. Thanks to the graphical wizard offered by PDM, you will be guided through the different steps. A “simple” mode performs the migration without customization, while an “advanced” mode allows you to choose the migration options.

It is highly recommended to use “advanced” mode to adjust the parameters (choice of vmbr, storage, destination node).

Once the VM was selected, the migration started and proceeded without incident. The VM was migrated live, which allowed minimizing service interruption.

To verify the proper functioning of the VM, I had to connect to it via the Proxmox web console from the new cluster CV2 - no issues. The VM had a virtual disk of 96 GB, with about 22 GB useful; the migration was completed in about ten minutes, which I found quite slow. This slowness raises a question: does the migration happen on the management network or on the data network? Alas, on the management network.

Limitations during this migration#

During the migration, I encountered several limitations:

• Some machines were created with OpenTofu, the deployment state is no longer up to date after the migration;
• The migration of cloud-init .yml files was not automatic and no option allows it;
• The PDM web interface does not allow choosing which network card the migration must use (this is my biggest concern);
• VMs with physical disks, GPUs, attached USBs, or mounted local directories cannot be migrated via PDM.

However, there are good points to remember:

• You can launch several migrations at the same time to different nodes;
• The wizard offered by PDM is simple and practical.
• Once the VM is migrated without activating the “delete source” option, on the old cluster the VM will be identified in the “migrate” state, preventing any action on it.

The result#

I am satisfied in a general way, but it was not the ideal solution for my environment. The PDM interface is simple, easy to use, functional and allowed successful migration of simple virtual machines.

However, the result is mixed, given my usage context (cloud-init and OpenTofu). Proxmox Datacenter Manager is a still young product that still needs time to reach maturity. Today, it works very well for centralizing the management of your Proxmox clusters, but it lacks features to be fully used in a production environment (particularly as in my use case).

The machines migrated via PDM are functional, the services are operational. There was no breakage or service interruption. I think VM migration via PDM is an interesting solution for migrating simple VMs, but not yet ideal for environments where automation is widespread.

Finally, I migrated more than a dozen different VMs via PDM, which allowed me to validate the solution and see its limits. For the other machines, I recreated them via OpenTofu.