Proxmox, OpenTofu and Talos, Kubernetes as code

Building a complete Kubernetes infrastructure as code on Proxmox with OpenTofu and Talos, fully automated and repeatable.

In a context where the management of infrastructure and information systems is becoming increasingly complex, Infrastructure as Code (IaC) (in French) has established itself as an essential solution to automate, standardize and optimize deployments.

Among the flagship tools of this ecosystem, Proxmox, OpenTofu and Talos stand out for their efficiency and their complementarity. I propose using these tools to deploy virtual machines in Proxmox with OpenTofu, and to spin up a Kubernetes cluster on them with Talos .

Proxmox is an open-source virtualization platform that lets you manage virtual machines and containers. OpenTofu, a fork of Terraform, is used for infrastructure automation, making it possible to define and provision resources declaratively. Talos, for its part, is a minimalist operating system designed specifically to run Kubernetes, offering a reduced footprint and enhanced security.

Still with the goal of hosting my services locally, I decided to use these tools to set up a complex, robust infrastructure that is able to scale.

To avoid repeating myself, I based this on several articles about Proxmox and OpenTofu:

Use Terraform or OpenTofu to create a VM in Proxmox — Following an article about using Terraform to create LXCs for Proxmox, I want to dig deeper into the subject, this time covering the creation of virtual machines. Prerequisites: prior knowledge of Terraform, administrative rights on Proxmox, and the ability to make changes without major constraints. J.HOMMET.NET Julien HOMMET Deploy multiple VMs with Terraform or OpenTofu in Proxmox — The power of tools like Terraform and OpenTofu lies in their ability to scale. Creating a single virtual machine is fairly quick, but let’s create several machines to make the most of the tool. I’ve been using OpenTofu since early 2025 on various Proxmox infrastructures, both online and air-gapped. The code examples below J.HOMMET.NET Julien HOMMET I deliberately assume that you already have knowledge of the IaC world, of OpenTofu and of Kubernetes.

As an example, I’ll create two virtual machines: one for the “control plane” and the other for the “worker”. Talos and Kubernetes will be preconfigured to use Cilium as the CNI. There will be no CSI, no example application deployment or observability. The goal is solely to set up the cluster and make it functional.

Proxmox context (9.x)#

I use only a single physical machine for virtualization, instead of having several dedicated to specific services. This machine runs Debian 13 and is equipped with various tools such as QEMU, bridge-utils and others. Combining these elements gives you Proxmox.

For this document, I’m keeping things simple. Proxmox was installed by following this guide , storage is local on an NVMe disk, and a single network bridge was created (the default one, vmbr0). The version of Proxmox, as of this article’s update (May 2026), is a 9.x (based on Debian 13 Trixie). The virtual machines will be created with the virtio drivers, without any optimization on the Proxmox side or in the VM. The principle is always the same: SIMPLICITY. Let’s keep everything as simple as possible, everywhere.

A small subtlety regarding the storage of Talos data: an OS-less virtual machine will be created with two virtual hard disks, which will be attached to each of the Talos virtual machines. Each Talos virtual machine will therefore have its system disk and a data disk. This data disk isn’t essential; it’s there to make use of the local-path-provisioner tool and to test various features later on.

Talos context (1.13.3)#

Talos, by Siderolabs, has a pragmatic and minimalist approach that I really appreciate. To minimize the system’s footprint and complexity, only the binaries strictly necessary to run Kubernetes and the Linux kernel are present. There’s no shell, no SSH, no snap: nothing superfluous.

The vendor provides the tools needed to customize the Talos ISO image or virtual disk to fit your needs and your system’s configuration. By browsing to « https://factory.talos.dev », you’ll find a form that guides you through the various options. In addition, a « schematic ID » is provided, a code that lets you keep the same options and customizations across Talos versions.

Here, I chose the “cloud server” machine type, version 1.13.3 and Nocloud, an AMD64 architecture (without secureboot), the extensions “amd-ucode ; amdgpu” (because my hypervisor has an AMD processor), “iscsi-tools” and “qemu-guest-agent”, and I entered the following kernel options: net.ifnames=0 biosdevname=0 console=tty0 console=ttyS0,115200n8 mitigations=off elevator=none. These kernel options force the network interface names to eth instead of predictable ens names, display kernel errors directly in the console (tty0), and disable two optimizations that are useless in a virtual context (mitigations and elevator). The final page of the wizard will give you a summary of your configuration and the famous schematic ID, in addition to the download links for the ISO image or the virtual disk.

Simplified management of the machines and the Kubernetes tools is another advantage. With Talos, all dependencies are up to date and compatible with each other. However, it’s important to check the compatibility between the Kubernetes version and your CNI, CSI, as well as dependencies like cert-manager and gateway-api.

OpenTofu context (1.12.x)#

Now let’s move on to the OpenTofu code. I’m going to create several files in this project to define the virtual machines and the Kubernetes cluster. I apologize in advance if the article’s readability isn’t optimal.

OpenTofu makes it easier to automate and scale machine deployments, and in our case, Kubernetes. By using configuration files like talos.tf and vm.tf, you can quickly deploy new nodes and update existing configurations. Horizontal scaling can be achieved by adding new worker nodes, while vertical scaling can be done by adjusting the resources (CPU, RAM, storage) allocated to each node.

The project tree is as follows:

shell
opentofu
> talos_nocloud
>> main.tf
>> variables.tf
>> vm.tf
>> talos.tf
>> talos-img.tf

The main.tf file will contain the provider declarations and their associated values:

hcl
terraform {
  required_providers {
    proxmox = {
      source  = "bpg/proxmox"
      version = "0.107.0"
    }
    talos = {
      source  = "siderolabs/talos"
      version = "0.11.0"
    }
    random = {
      source  = "hashicorp/random"
      version = "3.7.2"
    }
  }
}

provider "proxmox" {
  api_token = "user@realm!token_name=token_id"
  endpoint  = "https://your.proxmox.endpoint:8006/api2/json"
  insecure  = true # because self-signed TLS certificate is in use
  tmp_dir   = "/var/tmp/"

  ssh {
    agent    = true
    username = "root"
  }
}

provider "talos" {}

The variables.tf file contains the definition of the variables that will be used in the vm.tf and talos.tf files. I’ll use the “Use a .tfvars file with several resource blocks” approach from the article https://j.hommet.net/en/deploy-multiple-vm-terraform-opentofu-proxmox/ .

hcl
variable "kubernetes_version" {
  type = string
}

variable "talos_cluster_endpoint" {
  description = "The endpoint for the Talos cluster"
  type        = string
}
variable "talos_cluster_name" {
  description = "A name to provide for the Talos cluster"
  type        = string
}
variable "talos_version" {
  type = string
}

variable "nodes" {
  description = "Values of VM resources"
  type = map(object({
    data_vm_interface_disk         = string
    pve                            = string
    role                           = string
    usage                          = string
    vm_boot_disk_format            = optional(string, "raw")
    vm_boot_disk_size              = number
    vm_cpu_cores                   = number
    vm_cpu_flags                   = optional(list(string))
    vm_cpu_type                    = optional(string, "x86-64-v2-AES")
    vm_data_disk_interface         = string
    vm_datastore_id                = string
    vm_datastore_id_boot_disk      = string
    vm_datastore_id_data_disk      = string
    vm_datastore_id_efi_disk       = string
    vm_datastore_id_initialization = string
    vm_datastore_id_tpm            = string
    vm_description                 = string
    vm_dns                         = list(string)
    vm_domain                      = string
    vm_efi                         = optional(bool, true)
    vm_eth_rate_limit              = optional(number, 0)
    vm_gateway                     = string
    vm_id                          = number
    vm_install_disk                = string
    vm_ip                          = string
    vm_kvm_args                    = optional(string)
    vm_mac_address                 = string
    vm_memory_dedicated            = number
    vm_memory_floating             = number
    vm_name                        = string
    vm_on_boot                     = optional(bool, true)
    vm_pool_id                     = optional(string)
    vm_startup_order               = optional(number, 1)
    vm_started                     = optional(bool, true)
    vm_tags                        = optional(list(string))
    vm_timeservers                 = optional(list(string), ["0.pool.ntp.org", "1.pool.ntp.org", "2.pool.ntp.org", "3.pool.ntp.org"])
    vm_tpm                         = optional(bool, true)
  }))
}

variable "meta_config_metadata" {
  description = "Metadata for cloud-init configuration"
  type = map(object({
    snippet_datastore_id = string
    snippet_pve          = string
    data                 = string
  }))
}

The “nodes” variable is a set of values in which I specify the data for each VM. If you want to add more machines, you simply duplicate a nodes value block.

The vm.tf file is fairly standard (similar to what you can find in other articles on my site):

hcl
# Machines pour le stockage de données statiques pour Talos
resource "proxmox_virtual_environment_vm" "talos_vm_data" {
  description = "Managed by OpenTofu. Talos data disks."
  name        = "k8s-data"
  node_name   = "miniquarium"
  on_boot     = false
  protection  = true
  started     = false
  tags        = sort(["infra", "ne_pas_demarrer", "ne_pas_supprimer"])
  vm_id       = 9000

  disk { # control plane data disk 1
    backup       = true
    datastore_id = "local-nvme"
    file_format  = "raw"
    interface    = "scsi10"
    size         = 48
  }
  disk { # worker 1 data disk 1
    backup       = true
    datastore_id = "local-nvme"
    file_format  = "raw"
    interface    = "scsi11"
    size         = 64
  }
}

# Déclaration de machine virtuelle pour Talos
resource "proxmox_virtual_environment_vm" "talos_vm" {
  depends_on = [
    # proxmox_virtual_environment_download_file.talos_nocloud_image,
    proxmox_virtual_environment_file.meta_cloud_config
  ]
  for_each = var.nodes

  acpi                = true
  bios                = "ovmf"
  description         = each.value.vm_description
  keyboard_layout     = "fr"
  kvm_arguments       = each.value.vm_kvm_args
  machine             = "pc-q35-10.0"
  migrate             = true
  name                = each.value.vm_name
  node_name           = each.value.pve
  on_boot             = each.value.vm_on_boot ? true : false
  pool_id             = each.value.vm_pool_id
  scsi_hardware       = "virtio-scsi-single"
  started             = each.value.vm_started ? "true" : "false"
  stop_on_destroy     = true
  tablet_device       = false
  tags                = each.value.vm_tags
  timeout_create      = 180
  timeout_shutdown_vm = 30
  timeout_stop_vm     = 30
  vm_id               = each.value.vm_id

  agent {
    enabled = true
    timeout = "5m"
    trim    = true
  }

  cpu {
    cores   = each.value.vm_cpu_cores
    flags   = each.value.vm_cpu_flags
    numa    = true
    sockets = 1
    type    = each.value.vm_cpu_type
  }

  disk { # boot disk
    aio          = "native"
    backup       = false
    cache        = "none"
    datastore_id = each.value.vm_datastore_id_boot_disk
    discard      = "on"
    file_format  = each.value.vm_boot_disk_format
    file_id      = "local:iso/talos-v1.13.3-nocloud-amd64.img"
    interface    = "scsi0"
    iothread     = true
    replicate    = false
    size         = each.value.vm_boot_disk_size
  }

  dynamic "disk" { # data disk
    for_each = { for idx, disk in proxmox_virtual_environment_vm.talos_vm_data.disk : idx => disk if disk.interface == each.value.data_vm_interface_disk }
    iterator = data_disk
    content {
      datastore_id = data_disk.value.datastore_id
      discard      = "on"
      file_format  = data_disk.value.file_format
      size         = data_disk.value.size
      interface    = each.value.vm_data_disk_interface
    }
  }

  dynamic "efi_disk" {
    for_each = each.value.vm_efi ? [1] : []
    content {
      datastore_id      = each.value.vm_datastore_id_efi_disk
      file_format       = "raw"
      pre_enrolled_keys = false
      type              = "4m"
    }
  }

  initialization {
    datastore_id = each.value.vm_datastore_id_initialization
    dns {
      domain  = each.value.vm_domain
      servers = each.value.vm_dns
    }
    ip_config {
      ipv4 {
        address = "${each.value.vm_ip}/24"
        gateway = each.value.vm_gateway
      }
      ipv6 {
        address = "dhcp"
      }
    }
  }

  memory {
    dedicated = each.value.vm_memory_dedicated
    floating  = each.value.vm_memory_floating
  }

  network_device {
    bridge      = "vmbr0"
    firewall    = false
    mac_address = each.value.vm_mac_address
    model       = "virtio"
    rate_limit  = each.value.vm_eth_rate_limit
  }

  operating_system {
    type = "l26"
  }

  serial_device {}

  startup {
    order      = each.value.vm_startup_order
    up_delay   = 15
    down_delay = 60
  }

  dynamic "tpm_state" {
    for_each = each.value.vm_tpm ? [1] : []
    content {
      datastore_id = each.value.vm_datastore_id_tpm
      version      = "v2.0"
    }
  }

  vga {
    clipboard = "" # false if empty
    type      = "virtio"
  }
}

resource "proxmox_virtual_environment_file" "meta_cloud_config" {
  for_each = var.meta_config_metadata

  content_type = "snippets"
  datastore_id = each.value.snippet_datastore_id
  node_name    = each.value.snippet_pve

  source_raw {
    data      = each.value.data
    file_name = "${each.key}_ci_meta-data.yml"
  }
}

Let’s move on to a rather large file, parameters.auto.tfvars. Change the values according to your environment:

hcl
kubernetes_version     = "1.34.8"
talos_cluster_name     = "poctalos"
talos_cluster_endpoint = "172.16.255.1"
talos_version          = "1.13.3"

meta_config_metadata = {
  "ber" = {
    snippet_datastore_id = "local"
    snippet_pve          = "pve"
    data                 = <<-EOF
      instance-id: talos-cp
      local-hostname: ber
    EOF
  }

  "nar" = {
    snippet_datastore_id = "local"
    snippet_pve          = "pve"
    data                 = <<-EOF
      instance-id: talos-wkr
      local-hostname: nar
    EOF
  }
}

nodes = {
  "ber" = {
    data_vm_interface_disk = "scsi10" # port on the data VM disk
    pve                    = "pve"
    role                   = "controlplane"
    usage                  = "controlplane"
    vm_boot_disk_format    = "raw"
    vm_boot_disk_size      = 24
    vm_cpu_cores           = 2
    vm_cpu_type                    = "x86-64-v2-AES"
    vm_data_disk_interface         = "scsi1" # port on the control plane VM
    vm_datastore_id                = "local"
    vm_datastore_id_boot_disk      = "local"
    vm_datastore_id_data_disk      = "local"
    vm_datastore_id_efi_disk       = "local"
    vm_datastore_id_initialization = "local"
    vm_datastore_id_tpm            = "local"
    vm_description                 = "Managed by OpenTofu. Talos controlplane"
    vm_dns                         = ["172.16.255.253"]
    vm_domain                      = "dc.home.arpa"
    vm_efi                         = true
    vm_eth_rate_limit              = 0
    vm_gateway                     = "172.16.255.254"
    vm_id                          = 99121
    vm_install_disk                = "/dev/sda"
    vm_ip                          = "172.16.255.1"
    vm_kvm_args                    = ""
    vm_mac_address                 = "BC:24:11:CA:FE:21"
    vm_memory_dedicated            = 6144
    vm_memory_floating             = 6144
    vm_name                        = "ber"
    vm_on_boot                     = true
    vm_pool_id                     = ""
    vm_started                     = true
    vm_startup_order               = 2
    vm_tags                        = ["controlplane", "k8s", "opentofu", "talos"]
    vm_timeservers                 = ["fr.pool.ntp.org", "time.cloudflare.com"]
    vm_tpm                         = true
  }

  "nar" = {
    data_vm_interface_disk = "scsi11" # port on the data VM disk
    pve                    = "pve"
    role                   = "worker"
    usage                  = "infra"
    vm_boot_disk_format    = "raw"
    vm_boot_disk_size      = 24
    vm_cpu_cores           = 3
    vm_cpu_type                    = "x86-64-v2-AES"
    vm_data_disk_interface         = "scsi1" # port on the worker VM
    vm_datastore_id                = "local"
    vm_datastore_id_boot_disk      = "local"
    vm_datastore_id_data_disk      = "local"
    vm_datastore_id_efi_disk       = "local"
    vm_datastore_id_initialization = "local"
    vm_datastore_id_tpm            = "local"
    vm_description                 = "Managed by OpenTofu. Talos worker"
    vm_dns                         = ["172.16.255.253"]
    vm_domain                      = "dc.home.arpa"
    vm_efi                         = true
    vm_eth_rate_limit              = 0
    vm_gateway                     = "172.16.255.254"
    vm_id                          = 99122
    vm_install_disk                = "/dev/sda"
    vm_ip                          = "172.16.255.2"
    vm_kvm_args                    = ""
    vm_mac_address                 = "BC:24:11:CA:FE:22"
    vm_memory_dedicated            = 16384 # 16 GB
    vm_memory_floating             = 16384 # 16 GB
    vm_name                        = "nar"
    vm_on_boot                     = true
    vm_pool_id                     = ""
    vm_started                     = true
    vm_startup_order               = 3
    vm_tags                        = ["k8s", "opentofu", "talos", "worker"]
    vm_timeservers                 = ["0.pool.ntp.org", "1.pool.ntp.org", "2.pool.ntp.org", "3.pool.ntp.org"]
    vm_tpm                         = true
  }
}

The talos.tf file contains the Kubernetes cluster declaration. In addition, there are configurations specific to each node type, the addition of manifests to install Cilium (and the configurations needed for gateway-api and L2 announcement), and the creation of the configuration files for the control planes and worker. The code isn’t entirely mine; I reused part of the work by Vegard S. Hagen (https://blog.stonegarden.dev/ ) and also Quentin JOLY (https://une-tasse-de.cafe/blog/talos/ ).

hcl
# Getting the kubeconfig and talosconfig can be done with "terraform output -raw kubeconfig > $HOME/.kube/config"
# and "terraform output -raw talosconfig > $HOME/.talos/config"".
# source : https://github.com/siderolabs/contrib/tree/main/examples/terraform/basic
# source 2 : https://github.com/vehagn/homelab/blob/main/tofu/kubernetes/talos/config.tf

resource "talos_machine_secrets" "this" {}

data "talos_machine_configuration" "controlplane" {
  for_each = var.nodes

  cluster_endpoint   = "https://${var.talos_cluster_endpoint}:6443"
  cluster_name       = var.talos_cluster_name
  kubernetes_version = var.kubernetes_version
  machine_secrets    = talos_machine_secrets.this.machine_secrets
  machine_type       = "controlplane"
  talos_version      = var.talos_version

  config_patches = [
    yamlencode({
      machine = {
        kubelet = {
          nodeIP = {
            validSubnets = ["172.16.255.0/24"]
          }
        }
        features = {
          hostDNS = {
            enabled              = true
            forwardKubeDNSToHost = false
            resolveMemberNames   = true
          }
        }
        network = {
          hostname    = each.value.vm_name
          nameservers = each.value.vm_dns
          interfaces = [
            {
              addresses = ["${each.value.vm_ip}/24"]
              dhcp      = false
              interface = "eth0"
              routes = [
                {
                  network = "0.0.0.0/0"
                  gateway = each.value.vm_gateway
                }
              ]
            }
          ]
        }
        install = {
          disk = each.value.vm_install_disk
        }
        time = {
          servers = each.value.vm_timeservers
        }
      }
      cluster = {
        etcd = {
          advertisedSubnets = ["172.16.255.0/24"]
        }
        discovery = {
          enabled = false
          registries = {
            service = {
              disabled = true
            }
          }
        }
        network = {
          cni = { # Cilium will replace it
            name = "custom"
            urls = [
              "https://raw.githubusercontent.com/julienhmmt/homelab/refs/heads/main/kubernetes/00-cilium/00-cilium-custom.yaml"
            ]
          }
        }
        proxy = { # Cilium will replace it
          disabled = true
        }
        allowSchedulingOnControlPlanes = false
        extraManifests = [
          # Gateway API
          "https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.4.1/config/crd/standard/gateway.networking.k8s.io_gatewayclasses.yaml",
          "https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.4.1/config/crd/standard/gateway.networking.k8s.io_gateways.yaml",
          "https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.4.1/config/crd/standard/gateway.networking.k8s.io_httproutes.yaml",
          "https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.4.1/config/crd/standard/gateway.networking.k8s.io_referencegrants.yaml",
          "https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.4.1/config/crd/standard/gateway.networking.k8s.io_grpcroutes.yaml",
          "https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.4.1/config/crd/experimental/gateway.networking.k8s.io_tlsroutes.yaml",
          # Cilium
          "https://raw.githubusercontent.com/julienhmmt/homelab/refs/heads/main/kubernetes/00-cilium/00-cilium-gapi.yaml",
          "https://raw.githubusercontent.com/julienhmmt/homelab/refs/heads/main/kubernetes/00-cilium/00-cilium-l2announcement.yaml"
        ]
      }
    })
  ]
}

resource "talos_machine_configuration_apply" "controlplane" {
  depends_on = [data.talos_machine_configuration.controlplane]
  for_each   = { for key, value in var.nodes : key => value if value.role == "controlplane" }
  lifecycle { replace_triggered_by = [proxmox_virtual_environment_vm.talos_vm[each.key]] } # re-run config apply if vm changes

  client_configuration        = talos_machine_secrets.this.client_configuration
  machine_configuration_input = data.talos_machine_configuration.controlplane[each.key].machine_configuration
  node                        = each.value.vm_ip
}

data "talos_machine_configuration" "worker" {
  for_each = var.nodes

  cluster_endpoint   = "https://${var.talos_cluster_endpoint}:6443"
  cluster_name       = var.talos_cluster_name
  kubernetes_version = var.kubernetes_version
  machine_secrets    = talos_machine_secrets.this.machine_secrets
  machine_type       = "worker"
  talos_version      = var.talos_version

  config_patches = [
    yamlencode({
      machine = {
        network = {
          hostname = each.value.vm_name
        }
        install = {
          disk = each.value.vm_install_disk
        }
        time = {
          servers = ["fr.pool.ntp.org", "time.cloudflare.com"]
        }
      }
    })
  ]
}

resource "talos_machine_configuration_apply" "worker" {
  depends_on = [data.talos_machine_configuration.worker]
  for_each   = { for key, value in var.nodes : key => value if value.role == "worker" }
  lifecycle { replace_triggered_by = [proxmox_virtual_environment_vm.talos_vm[each.key]] } # re-run config apply if vm changes

  client_configuration        = talos_machine_secrets.this.client_configuration
  machine_configuration_input = data.talos_machine_configuration.worker[each.key].machine_configuration
  node                        = each.value.vm_ip
}

data "talos_client_configuration" "this" {
  cluster_name         = var.talos_cluster_name
  endpoints            = [for node in var.nodes : node.vm_ip if node.role == "controlplane"]
  client_configuration = talos_machine_secrets.this.client_configuration
}

resource "talos_machine_bootstrap" "this" {
  depends_on           = [talos_machine_configuration_apply.controlplane]
  client_configuration = talos_machine_secrets.this.client_configuration
  node                 = [for k, v in var.nodes : v.vm_ip if v.role == "controlplane"][0]
}

data "talos_cluster_health" "this" {
  depends_on = [
    talos_machine_configuration_apply.controlplane,
    talos_machine_configuration_apply.worker,
    talos_machine_bootstrap.this
  ]
  endpoints              = data.talos_client_configuration.this.endpoints
  client_configuration   = data.talos_client_configuration.this.client_configuration
  control_plane_nodes    = [for k, v in var.nodes : v.vm_ip if v.role == "controlplane"]
  skip_kubernetes_checks = true
  timeouts = {
    read = "10m"
  }
  worker_nodes = [for k, v in var.nodes : v.vm_ip if v.role == "worker"]
}

resource "talos_cluster_kubeconfig" "this" {
  depends_on = [
    talos_machine_bootstrap.this,
    data.talos_cluster_health.this
  ]
  client_configuration = talos_machine_secrets.this.client_configuration
  endpoint             = var.talos_cluster_endpoint
  node                 = [for k, v in var.nodes : v.vm_ip if v.role == "controlplane"][0]
  timeouts = {
    read = "1m"
  }
}

output "talosconfig" {
  value     = data.talos_client_configuration.this.talos_config
  sensitive = true # Empêche l’affichage en clair dans les logs
}

output "kubeconfig" {
  value     = talos_cluster_kubeconfig.this.kubeconfig_raw
  sensitive = true # Empêche l’affichage en clair dans les logs
}

A few explanations:

  • The talos_machine_secrets resource generates the secrets needed to secure communications between the Talos nodes. These secrets are used to encrypt communications and authenticate the nodes, ensuring that only authorized machines can join the cluster.
  • The talos_machine_configuration resource configures the Talos nodes with specific parameters, such as network configurations, Kubernetes settings, and configuration patches. These configurations make it possible to customize each node according to its role (control plane or worker) and its specific needs.
  • The talos_machine_configuration_apply resource then applies these configurations to the nodes, ensuring that each machine is correctly configured before joining the cluster.
  • Cilium is used as the CNI (Container Network Interface). Cilium is chosen for its high performance and its ability to provide advanced network security through the use of eBPF. Compared to other solutions like Calico or Flannel, Cilium offers better visibility and more granular control over network traffic. The network configurations applied include defining the network interfaces, IP addresses, and routes, ensuring that each node can communicate effectively within the cluster.

OpenTofu will run the manifests in the extraManifests section in the order given. For example, a Gateway manifest cannot be launched before GatewayClass, because the Gateway type requires the GatewayClass to be created. Be careful about the order of the lines.

In addition, in this section, I add the manifests corresponding to gateway-api (which replaces ingresses). For Cilium 1.19.4, Talos 1.13.3 and Kubernetes 1.34.8, the version of Gateway-API to use is 1.4.1 (check the Cilium documentation to verify the Cilium ↔ Gateway-API compatibility for your version). Here are the commands:

bash
curl -O https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.4.1/config/crd/standard/gateway.networking.k8s.io_gatewayclasses.yaml
curl -O https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.4.1/config/crd/standard/gateway.networking.k8s.io_gateways.yaml
curl -O https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.4.1/config/crd/standard/gateway.networking.k8s.io_httproutes.yaml
curl -O https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.4.1/config/crd/standard/gateway.networking.k8s.io_referencegrants.yaml
curl -O https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.4.1/config/crd/standard/gateway.networking.k8s.io_grpcroutes.yaml
curl -O https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.4.1/config/crd/experimental/gateway.networking.k8s.io_tlsroutes.yaml

I also add two other manifests to finalize the gateway-api configuration for Cilium. The benefit of this block lies in its ability to incorporate manifests to deploy applications or configurations into Kubernetes via OpenTofu, without needing the local kubectl tool. A very handy feature!

The last file of the project, talos-img.tf, is fairly easy to understand. It downloads the Talos virtual image directly into Proxmox storage.

hcl
locals {
  talos = {
    version = "v1.13.3"
    schema  = "ca84112fe212adcded1df4faf04156b49a915b14926c43e19fc09b06be2d06ae"
  }
}

resource "proxmox_virtual_environment_download_file" "talos_nocloud_image" {
  content_type            = "iso"
  datastore_id            = "local"
  node_name               = "miniquarium"
  file_name               = "talos-${local.talos.version}-nocloud-amd64.img"
  url                     = "https://factory.talos.dev/image/${local.talos.schema}/${local.talos.version}/nocloud-amd64.raw.gz"
  decompression_algorithm = "gz"
  overwrite               = false
}

In the locals block, change the value in schema to match your needs and the result obtained from the factory.talos.dev website form.

Now you have all the files to build your Kubernetes cluster with Talos, in virtual machines on your Proxmox host, all driven by OpenTofu. Run the tofu plan and tofu apply commands to start the setup. After several minutes, you should see the machines running, Talos started, and your Kubernetes cluster being instantiated. The installation time can vary depending on the performance of your physical and virtual machines. As an illustration, on a Ryzen 7 7700 processor, 64 GB of DDR5 RAM and a PCIe gen 5 NVMe disk, the total time, from creation to completion (including ready-to-use instances and application deployment), was about 3 minutes and 30 seconds.

Finally, don’t forget to retrieve the kubeconfig and talosconfig files via the tofu output commands:

bash
mkdir -p ~/.talos ~/.kube
tofu output -raw kubeconfig > ~/.kube/config
tofu output -raw talosconfig > ~/.talos/config

You can find an up-to-date version in the GitHub repository at the following address: https://github.com/julienhmmt/homelab/tree/main/opentofu/talos-nocloud .

And now?#

Once the deployment is complete, check that it’s working properly and that it’s accessible with the kubeconfig file by running kubectl get nodes or the kubectl get pods -n kube-system -o wide command. These two commands let you confirm that the Kubernetes cluster is working properly. The pods may take a little while to start and reach the running state, especially the Cilium ones.

FAQ and troubleshooting common problems#

Q: How do I resolve network connectivity problems between Talos nodes?

A: Check the network configurations in the parameters.auto.tfvars file, in particular the IP addresses, network interfaces, bridges, and routes. Make sure the nodes can reach each other and that the firewall rules don’t block the necessary traffic.

Q: How do I update the Talos node configurations?

A: Edit the configuration files in the OpenTofu project and apply the changes using the tofu plan and tofu apply commands. The nodes will be automatically updated with the new configurations.

Q: How do I add new worker nodes to the cluster?

A: Duplicate a node configuration block in the parameters.auto.tfvars file and apply the changes with tofu plan && tofu apply. The new node will be automatically configured and added to the cluster.

Q: How do I monitor the health of the Talos cluster?

A: Like any Kubernetes cluster, it’s essential to deploy monitoring tools such as a Prometheus + Grafana stack, VictoriaMetrics, or an Elasticsearch + Metricbeat stack. You can also use the talosctl tool:

  • talosctl health: This command runs a set of health checks on all nodes in the cluster and gives an overall overview of the cluster state (connectivity, etcd, Kubernetes, etc.)
  • talosctl cluster show: Displays general information about the local provisioned cluster, including the nodes, their role and their state
  • talosctl -n <NODE_IP> logs <service>: Retrieves the events of a node. For example, for etcd: talosctl -n <NODE_IP> logs etcd.
  • talosctl -n <NODE_IP> containers --kubernetes: Lets you check the state of the Kubernetes system pods (kube-apiserver, kube-controller-manager, etc.) if the control plane isn’t yet accessible via kubectl.
  • 📚 Official Talos documentation
  • 🛠️ OpenTofu (Terraform fork)
  • 🖥️ Proxmox VE

Talos Kubernetes on Proxmox using OpenTofu Talos is an immutable operating system designed to only run Kubernetes. The advantage of Talos is an out-of-the-box Kubernetes install, as well as a smaller attack surface, and easier maintenance. Stonegarden Vegard S. Hagen Talos - An immutable OS for Kubernetes Talos is an operating system for Kubernetes. It’s designed to be lightweight, secure and easy to use. In this article, I’ll introduce Talos and its specifics. Une tasse de café Quentin JOLY

Article info
Tags
#Linux #Opentofu #Terraform
Author
Julien HOMMET
Published
Reading time
18 minutes
Word count
3783 words
Category
Tuto , Kubernetes , Proxmox , Talos
Last modified

Related articles

#tuto No image
#linux

Create Proxmox LXC containers with Terraform/OpenTofu

These days, task automation and the creation of immutable infrastructures are essential. With this in mind, the use of tools such as Terraform or its open-source fork OpenTofu becomes particularly interesting for the creation of LXC, automated and in mass.

Read more

Latest in #linux