pve-imds

An OpenStack, EC2 and NoCloud IMDS (Instance Metadata Service) compatible metadata service for virtual machines running in Proxmox.

With pve-imds, an unmodified Linux cloud image running cloud-init can reach http://169.254.169.254 to retrieve not only instance metadata but also custom user data stored in the Proxmox VM Notes field. pve-imds also supports a signed identity document that a VM can use to authenticate to a service like Vault.

Quick start

1. Download the latest release of pve-imds and install it. This snippet will fetch the latest .deb package artifact URL from the GitHub API, download, and install it. It's all here for the sake of transparency over a curl ... | sudo bash alternative.

# Protip: hit the ⧉ button to copy to the clipboard then paste the whole command.
SUDO=$(command -v sudo >/dev/null 2>&1 && echo sudo || echo '') && \
tmp=$(mktemp --suffix=.deb) && \
curl -fL "$(curl -fsSL https://api.github.com/repos/wyattanderson/pve-imds/releases/latest \
    | python3 -c 'import sys,json;print(next(a["browser_download_url"] for a in json.load(sys.stdin)["assets"] if a["name"].endswith(".deb")))')" -o "$tmp" && \
$SUDO dpkg -i "$tmp" && \
rm -f "$tmp"

2. Download the latest cloud image from your distribution of choice. Here, we'll use Ubuntu 24.04 LTS:

curl -LO https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img

3. Import the cloud image as a new VM template. Substitute local-lvm and vmbr0 with your storage and bridge interface of choice.

Note

The --smbios1 configuration is necessary in order to convince cloud-init to look out over the network for configuration instead of expecting a CDROM.

qm create $(pvesh get /cluster/nextid) \
    --name noble-server-cloudimg-amd64 \
    --agent enabled=1 \
    --cpu cputype=host \
    --memory 2048 \
    --scsi0 local-lvm:0,import-from=$(pwd)/noble-server-cloudimg-amd64.img \
    --scsihw virtio-scsi-single \
    --smbios1 base64=1,product=$(echo -n 'OpenStack Nova' | base64) \
    --template 1 \
    --storage local-lvm \
    --net0 bridge=vmbr0,model=virtio

4. Note the VM ID of your new template. Set some user data and clone your template into a new VM:

cat << EOF > user-data
#cloud-config
package_update: true
packages:
  - qemu-guest-agent
runcmd:
  - [ systemctl, enable, --now, --no-block, qemu-guest-agent.service ]
ssh_authorized_keys:
  # replace this with your own SSH public key
  - ssh-ed25519 AAAAC3NzaC1lZ...
EOF

# pve-imds reads user data from the contents 
qm clone <TEMPLATE VMID> $(pvesh get /cluster/nextid) --description "$(echo '<!--#user-data'; cat user-data; echo '-->')" --name "my-test-vm"

5. Boot your new VM, look for the IP as reported by qemu-guest-agent, and try SSHing in. Verify that everything is working:

curl http://169.254.169.254/openstack/latest/meta_data.json

Why network-based configuration?

The ISO image approach that Proxmox uses (and that I used before this by generating my own ISO images with my own user data) isn't dynamic. With this approach, metadata is up-to-date immediately. Change a tag? Add user data? Immediately visible in the instance without regenerating the image.

Why not use the cloud-init support built-in to Proxmox?

Custom user data affords an incredibly powerful way to provision machines with cloud-init. It is extensively customizable and is an industry-standard method for complete unattended provisioning of machines.

Proxmox has basic support for cloud-init that works by generating an ISO image and attaching it to the VM. Unfortunately, Proxmox only lets you configure custom user data if you provide it via a snippet file from a storage source that supports snippets. This adds complexity, especially around replication.

With pve-imds, you can embed custom user data in the Notes field of a VM (inside a comment tag, so it isn't visible). This field is stored in the VM's replicated configuration file in /etc/pve/qemu-server/<VMID>.conf. Additionally, pve-imds serves all of the VM's metadata, including VM tags which can be fed into a config management tool like Ansible or Salt to specify things like machine roles or environments.

Instance identity documents and trust

For me, these capabilites alone were valuable enough to develop pve-imds. However, I also wanted a pathway to something like the AWS EC2 instance identity document, which provides an external mechanism for cryptographically verifying instance metadata. So, if you're using Proxmox tags to specify that a VM is a mysql server in the prod environment, that VM can prove it to another service that trusts a public key.

JWT-SVID identity tokens

VMs can use pve-imds to issue a short-lived JWT-SVID token. You can use these tokens to authenticate to a service like Vault.

# From inside the VM:
curl -s -X POST http://169.254.169.254/pve-imds/jwtsvid \
  -d "audience=https://vault.example.com"
# eyJhbGciOiJSUzI1NiIsImtpZCI6Ii4uLiIsInR5cCI6IkpXVCJ9...

When decoded, a token contains these claims:

{
  "aud": "foo",
  "exp": 1775087893,
  "hostname": "test-host",
  "iat": 1775087593,
  "jti": "6023722e-2af9-4f92-8a45-1fb98ccf4e11",
  "meta": {
    "another-tag": "",
    "centos-stream-9": "",
    "pve:node": "test-node",
    "pve:vmid": "106"
  },
  "name": "test-host",
  "sub": "spiffe://test-node/106",
  "uuid": "2c8c7c76-a26c-4eb7-b04c-bf52bc095b81"
}

A relying party can verify the signature of this token using the JWKS endpoint at http://169.254.169.254/.well-known/jwks.json. Rather than managing its own signing keys, pve-imds uses Proxmox's PKI. Tokens are signed using the node's private key, and the public keys of all nodes in the cluster are made available via the JWKS endpoint.

NoCloud datasource

pve-imds also serves the cloud-init NoCloud datasource over HTTP under http://169.254.169.254/nocloud/, for every VM and regardless of the emulate setting. This is what lets Talos Linux nocloud images, which support SecureBoot (unlike the Talos OpenStack image), and any cloud-init image provision from pve-imds.

Path	Response
/nocloud/ (or /nocloud)	List of the files below
/nocloud/meta-data	YAML with instance-id (the SMBIOS UUID, falling back to the VMID) and local-hostname (the VM name)
/nocloud/user-data	The same user data as the OpenStack and EC2 endpoints, after templating. Empty (200) when the VM has none
/nocloud/vendor-data	Empty
/nocloud/network-config	The VM's <!--#network-config Notes block as written (v1 or v2, with or without the top-level network: key), or, without one, DHCPv4 on every NIC matched by MAC address

Missing user-data is an empty 200 rather than a 404 because cloud-init's NoCloud datasource fails on a 404, while both cloud-init and Talos treat an empty body as "no user-data". network-config is always served because Talos retries a 404 for several minutes and only applies the hostname when it receives a network config.

To point a VM at it, set its SMBIOS serial number. Proxmox only accepts base64 in the serial field, so encode it and add base64=1. qm set --smbios1 replaces the whole smbios1 value, so keep the VM's existing UUID:

VMID=123
UUID=$(qm config $VMID | sed -n 's/^smbios1:.*uuid=\([^,]*\).*/\1/p')
SERIAL='ds=nocloud-net;s=http://169.254.169.254/nocloud/;h=talos-worker-1'
qm set $VMID --smbios1 "uuid=$UUID,serial=$(echo -n "$SERIAL" | base64 -w0),base64=1"

h= is optional; when set, it overrides the hostname from meta-data. The trailing / on the seed URL matters for older cloud-init releases; Talos and cloud-init 23.1+ add it if it is missing.

Static network config

To give a VM a static address instead of DHCP, add a <!--#network-config block to its Notes, next to any <!--#user-data block. It holds a standard cloud-init network config, v1 or v2:

<!--#network-config
network:
  version: 2
  ethernets:
    net0:
      match:
        macaddress: "bc:24:11:00:00:01"
      addresses:
        - 192.168.1.10/24
      routes:
        - to: 0.0.0.0/0
          via: 192.168.1.1
      nameservers:
        addresses: [192.168.1.1]
-->

The block is served as written; pve-imds only checks that it is YAML with version: 1 or 2. If it isn't, the VM gets the DHCP default instead and the problem is logged with the VMID, so check journalctl -u pve-imds if a static address doesn't apply.

Talos

With Talos, a VM without user-data gets its hostname and boots into maintenance mode, ready for talosctl apply-config. A VM with a machine config in its Notes boots straight into it. To keep the secrets in that config out of Notes, use templated user-data.

Set nocloud.enabled: false to disable the endpoint.

Templated user-data with OpenBao secrets

Notes are readable by anyone with VM.Audit on the VM and are included in backups, so they are not a good place for secrets such as a Talos machine config's tokens and keys. Instead, user data can contain placeholders that pve-imds fills in from an OpenBao (or Vault) KV v2 secrets engine when the VM fetches it:

<!--#user-data secrets=worker
version: v1alpha1
machine:
  type: worker
  token: {{ secret "machine_token" }}
  ca:
    crt: {{ secret "machine_ca_crt" }}
cluster:
  token: {{ secret "cluster_token" }}
  # ...the rest of your worker config...
-->

• secrets=<set> on the opening line names the secret set. Without it, user data is served exactly as written and no placeholders are processed, so cloud-init Jinja templates are unaffected. The attribute must be on the same line as <!--#user-data.
• The set is read from a KV path inside the VM's scope, which is either its resource pool or one of its tags (see Scoping secrets). Both come from Proxmox, never from anything the VM controls.
• {{ secret "<field>" }} is replaced with that field of the set. Whitespace inside the braces is allowed. Nothing else is interpreted: other {{ ... }} text is left as is, and values are inserted verbatim, so quote or indent them as your format requires (Talos expects base64 for certificates and keys, which keeps them on one line).
• Set names may contain only letters, digits, - and _ (up to 64 characters).
• Templated user-data applies to the NoCloud, OpenStack and EC2 user-data endpoints alike.
• The Notes field is limited to 8 KiB by Proxmox, which comfortably fits a worker config without its secrets.

For the example above, in pool k8s-dmz-01:

bao kv put kv/pve-imds/pools/k8s-dmz-01/worker \
    machine_token=... machine_ca_crt=... cluster_token=...

Scoping secrets: pools or tags

secrets.path_template decides what a VM's secret sets are scoped to. It must contain {name} (the set name) and exactly one of:

• {pool} (default, kv/data/pve-imds/pools/{pool}/{name}): the VM's Proxmox resource pool, read from /etc/pve/user.cfg. Nested pools (parent/child) become nested paths. A VM must be in a pool to use templating.

• {tag} (e.g. kv/data/pve-imds/tags/{tag}/{name}): the VM tag that starts with secrets.tag_prefix (default pve-imds-), without the prefix and lowercased. A VM tagged pve-imds-k8s-dmz-01 reads kv/data/pve-imds/tags/k8s-dmz-01/<set>. Other tags are ignored. A VM with no such tag, or with more than one, gets a 503.

  By default (secrets.require_registered_tags: true) the tag must also be listed in registered-tags in /etc/pve/datacenter.cfg (in the GUI under Datacenter → Options, or with pvesh set /cluster/options --registered-tags 'pve-imds-k8s-dmz-01;...'). Registered tags can only be set or removed by users with Sys.Modify on /. Without that, anyone with VM.Config.Options on any VM could tag it and read that scope's secrets. Turn the check off only if everyone who can edit VMs is trusted with all secrets.

  --registered-tags replaces the whole list, so include the tags that are already registered (pvesh get /cluster/options shows them). Tags are matched case-insensitively, as in Proxmox, unless tag-style sets case-sensitive=1.

  Tag scoping fits well with JWT-SVID roles that are bound to tags; register those tags for the same reason.

Pool membership, tags and registered-tags are re-read on every templated request, so changes, including ones made on other cluster nodes, take effect immediately.

Failure behaviour

Rendering fails closed. The user-data endpoints return 503 Service Unavailable with a Retry-After header, never a partially rendered or unrendered config, when:

• OpenBao is unreachable, returns an error, or rejects the token,
• the secret set or a referenced field doesn't exist (only string fields are used),
• the set name is invalid,
• the VM has no scope: it is not in a pool, or it has no, several, or an unregistered scope tag, or
• the Notes use secrets= but secrets.enabled is off.

cloud-init honours Retry-After; Talos keeps retrying until it gets the config rather than falling into maintenance mode. pve-imds logs the VMID, scope (pool or tag), set name and outcome of every templated request, never user-data or secret values.

Security model

• Secrets are bound to the VM's scope. A VM can only ever receive secrets from its own scope's prefix, whatever its Notes contain, so Proxmox permissions decide who can give a VM access to which secrets: Pool.Allocate on the pool for pool scoping, Sys.Modify on / for registered scope tags. Anyone who can put a VM into a scope, or edit the Notes of a VM in a scope, can obtain that scope's secrets through the VM.

• Keep secrets out of Notes. Notes are visible to VM.Audit and included in backups; set names and placeholders are not sensitive.

• Give pve-imds a read-only token. It needs nothing beyond read access to its prefix, for example:

  path "kv/data/pve-imds/pools/*" {   # or kv/data/pve-imds/tags/*
    capabilities = ["read"]
  }
  

• Separate clusters. If several Proxmox clusters share one OpenBao, give each cluster its own namespace with secrets.namespace (OpenBao 2.3+ or Vault Enterprise), so one cluster's token cannot read another's secrets. Without namespaces, use a different secrets.path_template prefix per cluster and scope each cluster's policy to it.

Token and OpenBao Agent

pve-imds has no authentication logic of its own. It reads a token from secrets.token_file on every request, so a token renewed by OpenBao Agent takes effect immediately. The service runs as a dynamic user that is a member of the pve-imds-token group (created by the package), so make the token file readable by that group, for example with an agent file sink in a setgid directory:

install -d -m 2750 -o root -g pve-imds-token /run/openbao-agent-pve-imds

sink "file" {
  config = {
    path = "/run/openbao-agent-pve-imds/token"
    mode = 0640
  }
}

The service unit only allows IPv4 sockets. If OpenBao is only reachable over IPv6, add a drop-in with systemctl edit pve-imds:

[Service]
RestrictAddressFamilies=AF_XDP AF_NETLINK AF_UNIX AF_INET AF_INET6

Setting it up

On every Proxmox node that runs pve-imds:

1. Run OpenBao Agent with a file sink that writes the token where the pve-imds-token group can read it, as above, and give its role a read-only policy for the secrets prefix (see Security model).

2. Enable templating in /etc/pve-imds/config.yaml (see Configuration) and restart the service:

   secrets:
     enabled: true
     address: https://openbao.example.com:8200
     token_file: /run/openbao-agent-pve-imds/token
     # for tag scoping, also:
     # path_template: kv/data/pve-imds/tags/{tag}/{name}
   

   systemctl restart pve-imds
   journalctl -u pve-imds -n 20   # a bad secrets config stops the service from starting
   

Then, once per scope and VM:

3. Store the secret set: bao kv put kv/pve-imds/pools/<pool>/<set> field=value ... (or kv/pve-imds/tags/<scope>/<set>).

4. Put the VM in the pool, or register the scope tag and add it to the VM:

   pvesh set /pools --poolid <pool> --vms <VMID>  # pool scoping
   qm set <VMID> --tags 'pve-imds-<scope>'        # tag scoping; replaces the VM's tags
   

5. Add secrets=<set> and the {{ secret "<field>" }} placeholders to the VM's <!--#user-data block.

6. Check the result from inside the VM with curl -i http://169.254.169.254/nocloud/user-data. A 503 means rendering failed; the pve-imds log on the VM's node has the reason as outcome (for example vm_not_in_pool, tag_not_registered or secret_fetch_failed).

PXE provisioning

pve-imds can provision VMs over iPXE. The VM's iPXE ROM chains http://169.254.169.254/pxe on every boot, and /pxe answers with an iPXE script that either provisions the VM or runs exit so the firmware falls through to the local disk. pve-imds stays read-only towards Proxmox: the desired state is in the VM's Notes, the available images are in a manifest, and the recorded state is in OpenBao.

There are two image types:

• disk: a raw.zst disk image. /pxe chains a live OS (a UKI served by pve-imds at /pxe/live.efi). The live OS fetches /pxe/job, writes the image to the VM's disk and reports it with POST /pxe/done, which records the VM as provisioned.
• chain: a remote iPXE target, such as a Talos Image Factory URL. /pxe answers chain <url>. With boot: once (the default) the VM is recorded as provisioned just before the chain, so it boots from disk afterwards; with boot: always it is chained on every boot and no state is kept.

The <!--#pxe block

The desired state is a YAML block in the VM's Notes:

<!--#pxe
image: debian13        # required
version: 2026.10.01    # optional; omit for the manifest's latest version
disk: /dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0   # optional, disk images only
-->

image and version may only contain a-z, 0-9, ., _ and -. disk must match ^/dev/disk/by-id/[A-Za-z0-9._:-]+$ and defaults to the VM's first SCSI disk. Any other key, invalid YAML, a missing image or an invalid value makes the block invalid, which is treated like no block and logged as a warning with the VMID and the reason.

The manifest

The manifest lists the images and their versions. pve-imds fetches it from pxe.manifest_url over HTTPS and reuses it for pxe.manifest_ttl. Redirects are not followed. Failed fetches are not cached, and an expired manifest is never served when a fetch fails: the VM boots from disk instead.

{
  "images": {
    "debian13": {
      "type": "disk",
      "latest": "2026.10.01",
      "versions": {
        "2026.10.01": {
          "url": "https://rgw.example.com/pxe/debian13/2026.10.01.raw.zst",
          "sha256": "<64 hex characters>",
          "size": 123456789
        }
      }
    },
    "talos": {
      "type": "chain",
      "boot": "once",
      "latest": "v1.14.2",
      "versions": {
        "v1.14.2": { "url": "https://pxe.factory.talos.dev/pxe/<schematic>/v1.14.2/nocloud-amd64-secureboot" }
      }
    }
  }
}

• Image IDs and versions use the same charset as the Notes block.
• A disk version needs an https url, a 64-hex sha256 and a size greater than 0.
• A chain version needs a url that starts with one of pxe.chain_allowed_prefixes, and nothing else. The allowlist is checked again whenever a URL is chained.
• boot is once (default) or always, and only valid for chain images.
• latest must be one of versions.
• URLs must be https, without user info, . or .. path segments, $ or whitespace, so they are safe to put into an iPXE script.
• Unknown keys are rejected. An invalid image or version is skipped and logged as a warning; the rest of the manifest is still used. An image whose latest is skipped is skipped as a whole. A manifest that isn't valid JSON of this shape is an error.
• With a private image bucket, a disk url must also be in that bucket.

Tools that write the manifest can validate it with the same code by importing github.com/wyattanderson/pve-imds/pkg/manifest and calling manifest.Parse with the same chain_allowed_prefixes and bucket URL; they should reject a manifest when any entry is skipped.

Private image bucket

Disk images don't have to be public. With pxe.images.bucket_url set, the manifest and the images live in a private S3 bucket (such as a Ceph RGW bucket) and pve-imds reads the access keys of a read-only user from OpenBao:

• The manifest is fetched with a SigV4-signed request. manifest_url must be in the bucket.
• Every disk url in the manifest must be in the bucket; other disk entries are skipped.
• /pxe/job returns a presigned URL for the image instead of the manifest's url. It is minted for each request and valid for pxe.images.url_lifetime (15 minutes by default). S3 checks the lifetime when a download starts, so a download may take longer. A live OS that retries a download should fetch /pxe/job again rather than reuse the old URL.
• If the keys can't be read, /pxe/job answers 409 with outcome image_url_error.

Signing is pure computation, so pve-imds never handles image data. The keys are re-read from OpenBao every 5 minutes, so rotating them needs no restart. Presigned URLs are never logged. Chain images are not affected.

Recorded state

The recorded state of a VM is a KV v2 entry at kv/data/pve-imds/pxe/<uuid>, keyed by the VM's SMBIOS UUID (lowercased). It never falls back to the VMID, because VMIDs get reused. The entry has the string fields image, version, sha256 (empty for chain images), provisioned_at (RFC 3339) and method (disk or chain). State reads are never cached.

Decision

/pxe first resolves the block's image and version (or the manifest's latest), then compares it with the record:

Situation	Response	outcome
The VM cannot be identified, or has no valid SMBIOS UUID	exit	no_identity, no_uuid
No valid <!--#pxe block	exit	no_block
The manifest cannot be fetched	exit	manifest_error
The image or version is not in the manifest	exit	manifest_miss
Chain image with boot: always	chain <url> (no state read or write)	chain_always
Reading the state fails	exit	state_error
The record matches: same image, and the pinned version (if any) is the recorded one	exit	provisioned
Otherwise, disk image	chain /pxe/live.efi (exit if live_uki is unreadable)	provision_new, provision_changed (live_uki_missing)
Otherwise, chain image with boot: once	write the record, then chain <url> (exit if the write fails)	chain_once (state_error)

A VM without a version is provisioned with the latest version once and then left alone: a newer latest never reprovisions it. Every /pxe response is a 200 with Content-Type: text/plain and Cache-Control: no-store, even when pve-imds hits an error, because iPXE treats any other status as a failure and retries instead of booting. Every request to a PXE endpoint logs one line with the VMID, UUID, image, version and outcome; failures also log a warning with the reason.

The live OS endpoints

These only serve the VM the request comes from, like every other endpoint.

Endpoint	Response
GET /pxe/live.efi	The live_uki file (application/octet-stream, Cache-Control: no-store), read on every request so it can be replaced without a restart. Only while the VM's decision is to provision a disk image; otherwise 404.
GET /pxe/job	{"image", "version", "url", "sha256", "size", "disk"} for the version resolved now; url is presigned with a private image bucket. Only while the VM's decision is to provision a disk image; otherwise 409 with {"error", "outcome"}.
POST /pxe/done	Body {"image", "version", "sha256"}. 204 once recorded; 409 if image isn't the block's, version isn't the pinned one, or the version or sha256 doesn't match the manifest; 503 with Retry-After if OpenBao or the manifest is unavailable; 400 for a malformed body. A repeated identical report is 204 without a new write.

No job state is kept between /pxe/job and /pxe/done: the report is checked against the Notes and the manifest as they are when it arrives. If latest moved while an unpinned VM was installing, the report of the older version is still accepted as long as that version is in the manifest.

Reprovisioning

To reprovision a VM, change image or version in its Notes, or delete its record:

bao kv delete kv/pve-imds/pxe/<uuid>

Setting it up

1. Configure /etc/pve-imds/config.yaml. The state store connects with the address, token_file, ca_file and namespace of the secrets section, which must have at least address and token_file; secrets.enabled (templating) does not need to be on.

   pxe:
     enabled: true
     manifest_url: https://rgw.example.com/pxe/manifest.json
     chain_allowed_prefixes:
       - https://pxe.factory.talos.dev/pxe/
   secrets:
     address: https://openbao.example.com:8200
     token_file: /run/openbao-agent-pve-imds/token
   

   pve-imds refuses to start if manifest_url isn't https, or if an allowed prefix isn't https or doesn't end with /. The trailing / keeps a prefix from matching a sibling path such as /pxe-other/.

2. Copy the live OS UKI to /var/lib/pve-imds/live.efi (the service's state directory) on every node and make it readable. It can be replaced at any time.

3. Add this to the policy of the pve-imds token:

   path "kv/data/pve-imds/pxe/*" {
     capabilities = ["read", "create", "update"]
   }
   

   delete is deliberately left out: deleting a record makes the VM reprovision on its next boot, so only an operator should be able to do that.

4. Optionally, keep the images private (see Private image bucket):

   # A bucket without public ACLs or policy, and a read-only user for pve-imds
   radosgw-admin user create --uid pve-imds-images --display-name "pve-imds images (read-only)"
   

   Grant that user read access with a bucket policy:

   {
     "Version": "2012-10-17",
     "Statement": [{
       "Effect": "Allow",
       "Principal": {"AWS": ["arn:aws:iam:::user/pve-imds-images"]},
       "Action": ["s3:GetObject"],
       "Resource": ["arn:aws:s3:::pxe-images/*"]
     }]
   }
   

   Store its keys in OpenBao, with the field names radosgw-admin prints, and let pve-imds read them:

   bao kv put kv/pve-imds/rgw-images access_key=... secret_key=...
   

   path "kv/data/pve-imds/rgw-images" {
     capabilities = ["read"]
   }
   

   Then set the bucket and move manifest_url into it:

   pxe:
     manifest_url: https://rgw.example.com/pxe-images/manifest.json
     images:
       bucket_url: https://rgw.example.com/pxe-images/
   

The service unit only allows IPv4 sockets. If the manifest server is only reachable over IPv6, add AF_INET6 with the same drop-in as for OpenBao.

Configuration

pve-imds reads /etc/pve-imds/config.yaml if it exists (or the file given with --config). Every option can also be set with an environment variable, e.g. PVE_IMDS_SECRETS_ENABLED=true. The options for NoCloud, templated user-data and PXE provisioning:

nocloud:
  enabled: true            # serve /nocloud/ (default: true)

secrets:
  enabled: false           # render secrets= user-data (default: false)
  address: https://openbao.example.com:8200   # must be https
  namespace: ""            # optional, sent as X-Vault-Namespace
  ca_file: /etc/pve-imds/openbao-ca.pem       # optional; default: system CAs
  token_file: /run/openbao-agent-pve-imds/token
  path_template: kv/data/pve-imds/pools/{pool}/{name}  # API path below /v1/; {pool} or {tag}
  tag_prefix: pve-imds-    # with {tag}: the tag that selects the scope
  require_registered_tags: true  # with {tag}: only trust registered tags
  cache_ttl: 60s           # how long a fetched secret set is reused

pxe:
  enabled: false           # serve /pxe and the live OS endpoints (default: false); uses the secrets connection settings
  state_path_template: kv/data/pve-imds/pxe/{uuid}  # API path below /v1/; must contain {uuid}
  manifest_url: https://rgw.example.com/pxe/manifest.json  # required when enabled; must be https
  manifest_ca_file: ""     # optional; default: system CAs
  manifest_ttl: 60s        # how long a fetched manifest is reused
  live_uki: /var/lib/pve-imds/live.efi  # live OS UKI served at /pxe/live.efi
  chain_allowed_prefixes:  # URL prefixes chain images may use; https, ending in /
    - https://pxe.factory.talos.dev/pxe/
  images:                  # private image bucket (optional)
    bucket_url: ""         # e.g. https://rgw.example.com/pxe-images/; https, ending in /; empty: no signing
    region: default        # SigV4 region; RGW accepts its zonegroup name
    credentials_path: kv/data/pve-imds/rgw-images  # API path below /v1/; fields access_key and secret_key
    url_lifetime: 15m      # lifetime of presigned image URLs

PVE_IMDS_PXE_CHAIN_ALLOWED_PREFIXES takes a comma-separated list.

path_template is the KV v2 API path, so it includes the mount's data/ segment. It must contain {name} and exactly one of {pool} or {tag}. Secret sets are cached per scope and set for cache_ttl; errors are never cached.

The configuration is read once at startup, so run systemctl restart pve-imds after changing it. The service runs as a dynamic user, so the file must be readable by other users (the default 0644 is fine; it holds no secrets) or by the pve-imds-token group. To use environment variables instead, add them in a drop-in with systemctl edit pve-imds:

[Service]
Environment=PVE_IMDS_SECRETS_ENABLED=true
Environment=PVE_IMDS_SECRETS_ADDRESS=https://openbao.example.com:8200

How it works

I think the EC2 IMDS approach (now shared by other hypervisors and clouds) is extremely elegant. A VM can make a request to http://169.254.169.254 and retrieve information about itself, like looking in a mirror. The VM doesn't even need a valid DHCP lease because 169.254.169.254 is a link-local IP address and the packets (at least in the case of pve-imds) never leave the hypervisor.

This does make the solution slightly more complicated. To provide a strong guarantee that a VM will be able to retrieve its identity and only its identity, the hypervisor must be able to intercept and directly respond to packets leaving the VM destined for 169.264.169.254. Doing this at line rate speeds is challenging. In the cloud, hypervisors (I'm guessing) offload this interception to hardware like AWS Nitro. In the case of pve-imds, we use XDP (eXpress Data Path) to intercept packets as soon as they hit the VM's tap interface before they hit the rest of the kernel networking stack (or whatever, I'm not an expert). I've been able to maintain full line rate from a VM bridged to a 25GbE network with this approach.

XDP intercepts the full Ethernet frame, so we use the gVisor userspace TCP stack to handle the path between the raw AF_XDP socket and the VM-specific HTTP handler. An in-memory metadata cache parses Proxmox VM configuration files for updates, refreshing as necessary via fsnotify.

Known limitations

My goal with releasing this is to hopefully improve it with community feedback. I find it useful in my homelab. There is absolutely no warranty, express or implied, that it is suitable in its current or future form for any use case of any import. Use at your own risk.

I have only tested this with VMs running on Proxmox 9 using virtio interfaces (though I don't think the interface type matters). This will not work with VMs using SR-IOV, unless the VM has a secondary interface with a link-local address.

Future work

• hardware offload using ASAP² direct

AI disclosure

This was not vibe-coded. I used Claude Code to author most of the code here, but by specifying exactly the architecture and approach that I personally designed as an experienced software and infrastructure engineer, and with thorough review. I could have written all of this on my own (and I did for the prototype), but this approach saved me a fuckton of time and enabled me to do things like write conformance tests that test pve-imds against cloud-init.

License

This fork is distributed as a whole under the GNU Affero General Public License, version 3 or (at your option) any later version. See LICENSE.

It builds on wyattanderson/pve-imds, which is MIT licensed. The MIT license allows its code to be redistributed under the AGPL as part of this fork, provided its copyright and permission notice is kept; that notice is in LICENSES/MIT.txt.

Each file says which applies in its SPDX-License-Identifier header:

• AGPL-3.0-or-later: written for this fork.
• MIT AND AGPL-3.0-or-later: upstream code with changes made in this fork.
• No header: upstream code unchanged in this fork, still available under the MIT license.

Because pve-imds is a network service, section 13 of the AGPL applies if you run a modified version for other users: you must offer them its source code.