Erasure coding on Proxmox VE

k and m

An erasure coded pool splits every object into k data chunks and computes m additional coding chunks from them, so that any k of the k + m total chunks are enough to reconstruct the original object. m is, in effect, how many simultaneous chunk losses the pool tolerates before data is actually gone — conceptually similar to what size - 1 means for a replicated pool, but computed very differently and, crucially, at a much better ratio of raw to usable capacity.

efficiency = k / (k + m)
Usable bytes per raw byte, for an erasure coded pool.

A 4+2 pool (k = 4, m = 2) keeps efficiency 4 / 6 ≈ 0.667 — two thirds of raw capacity is usable — while tolerating two simultaneous chunk losses. A replicated pool would need size 3 to tolerate even one full host loss with data still available, at efficiency 1 / 3 ≈ 0.333. That gap in efficiency, at a comparable or better level of protection, is the entire reason erasure coding exists.

min_size for an erasure coded pool

Proxmox derives min_size for an erasure coded pool rather than asking for it directly: k + 1 when m > 1, or exactly k when m = 1.

Failure domain host: how many hosts are actually needed

Under failure domain host, CRUSH places at most one chunk of any given object per host, exactly as it does for a replicated pool’s copies. An erasure coded pool with width k + m therefore needs at least k + m hosts to be placeable at all — fewer than that and Ceph cannot spread the chunks, which the planner reports as an error, not a warning, because the layout genuinely cannot be created.

hostsplaceable?self-healing after one host fails?
fewer than k + mno — errorn/a
exactly k + myesno spare failure domain to rebuild into
k + m + 1yesyes — exactly one spare failure domain
more than k + m + 1yesyes, with more than one spare

Exactly k + m hosts is enough to place the pool, and data stays available after one host fails as long as the surviving chunk count still meets min_size. What k + m hosts alone does not provide is self-healing: with every host already holding a chunk, there is nowhere for Ceph to rebuild the missing one, so the cluster stays degraded until the failed host returns. The recommendation, exactly mirroring the size vs. min_size case for replicated pools, is hosts ≥ k + m + 1: one spare host is what turns "still readable" into "Ceph fixes it on its own".

The metadata pool Proxmox creates alongside it

RBD images cannot keep their metadata (the block allocation bitmap and similar bookkeeping, technically omap data) on an erasure coded pool at all — object maps require the partial overwrites replication supports and EC does not. Because of that, pveceph pool create NAME --erasure-coding k=K,m=M creates two pools in one step: the erasure coded data pool itself, and a small, separately named, replicated metadata pool at the ordinary Proxmox default of size 3 / min_size 2 next to it. The planner shows both pools in its result — the metadata pool is genuinely tiny in real clusters, so the planner models its share as a deliberately generous placeholder rather than trying to compute the real figure precisely.

What erasure coding is good for

The following is common Ceph operating practice, not a hard rule enforced anywhere: erasure coding works technically on Proxmox for VM disks (rbd) as much as for anything else, but it costs meaningfully more write amplification and CPU per write than replication does, because every write has to recompute coding chunks rather than simply duplicating bytes. The usual recommendation is to reserve erasure coding for workloads that are comparatively write-light and read-heavy or append-mostly — backup targets, archives, and CephFS data pools are the classic fit — and to keep latency-sensitive, write-heavy VM storage on a replicated pool instead. The planner surfaces this as an informational note whenever an erasure coded pool is set to application rbd, precisely because it is a trade-off worth a deliberate decision, not a default to fall into.

Worked examples

7 nodes, EC 4+2 — 7 × 6 × 8 TB HDD nodes, one CephFS pool at EC 4+2: 7 hosts is exactly k + m + 1, so the planner reports one spare failure domain for self-healing; min_size is 5 (k + 1, since m = 2 > 1).

The same layout with only 5 hosts instead of 7 — everything else unchanged — is the planner’s canonical "not placeable" example: with k + m = 6 and only 5 hosts, CRUSH has nowhere to put the sixth chunk, so the planner reports an error and generates no commands at all, even though the raw and usable capacity figures are still computed and shown.

Choosing k, m and a failure domain

  • Efficiency improves as k grows relative to m, but so does the CPU cost of every recovery and every degraded read.
  • Prefer m ≥ 2 unless the pool can tolerate a supervised, no-reserve window during recovery.
  • Size the cluster to hosts ≥ k + m + 1, not just k + m, if self-healing without intervention matters.
  • Reserve EC for backup, archive and CephFS data; keep VM disks on replication unless there is a specific, measured reason not to.
  1. Pick k and m for the tolerated simultaneous loss and the wanted efficiency.
  2. Check hosts against k + m (placeable) and k + m + 1 (self-healing).
  3. Expect a small replicated metadata pool alongside the EC pool automatically.
k
Number of data chunks an erasure coded object is split into.
m
Number of coding chunks computed alongside the data chunks.

Open in the planner

Not affiliated with Proxmox Server Solutions GmbH, the Ceph Foundation or Red Hat/IBM. Proxmox and Ceph are trademarks of their respective owners.

Everything is calculated in your browser. Nothing is sent to a server.