Placement groups: how Ceph arrives at pg_num

What a placement group is

A placement group, or PG, is the unit Ceph actually spreads across OSDs. Every object written to a pool is hashed into exactly one PG, and every PG is then mapped by CRUSH onto a set of OSDs: size copies for a replicated pool, or k + m chunks for an erasure coded one. Objects are never mapped onto OSDs directly, and that is deliberate. Tracking the placement of millions of individual objects would be far too expensive for CRUSH to compute on every change and for every OSD to keep in memory. A few hundred PGs per pool are cheap to track, cheap to rebalance and, done right, spread evenly across the cluster; millions of individually placed objects would not be.

Choosing pg_num — the number of PGs a pool is created with — is therefore a balance, not a single "more is better" knob. Too few PGs and data does not spread evenly across OSDs and hosts: some drives fill up noticeably faster than others, and a recovery after a failure has fewer independent units of work to parallelise across the surviving drives. Too many PGs and every OSD carries more per-PG memory and CPU overhead, peering after a restart takes longer, and past a hard limit Ceph refuses to create further pools outright. The planner exists to land in the middle of that range for the exact cluster entered, rather than leaving it to a guess.

The pgcalc formula

ideal = targetPgsPerOsd × osdCount × (percent / 100) / width
width = size for a replicated pool, k + m for an erasure coded one.

targetPgsPerOsd is either 100 or 200, following the pgcalc convention: 100 while the cluster is still expected to grow, 200 once it is at its final size. It is a planning input the person sizing the cluster chooses, not a value Ceph enforces. osdCount is every OSD in the cluster — the planner assumes a single CRUSH root, so device-class-specific roots are a later extension, not part of this formula today. percent is the share of the cluster’s raw capacity this particular pool is planned to hold. Dividing by width accounts for the fact that each of a pool’s "logical" PGs actually generates size (or k + m) separate PG replicas or chunks that all need placing.

Rounding to a power of two, and the 25 % rule

pg_num must always be a power of two. pgcalc’s rule takes the power of two nearest to the raw rawPgs value; if that nearest value sits more than 25 % below rawPgs, the next power of two up is used instead, so a pool is never left meaningfully under-provisioned just because it happened to fall on the wrong side of a rounding boundary.

roundToPgNum(), as implemented — doubling instead of log2 avoids floating-point rounding pushing an exact power of two to the wrong side.
lower = largest power of two <= rawPgs
upper = 2 * lower
candidate = (rawPgs - lower <= upper - rawPgs) ? lower : upper
if (candidate < rawPgs * 0.75) candidate = upper
pgNum = candidate
Selected rows of the pgcalc rounding table; the full derivation, including the tie-breaking rule, is on the methodology page.
rawnearest power of twogap vs. raw25 % guardpg_num
400512512 is nearer than 256not needed512
700512700 → 512 is 26.9 % belowtriggered → 10241024
320256320 → 256 is 20.0 % belownot triggered (≤ 25 %)256
606464 is nearer than 32not needed64
201620 → 16 is 20.0 % belownot triggered (≤ 25 %)16

700 rounding up to 1024 and 400 rounding up to 512 are the two worked examples the implementation plan calls out explicitly, and both come from exactly this rule: 700 is closer to 512 than to 1024 in absolute terms, but 512 undershoots 700 by more than a quarter, so the guard pushes the result to 1024 anyway. 400 needed no guard at all — 512 was already the nearer power of two.

PGs per OSD: the 200 and 250 thresholds

Every pg_num the planner recommends also feeds a cluster-wide figure, PGs per OSD, computed per pool as pgNum × width / osdCount and summed across every pool — including, for an erasure coded pool, the small replicated metadata pool Proxmox creates alongside it (see erasure coding on Proxmox VE). Above 200 PGs per OSD the planner warns that memory and CPU overhead is adding up on every OSD in the cluster. Above 250 — Ceph’s own mon_max_pg_per_osd — Ceph refuses outright to create further pools without an explicit override; the planner marks this severity danger. Below 50 PGs per OSD the planner flags the opposite problem instead: too little parallelism and an uneven spread of data across the cluster.

The autoscaler and target_size_ratio

Next to the static pg_num, the planner always shows the autoscaler alternative: setting pg_autoscale_mode on for the pool together with target_size_ratio = percent / 100. Ceph’s PG autoscaler then adjusts pg_num over time on its own, aiming for roughly 100 PGs per OSD (mon_target_pg_per_osd), and rebalances automatically as data grows or OSDs are added. The static number this page has walked through is best read as a sensible, deliberate starting point — not a number to defend forever once the autoscaler is switched on.

Proxmox defaults

A pool created straight from the Proxmox web interface, or with pveceph pool create and no explicit --pg_num, starts at 128 PGs with the autoscaler set to warn — not on. In warn mode Proxmox tells the administrator a resize would help, but it does not resize the pool automatically until the mode is switched. For anything beyond a first test pool, working out pg_num from the formula above, and deciding deliberately whether to also enable the autoscaler, is worth the extra minute.

Worked examples

Homelab: 3 nodes, one pool — 3 nodes × 4 OSDs × 1.92 TB, target 100 PGs/OSD, one pool at 100 %: pgcalc raw value 400 rounds to 512, which works out to 128 PGs per OSD.

SMB: 3 nodes, two pools — 3 nodes × 4 OSDs × 3.84 TB SSD, target 100 PGs/OSD, vmstore at 80 % and cephfs-data at 20 %: pgcalc rounds the two raw values, 320 and 80, to 256 and 64 PGs respectively — a reminder that PGs per OSD is summed across every pool, not judged per pool.

Archive: 7 nodes, erasure coded 4+2 — 7 nodes × 6 OSDs × 8 TB, one erasure coded pool at 100 %: pgcalc raw value 700 rounds to 1024 (the 25 % guard triggers), for roughly 146 PGs per OSD once the pool’s own metadata pool is counted.

Frequently confused points

  • A pool never gets fewer than one PG participation per OSD, even at a tiny percent — that is the minimum rule above, not the rounding rule.
  • pg_num is set once at pool creation (or changed explicitly); the autoscaler changes it later, but only when it is switched to on.
  • PGs per OSD is a cluster-wide figure summed across all pools, not a per-pool number — a cluster with several pools can cross 200 even though each individual pool looks modest.
PG
A group of objects Ceph places and moves as one unit; the input to CRUSH placement.
pg_num
The number of PGs a pool is created with; always a power of two in this planner.
  1. Estimate the raw pgcalc value for every pool.
  2. Round each to a power of two using the 25 % rule.
  3. Add up each pool’s contribution to PGs per OSD and check it against 50/200/250.

Open in the planner

Not affiliated with Proxmox Server Solutions GmbH, the Ceph Foundation or Red Hat/IBM. Proxmox and Ceph are trademarks of their respective owners.

Everything is calculated in your browser. Nothing is sent to a server.