Network & Infrastructure / Storage Networks & Distributed Data
CRUSH failure domains and replicas sharing one physical risk
A pool reports its configured replica count, yet multiple copies reside in the same rack-level power domain. Logical copy count can overstate resilience to a shared physical event.
Environment and inputs: Ceph and storage transport dependencies; match health identifiers and recovery behavior to the installed release. Diagnostic/design example; not tested on production equipment.
Collect the evidence
Map CRUSH hierarchy, rule selection, pool settings and actual object placement to racks, hosts and power boundaries. Include device classes and any manually overridden placement.
Explain the result
Placement rules determine which failures the copies can survive independently. Three copies are not three independent fault domains when the hierarchy does not represent actual shared risk.
Plan the fix
Correct the physical inventory and reviewed CRUSH rule, planning the data movement and required headroom. Do not change failure domains without confirming enough eligible capacity exists.
Check the result and rollback
Inspect representative and sampled placement, then simulate the intended domain loss in a safe environment. Revert if placement cannot satisfy the rule or recovery would exceed available surviving capacity.