Network & Infrastructure / High Availability & Recovery Design
Preemption and failback oscillation after partial recovery
A preferred node repeatedly regains ownership before its dependencies are ready, then fails again. Automatic failback can create more disruption than remaining on a healthy secondary.
Environment and inputs: Routed and clustered services; define failure ownership, fencing and application recovery requirements before testing. Diagnostic/design example; not tested on production equipment.
Collect the evidence
Collect resource health, dependency readiness, preemption settings, recovery delays and ownership history. Distinguish a genuinely stable recovered node from one whose heartbeat merely returned.
Explain the result
A higher priority expresses preference, not complete application readiness. Without suitable readiness and hysteresis, each transient recovery can trigger another disruptive move.
Plan the fix
Choose a documented failback policy based on service state, operational risk and supported cluster controls. Consider manual return where appropriate rather than assuming immediate preemption is always desirable.
Check the result and rollback
Test slow dependency recovery and intermittent health in isolation. Revert if the new policy strands service on a failed node or allows repeated moves without sustained readiness.