Network & Infrastructure / High Availability & Recovery Design
Anycast health withdrawal and the difference between process and service
An anycast node keeps advertising after its backend fails because the routing daemon is still healthy. Requests continue reaching a node that cannot complete the service.
Environment and inputs: Routed and clustered services; define failure ownership, fencing and application recovery requirements before testing. Diagnostic/design example; not tested on production equipment.
Collect the evidence
Map the advertised prefix to application readiness, upstream dependencies and the withdrawal mechanism. Record when the backend fails, health changes, routing updates and users recover.
Explain the result
Anycast routing selects a reachable advertising node, not inherently a functioning application. The advertisement must be connected to an appropriate service-health contract to avoid attracting failed work.
Plan the fix
Implement supported bounded health-based withdrawal with protection against transient flapping. Keep node liveness, backend readiness and global route availability as separate signals.
Check the result and rollback
Test backend-only and whole-node failures, including recovery and route reentry. Revert if health oscillation causes excessive route churn or all nodes withdraw for a shared noncritical dependency.