Pooled Memory Can Pool the Tail
Memory pooling addresses a familiar inefficiency: one server exhausts capacity while another leaves expensive memory idle. Sharing capacity can reduce stranded bytes and make provisioning less sensitive to individual demand estimates. The inherited assumption is that independent demand will average out favorably. That assumption belongs to a workload distribution, not to the pooling mechanism. If a shared service event activates many consumers together, the pool can consolidate both available capacity and the queueing that determines their worst latency.
Imagine eight illustrative hosts, each requiring 10 GB/s during normal operation and 30 GB/s during a correlated recovery event. A 160 GB/s pool serves the 80 GB/s baseline comfortably, but the recovery demands 240 GB/s. Extra capacity in bytes does not resolve the bandwidth deficit. Queues grow unless admission or service rates change. Recovery traffic may be unusually sensitive because it competes with the foreground work whose failure triggered recovery, creating a common bottleneck where local memory previously provided partial isolation.
The necessary admission condition is not merely that average demand stays below pool bandwidth. For demand processes b_i(t), the controller must bound the probability and duration of Σb_i(t) exceeding the sustainable service rate. It must also consider request granularity, read and write mix, path asymmetry, and multiple links feeding the same controller. Even below saturation, an extra hop can lengthen a dependent load's critical path. Capacity pooling and latency improvement are therefore separate claims, and aggregate link rates do not establish either a tail bound or uniform access cost.
Design the pool with explicit failure and contention domains. Reserve service for critical traffic, keep latency sensitive working sets local, and place cold or elastic allocations where added delay is acceptable. Expose capacity availability separately from bandwidth entitlement. A scheduler that sees only free bytes can overcommit a path that already has enough resident clients. Tier placement also needs recovery policy: migration itself consumes the shared bandwidth that a stressed application hopes to acquire. In some deployments, smaller independently controlled pools can outperform a larger statistically efficient pool by containing synchronized demand.
The experiment should include host restart, cache warmup, checkpoint restore, and simultaneous tenant bursts alongside steady state utilization. Measure deadline misses and allocation failures jointly with per-path queueing and migration traffic. Compare the memory saved against any additional replicas, local reserves, or slower recovery required to preserve service. Pooling remains attractive when demand is sufficiently complementary and the sharing cost is controlled. Does your memory pool remove stranded capacity, or exchange isolated shortages for a synchronized tail that the whole fleet must provision around?