KnapsackLB: Enabling Performance-Aware Layer-4 Load Balancing

Layer-4 load balancer (LB) is a key building block of online services. Inthis paper, we empower such LBs to adapt to different and dynamic performanceof backend instances (DIPs). Our system, KNAPSACKLB, is generic (can work withvariety of LBs), does not require agents on DIPs, LBs or clients, and scales tolarge numbers of DIPs. KNAPSACKLB uses judicious active probes to learn amapping from LB weights to the response latency of each DIP, and then appliesInteger Linear Programming (ILP) to calculate LB weights that optimize latency,using an iterative method to scale the computation to large numbers of DIPs.Using testbed experiments and simulations, we show that KNAPSACKLB loadbalances traffic as per the performance and cuts average latency by up to 45compared to existing designs.