UC Davis

Microservice-based deployments of Latency-Critical cloud services (LC-Services) pose a well-studied Tail-Energy co-optimization challenge: meeting Service-Level Objective (SLO) for tail latency of requests while minimizing energy consumption. Prior works have shown that CPU heterogeneity can be exploited at server-level to improve the overall energy-efficiency of LC-Services. In my research work, I show that exploiting CPU heterogeneity at the cluster-level can reap additional benefits. As a part of my research, I propose two control-plane strategies to exploit the CPU heterogeneity at the cluster-level. First, a Reinforcement Learning based technique to perform load balancing across a homogeneous cluster of Heterogeneous Multi-Processors (HMPs). Second, a heuristic-based instance scaling and load balancing technique to perform the co-optimization on a heterogeneous cluster of Symmetric Multi-Processors (SMPs). In addition, I also explore OS-level approaches to achieve tail latency predictability and perform Tail-Energy co-optimization by exploiting the CPU frequency scaling (or, as I call it, the Core-frequency heterogeneity).

During my research, I also inferred that the currently enforced SLOs for LC-Services are cloud-centric instead of being user-centric, i.e. cloud SLOs guarantee intra-cloud latency, but not the user QoE. To guarantee statistical bounds on user QoE, the control plane strategies must account for the external network delay information of requests. I propose a novel user-centric paradigm and implement a user-centric SLO-enforcement framework built on top of Kubernetes that performs QoE-aware instance scaling, load balancing and task scheduling on a homogeneous cluster. I also demonstrate that significant energy saving can be achieved in this novel paradigm too by exploiting cluster heterogeneity.