The role of software in computer networks has never been more crucial than today, with the advent of Internet-scale services and cloud computing. The trend toward software-based network dataplane—as in network function virtualization—requires software packet processing to meet challenging perfomance requirements, such as supporting exponentially increasing link bandwidth and microsecond-order latency. Many architectural aspects of existing software systems for packet processing, however, are decades old and ill-suited to

today’s network I/O workloads.

In this dissertation, we explore the design space of high-performance software packet processing systems in the context of two application domains, . First, we start by discussingthe limitations of BSD Socket, which is a de-facto standard in network I/O for server applications. We quantify its performance limitations and propose a clean-slate API, called MegaPipe, as an alternative to BSD Socket. In the second part of this dissertation, we switch our focus to in-network software systems for network functions, such as network switches and middleboxes. We present Berkeley Extensible Software Switch (BESS), a modular framework for building extensible network functions. BESS introduces various novel techniques to achieve high-performance software packet processing, without compromising on either programmability or flexibility.

There has been much research devoted to improving the performance of data analytics frameworks, but comparatively little effort has been spent systematically identifying the performance bottlenecks of these systems. Without an understanding of what factors are most important to performance, users do not know how to choose a software and hardware configuration to optimize runtime, and developers do not know which optimizations are most important to implement.

This thesis explores how to architect systems for performance clarity: the ability to understand where bottlenecks lie and the performance implications of various system changes. First, we focus on incrementally adding performance clarity to current data analytics frameworks. We develop blocked time analysis, a methodology for quantifying performance bottlenecks in parallelized systems, and use it to analyze the Spark framework’s performance on two SQL benchmarks and one production workload. Contrary to commonly-held beliefs about performance, we find that (i) CPU (and not I/O) is often the bottleneck, (ii) improving network performance can improve job completion time by at most 2%, and (iii) the causes of most stragglers can be identified.

Blocked time analysis helped to understand performance bottlenecks in today’s frameworks, but fell short of enabling users to reason about the impact of potential hardware and software configuration changes. Given the challenges to providing performance clarity in current architectures, the second part of this thesis focuses on a new system architecture built from the ground up for performance clarity. Rather than breaking jobs into tasks that pipeline many resources, as in today’s frameworks, we propose breaking jobs into units of work that each use a single resource, called monotasks. We demonstrate that explicitly separating the use of different resources simplifies reasoning about performance without sacrificing fast runtimes. Our implementation of monotasks provides job completion times within 9% of Apache Spark, and leads to a model for job completion time that predicts runtime under different hardware and software configurations with at most 28% error for most predictions.

Today's networks do much more than merely deliver packets. Through the deployment of middleboxes, enterprise networks today provide improved security -- e.g., filtering malicious content -- and performance capabilities -- e.g., caching frequently accessed content. Although middleboxes are deployed widely in enterprises, they bring with them many challenges: they are complicated to manage, expensive, prone to failures, and challenge privacy expectations.

In this thesis, we aim to bring the benefits of cloud computing to networking.

We argue that middlebox services can be outsourced to cloud providers in a

similar fashion to how mail, compute, and storage are today outsourced. We begin

by presenting APLOMB, a system that allows enterprises to outsource middlebox

processing to a third party cloud or ISP. For enterprise networks, APLOMB can

reduce costs, ease management, and provide resources for scalability and

failover. For service providers, APLOMB offers new customers and business

opportunities, but also presents new challenges. Middleboxes have tighter

performance demands than existing cloud services, and hence supporting APLOMB

requires redesigning software at the cloud. We re-consider classical cloud

challenges including fault-tolerance and privacy, showing how to implement

middlebox software solutions with throughput and latency 2-4 orders of magnitude

more efficient than general-purpose cloud approaches. %Some of the technologies discussed in this thesis are presently being adopted by industrial systems used by cloud providers and ISPs.

Today’s networks provide more than connectivity. Network functions such as firewalls, caches, WAN optimizers play a crucial role in improving security and performance capabilities. Although network functions traditionally have been implemented as dedicated hardware middleboxes, a recent effort commonly referred to as Network Function Virtualization (NFV) promises to bring the advantages of cloud computing to network packet processing by moving network appliance functionality from proprietary hardware to software. However, while NFV has quickly gained remarkable momentum in the industry, accepted NFV approaches are merely replacing monolithic hardware with monolithic software.

In this dissertation, we argue that current approaches to NFV are ill-suited to the original vision of NFV. Instead, NFV needs a framework that serves as a common runtime for network functions. We present E2 – an NFV framework that provides placement and elastic scaling with high-level network function composition interface. We further consider the privacy challenge of outsourcing NFV deployments in public clouds and present a functional cryptographic technique for privacy-preserving packet classification. Finally, we discuss optimizing NF data-plane scheduling for performance guarantees.

Modern datacenters are the foundation of large scale Internet services, such as search engines, cloud computing and social networks. In this thesis, I will investigate the new challenges in building and managing large scale datacenters. Specifically, I will show trends and challenges in the software stack, the hardware stack and the network stack of modern datacenters, and propose new approaches to cope with these challenges.

In the software stack, there is a movement to serverless computing where cloud customers can write short-lived functions to run their workloads in the cloud without the hassle of managing servers. Yet the storage stack has not changed to accommodate serverless workloads. We build a storage system called Savanna that significantly improves the serverless applications performance. In the hardware stack, with the end of Moore's Law, researchers are proposing disaggregated datacenters with pools of standalone resource blades. These resource blades are directly connected by the network fabric, and I will present the requirements of building such a network fabric. Lastly, in the network stack, new congestion control algorithms are proposed (e.g. pFabric) to reduce the flow completion time. However, these algorithms require specialized hardware to achieve desirable performance. I designed pHost, a simple end-host based congestion control scheme that has comparable performance with pFabric without any specialized hardware support.

Modern datacenter applications need to provide resiliency to mask failures. These applications widely use techniques like Replicated State Machines to provide fault tolerance. Replicated State Machines typically rely on consensus protocols to provide availability and consistency. These applications also require high throughput and low latency from the underlying consensus protocols. Furthermore, in an effort to further reduce latency experienced by clients, we are seeing the emergence of edge computing; storage and computational resources are placed in between the clients and servers in datacenters (typically closer to the client). This placement provides many benefits: lower-latency responses to clients, lower bandwidth demands on the backbone and increased privacy. Stateful applications running on the edge pose a problem of losing state when an edge node fails. This dissertation looks at fault tolerance for datacenters and edge computing. First, RingWorld looks at datacenter fault tolerance for highly available lock services. RingWorld adapts ring-based consensus protocols to leverage programmable switches and datacenter topology. This allows RingWorld to provide higher throughput and comparable latency to existing lock services. Second, to provide fault tolerance for edge computing, CESSNA provides a mechanism to recover from edge failures for strongly stateful applications that ensures correctness and good performance. To do this, CESSNA defines the consistency guarantee for correctness in the face of edge failures, and recovers from failures by adapting techniques like log-replay.

There have been many recent proposals to change the network infrastructure in order to meet different performance objectives. These changes are often difficult to deploy, either requiring specialized network switching hardware or greatly complicating network management. Rather than continuing to add new features to the network in an adhoc manner, we advocate a more principled approach for meeting different performance objectives, that leads to a more stable network infrastructure. This approach is based on the following two questions:

(1) Can we avoid making changes to the network infrastructure by finding solutions that only change the end-points? Here, we focus on congestion control for both wide-area and datacenter networks, showing how the end-points can be updated to achieve near-optimal performance using commodity switches, and on redesigning RDMA NICs to eliminate their reliance on the in-network mechanism for loss avoidance.

(2) When infrastructure changes are needed, can we make them universal in nature? Here, we focus on packet scheduling, examining whether we can have a universal packet scheduling algorithm that can mimic all others. We show, both theoretically and practically, that we can have an almost-universal packet scheduling algorithm that can closely mimic other scheduling algorithms and can achieve a variety of network-wide performance objectives.