DRAM costs in modern data centers are rising rapidly due to increasingdemand and flattening cost per byte from the slowdown in device scaling. Recent efforts to address this problem include extending server DRAM with cost-effective options like compressed memory tiers, NVMe SSDs, and rack-level DRAM pools enabled by fast interconnects like RoCE. However, these far-memory extensions are much slower than local DRAM, causing application slowdown unless data is efficiently managed between local and far memory.

Existing systems support far memory in two ways. OS paging-basedsystems use virtual memory and hardware guards, requiring no application changes but suffering from inflexible kernel memory management policies based on limited page access information. App-integrated systems enable application-aware optimizations with a custom interface but require significant application redesign and/or introduce overhead on local accesses due to software guards. Compiler-guided efforts to automate application redesign are promising but face new challenges, such as requiring source code access for all dependencies and nascent support for various language frontends.

In this dissertation, we argue that paging-based systems can offerpolicy flexibility, including application guidance, without requiring disruptive changes. We present two new systems, PageFlex and Eden, to support this argument. PageFlex externalizes policies to enable general improvements for existing OS-based hyperscale deployments. For application guidance, we show that applications typically access far memory at a few easily identifiable code locations. Eden leverages lightweight hints at these locations to convey application semantics, achieving similar performance as the existing app-integrated systems but with significantly fewer code changes.