UC Berkeley

Large language models (LLMs) are as capable as the context that they are given. This dissertation studies how to structure context, improving LLMs by orchestrating and engineering the contexts that they are given. First, we present context decomposition, a technique for breaking complex contexts into simpler contexts that specialized models are more capableof handling. Second, we show in a comparative study that, context rewriting, a method for re-representing conversational utterances into simpler contexts improves data labeling efficiency and modularity. Finally, we present context tuning, a technique to finetune LLMs to better handle input contexts.