Managing power consumption and improving energy efficiency is a key driver in the design of computing devices today. This is true for both battery powered mobile devices as well as mains-powered desktop PCs and servers. In case of mobile devices, the focus of optimization is on energy efficiency to maximize battery lifetime. In case of mains- powered devices, we seek to optimize power consumption to reduce energy costs, thermal and environmental concerns. Traditionally, there are two main mechanisms to improve energy efficiency in systems: slowdown techniques that seek to reduce processor speed or radio power against the rate of work done, and shutdown techniques that seek to shut down specific components or subsystems - such as processor, radio, memory - to reduce power used by these components when not in use. The adverse effect of using these techniques is either reduced performance (e.g., increase in latency) and/or usability or loss of functionality. The thesis behind this dissertation is that improved energy efficiency can be achieved through system architectures that seek to design and exploit "collaboration" among heterogeneous but functionally similar subsystems. For instance, multiple radio interfaces with different power/performance characteristics can collaborate to provide an energy- efficient wireless communication subsystem. Furthermore, we show that in systems where such heterogeneity is not naturally present, we can introduce heterogeneous components to improve overall energy efficiency. We show that using collaboration, individual subsystems and even entire platforms can be shut down more aggressively to reduce energy consumption, while reducing adverse impacts on performance or usability. We have used collaboration to do energy efficient operation in several contexts. For battery powered mobile devices we show that wireless radios are the dominant power consumers, and then describe several techniques that use various heterogeneous radios present on these devices in a collaborative manner to improve their battery lifetime substantially, on average by two to three times and in some cases up to 8 times. First we present "Cell2Notify", a technique in which a lower power radio is used purely to wakeup a higher power radio. Next, we present "CoolSpots" and "SwitchR", systems that build a hierarchy of collaborative radios to choose the most appropriate radio interface, taking into account application characteristics as well as various energy and performance metrics. In the case of wall-powered desktop and laptop Personal Computers (PCs) we show that the dominant power consumers are the processors themselves. We then describe "Somniloquy", an architecture that augments a PC with a separate low power secondary processor that can perform some of the functions of the host PC on its behalf. We show that by using the primary processor (i.e. the PC) collaboratively with the secondary processor we can shut down PCs opportunistically, and as a result reduce the overall energy consumption by up to 80% in most cases