Recent years have witnessed a rapid growth in the amount of generated data, owing to the emergence of Internet of Things (IoT). Processing such huge data on traditional computing systems is highly inefficient, mainly due to the limited cache capacity and memory bandwidth. Processing in-memory (PIM) is an emerging paradigm which tries to address this issue. It uses memories as computing units, hence reducing the data transfers between memory and processing cores. However, the application of present PIM techniques is restricted by their limited functionality and inability to process large amounts of data efficiently. In this thesis, we propose novel techniques which exploit the analog properties of emerging memory technologies. Not only do these support more complex functions such as addition, multiplication, and search but also manage and process large data more efficiently. We present a new blocked PIM architecture which uses inter-block interconnects to accelerate data intensive processing. We also introduce a heterogeneous architecture having general purpose cores and PIM-enable memory and a data-dependent task allocation scheme for it. We also apply application specific optimizations and approximation techniques to further design accelerators for neural networks and database query systems. While we design a multiplication-by-constant hardware for neural networks, query processing is accelerated by a novel in-memory nearest search technique. Our neural network accelerator achieves 113.9x higher energy efficiency and 56.3x speedup as compared to AMD GPU. Also, the query accelerator provides 49.3x performance speedup and 32.9x energy savings as compared to recent Intel CPU.

Learning with data from multiple domains is a longstanding topic in machine learning research. In recent years, deep neural networks (DNN) have shown remarkable performance on different machine learning tasks. However, how to efficiently utilize deep neural networks for learning with multiple domains is largely unexploited. A model aware of the relationships between different domains can be trained to work on new domains with fewer resources and achieve better performance. However, to identify and leverage the transferable structure is challenging.

In this dissertation, we propose novel methods which allow efficient learning across multiple domains in several different scenarios. First, we address the problem of learning across two image domains with deep neural networks. We propose two adaptive methods which allow different images to fine-tune and reuse different residual blocks and convolutional filters of the pre-trained model. Experimental results show that the proposed SpotTune outperforms the standard fine-tuning on 12 out of 14 datasets. Second, we consider the case that the target domain only has few examples per category which is referred to as the cross-domain few-shot problem. We establish a new benchmark for cross-domain few-shot learning and propose a multi-model selection algorithm which achieves an average improvement of 2\% compared with the state-of-the-art approach on the proposed benchmark. Third, we consider learning with multiple domains simultaneously. We propose a multi-domain learning method based on depthwise separable convolution which achieves the highest score on the Visual Decathlon Challenge and reduces the number of parameters by 50% compared with the state-of-the-art approach. We further propose an efficient multi-domain learning method for distributed training in sensor networks. The proposed method can reduce the communication cost by up to 53% and energy consumption by up to 67% without accuracy degradation compared with conventional approaches. Finally, we address learning with multiple domains sequentially. We propose an algorithm called mixed stochastic gradient descent (MEGA) which allows the model to maintain the performance on old domains while being trained on a new domain. MEGA achieves an average accuracy of 91.21±0.10% on Permuted MNIST, which is 2% better than the previous state-of-the-art model. On Split CIFAR, the proposed MEGA achieves an average accuracy of 66.12±1.93%, which is about 5% better than the state-of-the-art method.

Recent years have witnessed a rapid growth in the amount of generated data. Learning algorithms, like hyperdimensional (HD) computing, promise to reduce the computation complexity of processing such a huge amount of data. However, traditional computing systems are highly inefficient for such algorithms, mainly due to the limited cache capacity and memory bandwidth. Processing in-memory (PIM) is an emerging paradigm which tries to address these issues by using memories as computing units. In this dissertation, we propose a PIM-based HD computing architecture that accelerates all phases of the HD computing pipeline namely, encoding, training, retraining, and inference. Our architecture is enabled by fast and energy-efficient in-memory logic operations, combined with a hardware-friendly distance metric. However, the improvements from PIM decrease as the size of the dataset increases beyond the memory capacity. Hence, we also design an in-storage computing (ISC) solution. Our ISC design includes on-flash-chip acceleration of HD encoding, which encodes multiple data points in parallel across different flash chips, exploiting the high parallelism provided by the flash hierarchy. This is supported by a controller-level accelerator that performs HD training, retraining, inference, and clustering. Our proposed PIM and ISC solutions provide 434x and 222x speedup as compared to the state-of-the-art HD computing implementations on CPU.Many applications, most notably in healthcare, finance, and defense, rely on cloud computing for learning tasks and demand privacy which today’s solutions cannot fully provide. Fully homomorphic encryption (FHE) elevates the bar of today’s solutions by adding confidentiality of data during processing, while introducing noticeable data size expansion - the ciphertext is 5000x bigger than the aggregate of native data types. In this dissertation, we present a design of the first PIM-based accelerator of both client and server using the latest Ring-GSW based homomorphic encryption schemes. Our design supports various security levels and provides on average 2007x higher throughput than CPU while running FHE-enabled neural networks. This improvement comes from a significant reduction in total data-transfers and the high number of processing in-memory cores, which enables higher parallelism and deeper pipelining in our design.

The smart grid is driven by distributed, heterogeneous energy sources and controllable loads matched to sources to optimize energy usage. Ubiquitous data collection of energy generation and consumption is used for prediction of sources and loads, which in turn is used for net metering, and controlling retail energy prices and loads. This thesis proposes using context — additional high-level information — about elements of the smart grid (sources, loads, and storage) to improve the efficiency of its operation.

We first investigate the integration of local renewables in data centers, some of the largest consumers in the grid. We mitigate the variability of renewables with short-term prediction of solar and wind energy using additional environmental data. This allows us to use renewable energy for computing loads, the largest segment of energy consumption while maintaining quality of service requirements of the data center. We extend this work to a networked set of data centers, allocating and migrating jobs to improve energy efficiency.

While individual large consumers are of particular importance to the grid, there is relatively little focus on automated regulation of smaller consumers like individual houses, despite the residential sector consuming over 1/3 of national energy consumption. We propose a residential electrical energy simulation platform that enables investigating the impact of technologies such as renewable energy, different battery types, centralized vs. distributed in-home energy storage, and smart appliances.

While batteries are typically used as a buffer against inconsistent output from renewables, the growing deployment of reverse-power-operation systems provides residences with the ability to profit from buying and selling energy at the retail time-of-use rate. We develop a formulation for battery usage based on more realistic battery models, optimizing the benefit of discharging the battery and minimizing the time to return on investment.

Finally, unlike data centers, residential energy is directly driven by the activities and behavior of the users. The growing advent of sensing in the IoT presents a unique opportunity: using general-purpose modular data processing to generate grid-related context: energy prediction and flexibility, which can be used to drive individualized automated actuation that scales to the size of the grid.

Today, there are more mobile devices than human beings on the planet. Mobiles execute a wide variety of applications and are expected to perform in many different environments, from the deserts to the mountain tops. Their goal is to meet user’s expectations and deliver high quality of experience. Unfortunately, to achieve this they face a set of interrelated problems. The heterogeneous components that execute varied tasks can drain a battery in a matter of hours. Power dissipation raises the device temperature, which can be a source of discomfort for the user. Temperature stress also dramatically increases the impact of reliability degradation mechanisms on transistors and interconnects, which can lead to early failure. These problems only worsen with CMOS scaling, which reduces the accuracy of the fabrication process and increases the variability in power, performance and degradation rate. Dynamic management mitigates such issuesby adapting the operating conditions at runtime. Strategies have been very well studied for traditional systems like desktops and servers, but unfortunately they cannot be applied to mobiles, because they do not consider user experience. Existing dynamic management for mobiles only begins to target the problems of power and thermal management, and offers limited solutions for reliability and variability management. This is fundamental to guarantee power savings and prevent early failure in a scenario of increasing variability. In this dissertation, we propose the design and real implementation of a novel unified framework for the comprehensive dynamic management of power, temperature, reliability and variability in mobile systems, subject to user experience requirements. We develop novel lightweight solutions for power and thermal management that optimize for both application behavior and battery lifetime. We complement this with our framework for the online emulation of reliability degradation and variability, which enables the development and rapid prototyping of hardware management solutions. We leverage our emulation framework to design and integrate dynamic reliability and variability management for mobile systems. The presented solutions have been implemented and tested on real devices. Our strategy can meet user experience requirements with a selected target battery lifetime extension of at least 25%. Also, it achieves up to 35% savings in power consumption at the device level, with up to 1 year reliability lifetime improvement for a multicore platform and up to 100% of performance improvement on cluster architectures. Finally, on devices affected by variability it achieves up to 160% performance improvement over the state-of-the-art while meeting the lifetime constraints.

The Internet of Things (IoT) is a growing network of heterogeneous devices, combining various sensing and computing nodes at different scales, which creates a large volume of data. Many IoT applications use machine learning (ML) algorithms to analyze the data. The high computational complexity of ML workloads poses significant computational challenges to IoT computing platforms, which tend to be less-powerful and resource-constrained devices. Transmitting such large volumes of data to the cloud also have various issues such as scalability, security and privacy. In this dissertation, we propose efficient solutions to perform the ML tasks while decreasing power consumption and improving performance. We first leverage the heterogeneous and interconnected nature of the IoT systems, where IoT applications run on many different architectures (e.g., X86 server or ARM-based edge device) while communicating with each other. We present a cross-platform power and performance prediction technique for intelligent task allocation. The proposed technique estimates the time-variant energy consumption with only 7% error across completely different architectures, enabling the intelligent task allocation that saves the energy consumption of 16.5% for state-of-the-art ML workloads.

We next show how to further advance the learning procedures towards real-time and online processing by distributing such learning tasks onto the hierarchy of IoT devices. Our solution leverages brain-inspired high-dimensional (HD) computing to derive a new class of

learning algorithms that can easily run on IoT devices, while providing high accuracy comparable to the state-of-the-arts. We present that the HD-based learning algorithms can cover various real-world problems from conventional classification to other cognitive tasks beyond classical MLs such as DNA pattern matching. We demonstrate that the HD-based learning can enable secure, collaborative learning by efficiently distributing a large volume of learning tasks into heterogeneous computing nodes. We have implemented the proposed learning solution on various platforms while offering superior computing efficiency. For example, our solution achieves 486×

and 7× performance improvements for each of the training and inference phases on a low-power ARM processor, as compared to state-of-the-art deep learning.

The Internet of Things (IoT) envisions a web-connected infrastructure comprising billions of sensors and actuation devices that collect contextual data about the environment and respond accordingly. Machine and environmental context-aware computing opens up opportunities to learn new abstract information from sensed data, and is key for both standalone and interconnected applications to run efficiently. Since the volume of such data is growing exponentially, it is essential to develop automated techniques to derive useful data for design of scalable systems. We present a middleware framework along with analytical proofs that reduce the computing complexity of vast context collection and communication of abstract data among distributed nodes.

Effective and scalable distributed systems rely on accurate and flexible modeling of the devices and their interactions. In this dissertation, we study the use of context extraction across a hierarchy of devices, including resource-constrained embedded platforms as well as more powerful edge servers. We show that relevant context for optimizing performance and energy consumption, even for a single server executing database workloads, goes well beyond traditional variables like performance counters. In fact, we are the first to show that the speed of cooling fans can strongly affect the energy-delay product of a system. We leverage this additional contextual information as input to convex optimization that reduces the energy consumption of a server by 65% compared to basic PID controllers, or by 19% in comparison to advanced hardware management techniques proposed in literature. At an ecosystem level, we compare our distributed infrastructure to current monolithic implementations with single- point communications between sensor nodes and the cloud servers, demonstrating a reduction in combined system energy by 22-45%, and increasing the battery lifetime of power-constrained devices by at least 22x.

Faster and more efficient hardware is needed to handle the rapid growth of Big Data processing. Applications such as multimedia, medical analysis, computer vision, and machine learning can be parallelized and accelerated using General-Purpose Computing on Graphics Processing Units (GPGPUs). However, these are power intensive and novel approaches are needed to improve their efficiency. Many of applications also show a tolerance to noise within their computation. Approximate computing is a design strategy in which energy savings and speedup can be achieved at the expense of accuracy. If carefully controlled, many applications can accept small amounts of error and still produce acceptable results. This thesis proposes a number of methods to enable approximate computing for GPUs.

We first examine a number of approaches for approximating operations at the core level. Floating point arithmetic, specifically multiplies, make up the majority of instructions computed on GPUs. In this dissertation we propose a configurable floating point unit (CFPU) which eliminates the costly manitassa multiply by copying one of the input mantissa directly to the output. For applications with a higher amount of temporal similiarity we propose adaptive lookup (ALook) to use small dynamic look up tables to store recently computed operations. This low power look up table provides nearest distance matches to provide results rather than computing on the exact hardware.

GPUs issue threads in groups, commonly 32, called warps. Cores in a warp run the same instructions in lock-step. Every instruction within a warp must be accelerated to provide performance improvements. To control accuracy, we run the most erroneous approximate results on the exact hardware. Bottlenecks can arise as some threads spend time computing exact results while others use approximate solutions. We propose two methods to handle this problem. First, we use warp pass through to target warps in which a very small fraction of threads must be computed exactly. To handle warps with a larger percentage of exact computations, we utilize warp value trading (WVT). Under WVT, operations are traded between warps running on the same multiprocessor to create uniform groups of either exact or approximate operations.

Finally, we focus on application specific approximation. We show approximation can be used to accelerate neural networks during training and inference. Early stages of training tolerate more error than later ones, so we adjust the level of approximation over time. To accelerate inference we approximate larger operations to a lesser degree than larger ones to increase hit rate. For training we show that gradually reducing the maximum allowed error per operation results in 7.13x EDP improvement and 4.64x speedup training of four different neural network applications with less than 2% quality loss. For inference we are able to automatically select parameters based on user prediction requirements for neural networks and improves speedup by 2.9x speedup and EDP by 6.2x of inference across six neural networks.

Advancements in communication and computation, energy storage and renewable generation, along with the recent support of policy makers have been transforming the power grid into the Smart Grid. In this dissertation, we propose an optimized control framework that addresses four major interconnected problems present in today's grid. We develop a fast, low complexity and accurate forecast algorithm that supplies solar energy predictions to the control algorithms. We improve the accuracy by up to 50% as compared to the state of the art solutions. We complement this with an optimal distributed nonlinear battery control algorithm that uses a high-accuracy nonlinear battery model. Our solution reduces the utility bill of an actual building by up to 50%, a 25% improvement over heuristic solutions. Furthermore, we show that state of the art linear optimal control algorithms incur up to 250% higher costs due to model inaccuracies accumulated over time. We develop a smart grid simulator, S2Sim, that enables the co-simulation of distributed control algorithms. Using S2Sim we show that our battery control algorithm can improve the stability of the grid by up to 45%. Finally, we develop an optimal packet aggregation solution that can optimize individual goals, such as information freshness or energy, in a network-wide manner. We show through case studies that our solution has constant performance under increasing congestion, reduces energy consumption by up to 60%, increases information freshness by up to 55% and adapts to dynamic conditions in real-time, an important quality for distributed control in future Smart Grid applications.