Clinicians are routinely faced with the practical challenge of integrating high-dimensional data in order to make the most appropriate clinical decision from a large set of possible actions for a given patient. Current clinical decisions continue to rely on clinical practice guidelines, which are aimed at a representative patient rather than an individual patient who may display other characteristics. Unfortunately, if it were necessary to learn everything from the limited medical data, the problem would be completely intractable because of the high-dimensional feature space and large number of medical decisions. My thesis aims to design and analyze algorithms that learn and exploit the structure in the medical data -- for instance, structures among the features (relevance relations) or decisions (correlations). The proposed algorithms have much in common with the works in online and counterfactual learning literature but unique challenges in the medical informatics lead to numerous key differences from existing state of the art literature in Machine Learning (ML) and require key innovations to deal with large number of features and treatments, heterogeneity of the patients, sequential decision-making, and so on.

Video-based object or face recognition services on mobile devices have recently garnered significant attention, given that video cameras are now ubiquitous in all mobile communication devices. In one of the most typical scenarios for such services, each mobile device captures and transmits video frames over wireless to a remote computing cluster (a.k.a. "cloud" computing infrastructure) that performs the heavy-duty video feature extraction and recognition tasks for a large number of mobile devices. A major challenge of such scenarios stems from the highly-varying contention levels in the wireless transmission, as well as the variation in the task-scheduling congestion in the cloud. In order for each device to adapt the transmission, feature extraction and search parameters and maximize its object or face recognition rate under such contention and congestion variability, we propose a systematic learning framework based on multi-user multi-armed bandits. The performance loss under two instantiations of the proposed framework is characterized by the derivation of upper bounds for the achievable short-term and long-term loss in the expected recognition rate per face recognition attempt against the "oracle" solution that assumes a-priori knowledge of the system performance under every possible setting. Unlike well-known reinforcement learning techniques, such as Q-learning, that exhibit very slow convergence when operating in highly-dynamic environments, the proposed bandit-based systematic learning quickly approaches the optimal transmission and cloud resource allocation policies based on feedback on the experienced dynamics (contention and congestion levels).

To validate our approach, time-constrained simulation results are presented via: (i) contention-based H.264/AVC video streaming over IEEE 802.11 WLANs and (ii) principal-component based face recognition algorithms running under varying congestion levels of a cloud-computing infrastructure. Against state-of- the-art reinforcement learning methods, and for the same recognition accuracy rate set for each face recognition transaction between a mobile and the cloud, our bandit-based framework is shown to provide 17.8% ~ 44.5% reduction of the number of video frames that must be processed by the cloud for recognition and 11.5% ~ 36.5% reduction in the video traffic over the WLAN.

The popularity of machine learning in both academia and industry has experienced unparalleled growth.This has been driven by many factors, including the proliferation and availability of digitized data, the recent growth of computational power available, such as graphical processing units, and the powerful machine learning software libraries that leverage them. The overwhelming majority of existing and current research focuses on learning correlations between data rather than leveraging cause-effect relationships.

In parallel to the machine learning revolution, the study of cause-effect relationships, causality has been well-studied but often overlooked in current practice. These two disciplines are often accepted as orthogonal approaches to data modeling. This dissertation focuses on the confluence of these two approaches in an attempt to advance current machine learning techniques with fundamental concepts from causality. Namely, we identify several strategies to leverage causal structure (in the form of a directed acyclic graph) to improve machine learning performance.

Our technical contributions touch several fundamental and widespread machine learning problems. We first present a regularization method, called CASTLE (Causal Structure Learning), that simultaneously learns the causal graph/structure in the input layers of a neural network, allowing for improved predictive performance on out-of-sample data. Next, using a similar method, we develop a method for missing data imputation called MIRACLE (Missing data Imputation Refinement and Causal Learning). Similar to CASTLE, MIRACLE simultaneously learns the underlying causal structure to improve missing data imputation by refining its predictions in a unique ``bootstrapping'' manner. Next, we introduce a method, DECAF (Debiasing Causal Fairness), that introduces causal structure into Generative Adversarial Networks (GANs) to generate synthetic data that is fair for any downstream model. Next, we focus on the problem of unsupervised domain adaptation (UDA), where we leverage the invariance of causal structure to select models that best generalize to an unlabeled target domain. Lastly, we focus on extending our model selection method to individualized treatment effect (ITE) models, which are commonly used in the healthcare setting.

To demonstrate the utility of our models, we evaluate their performance on a variety of synthetic datasets, semi-synthetic datasets (for ITE models), and real-world datasets that include publicly available UCI datasets and healthcare datasets for heart failure, COVID-19, and prostate cancer among many others. We show that, compared to existing machine learning models that are agnostic to causality, our causally-aware models can improve regularization, missing data imputation, synthetic data quality, and UDA model selection.

Disease progression manifests through a broad spectrum of statically and longitudinally linked clinical features and outcomes. This leads to heterogeneous progression patterns that may vary greatly across individual patients and makes the survival and quality of a patient’s life substantially different. Recently, the rapid increase of healthcare databases, such as electronic health records (EHRs) and disease registries, has opened new opportunities for "data-driven" approaches to clinical decision support systems. This dissertation addresses the question of how machine learning (ML) techniques can capitalize on these data resources and provide actionable intelligence to move away from a rules-based clinical care toward a more data-driven and personalized model of care.

To this end, we develop a set of data-driven ML frameworks that can better predict and understand disease progression under two broad clinical setups: (I) the static setup where patients’ observations are collected at a particular point of time and (II) the longitudinal setup where observations of each patient are repeatedly collected over a period of time. In these setups, we focus on building ML methods that are (i) accurate by providing better performance in predicting disease-related outcomes, (ii) automated by freeing clinicians from the concern of choosing one particular model for a given dataset at hand, and (iii) actionable in a sense that the model is capable of answering "what if" questions and discovering subgroups of patients with similar progression patterns and outcomes.

We highlight the following technical contributions. In the static setting, we present a set of novel ML algorithms for survival analysis, a framework that informs the relationships between the clinical features and the events of interest (such as death, onset of a certain disease, etc.), and predicts what type of event will occur and when it will occur. We start off by developing a deep learning (DL) method that makes no modeling assumptions about the underlying survival process and that flexibly allows for competing events. Then, we propose an automated ML for survival analysis that combines the collective intelligence of different survival models to produce a valid survival function that is both discriminative and well-calibrated. Lastly, we develop a DL model that can accurately estimate heterogeneous treatment effects in survival analysis by adjusting for covariate shifts from multiple sources which makes the problem unique and challenging. In the longitudinal setting, we first develop a DL model for dynamic survival analysis which provides personalized and event-specific survival predictions based on a patient’s heterogeneous and historical context. Then, we provide a novel temporal clustering method that can transform the raw information in the complex longitudinal observations into clinically relevant and interpretable information to recognize future outcomes as well as life-changing disease manifestations which may cause a patient to transit between clusters.

To show the utilities of the proposed models, we evaluate the performance on various real-world medical datasets on breast cancer, prostate cancer, and cystic fibrosis patient cohorts. We demonstrate that the proposed models consistently outperform clinical scores and state-of-the-art ML methods in predicting disease progression, estimating the heterogeneous treatment effects, and providing insights into underlying disease mechanisms.

Timely prediction of clinical adverse events is a ubiquitous and important problem. We present here a timely risk scoring algorithm (Deep Risk) based on a novel Deep Learning architecture that solves the following key challenges: 1) the statistical properties of the physiological time-series data streams are not constant over time; 2) timely prediction is of the essence; 3) different patients exhibit different physiological trajectories; 4) the data is unbalanced (adverse events are uncommon). Deep Risk employs a Gated Recurrent Unit (GRU)-based Recurrent Neural Network (RNN) to aggregate the predictions of a family of GRU-based RNN's which operate on time windows of varying lengths. We show that shorter windows cope better with the non-stationary data but longer windows are capable of issuing more timely predictions. Using both shorter and longer windows enables Deep Risk to do both. Each "lower level" RNN uses the information in its time window to make a prediction; the "higher level" RNN uses the information in the longest of these time intervals to aggregate the predictions of the "lower level" RNN's and issue a final prediction. We perform simulations based on real-world medical data sets and show that Deep Risk achieves large and significant performance improvement over other methods, including clinical risk scores and state-of-the-art machine learning algorithms.

Socio-technical networks (e.g. social networking services, peer-to-peer systems, etc.) provide a popular, cost-effective and scalable framework for sharing user-generated resources or services. Achieving resource sharing efficiency in socio-technical networks is a challenging problem, because the information available about the various resources is decentralized and it is changing dynamically; the agents may be heterogeneous and have different learning abilities; the agents may make proactive decisions on link formation; and most importantly, the agents may be self-interested, i.e. they take actions which maximize their individual utilities rather than the collective social welfare and thus choose to free-ride rather than share their resources.

The overarching goal of my dissertation is to develop a rigorous and unified paradigm for the joint design of efficient incentive mechanisms and resource management schemes in socio-technical networks. It can be generally divided into two parts.

The first part focuses on the efficient resource sharing in socio-technical networks. Existing distributed network optimization techniques that enable efficient resource allocation when agents are obedient or cooperative are no longer suitable in socio-technical networks which are formed by self-interested agents. The strategic interactions of such self-interested agents lead in numerous socio-technical networks to (Nash) equilibria that are highly inefficient from a social perspective. To achieve social efficiency, incentives need to be provided to agents such that they find in their own self-interest to cooperate and thus act in a socially-optimal way. I propose a general methodology for the design and analysis of rating protocols and associated multi-agent learning algorithms to sustain cooperation in socio-technical networks. Under a rating protocol, an agent is rated based on its behavior. The rating affects the agent's rewards received in the network, which are typically determined according to a differential resource management scheme: compliant agents receive higher ratings and are rewarded by gaining more access to resources compared to non-compliant agents. This preferential treatment thus provides an incentive for agents to cooperate. I rigorously formalize and study the design of rating protocols to optimize the social resource sharing efficiency while encompassing various unique features of socio-technical networks, including the anonymity of agents, asymmetry of interests between different parties in the network, imperfect monitoring, dynamics in the agent population, and white-washing effects (i.e., an individual agent creating multiple identities in the network).

Different from the first part where the underlying network topology is exogenously determined, the second part of my dissertation augments the proposed rating protocols by investigating the endogenous formation of network topologies by the strategic, self-interested agents who produce, disseminate or collect resources. I propose a novel game-theoretic framework to model and analyze the trade-offs (of each individual agent) between the costs and benefits of producing resources personally and forming links to acquire and disseminate resources. A central point of my analysis, which departs from the existing literature on social network formation, is the assumption that the strategic agents are heterogeneous and that agents value this heterogeneity. The heterogeneity of agents and the ability of agents to strategically produce, disseminate or collect resources have striking consequences on the endogenously emerging topology, which provide important guidelines for the design of effective incentive mechanisms and resource management schemes in endogenous socio-technical networks. I first show that the network topology that emerges (at equilibrium) necessarily displays a core-periphery type: hub agents (at the core of the network) produce most of the resources and also create and maintain links to the agents at the periphery, while spoke agents (at the periphery of the network) derive most of their resources from hub agents, producing little of it themselves. As the population becomes larger, the number of hub agents and the total amount of resources produced grow in proportion to the total population. I then show that the networks that emerge at equilibrium are frequently minimally connected and have short network diameters. These ``scale-free'' conclusions had been conjectured for many networks, such as the ``small-world'' phenomenon in the World-Wide-Web, but not derived in any formal framework, and are in stark contradiction to the ``law of the few'' that had been established in previous work, under the assumption that agents solely benefit by forming links for resource acquisition, while resources are homogeneous and part of the endowment of agents, rather than heterogeneous and produced.

With the advent of electronic health records, more data is continuously collected for individual patients and more data is available for review from past patients. Despite this, it has not yet been possible to successfully use this data to systematically build clinical decision support systems that can produce personalized clinical recommendations to assist clinicians in providing individualized healthcare. In this paper, we present a novel approach, Discovery Engine (DE) that discovers which patient characteristics are most relevant for predicting the correct diagnosis and/or recommending the best treatment regimen for each patient. We demonstrate the performance of DE in two clinical settings: diagnosis of breast cancer as well as personalized recommendation for a specific chemotherapy regimen for breast cancer patients. For each distinct clinical recommendation, different patient features are relevant; DE can discover these different relevant features and use them to recommend personalized clinical decisions. The DE approach achieves a 16.6% improvement over existing state-of-the-art recommendation algorithms in terms of kappa coefficients for recommending the personalized chemotherapy regimens. For diagnostic predictions, the DE approach achieves a 2.18% and 4.20% improvement over existing state-of-the-art prediction algorithms in terms of prediction error rate and false positive rate, respectively. We also demonstrate that the performance of our approach is robust against missing information, and that the relevant features discovered by DE are confirmed by clinical references.

Enabled by ubiquitous broadband connectivity and seamless wireless connections, we have witnessed in the past few years the emergence of a plethora of wireless applications, ranging from data communications and social networking to the more recently wireless cloud computing. The growing tension between the exploding demand for such wireless applications and the increasingly scarce network resources (e.g., spectrum, power) has urged a rethinking of the service providers' pricing strategies and network resource management techniques to cope with potential threats of quality-of-service degradation and revenue decreases. Specifically, it has become of paramount importance for service providers to strategically redesign their pricing policies and to understand how various pricing policies will affect the service demand, competition in the market, as well as the network resource management.

In this dissertation, I propose a novel framework to optimize a service provider's pricing policy as well as its network resource allocation decision for profit maximization, in the presence of self-interested participating users that strategically respond to the charged price to maximize their own benefits. Applicable to both static and stochastic environments, the proposed framework explicitly takes into account user heterogeneity, which is observed in a wide range of applications. Based on the framework, I investigate the problem of optimizing pricing and resource allocation for the service provider's profit maximization in various contexts, including cooperative relay networks, communications markets, online user-generated content platforms, and mobile cloud computing systems.

Central to the design of many engineering systems and social networks is to solve the underlying resource sharing and exchange problems, in which multiple decentralized agents make sequential decisions over time to optimize some long-term performance metrics. It is challenging for the decentralized agents to make optimal sequential decisions because of the complicated coupling among the agents and across time. In this dissertation, we mainly focus on three important classes of multi-agent sequential resource sharing and exchange problems and derive optimal solutions to them.

First, we study multi-agent resource sharing with imperfect monitoring, in which self-interested agents have imperfect monitoring of the resource usage and inflict strong negative externality (i.e. strong interference and congestion) among each other. Despite of the imperfect monitoring, the strong negative externality, and the self-interested agents, we propose an optimal, distributed, easy-to-implement resource sharing policy that achieves Pareto optimal outcomes at the equilibrium. A key feature of the optimal resource sharing policy is that it is nonstationary, namely it makes decisions based on the history of past (imperfect)

monitoring of the resource usages. The applications of our proposed design in wireless spectrum sharing problems enable us to improve the spectrum efficiency by up to 200% and achieve up to 90% energy saving, compared to state-of-the-art (stationary) spectrum sharing policies.

Second, we study multi-agent resource sharing with decentralized information, in which each agent has a private, independently and stochastically changing state (whose transition may depend on the agent's action), and the agents' actions are coupled through resource sharing constraints. Despite of the decentralized information (i.e. private states), we propose distributed resource sharing policies that achieve the social optimum, and apply the proposed policies to demand-side management in smart grids, and joint resource allocation and packet scheduling in wireless video transmissions. The proposed policies demonstrate significant performance gains over existing myopic policies that do not take into account the state dynamics and the policies based on Lyapunov optimization that were proposed for single-agent problems.

Finally, we study multi-agent resource exchange with imperfect monitoring, in which self-interested, anonymous agents exchange services (e.g. task solving in crowdsourcing platforms, file sharing in peer-to-peer networks, answering in question-and-answer forums). Due to the anonymity of the agents and the lack of fixed partners, free-riding is prevalent, and can be addressed by rating protocols.

We propose the first rating protocol that can achieve the social optimum at the equilibrium under imperfect monitoring of the service quality. A key feature of the optimal rating protocol is again that it is nonstationary, namely it recommends desirable behaviors based on the history of past rating distributions of the agents.

Much of society is organized in networks: autonomous communication networks, social networks, economic networks. However, to enable the efficient and robust operation of networks several key challenges need to be overcome: the interacting agents (people, devices, software, companies, etc.) are strategic, heterogeneous and have incomplete information about the other agents. This dissertation develops systematic solutions to address these challenges.

The first part of this dissertation studies how to incentivize self-interested agents to take socially optimal actions. In many service exchange networks, agents connect to other agents to request services (e.g. favors, goods, information etc.); however, since agents who provide service gain no (immediate) benefit but only incur costs, they have an incentive to withhold their service. This dissertation designs and analyzes incentives mechanisms that rely on various types of social reciprocation, including exchange of fiat money and rating systems. The analysis builds on the theory of repeated and stochastic games with imperfect monitoring, but requires significant innovations to address the unique characteristics and requirements of online communities and networks: the anonymity and heterogeneity of agents, informational constraints (for both agents and the network manager), real-time constraints, network topology constraints, etc.

The second part of this dissertation studies how agents learn in networks. In many networks, agents need to learn how to cooperate with each other to achieve a common goal. This dissertation designs the first multi-agent learning algorithm that is able to achieve cooperation without requiring any explicit message exchange with other agents and to provide performance guarantees, including characterizing the speed of convergence.

A final part of the dissertation aims to address the problem of adverse selection in networks. The goal is to design and analyze reputation-based social norms that aim to eliminate agents of low qualities from participating in networks and communities. For this, a system of reputation in which agents’ reputation is determined based on their productivity when working alone or with others. If the agents’ reputation at the time of their evaluation (determined by the social norm) is higher than a quality/productivity level (determined by the social norm) they can remain in the network; otherwise they are expelled. The dissertation designs and analyzes social norms that maximize the productivity of the society.