Non-invasive assessment of physical and mental health has been the core of an extensive body of research in recent years. Due to the development of wearable bio-sensors and the feasibility of wearing them in long time-frames (e.g. smartwatches), a tremendous opportunity lies in development and improvement of monitoring and detection methods and models for physical and mental health, based on biophysical data.We start by developing an advanced personalized data-driven approach that uses a three-layer deep neural network to estimate Blood Pressure (BP) based on photoplethysmography (PPG) signals. The model takes windows of PPG signals, and automatically extracts features which it uses for the regression problem. The proposed model outperforms prior methods and passes two long-standing standards for BP monitoring systems. In the next steps, we move forward with the development of a system for mental health assessment. Most existing health signal based tracking and monitoring approaches achieve better results presuming a large pre-labeled dataset is available and can be utilized for training the models. This is not a challenge-free assumption since these datasets usually do not exist and are not easy to collect particularly for new applications. Furthermore, these signals and labels are highly subjective and show different data distributions among different subjects. We propose a Q-learning based human-in-the-loop active learning model which releases the restriction of having a large pre-labeled dataset and aims at collecting labels for instances that are expected to improve the model's performance more efficiently. The goal is to not only minimize human annotation while maximizing the model performance through personalizing the models, but also analyzing temporal correlations to optimize the query times. Given that the experiment occurs in everyday settings (also known as \textit{in-the-wild}) and over long periods of time (time frame of few weeks to few months), maximizing user engagement is another important factor we considered in the proposed model. The proposed Q-learning based agent that selects certain instances to be labeled by the users, through analyzing the instance itself as well as the contextual and behavioral information from the subject, all in real time. The framework iteratively updates the agent based on subjective response behavior and also updates the model with the new subjective data. We performed three rounds of field studies with groups of volunteers, to collect data and evaluate the performance of the proposed framework. Extensive experiments demonstrate the superiority of the proposed model to existing unsupervised and active learning models. The proposed context-aware agent is able to provide the same level of personalization as random selection and a standard active learning method, with up to 88% and 32% fewer queries from users, respectively.

Recent advances in the Internet of Things (IoT) technologies have enabled the use of wear- ables for remote patient monitoring. Wearable sensors capture the patient’s vital signs, and provide alerts or diagnosis based on the collected data. Unfortunately, wearables typically have limited energy and bandwidth, making their use challenging for healthcare applications where monitoring must continue uninterrupted long time, without the need to charge or change the battery. Edge computing can alleviate these problems by offloading computa- tionally intensive tasks from the sensor layer to edge servers, thereby not only meeting the sensors’ limited computational capacity but also enabling the use of local closed-loop energy and bandwidth optimization algorithms to increase the battery life. By incorporating the patient’s contextual information, a desired quality of experience can be achieved by creat- ing a dynamic balance between energy-efficiency and measurement accuracy. In addition, transmitting representation of a signal instead of the raw signal can minimize the bandwidth consumption of the sensor. We first present a run-time distributed control-based solution to find the most energy-efficient system state for a given context while keeping the accuracy of decision making process over a certain threshold. We propose two approaches, myopic and Markov Decision Processes (MDP) to consider both energy constraints and risk factor requirements for achieving a two-fold goal: energy savings while satisfying accuracy require- ments of abnormality detection in a patient’s vital signs. Vital signs, including heart rate,respiration rate and oxygen saturation (SpO2), are extracted from a Photoplethysmogram (PPG) signal and errors of extracted features are compared to a ground truth that is modeled as a Gaussian distribution. We control the sensor’s sensing energy to minimize the power consumption while meeting a desired level of satisfactory detection performance. We present experimental results on realistic case studies using a reconfigurable PPG sensor in an IoT system, and show that compared to non-adaptive methods, myopic reduces an average of 16.9% in sensing energy consumption with the maximum probability of abnormality mis- detection on the order of 0.17 in a 24-hour health monitoring system. In addition, during four weeks of monitoring, we demonstrate that our MDP policy can extend the battery life on average of more than 2x while fulfilling the same average probability of misdetection com- paring to myopic method. We illustrate results comparing, myopic, MDP and non-adaptive methods to monitor 14 subjects during one month. Implementation of myopic, and Markov Decision Processes scheme are based upon implementation of optimization schemes using Matlab coding. We then, propose a framework to dynamically optimize the resolution of an electrocardiogram (ECG) signal transferred to an edge server to detect heart diseases in real-time. Our objective is two fold: maximizing the detection probability of abnormal ECG cycles while satisfying requirements on wireless channel usage. Based on sequential hypoth- esis testing and hierarchical wavelet representation, the framework establishes a control loop between the sensor and the edge server to iteratively determine the subset of wavelet coef- ficients of the ECG heart cycle based on the signal itself. Numerical results show that the sequential hypothesis testing selects different subsets for normal and abnormal ECG cycles. In addition, we show that the signal representation should be personalized, as the system selects different sets of coefficients for different patients. Results over 14 subjects indicate that average channel usage is reduced an average of 43% over non-adaptive optimization to achieve the same accuracy of classification. This work is implemented using Convolutional Neural Network using TensorFlow library and Python programming to achieve the goal.

Next generation of wireless systems and internet of things (IoT), together with smart energy networks and artificial intelligence (AI) will create a new paradigm to enable novel application scenarios and services to be deployed in future smart cities and intelligent transportation by effectively sharing information among different entities in a fast, flexible, power efficient and reliable manner. Given the huge amount of data which is generated every second, fast environment condition variability and demand for more efficient and high response control, the future networks present inherent prediction and control challenges that defy traditional model-based approaches. To address these daunting challenges, we would utilize advanced machine learning and deep reinforcement learning algorithms which enable fast data-driven adaptation to different operating conditions.

This dissertation proposes how advanced machine learning tools could be utilized in the existing wireless and smart energy networks problems. We will consider different and diverse range of tasks including anomaly and fault detection for predictive maintenance in smart grids, distributed resource allocation and network control using federated deep reinforcement learning in wireless networks and best wireless channels selection for video streaming. Specifically the main contributions of this thesis could be summarized as these four parts:

In the first part we would study the smart wireless channel selection using convolutional neural network-based predictive model for high quality video streaming. The second part would be on using AI for smart energy systems and power networks, where we propose a framework based on a combination of signal processing and deep learning techniques for anomaly and fault detection in smart grids. In the third part we would talk on federated multi-agent deep reinforcement learning for distributed power control in multi cell mobile wireless networks. The last part we would consider a network slicing environment where different mobile operators compete for physical resource block while they want to satisfy their clients demands in terms of latency requirement and we propose a deep reinforcement learning solution and a reward based personalization method to tackle this problem.

The operations of Mobile Autonomous Systems (MAS) rely on real-time data analysis. For instance, autonomous vehicles’ navigation requires the low-latency analysis of high resolution images to detect, and avoid, objects. Unfortunately, many Mobile Autonomous Systems (MASs) have constrained computing and energy resources, and the continuous execution of state-of-the-art algorithms is out of their reach. By offloading the processing load to compute-capable device located at the network edge, the edge computing paradigm can mitigate this issue. However in practical real-world settings, the wireless channel connecting the mobile devices to the edge server often presents erratic capacity patterns due to mobility.As a result, the overall delay perceived by the mobile application may be affected by large variations, which in turn harm control.This thesis explores solutions to the problem described above. To this aim, in addition to new concepts and edge offloading strategies, a complete real-world platform – the HyDRA platform, was developed to support design and evaluation, as well as dataset collection.HyDRA is a fully open source software and hardware platform realizing flexible machine learning-empowered computing for MAS. From a hardware perspective, HyDRA is composed of several Unmanned Aerial Vehicles (UAVs) and ground devices collaboratively performing data analysis to accomplish system-wide goals. From a software perspective, the HyDRA middleware enables real-time control of data and task routing within the system, organized as a distributed set of modules transforming the data captured by the MAS into actuable control.Taking HyDRA as a starting point, this thesis makes the following conceptual contributions:‚the end-to-end delay in remote computing settings for MAS – and specifically autonomous quad-copters – were characterized by means of real-world experiments that produced a comprehensive dataset focused on object detection from images. The study considered both Wi-Fi and Long-Term Evolution connectivity, and several embedded computing platforms. The results demonstrates the instability of application level delay even in line-of-sight settings and relatively slow vehicle motion. A framework for the dynamic control of task offloading in MASs with extreme temporal variations is developed. The frameworks is based on a preliminary experimental analysis, which indicates that there is no dominant feature, including obvious features such as channel quality, and that prediction necessitates an ensemble of weaker features. We first mathematically formulate a Redundant Task Offloading Problem. Then, we create predictors that can help manage the resource usage/performance trade-off. Specifically, we propose a myopic predictor as baseline and a DRL-based approach, which operates on a set of features from application, network and device-level components. To the best of our knowledge, this is the first framework addressing the problem of redundant task offloading in MAS with a data-driven approach which efficacy is verified in a real-world testbed and with replicable dataset-based experiments.‚A modeling and optimization framework based on Markov Decision Processes (MDP) was developed to analyze the structural properties of dynamic control strategies determining where information is processed in collaborative computing scenarios for MAS. In this section of the thesis, the focus of control is primarily between local and remote analysis. Using recent split Deep Neural Networks deep neural network (DNN) techniques, the framework also controls at a fine-grain how the analysis task (the DNN in this case) is divided between the mobile device and the edge server based on current system parameters.

The design choices of the networked systems for autonomous & connected vehicles (CAV) play a critical role in the security and reliability of their operations and performance. Mainly, the networked systems for CAV should be resistant to disruptions, whether they are intentional or unintentional. For instance, the security of a given restricted airspace might be governed by an authentication system for authorizing drone activity over such airspace. However, having an open authentication channel for drones would leave them vulnerable for stalking due to the broadcast nature of authentication beacons. Furthermore, such authentication system would be fruitless without the means of detecting unauthorized and disruptive drones in the first place. On the other hand, the latency-sensitive nature of CAV applications requires reliable packet delivery schemes for resisting any foreseen disruptions.

In this Dissertation, we discuss the security and reliability perspectives of CAV. In the first Chapter of the Dissertation, we start by first proposing a drone detection framework that can detect the presence of flying drones by exploiting their emitted wireless traffic. Our work shows that the proposed framework achieves a detection accuracy up to 99% for both profiled and unprofiled drones, even in the presence of network interference and benign drone-like wireless traffic. Next we propose a privacy-preserving authentication scheme for drones which is influenced by the Security Credential Management System (SCMS), the official authentication system for V2V and V2I adopted by the U.S. Department of Transportation (USDoT). Our authentication scheme is then re-designed to be fitted with Remote-ID, the official identification framework for drones adopted by the Federal Aviation Administration (FAA). Finally, we propose our localized-authentication scheme that can pinpoint the physical location of authenticated vehicles and therefore differentiate between authorized and unauthorized vehicles.

In the second Chapter of the Dissertation, we go through different tools and techniques for testing and implementing reliable networked systems tailored to latency-sensitive applications such as CAV applications. First we present our packet injection tool that has various features such as generating customized traffic profiles using real data instead of padded data, creating different packetization policies, and producing fine-grained datasets. Then we go through our computing slicer module that can slice the computing resources of a device among different incoming traffic types based on different criteria such as customized scheduling policies. Next, we propose a state-recovery protocol for recovering the state of a channel in the case of channel disruptions using an adaptive loopback channel. Finally, we discuss our work on physical network slicing for 5G systems which is implemented on Colosseum, world’s largest RF emulator.