Machine Learning (ML) models, in particular Deep Neural Networks (DNNs), have been evolving exceedingly fast in the past few decades although the idea of DNNs was proposed in the nineteenth century. The success of contemporary ML models can be attributed to two key factors: (i) Data of various modalities is becoming more abundant for designers, which makes data-driven approaches such as DNNs more applicable in real-world settings; (ii) The computing power of emerging hardware platforms (e.g., GPUs, TPUs) is becoming stronger due to the architecture advance. The increasing computation capability makes the training of large-scale DNNs practical for complex data applications. While ML has enabled a paradigm shift in various fields such as autonomous driving, natural language processing, and biomedical diagnosis, training high-performance ML models can be both time and resource-consuming. As such, commercial ML models (which typically contain a tremendous amount of parameters to learn complex tasks) are trained by large tech companies and then distributed to the end users or deployed on the cloud for Machine Learning as a Service (MLaaS).

This supply chain of ML models raises concerns for both model designers and end users. From the model developer’s perspective, he/she wants to ensure ownership proof of the trained model in order to prevent copyright infringement and preserve the commercial advantage. For the end user, he/she needs to verify the obtained ML model is not maliciously altered before deploying the model. This dissertation introduces holistic algorithm-level and hardware-level solutions to resolving the Intellectual Property (IP) protection and security assessment challenges of ML models, thus facilitating safe and reliable ML deployment.

The key contributions of this dissertation are as follows:

• Devising an end-to-end collusion-secure DNN fingerprinting framework named DeepMarks that enables the model owner to prove model authorship and identify unique users in the context of Deep Learning (DL). I design a fingerprint embedding technique that combines anti-collusion codes and weight regularization to ensure the fingerprint is encoded in the marked DL model in a robust manner while preserving the main task accuracy. • Designing a hardware-level IP protection and usage control technique for DL applications using on-device DNN attestation. The proposed framework DeepAttest leverages device-specific fingerprints to ‘mark’ authentic DNNs and verifies the legitimacy of the deployed DNN with the support of the Trusted Execution Environment (TEE). The algorithm and hardware architecture of DeepAttest are co-optimized to ensure the process of on-device DNN attestation is lightweight and secure. * Developing a spectral-domain DNN watermarking framework named SpecMark that removes the requirement of model re-training for watermark embedding and is robust against transfer learning. I adapt the idea of spread spectrum watermarking in the conventional multi-media domain to protect the IP of model designers using spectral watermarking. The effectiveness and robustness of SpecMark are corroborated on various automatic speech recognition datasets. * Demonstrating a targeted Trojan attack against DNNs named ProFlip that exploits bit flipping techniques (particularly Row Hammer attacks) for Trojan insertion. Compared to previous Neural Trojan attacks that require poisoned training to backdoor the model, ProFlip can embed the Trojan after model deployment. To this end, I develop a new layer-wise sensitivity analysis technique to pinpoint the vulnerable layer for attack and a novel critical bit search algorithm that identifies the most susceptible weights bits. * Designing a black-box Trojan detection and mitigation framework called DeepInspect that can assess a pre-trained DL model and determines if it has been backdoored. DeepInspect defense scheme identifies the footmark of Trojan insertion by learning the probability distribution of potential triggers with a conditional generative model. DeepInspect further leverages the trained generator to patch the model for higher Trojan robustness. * Proposing a genetic algorithm-based logic unlocking scheme named GenUnlock that outperforms prior satisfiability (SAT)-based counterpart with better runtime efficiency. GenUnlock performs fast and effective key searching by algorithm/hardware co-design and an ensemble-based method. Empirical results show that GenUnlock reduces the attack runtime by an average of 4.68× compared to SAT-based attacks. * Introducing a new logic testing-based Hardware Trojan detection framework named AdaTest that combines Reinforcement Learning (RL) and adaptive sampling. AdaTest achieves dynamic and progressive test pattern generation by defining a domain-specific reward function for circuits that characterizes both the static and dynamic properties of the circuit status. Experimental results show that AdaTest obtains a higher Trojan coverage with a shorter test pattern generation time compared to prior arts.