This dissertation considers the random homogenization of coercive Hamilton-Jacobi equations and it gives the most generalized result in 1-D. Basically, we can prove that in the stationary ergodic media, the random homogenization holds as long as the Hamiltonian is coercive. This is an extension of the result by Armstrong, Tran and Yu when the Hamiltonain is separable. We also provide some application of random homogenizaton in front propagation based on the analysis of inviscid G-equation model, it is proved that with 2-d random shear flows, the strain effect reduces the propagation of the flame front.

In chaotic advection generated by a class of time periodic cellular flows, the residual diffusion refers to the non-zero effective (homogenized) diffusion in the limit of zero molecular diffusion as a result of chaotic mixing of the streamlines. We study the residual diffusion phenomenon computationally and analytically.

We make use of the Poincar e map of the advection-diffusion equation to bypass long time simulation and gain accuracy in computing effective diffusivity and learning adaptive basis. We observe a non-monotone relationship between residual diffusivity and the amount of chaos in the advection, though the overall trend is that sufficient chaos leads to higher residual diffusivity. The adaptive orthogonal basis with built-in sharp gradient structures is constructed by taking snapshots of solutions in time, preprocessing with deep neural network (DNN) if necessary and performing singular value decomposition of the matrix consisting of those snapshots as column vectors. The trained orthogonal adaptive basis makes possible low cost computation of the effective diffusivities at smaller molecular diffusivities. The testing errors decrease as the training occurs at smaller molecular diffusivities.

We also study the enhanced diffusivity in the so called elephant random walk model with stops by including symmetric random walk steps at small probability $\epsilon$. At any $\epsilon > 0$, the large time behavior transitions from sub-diffusive at $\epsilon = 0$ to diffusive in a wedge shaped parameter regime where the diffusivity is strictly above that in the un-perturbed model in the $\epsilon \downarrow 0$ limit. The perturbed model is shown to be solvable with the first two moments and their asymptotics calculated exactly in both one and two space dimensions. The model provides a discrete analytical setting of the residual diffusion phenomenon as molecular diffusivity tends to zero. On a related nonlinear case, we give theoretical proof that the turbulent flame speed as an effective burning velocity is decreasing with respect to the curvature diffusivity (Markstein number) for shear flows in the well-known G-equation model. Besides, we solve the selection problem of weak solutions when the Markstein number goes to zero and solutions approach those of the inviscid G-equation model. The limiting solution is given by a closed form analytical formula.

Finally for the dimensionality reduction on DNNs, we propose BinaryRelax, a simple two-phase algorithm, for training DNNs with quantized weights. We relax the hard constraint that characterizes the quantization of weights into a continuous regularizer via Moreau envelope, which turns out to be the squared Euclidean distance to the set of quantized weights. The pseudo quantized weights are obtained by linearly interpolating between the float weights and their quantizations. A continuation strategy is adopted to push the weights towards the quantized state by gradually increasing the regularization parameter. We test BinaryRelax on the benchmark CIFAR and ImageNet color image datasets to demonstrate the superiority of the relaxed quantization approach and the improved accuracy over the state-of-the-art training methods. Moreover, we prove the convergence of BinaryRelax under an approximate orthogonality condition.

We study the dynamics of gradient descent in learning neural networks for classification problems. Unlike in existing works, we consider the linearly non-separable case where the training data of different classes lie in orthogonal subspaces. Deep neural networks (DNNs) are quantized for efficient inference on resource-constrained platforms. However, training deep learning models with low-precision weights and activations involves a demanding optimization task, which calls for minimizing a stage-wise loss function subject to a discrete set-constraint. While numerous training methods have been proposed, existing studies for full quantization of DNNs are mostly empirical. From a theoretical point of view, we study practical techniques for overcoming the combinatorial nature of network quantization. Specifically, we investigate a simple yet powerful projected gradient-like algorithm for quantizing two-layer convolution networks, by repeatedly moving one step at float weights in the negative direction of a heuristic fake gradient of the loss function (so-called coarse gradient) evaluated at quantized weights. For the first time, we prove that under mild conditions, the sequence of quantized weights recurrently visit the global optimum of the discrete minimization problem for training fully quantized network. We also show numerical evidence of the recurrence phenomenon of weight evolution in training quantized deep networks.

Purpose

Methods

Results

Conclusions