Role of Nonlocal Operators in Inverse and Deep Learning Problems
Date
2020
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
In this work, we will discuss several areas in which we harness the power of nonlocal operators. In the first part, we discuss an inverse problem from the imaging science domain. Here, we propose to use the fractional Laplacian as a regularizer to improve the reconstruction quality. In addition, inspired by residual neural networks, we develop a bilevel optimization neural network (BONNet) to learn the optimal regularization parameters, like the strength of regularization and the exponent of fractional Laplacian. As our model problem, we consider tomographic reconstruction and show an improvement in the reconstruction quality, especially for limited data, via fractional Laplacian regularization. In the second part, we propose a mathematical framework for a fractional deep neural network (fractional-DNN) for classification problems in supervised machine learning. First we formulate the deep learning problem as an ordinary differential equation (ODE) constrained optimization problem, and then we introduce a fractional time derivative based dynamical system (Neural Network) for the state equation. This architecture allows us to incorporate history (or memory) into the network by ensuring each layer is connected to the subsequent layers. The key benefits are a significant improvement to the vanishing gradient issue due to the memory effect,and better handling of nonsmooth data due to the network's ability to approximate non-smooth functions. We test our network on several datasets for classification problems. In the third part, we introduce a new class of inverse problems for external control/source identification problems with fractional partial differential equation (PDE) as constraints. Our motivation to introduce this new class of inverse problems stems from the fact that the classical PDE models only allow the source/control to be placed on the boundary or inside the observation domain where the PDE is fulfilled. Our new approach allows us to place the source/control outside and away from the observation domain.