Artificial neural networks for approximating the solutions to nonlinear ordinary differential equations

Artificial neural networks (ANNs) are the computer archetype of biological networks in the human brain. An ANN is a group of interconnected nodes stacked in layers and each layer is connected with its preceding and succeeding one via specified weights. The numerical algorithms based on ANN have been shown to perform well in approximating the differential and integral operators. In this thesis investigation, a neural network architecture is proposed for approximating the numerical solutions to the nonlinear ordinary differential equations. A comparative study is performed between the ANN predictions and the approximation obtained from finite element method (FEM). The solution finding problem using ANN is formulated as a minimization of a total loss function, an L2-type function or root mean square type function, which is a sum of differential equation loss and the boundary loss terms. For the minimization, a feedforward-type unsupervised neural network architecture is examined in this thesis. Recent works have shown that such an unsupervised minimization yields highly accurate prediction which can approximate the numerical solution to the differential equation. However, currently no study is available in the literature on a comprehensive unified ANN method with particular choice of the loss function and network’s hyperparameters and how such choices influence the accuracy of the network prediction. In this thesis, we addressed some issues concerning the design of the network to obtain highly accurate numerical results. The sensitivity in the network’s accuracy with respect to the size of the training data, activation functions, optimizers, number of hidden layers, and the number of neurons in each hidden layer is also studied. Our trail solution consists of two parts: the first part satisfying the differential equation; the second part stems from satisfying boundary conditions. At the training phase access to the exact solution is not needed, however, the network adjusts its training based on the linear interpolation of the randomly chosen data points from the computational domain. A backpropogation step is needed for a calibration of the network parameters to obtain accurate prediction. We investigate the proposed ANN method to approximate the numerical solutions to two nonlinear boundary value problems from fluid dynamics: Electrohydrodynamic fluid model; one-dimensional Darcy-Brinkman-Forchheimer model. A comparison of the network solution is made with that of the one obtained from classical continuous finite elements. We report that ANN method developed in this thesis performs better in achieving higher accuracy within the fewer number of data points. We believe that the proposed architecture along with the “correctly” chosen hyperparameters constitute a better numerical approximator for the partial differential equations in higher dimensions.