A 3D ray traced biological neural network learning model and applications
Date
2024
Authors
Yuen, Brosnan
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Training large neural networks on big datasets requires significant computational resources and time. Transfer learning reduces training time by pre-training a base model on one dataset and transferring the knowledge to a new model for another dataset. However, current transfer learning algorithms are very limited because the transferred models have to adhere to the dimensions of the base model and can not easily change the neural architecture for new datasets. This dissertation presents a novel 3D ray-traced biological neural network (RayBNN) inspired by brain neural structures, that provides flexibility and adaptability surpassing the state-of-the-art models.
The dissertation begins with the design of a privacy-preserving contact tracing system based on wireless indoor localization. This system seamlessly localizes and tracks user devices using Wi-Fi and Bluetooth signals but does not trace user identity. As the received signals at the routers side are used for wireless fingerprinting, the system does not require smartphone applications, joining wireless networks, custom user devices, or any user actions at all. A bidirectional long short-term memory (BiLSTM) neural network is built and trained with data pre-collected by an autonomous robot, resulting in the absolute position precision of user devices within 1.0 m. This dataset is later used for RayBNN application.
One issue in a machine learning application, for example, the BiLSTM network for contact tracing, is selecting the optimal activation function because there are many activation functions in the literature. A universal activation function (UAF) is proposed in the dissertation to solve the optimal activation function problem by evolving the UAF to a suitable activation function via tuning the UAF’s parameters. For the CIFAR-10 classification using the VGG-8 neural network, the UAF converges to the Mish-like activation function, which has near-optimal performance when compared to other activation functions. In the graph convolutional neural network on the CORA dataset, the UAF evolves to the identity function. For the quantification of simulated 9-gas mixtures in 30 dB signal-to-noise ratio (SNR) environments, the UAF converges to the identity function, which has near-optimal root-mean-square error. In the ZINC molecular solubility quantification using graph neural networks, the UAF morphs to a LeakyReLU/Sigmoid hybrid. For the BipedalWalker-v2 RL dataset, the UAF leads to a brand-new activation function, which gives the fastest convergence rate among the commonly used activation functions.
Biological neural networks (BNNs) are adept at rearranging themselves to tackle completely different problems using transfer learning. Inspired by BNN, the designed RayBNN is a neural network that is transferable to any other network architecture and can accommodate many datasets. The novel approach uses raytracing to connect neurons in a three-dimensional space, allowing the network to grow into any shape or size. Moreover, the UAF is applied to every neuron for added flexibility. In the Alcala dataset, the RayBNN transfer learning algorithm trains the fastest across changing environments and input sizes. In the future, this network may be considered for implementation on real biological neural networks to decrease power consumption.
Description
Keywords
neural networks, transfer learning, machine learning, ray tracing, EEG, wi-fi localization, BLE localization, activation function