Neural networks and neural fields: discrete and continuous space neural models




Edwards, Roderick

Journal Title

Journal ISSN

Volume Title



'Attractor' neural network models have useful properties, but biology suggests that more varied dynamics may be significant. Even the equations of the Hopfield network, without the constraint of symmetry, can have complex behaviours which have been little studied. Several new ideas or approaches to neural network theory are examined here, focussing on the distinction between discrete and continuous space neural models. First, simple chaotic dynamical systems are examined, as candidates for more natural neural network models, including coupled systems of Lorenz equations and a Hopfield equation model with a balance of inhibitory and excitatory neurons. Also, continuous space models with a structure like that of the Hopfield network are briefly explored, with interesting training possibilities. The main results deal with the approximation of Hopfield network equations with a particular class of connection structures (allowing asymmetry), by a reaction-diffusion equation, using techniques borrowed from particle methods used in the numerical solution of fluid-dynamical equations. It is shown that the approximation holds rigorously only in certain spatial regions but the small regions where it fails, namely within transition layers between regions of high and low activity, are not likely to be critical. The result serves to classify connectivities in Hopfield-type models and sheds light on the limiting behaviour of networks as the number of neurons goes to infinity. Standard discretizations of the reaction-diffusion equations are analyzed to clarify the effects which can arise in the limiting process. The discrete space systems can have stable patterned equilibria which must be close to metastable patterns of the continuous systems. Our results also suggest that the fine structure of neural connections is important, and to obtain complex behaviour in the Hopfield network equations, a predominance of inhibition or wildly oscillating connection matrix entries are indicated.



Neural circuitry, Neural networks (Computer science)