Odense, SimonEdwards, Roderick2021-07-262021-07-2620162016Odense, S. & Edwards, R. (2016). Universal approximation results for the temporal restricted Boltzmann machine the recurrent temporal restricted Boltzmann machine. Journal of Machine Learning Research, 17(158), 1-21. http://jmlr.org/papers/v17/15-478.htmlhttp://jmlr.org/papers/v17/15-478.htmlhttp://hdl.handle.net/1828/13176The Restricted Boltzmann Machine (RBM) has proved to be a powerful tool in machine learning, both on its own and as the building block for Deep Belief Networks (multi-layer generative graphical models). The RBM and Deep Belief Network have been shown to be universal approximators for probability distributions on binary vectors. In this paper we prove several similar universal approximation results for two variations of the Restricted Boltzmann Machine with time dependence, the Temporal Restricted Boltzmann Machine (TRBM) and the Recurrent Temporal Restricted Boltzmann Machine (RTRBM). We show that the TRBM is a universal approximator for Markov chains and generalize the theorem to sequences with longer time dependence. We then prove that the RTRBM is a universal approximator for stochastic processes with nite time dependence. We conclude with a discussion on e ciency and how the constructions developed could explain some previous experimental results.enTRBMRTRBMmachine learninguniversal approximationArticleDepartment of Mathematics and Statistics