Renormalization Group connected to Neural Networks

Abstract

Even though deep learning has proved to be very powerful as the core method of machine learning, theoretical understanding behind its success is still unclear. It is been pointed out in recent years that the behaviour of deep neural networks is reminiscent of a fundamental framework in statistical physics: the renormalization group (RG). Motivated by this analogy, we develop an analytical method of directly obtaining the trained weights W resulting from a training set of 1D-Ising samples with coupling J and temperature T. The found relation W(J) drives a flow that takes 1D-Ising configurations at non-zero temperature to the critical point at T = 0. This behaviour is opposite to what a typical RG-flow dictates.