I will discuss the asymptotic behavior at initialization of fully connected deep neural networks with Gaussian weights and biases when the widths of the hidden layers go to infinity. The focus of the talk will be on the one-dimensional case and optimal bounds for the total variation distance obtained by means of Stein's method. This is based on a joint work with S. Favaro, B. Hanin, D. Marinucci and G. Peccati.