Are deep learning networks getting too deep?

AlexNet, the network that demonstrated that deep learning could recognize images far better than existing technologies, used a network with eight layers when it won the ImageNet Large-Scale Visual Recognition Challenge (ILSVRC) competition in 2012. Eight layers was quite deep by the standards of the times. For a long time before that the depth of […]

Can the same deep network memorize with brute force and still generalize?

A paper presented a year ago, at the 2017 International Conference on Machine Learning, is still being talked about and inciting more papers to debunk or confirm it. The paper was titled “Understanding deep learning requires rethinking generalization.” (Zhang, et al, 2017.) The paper from the start seemed to be quite polarizing. The official conference […]

A tutorial on global average pooling

m When convolutional networks were first created, the typical practice was to use convolutional layers in the lower part of the network, and a few fully connected layers in the higher part of the network. AlexNet, for example, the network that demonstrated that deep networks could recognize images better than conventional alternatives (Krizhevsky, et al, […]

About this site

Deep learning has been criticized for being “like alchemy.” We in fact don’t fully understand how the networks being built today work. But many researchers are working on this problem. The goal of this site is to understand the principles upon which this technology is based and to communicate them to a broad audience.