Neural network types

From stats++ wiki
Jump to: navigation, search

There are several types of neural networks.

In the context of supervised neural networks, neural networks can be classified into two types.

Neural networks are often also described according to their depth.

Feedforward neural network

In feedforward neural network, information moves only in the forward direction.

Recurrent neural network

In recurrent neural networks (RNNs), there is bi-directional flow of information.

Depth

The depth of a Neural network refers to the number of hidden layers.

TODO: Add discussion of shallow vs deep networks. I think that there are older references indicating that shallow networks are fine. However, I think there are/is recent reference(s) that shallow networks need an exponentially number of hidden nodes, whereas deep networks do not.

Training of Deep neural networks (DNN) traditionally was problematic. However, pre-training from deep learning is an effective solution.