Neural network types
There are several types of neural networks.
In the context of supervised neural networks, neural networks can be classified into two types.
Neural networks are often also described according to their depth.
In feedforward neural network, information moves only in the forward direction.
In recurrent neural networks (RNNs), there is bi-directional flow of information.
The depth of a Neural network refers to the number of hidden layers.
- Shallow networks:
- Deep neural networks (DNN)
TODO: Add discussion of shallow vs deep networks. I think that there are older references indicating that shallow networks are fine. However, I think there are/is recent reference(s) that shallow networks need an exponentially number of hidden nodes, whereas deep networks do not.
Training of Deep neural networks (DNN) traditionally was problematic. However, pre-training from deep learning is an effective solution.