Before RNN came and
Hidden Markov Models can be used in sequence modeling, that is, list elements from a family of strings. For example, if you have an HMM that models a set of sequences, you would be able to generate members of this family by listing sequences that would fall into the group of sequences we are modeling.
Recurrent neural networks eliminates the reliance on the Markov Assumption that was used in sequence models, allowing to condition on arbitrarily long sequences and produce effective feature extractions.
Recurrent Neural Networks allow abandoning the Markov assumption that was prevalent in natural language processing for
The power of Recurrent Neural Networks
Recurrent Neural Networks are state of the art model for dealing with sequence data ( or time series data).
Recurrent Neural Networks are rarely used as standalone component.
The Power of Recurrent Neural Network is in being trainable. Components can be fed into other network components. For example, the output of a recurrent network can be fed into a feed-forward network that will try to predict some value.
The recurrent network is used as an input-transformer that is trained to produce informative representations for the feed-forward network that will operate on top of it.
Recurrent Networks are very impressive models for sequences and are arguably the most exciting over other neural-networks for language processing. They perform exceptionally in language-modeling, the task of predicting the probability of the next word in a sequence.