The power of Recurrent Neural Networks in NLP

26th April 2019


Before RNN came and revolutionised the natural language processing, Hidden Markov models were used for sequence data.

Hidden Markov Models can be used in sequence modeling, that is, list elements from a family of strings. For example, if you have an HMM that models a set of sequences, you would be able to generate members of this family by listing sequences that would fall into the group of sequences we are modeling.

Recurrent neural networks eliminates the reliance on the Markov Assumption that was used in sequence models, allowing to condition on arbitrarily long sequences and produce effective feature extractions.

Recurrent Neural Networks allow abandoning the Markov assumption that was prevalent in natural language processing for decades, and designing models that can condition on entire sentences. All this while taking word order into account when it is needed, and not suffering much from statistical estimation problems stemming from data sparsity.

The power of Recurrent Neural Networks

Recurrent Neural Networks are state of the art model for dealing with sequence data ( or time series data).

Recurrent Neural Networks are rarely used as standalone component.

The Power of Recurrent Neural Network is in being trainable. Components can be fed into other network components. For example, the output of a recurrent network can be fed into a feed-forward network that will try to predict some value.

The recurrent network is used as an input-transformer that is trained to produce informative representations for the feed-forward network that will operate on top of it.

Recurrent Networks are very impressive models for sequences and are arguably the most exciting over other neural-networks for language processing. They perform exceptionally in language-modeling, the task of predicting the probability of the next word in a sequence.

Read also: How to develop a Data Science platform upon Kubernetes

Tagged with:

Manish Prasad

An experienced data scientist with a passion to work on new challenges

To give you the best possible experience, this site uses cookies. Continuing to use means you agree on our use of cookies.