skip-gram model

浅浅的花香味﹌ 2021-12-15 18:17 220阅读 0赞

from: https://www.kdnuggets.com/2018/04/implementing-deep-learning-methods-feature-engineering-text-data-skip-gram.html

[Deep Learning][], [Feature Engineering][], [NLP][], [Python][], [Text Mining][], [Word Embeddings][]

Just like we discussed in the CBOW model, we need to model this Skip-gram architecture now as a deep learning classification model such that we take in the target word as our input and try to predict the context words.

**By [Dipanjan Sarkar][], RedHat.**

![c][] [comments][]

> **Editor's note:** This post is only one part of a far more thorough and in-depth original, [found here][], which covers much more than what is included here.

### The Skip-gram Model ###

The Skip-gram model architecture usually tries to achieve the reverse of what the CBOW model does. It tries to predict the source context words (surrounding words) given a target word (the center word). Considering our simple sentence from earlier, ***“the quick brown fox jumps over the lazy dog”.*** If we used the CBOW model, we get pairs of ***(context\_window, target\_word)***where if we consider a context window of size 2, we have examples like ***(\[quick, fox\], brown), (\[the, brown\], quick), (\[the, dog\], lazy) ***and so on. Now considering that the skip-gram model’s aim is to predict the context from the target word, the model typically inverts the contexts and targets, and tries to predict each context word from its target word. Hence the task becomes to predict the context ***\[quick, fox\]*** given target word ***‘brown’ ***or ***\[the, brown\]*** given target word ***‘quick’*** and so on. Thus the model tries to predict the context\_window words based on the target\_word.

![1_SR6l59udY05_bUICAjb6-w.png][]  
The Skip-gram model architecture (Source: [https://arxiv.org/pdf/1301.3781.pdf][https_arxiv.org_pdf_1301.3781.pdf] Mikolov el al.)

Just like we discussed in the CBOW model, we need to model this Skip-gram architecture now as a deep learning classification model such that we take in the *target word as our input *and try to *predict the context words.*This becomes slightly complex since we have multiple words in our context. We simplify this further by breaking down each ***(target, context\_words) pair*** into ***(target, context) pairs*** such that each context consists of only one word. Hence our dataset from earlier gets transformed into pairs like ***(brown, quick), (brown, fox), (quick, the), (quick, brown)*** and so on. But how to supervise or train the model to know what is contextual and what is not?

For this, we feed our skip-gram model pairs of ***(X, Y)*** where ***X ***is our ***input ***and ***Y ***is our ***label***. We do this by using ***\[(target, context), 1\] ***pairs as ***positive input samples*** where ***target ***is our word of interest and ***context ***is a context word occurring near the target word and the ***positive label 1*** indicates this is a contextually relevant pair. We also feed in ***\[(target, random), 0\]*** pairs as ***negative input samples*** where ***target ***is again our word of interest but ***random ***is just a randomly selected word from our vocabulary which has no context or association with our target word. Hence the ***negative label 0***indicates this is a contextually irrelevant pair. We do this so that the model can then learn which pairs of words are contextually relevant and which are not and generate similar embeddings for semantically similar words.

### Implementing the Skip-gram Model ###

Let’s now try and implement this model from scratch to gain some perspective on how things work behind the scenes and also so that we can compare it with our implementation of the CBOW model. We will leverage our Bible corpus as usual which is contained in the `norm_bible` variable for training our model. The implementation will focus on five parts

*  **Build the corpus vocabulary**
 *  **Build a skip-gram \[(target, context), relevancy\] generator**
 *  **Build the skip-gram model architecture**
 *  **Train the Model**
 *  **Get Word Embeddings**

Let’s get cracking and build our skip-gram Word2Vec model!

**Build the corpus vocabulary**

To start off, we will follow the standard process of building our corpus vocabulary where we extract out each unique word from our vocabulary and assign a unique identifier, similar to what we did in the CBOW model. We also maintain mappings to transform words to their unique identifiers and vice-versa.

[view raw][][feature\_engg\_text\_21.py][feature_engg_text_21.py] hosted with ❤ by [GitHub][]

Vocabulary Size: 12425
    Vocabulary Sample: [('perceived', 1460), ('flagon', 7287), ('gardener', 11641), ('named', 973), ('remain', 732), ('sticketh', 10622), ('abstinence', 11848), ('rufus', 8190), ('adversary', 2018), ('jehoiachin', 3189)]

Just like we wanted, each unique word from the corpus is a part of our vocabulary now with a unique numeric identifier.

**Build a skip-gram \[(target, context), relevancy\] generator**

It’s now time to build out our skip-gram generator which will give us pair of words and their relevance like we discussed earlier. Luckily, `keras` has a nifty `skipgrams` utility which can be used and we don’t have to manually implement this generator like we did in CBOW.

> **Note:** The function `skipgrams(…)` is present in `keras.preprocessing.sequence`
> 
> This function transforms a sequence of word indexes (list of integers) into tuples of words of the form:  
> \- (word, word in the same window), with label 1 (positive samples).  
> \- (word, random word from the vocabulary), with label 0 (negative samples).

[view raw][view raw 1][feature\_engg\_text\_22.py][feature_engg_text_22.py] hosted with ❤ by [GitHub][]

(james (1154), king (13)) -> 1 (king (13), james (1154)) -> 1 (james (1154), perform (1249)) -> 0 (bible (5766), dismissed (6274)) -> 0 (king (13), alter (5275)) -> 0 (james (1154), bible (5766)) -> 1 (king (13), bible (5766)) -> 1 (bible (5766), king (13)) -> 1 (king (13), compassion (1279)) -> 0 (james (1154), foreskins (4844)) -> 0

Thus you can see we have successfully generated our required skip-grams and based on the sample skip-grams in the preceding output, you can clearly see what is relevant and what is irrelevant based on the label (0 or 1).

**Build the skip-gram model architecture**

We now leverage `keras` on top of `tensorflow` to build our deep learning architecture for the skip-gram model. For this our inputs will be our target word and context or random word pair. Each of which are passed to an embedding layer (initialized with random weights) of it’s own. Once we obtain the word embeddings for the target and the context word, we pass it to a merge layer where we compute the dot product of these two vectors. Then we pass on this dot product value to a dense sigmoid layer which predicts either a 1 or a 0 depending on if the pair of words are contextually relevant or just random words (***Y’***). We match this with the actual relevance label (***Y***), compute the loss by leveraging the `mean_squared_error` loss and perform backpropagation with each epoch to update the embedding layer in the process. Following code shows us our model architecture.

<table> 
 <tbody> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:14px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">from keras.layers import Merge</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:14px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">from keras.layers.core import Dense, Reshape</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:14px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">from keras.layers.embeddings import Embedding</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:14px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">from keras.models import Sequential</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:14px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">&nbsp;</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:14px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;"># build skip-gram architecture</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:14px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">word_model = Sequential()</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:14px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">word_model.add(Embedding(vocab_size, embed_size,</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:14px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">embeddings_initializer="glorot_uniform",</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:14px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">input_length=1))</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:14px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">word_model.add(Reshape((embed_size, )))</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:14px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">&nbsp;</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:14px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">context_model = Sequential()</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:14px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">context_model.add(Embedding(vocab_size, embed_size,</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:14px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">embeddings_initializer="glorot_uniform",</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:14px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">input_length=1))</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:14px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">context_model.add(Reshape((embed_size,)))</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:14px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">&nbsp;</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:14px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">model = Sequential()</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:14px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">model.add(Merge([word_model, context_model], mode="dot"))</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:14px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">model.add(Dense(1, kernel_initializer="glorot_uniform", activation="sigmoid"))</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:14px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">model.compile(loss="mean_squared_error", optimizer="rmsprop")</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:14px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">&nbsp;</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:14px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;"># view model summary</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:14px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">print(model.summary())</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:14px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">&nbsp;</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:14px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;"># visualize model structure</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:14px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">from IPython.display import SVG</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:14px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">from keras.utils.vis_utils import model_to_dot</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:14px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">&nbsp;</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:14px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">SVG(model_to_dot(model, show_shapes=True, show_layer_names=False,</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:14px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">rankdir='TB').create(prog='dot', format='svg'))</td> 
 </tr> 
 </tbody> 
</table>

[view raw][view raw 2][feature\_engg\_text\_23.py][feature_engg_text_23.py] hosted with ❤ by [GitHub][]

![1_tXimFtA5R2jswNjVT80Q5Q.png][]  
Skip-gram model summary and architecture

Understanding the above deep learning model is pretty straightforward. However, I will try to summarize the core concepts of this model in simple terms for ease of understanding. We have a pair of input words for each training example consisting of ***one input target word*** having a unique numeric identifier and ***one context word*** having a unique numeric identifier. If it is ***a positive sample*** the word has contextual meaning, is ***a context word***and our ***label Y=1***, else if it is a ***negative sample***, the word has no contextual meaning, is just ***a random word*** and our ***label Y=0***. We will pass each of them to an ***embedding layer*** of their own, having size `(vocab_size x embed_size)` which will give us ***dense word embeddings*** for each of these two words `(1 x embed_size for each word)`. Next up we use a ***merge layer*** to compute the ***dot product*** of these two embeddings and get the dot product value. This is then sent to the ***dense sigmoid layer*** which outputs either a 1 or 0. We compare this with the actual label Y (1 or 0), compute the loss, backpropagate the errors to adjust the weights (in the embedding layer) and repeat this process for all ***(target, context)*** pairs for multiple epochs. The following figure tries to explain the same.

![1_4Uil1zWWF5-jlt-FnRJgAQ.png][]  
Visual depiction of the Skip-gram deep learning model

Let’s now start training our model with our skip-grams.

**Train the Model**

Running the model on our complete corpus takes a fair bit of time but lesser than the CBOW model. Hence I just ran it for 5 epochs. You can leverage the following code and increase it for more epochs if necessary.

[view raw][view raw 3][feature\_engg\_text\_24.py][feature_engg_text_24.py] hosted with ❤ by [GitHub][]

Epoch: 1 Loss: 4529.63803683
    Epoch: 2 Loss: 3750.71884749
    Epoch: 3 Loss: 3752.47489296
    Epoch: 4 Loss: 3793.9177565
    Epoch: 5 Loss: 3716.07605051

Once this model is trained, similar words should have similar weights based off the embedding layer and we can test out the same.

**Get Word Embeddings**

To get word embeddings for our entire vocabulary, we can extract out the same from our embedding layer by leveraging the following code. Do note that we are only interested in the target word embedding layer, hence we will extract the embeddings from our `word_model` embedding layer. We don’t take the embedding at position 0 since none of our words in the vocabulary have a numeric identifier of 0 and we ignore it.

[view raw][view raw 4][feature\_engg\_text\_25.py][feature_engg_text_25.py] hosted with ❤ by [GitHub][]

![1_guUd_CKJHYOwNIIt_6YchA.png][]  
Word Embeddings for our vocabulary based on the Skip-gram model

Thus you can clearly see that each word has a dense embedding of size `(1x100)` as depicted in the preceding output similar to what we had obtained from the CBOW model. Let’s now apply the euclidean distance metric on these dense embedding vectors to generate a pairwise distance metric for each word in our vocabulary. We can then find out the n-nearest neighbors of each word of interest based on the shortest (euclidean) distance similar to what we did on the embeddings from our CBOW model.

<table> 
 <tbody> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:7px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">from sklearn.metrics.pairwise import euclidean_distances</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:7px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">&nbsp;</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:7px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">distance_matrix = euclidean_distances(weights)</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:7px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">print(distance_matrix.shape)</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:7px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">&nbsp;</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:7px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">similar_words = {search_term: [id2word[idx] for idx in distance_matrix[word2id[search_term]-1].argsort()[1:6]+1]</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:7px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">for search_term in ['god', 'jesus', 'noah', 'egypt', 'john', 'gospel', 'moses','famine']}</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:7px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">&nbsp;</td> 
 </tr> 
 <tr> 
 <td style="text-align:right;vertical-align:top;width:7px;">&nbsp;</td> 
 <td style="text-align:left;vertical-align:top;">similar_words</td> 
 </tr> 
 </tbody> 
</table>

[view raw][view raw 5][feature\_engg\_text\_26.py][feature_engg_text_26.py] hosted with ❤ by [GitHub][]

(12424, 12424)
    
    {'egypt': ['pharaoh', 'mighty', 'houses', 'kept', 'possess'], 'famine': ['rivers', 'foot', 'pestilence', 'wash', 'sabbaths'], 'god': ['evil', 'iniquity', 'none', 'mighty', 'mercy'], 'gospel': ['grace', 'shame', 'believed', 'verily', 'everlasting'], 'jesus': ['christ', 'faith', 'disciples', 'dead', 'say'], 'john': ['ghost', 'knew', 'peter', 'alone', 'master'], 'moses': ['commanded', 'offerings', 'kept', 'presence', 'lamb'], 'noah': ['flood', 'shem', 'peleg', 'abram', 'chose']}

You can clearly see from the results that a lot of the similar words for each of the words of interest are making sense and we have obtained better results as compared to our CBOW model. Let’s visualize these words embeddings now using [**t-SNE**][t-SNE]** **which stands for [***t-distributed stochastic neighbor embedding***][t-SNE]** **a popular [dimensionality reduction][] technique to visualize higher dimension spaces in lower dimensions (e.g. 2-D).

[view raw][view raw 6][feature\_engg\_text\_27.py][feature_engg_text_27.py] hosted with ❤ by [GitHub][]

![1_5N5p0cqCitIiGpfbQufc-A.png][]  
Visualizing skip-gram word2vec word embeddings using t-SNE

I have marked some circles in red which seemed to show different words of contextual similarity positioned near each other in the vector space. If you find any other interesting patterns feel free to let me know!

**Bio: [Dipanjan Sarkar][Dipanjan Sarkar 1]** is a Data Scientist @Intel, an author, a mentor @Springboard, a writer, and a sports and sitcom addict.

[Original][found here]. Reposted with permission.

**Related:**

*  [Text Data Preprocessing: A Walkthrough in Python][Text Data Preprocessing_ A Walkthrough in Python]
 *  [A General Approach to Preprocessing Text Data][]
 *  [A Framework for Approaching Textual Data Science Tasks][]

[Deep Learning]: https://www.kdnuggets.com/tag/deep-learning
[Feature Engineering]: https://www.kdnuggets.com/tag/feature-engineering
[NLP]: https://www.kdnuggets.com/tag/nlp
[Python]: https://www.kdnuggets.com/tag/python
[Text Mining]: https://www.kdnuggets.com/tag/text-mining
[Word Embeddings]: https://www.kdnuggets.com/tag/word-embeddings
[Dipanjan Sarkar]: https://www.kdnuggets.com/author/dipanjan-sarkar
[c]: /images/20211215/7211bc118a6741be9b190888d858d0c8.png
[comments]: https://www.kdnuggets.com/2018/04/implementing-deep-learning-methods-feature-engineering-text-data-skip-gram.html#comments
[found here]: https://towardsdatascience.com/understanding-feature-engineering-part-4-deep-learning-methods-for-text-data-96c44370bbfa
[1_SR6l59udY05_bUICAjb6-w.png]: https://cdn-images-1.medium.com/max/800/1*SR6l59udY05_bUICAjb6-w.png
[https_arxiv.org_pdf_1301.3781.pdf]: https://arxiv.org/pdf/1301.3781.pdf
[view raw]: https://gist.github.com/dipanjanS/84f415ae0781c1d42f6fb0ac843bf63e/raw/5dbeaa06c518f58e3e78a88ae81f1965ed6f2ce9/feature_engg_text_21.py
[feature_engg_text_21.py]: https://gist.github.com/dipanjanS/84f415ae0781c1d42f6fb0ac843bf63e#file-feature_engg_text_21-py
[GitHub]: https://github.com/
[view raw 1]: https://gist.github.com/dipanjanS/865248edadb721f754bad10e874a9866/raw/f9e0dda123f5e432c28cfb671381817af51f3865/feature_engg_text_22.py
[feature_engg_text_22.py]: https://gist.github.com/dipanjanS/865248edadb721f754bad10e874a9866#file-feature_engg_text_22-py
[view raw 2]: https://gist.github.com/dipanjanS/dbff31351127c893a4579c9be11e113d/raw/863005f33025d6764fc1945579dde98ef52f0f0b/feature_engg_text_23.py
[feature_engg_text_23.py]: https://gist.github.com/dipanjanS/dbff31351127c893a4579c9be11e113d#file-feature_engg_text_23-py
[1_tXimFtA5R2jswNjVT80Q5Q.png]: https://cdn-images-1.medium.com/max/800/1*tXimFtA5R2jswNjVT80Q5Q.png
[1_4Uil1zWWF5-jlt-FnRJgAQ.png]: https://cdn-images-1.medium.com/max/800/1*4Uil1zWWF5-jlt-FnRJgAQ.png
[view raw 3]: https://gist.github.com/dipanjanS/a820a598b2d93c2d57e2dd8319936095/raw/f111b0b9150f79e410c8698036f9fd06d4afff30/feature_engg_text_24.py
[feature_engg_text_24.py]: https://gist.github.com/dipanjanS/a820a598b2d93c2d57e2dd8319936095#file-feature_engg_text_24-py
[view raw 4]: https://gist.github.com/dipanjanS/e21c2fe26a54c3bbf90fe751989a7560/raw/3f38e24ea40402fcac6a09d6d39cd3657c7ba7ed/feature_engg_text_25.py
[feature_engg_text_25.py]: https://gist.github.com/dipanjanS/e21c2fe26a54c3bbf90fe751989a7560#file-feature_engg_text_25-py
[1_guUd_CKJHYOwNIIt_6YchA.png]: https://cdn-images-1.medium.com/max/1000/1*guUd_CKJHYOwNIIt_6YchA.png
[view raw 5]: https://gist.github.com/dipanjanS/3557b2a27d4365e16890aee3e6e35dbe/raw/41e923520ecfc0390d27495db171998b957f429b/feature_engg_text_26.py
[feature_engg_text_26.py]: https://gist.github.com/dipanjanS/3557b2a27d4365e16890aee3e6e35dbe#file-feature_engg_text_26-py
[t-SNE]: https://en.wikipedia.org/wiki/T-distributed_stochastic_neighbor_embedding
[dimensionality reduction]: https://en.wikipedia.org/wiki/Dimensionality_reduction
[view raw 6]: https://gist.github.com/dipanjanS/7aa50bfdc663afdb106c1d842332f637/raw/5660dc7f9d587ef7773f2f5b0d4ad26498553ea0/feature_engg_text_27.py
[feature_engg_text_27.py]: https://gist.github.com/dipanjanS/7aa50bfdc663afdb106c1d842332f637#file-feature_engg_text_27-py
[1_5N5p0cqCitIiGpfbQufc-A.png]: https://cdn-images-1.medium.com/max/800/1*5N5p0cqCitIiGpfbQufc-A.png
[Dipanjan Sarkar 1]: https://www.linkedin.com/in/dipanzan
[Text Data Preprocessing_ A Walkthrough in Python]: https://www.kdnuggets.com/2018/03/text-data-preprocessing-walkthrough-python.html
[A General Approach to Preprocessing Text Data]: https://www.kdnuggets.com/2017/12/general-approach-preprocessing-text-data.html
[A Framework for Approaching Textual Data Science Tasks]: https://www.kdnuggets.com/2017/11/framework-approaching-textual-data-tasks.html