Before you begin make sure you have installed the following libraries: nltk, genism, tensorflow and numpy. In contrast to traditional NLP approaches which associate words with discrete representations, vector space models of meaning embed each word into a continuous vector space in which words can be easily compared for similarity. They are based on the distributional hypothesis stating that a word’s meaning can be inferred from the contexts it appears in. For example, lemon would be defined in terms of words such as juice, zest, curd or squeeze, providing an indication that it is a type of fruit.įollowing this hypothesis, words are represented by means of their neighbours - each word is associated with a vector that encodes information about its co-occurrence with other words in the vocabulary. Representations built in such a way demonstrate a useful property: vectors of words related in meaning are similar - they lie close to one another in the learned vector space. One common way of measuring this similarity is to use the cosine of the angle between the vectors. The exact method of constructing word embeddings differs across the models, but most approaches can be categorised as either count-based or predict-based, with the latter utilising neural models.
In this tutorial we will focus on one of the most popular neural word-embedding models - Skip-gram.
0 kommentar(er)
