What are the common pre-trained word embeddings models available for NLP?

In natural language processing (NLP), pre-trained word embedding models are a crucial component, enabling models to understand and process language data. Common pre-trained word embedding models include:

1. Word2Vec: Developed by Google researchers in 2013, the Word2Vec model uses shallow neural networks to generate word vectors by learning context relationships from large text datasets. It features two training architectures: Skip-gram, which predicts context from the current word, and CBOW (Continuous Bag of Words), which predicts the current word from context. For example, Google utilized a large corpus of news articles to train its Word2Vec model.

2. GloVe (Global Vectors for Word Representation): Developed by Stanford University in 2014 as a statistical word embedding technique, GloVe constructs a global co-occurrence matrix to statistically capture word frequencies and then decomposes this matrix to obtain word vectors. This approach combines the strengths of matrix decomposition and local window methods, effectively capturing relationships between words.

3. fastText: Developed by Facebook's research team in 2016, fastText is similar to Word2Vec but incorporates subword structures (i.e., word subwords) in addition to whole words. This makes it particularly suitable for morphologically rich languages (such as German or Turkish) and better handles out-of-vocabulary (OOV) words.

These models operate under different assumptions and techniques to process and understand words. Their common goal is to convert words into numerical forms that computers can process (i.e., word vectors), which encode rich semantic information and linguistic structures. In practical applications, the choice of word embedding model typically depends on specific task requirements and available computational resources.