Models

Word2Vec

• Papers
  • Efficient Estimation of Word Representations in Vector Space
  • Distributed Representations of Words and Phrases and their Compositionality
  • Linguistic Features in Turkish Word Representations
  • Intrinsic and Extrinsic Evaluation of Word Embedding Models
• Implementation
  • Gensim Word2Vec
  • Turkish Word2Vec

GloVe (Global Vectors)

• Paper
• Paper explanations (unofficial)
• Github Repository
• Türkçe GloVe - Repository for Turkish GloVe Word Embeddings
• glove-python: Contains a script for processing a corpus and calculating similarity

FastText

• Papers
  • Enriching Word Vectors with Subword Information
• Gensim FastText
• Reference Turkish Word Embeddings

BERT (Bidirectional Encoder Representations from Transformers)

• Github Repository: Contains a guide for finetuning, "BERT FineTuning with Cloud TPUs" (There is a pretrained multilingual model, probably we will fine tune that.).
  • Fine-tuning a BERT Model: TensorFlow tutorial for fine-tuning a BERT model.
• turkish-bert: BERT models trained with Turkish corpus. Maybe we can fine tune these models.
• Train BERT From Scratch: A guide on how to train a BERT model from scratch.

ELMo (Embeddings from Language Models)

• Papers
  • Deep contextualized word representations
• ELMo TensorFlow
• ELMo PyTorch
• PyTorch Implementation Wrapper