Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
Classic word representation cannot handle unseen word or rare word well. Character embeddings is one of the solution to overcome out-of-vocabulary (OOV). However, it may be too fine-grained and miss some …
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
Word Embedding Character Embedding Subword Embeddling Tokenization
General Language Understanding Evaluation (GLUE)
Natural Language Generation (NLG) Natural Language Generation, as defined by Artificial Intelligence: Natural Language Processing Fundamentals, is the “process …
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
https://nlpoverview.com/#a-pos-tagging
Andor et al. (2016) showed a transition-based approach that produces competitive result with a simple feed-forward neural network.
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
Code for the paper "Language Models are Unsupervised Multitask Learners"
The project huggingface/transfomer has implementation of transfomer based modles (such as GPT-2 ) .
https://openai …
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
http://nlp.seas.harvard.edu/2018/04/03/attention.html
https://blog.floydhub.com/the-transformer-in-pytorch/
http://jalammar.github.io/illustrated-transformer/
https://towardsdatascience.com/transformers-141e32e69591
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
http://www.wildml.com/2016/01/attention-and-memory-in-deep-learning-and-nlp/
https://medium.com/@joealato/attention-in-nlp-734c6fa9d983