Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
Tips on NLP
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
https://blog.floydhub.com/ is a great place for deep learning blogging.
Overview of NLP
Deep Learning for NLP: An Overview of Recent Trends Chapter 8 of the book (Performance of …
Nature Language Processing Using NLTK
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
nltk.util.ngrams nltk.bigrams nltk.PorterStemmer
from nltk.util import ngrams
sentence = 'this is a foo bar sentences and i want to ngramize it'
n = 6
sixgrams = ngrams(sentence.split …Keywords Extracting from Text
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
Word Stemming
-
existing stemming method such as NLTK.PorterStem, etc.
-
didn't -> did not, there's -> there is, etc. Mr. -> Mister Mrs. -> ... Ms. -> ...
Other things
-
it seems that it is hard to get …
Tips on the Transformers Python Library for NLP
Tokenization in NLP
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
Libraries
SentencePiece
SentencePiece is an unsupervised text tokenizer for Neural Network-based text generation.