Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
Entropy and Related Concepts
- Entropy
- Shannon Entropy
- Cross Entropy
- K-L divergence
Tips
-
The entropy concept was first introduced for discrete distributions (called Shannon entropy), which is defined as
$$H(X) = E …