I must admit that my first encounter with the so called Kullback-Leibler divergence was underwhelming. Whether it was the esoteric formula, or the fact that it was grouped together with what seemed like a dozen other entropy formulas; all I can say is that I didn’t like it.
I am in the process of migrating my old blog posts to this new github.io blog. I’m using a slightly modified version of Not Pure Poole. All posts prior to this one are from my old github.io blog.
