Cross-Entropy Loss

We use this for Classification problems, i am pretty sure?

The cross-entropy between a “true” distribution $p$ and an estimated distribution $q$ is defined as: $H(p,q)=-\sum _{x}p(x)\log q(x)$

The cross-entropy loss is given by $L_i=-\log \frac{e^{f_{y_i}}}{\sum _j{e}^{f_j}}$

Is this a similar idea of using the negative log to the Negative Log Likelihood??

https://aboveintelligent.com/deep-learning-basics-the-score-function-cross-entropy-d6cc20c9f972

Also see Softmax Classifier

Related

• Cross-Entropy Method