Softmax Classifier
The Softmax Classifier is another popular choice of a linear classifier compared to SVM. It uses the Softmax Function as its loss function (we call it the Softmax Function as its loss function (we call it the Cross-Entropy Loss loss), where Softmax classifier provides “probabilities” for each class.
Diffference with SVM
With a SVM, the Loss becomes 0 once there is enough distance between the correct class score and the incorrect class scores.
However, with a Softmax Loss, we will always have a non-zero loss, unless we have the the correct class score is infinity, and the incorrect class scores are negative infinity.
The sum of the values should be 1