Author: adminmautiste

26 Jun So we need esatto compute the gradient of CE Loss respect each CNN class conteggio in \(s\)

Defined the loss, now we'll have esatto compute its gradient respect esatto the output neurons of the CNN sopra order puro backpropagate it through the net and optimize the defined loss function tuning the net parameters. The loss terms coming from the negative classes are zero. However, the loss gradient respect those negative classes is not cancelled, since the Softmax of the positive class also depends on the negative classes scores.

The gradient expression will be the same for all \(C\) except for the ground truth class \(C_p\), because the punteggio of \(C_p\) (\(s_p\)) is mediante the nominator.

• Caffe: SoftmaxWithLoss Layer. Is limited preciso multi-class classification.
• Pytorch: CrossEntropyLoss. Is limited sicuro multi-class classification.
• TensorFlow: softmax_cross_entropy. Is limited sicuro multi-class classification.

Durante this Facebook rete di emittenti they claim that, despite being counter-intuitive, Categorical Cross-Entropy loss, or Softmax loss worked better than Binary Ciclocampestre-Entropy loss mediante their multi-label classification problem.

> Skip this part if you are not interested con Facebook or me using Softmax Loss for multi-label classification, which is not norma.

When Softmax loss is used is verso multi-label scenario, the gradients get per bit more complex, since the loss contains an element for each positive class. Consider \(M\) are the positive classes of per sample. The CE Loss with Softmax activations would be:

Where each \(s_p\) durante \(M\) is the CNN conteggio for each positive class. As sopra Facebook paper, I introduce a scaling factor \(1/M\) onesto make the loss invariant onesto the number of positive classes, which ple.

As Caffe Softmax with Loss layer nor Multinomial Logistic Loss Layer accept multi-label targets, I implemented my own PyCaffe Softmax loss layer, following the specifications of the Facebook paper.