Binary Cross-Entropy Loss

Cross-Entropy loss for a mulit-label classifier (taggers)

Binary Cross-Entropy loss is a special case of Cross-Entropy loss used for multilabel classification (taggers). It is reliant on Sigmoid activation functions.
Code  implementation
PyTorch
TensorFlow
PyTorch
# importing the library
import torch
import torch.nn as nn
​
input = torch.randn(3, 5, requires_grad=True)
​
# Binary Cross-Entropy Loss
​
target = torch.ones([10, 64], dtype=torch.float32)  # 64 classes, batch size = 10
output = torch.full([10, 64], 1.5)  # A prediction (logit)
​
pos_weight = torch.ones([64])  # All weights are equal to 1
criterion = torch.nn.BCEWithLogitsLoss(pos_weight=pos_weight)
criterion(output, target)  # -log(sigmoid(1.5))
TensorFlow
# importing the library
import tensorflow as tf
​
y_true = [[0., 1.], [0., 0.]]
y_pred = [[0.6, 0.4], [0.4, 0.6]]
​
# Using 'auto'/'sum_over_batch_size' reduction type.
bce = tf.keras.losses.BinaryCrossentropy()
bce(y_true, y_pred).numpy()
​
​
# Calling with 'sample_weight'.
bce(y_true, y_pred, sample_weight=[1, 0]).numpy()
Further Resources 
​Cross-Entropy Loss​
​Classifier / Tagger​

Loss 📉 - Previous

Cross-Entropy Loss

Next - Solvers / Optimizers 🧮

Momentum (SGD)

Last updated 2 months ago