Neural Collapse with Cross-Entropy Loss
share
We consider the variational problem of cross-entropy loss with n feature vectors on a unit hypersphere in ℝ^d. We prove that when d ≥ n - 1, the global minimum is given by the simplex equiangular tight frame, which justifies the neural collapse behavior. We also show a connection with the frame potential of Benedetto Fickus.
READ FULL TEXT