Let me explicitly state at the very beginning that I'm a complete beginner in neural networks. So, please bear with me if this question appears to be too naive.
There are lots of loss functions available. My problem is that I don't know all of them. Hence, I'm confused to choose between them while implementing a model. I understand that it's preferable to use MSE, if I want to minimise the Euclidean distance between the source and the target. Likewise for Manhattan distance, the choice should be MAE.
So, I was wondering whether there are some references where one can get the basic ideas (definition) about different loss functions, and the situations in which they perform better (or at least, theoretically more justifiable and/or preferable) than the others, and more importantly, why. Unfortunately, I couldn't find anything as such. The best I found is in the PyTorch documentation. This provides definitions almost all the loss functions provided in PyTorch, but not their justifications.
So, if someone can provide me such references, or can explain the loss functions (at least some of them), it will be really helpful.