Angular Margin-Based Softmax Losses: Toward Discriminative Deep Metric Learning

Abstract

Discriminative deep metric learning aims to construct an embedding space in whichinstances of the same class can be grouped together while being effectivelydistinguished from instances belonging to other classes by deeply learnedrepresentations. In this context, angular deep metric learning emerges as a specializedsubset of discriminative deep metric learning, which is characterized by focusing onthe angles between the feature vectors rather than their magnitudes.Classical methods such as ArcFace and CosFace are considered pioneers in the fieldof angle-dependent metric learning as they introduce angle-dependent margins intothe softmax loss function. This strategic approach aims to promote more coherentclustering within classes while achieving greater angular separation between differentclasses. These methods have been applied specifically in the context of facerecognition.This thesis presents several research contributions consisting of novel softmax lossfunctions based on angular margins. The first contribution is to extend theapplicability of these loss functions beyond the field of face recognition. Theeffectiveness of these functions is investigated in challenging contexts with limitedlabeled data. Topics such as fashion image retrieval, fashion style recognition andclassification of histopathologic breast cancer images are covered.In the area of fashion image retrieval, Discriminative Margin Loss (DML) is proposedto investigate the adjustment of margin penalties for positive and negative classes.The underlying goal is to improve the discriminative power of the learnedembeddings specifically for fashion image retrieval.For the challenges related to fashion style and face recognition, Additive CosineMargin Loss (ACML) is introduced. ACML simplifies the fine-tuning of marginpenalties while strengthening the separation between classes and the cohesion withinclasses. This approach leads to performance improvements in these specialrecognition tasks.The Boosted Additive Angular Margin Loss (BAM) method is proposed for the fieldof breast cancer diagnosis using histopathological images. BAM not only penalizesthe angle between the deep feature and its corresponding weight from the target class,but also considers the angles between deep features and their corresponding weightsfrom non-target classes. This approach aims to facilitate the detection of highlydiscriminative features for accurate diagnosis while improving the intra-classcohesion and increasing the inter-class discrepancy to take advantage of marginconstraints.Overall, these new loss functions, including DML, ACML, and BAM, contributesignificantly to the field of softmax losses based on angular margins by extendingtheir application beyond conventional constraints. With this expanded scope, theseloss functions are able to effectively address the distinct challenges inherent in diversedomains.The proposed loss functions have been rigorously tested and validated in variousstudies addressing data limitations. The robustness of the results was demonstratedthrough various metrics, feature visualizations and statistical analysis. It is importantto mention that these loss functions have improved the performance of numerous deeplearning architectures. The superiority of these loss functions over various complexfeature-based architectures containing significant parameters has been confirmedthrough extensive experiments and comparisons with angular margin-based losses onvarious benchmark datasets. In addition, the performance of the models was evaluatedagainst other methods on large datasets to obtain a comprehensive assessment of theircapabilities.