Date of Award
Fall 11-22-2024
Document Type
Dissertation
Degree Name
Ph.D.
Organizational Unit
Daniel Felix Ritchie School of Engineering and Computer Science, Electrical and Computer Engineering
First Advisor
Mohammad H. Mahoor
Second Advisor
Stephen Hutt
Third Advisor
Timothy Sweeny
Fourth Advisor
Haluk Ogmen
Copyright Statement / License for Reuse
All Rights Reserved.
Keywords
Artificial intelligence, Custom loss functions, Deep learning, Machine learning
Abstract
This dissertation explores the critical role of loss functions in enhancing the predictive performance of deep machine learning models. Loss functions are an integral element of all the ongoing advances we witness daily in this domain. I design custom loss functions and their impacts on various machine learning tasks, particularly in computer vision.
In the first stage of my research, I aim to improve the prediction performance of deep learning models by providing them with more precise feedback associated with task requirements. This led me to create the concept of assistive loss functions. My first proposed loss function, inspired by the Active Shape Model (ASM), is ASM Loss, designed to improve the detection of facial landmarks and head pose accuracy in facial images. Additionally, influenced by Knowledge Distillation (KD), I introduce KD-Loss, a loss designed specifically for regression tasks, focusing on detecting facial landmarks.
Proceeding to the second phase of my research, I focus on regression problems. Building upon my prior work, I introduce Adaptive Coordinate-based Regression (ACR)-Loss, a context-aware loss function with face alignment implication. I provide a mechanism to dynamically measure how hard the localization of each landmark point is, and accordingly, ACR-Loss provides unique feedback to the model. Furthermore, I introduce Intensity-aware Categorical (IC)-Loss for detecting sagittal cervical spine landmark points in medical images. It effectively deploys characteristics from classification problems to enhance regression optimization. Both ACR-Loss and IC-Loss comprise contextual information, leading to a significant enhancement in localization precision.
In the third phase, I turned my attention toward classification problems within the domain of deep machine learning. I introduce Adaptive Correlation-based (Ad-corre) Loss, designed to enhance the discriminative capabilities of deep learning models, with a specific focus on improving classification accuracy in automatic facial expression recognition. Furthermore, I introduce Informative (Info) Loss, a specialized loss function developed for medical applications, specifically aimed at distinguishing between Mild Cognitive Impairment and Normal Cognition groups based on transcriptions of interviews conducted as part of the I-CONECT study.
Overall, I have proposed custom loss functions to improve the predictive performance in both regression and classification problems. Furthermore, I have introduced assistive loss functions, designed to enhance the performance of the main loss functions. The collective outcomes of this extensive exploration highlight the critical role of loss functions, adapting and optimizing them for various domains in deep machine learning. This research suggests and proposes a variety of methodologies for designing adaptive and context-aware loss functions within deep machine learning.
Copyright Date
11-2024
Publication Statement
Copyright is held by the author. User is responsible for all copyright compliance.
Rights Holder
Ali Pourramezan Fard
Provenance
Received from author
File Format
application/pdf
Language
English (eng)
Extent
275 pgs
File Size
47.4 MB
Recommended Citation
Pourramezan Fard, Ali, "Designing Customized Loss Functions for Training Deep Neural Networks" (2024). Electronic Theses and Dissertations. 2501.
https://digitalcommons.du.edu/etd/2501
