Does the Fairness of Your Pre-Training Hold Up? Examining the Influence of Pre-Training Techniques on Skin Tone Bias in Skin Lesion Classification

Pratinav Seth, Abhilash K. Pai

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Deep Neural Networks (DNNs) have found widespread application in various domains, but the challenge of addressing Algorithmic bias and ensuring fairness in their decision-making processes has emerged as a critical concern, particularly in mission-critical contexts. One of the main reasons for this concern is the inadequate representation of certain groups in the available datasets used for training. Pre-Training is a powerful technique for training DNNs, but it can be affected by pre-existing biases in the dataset. These biases can be transferred to the DNN during Pre-Training, leading to the DNNs making biased decisions, even when trained on unbiased datasets. This study investigates the impact on the fairness of popular Pre-Training methods, such as Masked Image Modeling (MAE, SimMIM) and Self-Supervised Learning (BYOL, MoCo, SimCLR, VICRegL), when used on skin lesion classification datasets with underrepresented demographic groups. The study compares the performance of pre-trained models to supervised learning backbones on two skin lesion datasets (ISIC-2019 and Fitzpatrickl17k) with different skin tone distributions. The findings of this study reveal that Pre-Training improves performance but has a trade-offwith fairness, which can be a potential danger associated with the model when applied in the real world. This study is one of the first to investigate how Self-Supervised Learning and Masked Image Modeling Pre-Training methods affect fairness in both in-distribution and out-of-distribution scenarios. Code is available at https://github.com/ptnv-s/PretrainingImpactOnSkinBias.

Original languageEnglish
Title of host publicationProceedings - 2024 IEEE Winter Conference on Applications of Computer Vision Workshops, WACVW 2024
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages580-587
Number of pages8
ISBN (Electronic)9798350370287
DOIs
Publication statusPublished - 2024
Event2024 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops, WACVW 2024 - Waikoloa, United States
Duration: 04-01-202408-01-2024

Publication series

NameProceedings - 2024 IEEE Winter Conference on Applications of Computer Vision Workshops, WACVW 2024

Conference

Conference2024 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops, WACVW 2024
Country/TerritoryUnited States
CityWaikoloa
Period04-01-2408-01-24

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Computer Vision and Pattern Recognition
  • Media Technology

Fingerprint

Dive into the research topics of 'Does the Fairness of Your Pre-Training Hold Up? Examining the Influence of Pre-Training Techniques on Skin Tone Bias in Skin Lesion Classification'. Together they form a unique fingerprint.

Cite this