Description
- Abstract:
- Deep neural networks (DNNs), which are based loosely on the ventral stream pathway of the primate visual system, are good models of the visual system because they fit neural data of the ventral steam (Yamins et al., 2013). Serre (2019) details that DNNs are also able to mimic functions of the human visual system such as image categorization, where they have achieved high accuracy. However, these models are vulnerable to adversarial attacks–small changes in image pixel values which have no effect on the accuracy of human vision. They have decreased performance when exposed to images from the ObjectNet dataset that contains objects of different rotations, viewpoints and backgrounds. The low adversarial and ObjectNet robustness of models mark a processing difference between models and the human visual system. Work by Fel et al (2022) has resulted in harmonized models, which are models that are trained on both categorization accuracy loss and a loss in alignment of human feature importance maps that capture the features in an image important for classification. Harmonized models are able to achieve high categorization accuracy as well as alignment to human visual strategies. To further explore the capabilities of harmonized models, we compare the adversarial robustness of harmonized models compared to their corresponding baseline models (models without neural harmonizer) by comparing model accuracy on the Projected Gradient Descent (LinfPGD) adversarial attack from the Foolbox toolbox. Overall, we found that although accuracy drops with increasing image perturbation, harmonized models show greater adversarial robustness compared to baseline models. Furthermore, we compare the accuracy of the models on the ObjectNet dataset and find that harmonization does not increase real world image robustness which we attribute to model limitations due to training on ImageNet.
- Notes:
- Senior thesis (ScB)--Brown University, 2023
- Concentration: Neuroscience
Access Conditions
- Use and Reproduction
- All rights reserved
- Rights
- In Copyright
- Restrictions on Use
- All Rights Reserved
Citation
Olaiya, Stephanie O.,
"Adversarial and Real World Image Robustness of Computer Vision Models With Better Human Visual Strategy Alignment"
(2023).
Neuroscience Theses and Dissertations.
Brown Digital Repository. Brown University Library.
https://repository.library.brown.edu/studio/item/bdr:mw5ra9fe/
