2022
Pestana, Camilo; Akhtar, Naveed; Rahnavard, Nazanin; Shah, Mubarak; Mian, Ajmal
Transferable 3D Adversarial Textures using End-to-end Optimization Conference
IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2022.
Abstract | Tags: Adversarial Attacks, WACV | Links:
@conference{Pestana2022,
title = {Transferable 3D Adversarial Textures using End-to-end Optimization},
author = {Camilo Pestana and Naveed Akhtar and Nazanin Rahnavard and Mubarak Shah and Ajmal Mian},
url = {https://www.crcv.ucf.edu/wp-content/uploads/2018/11/WACV_2022_Deceptive_Textures-1.pdf},
doi = {10.1109/WACV51458.2022.00080},
year = {2022},
date = {2022-02-15},
urldate = {2022-02-15},
pages = {727-736},
publisher = {IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
abstract = {Deep visual models are known to be vulnerable to adversarial attacks. The last few years have seen numerous techniques to compute adversarial inputs for these models. However, there are still under-explored avenues in this critical research direction. Among those is the estimation of adversarial textures for 3D models in an end-to-end optimization scheme. In this paper, we propose such a scheme to generate adversarial textures for 3D models that are highly transferable and invariant to different camera views and lighting conditions. Our method makes use of neural rendering with explicit control over the model texture and background. We ensure transferability of the adversarial textures by employing an ensemble of robust and non-robust models. Our technique utilizes 3D models as a proxy to simulate closer to real-life conditions, in contrast to conventional use of 2D images for adversarial attacks. We show the efficacy of our method with extensive experiments.},
keywords = {Adversarial Attacks, WACV},
pubstate = {published},
tppubtype = {conference}
}
Deep visual models are known to be vulnerable to adversarial attacks. The last few years have seen numerous techniques to compute adversarial inputs for these models. However, there are still under-explored avenues in this critical research direction. Among those is the estimation of adversarial textures for 3D models in an end-to-end optimization scheme. In this paper, we propose such a scheme to generate adversarial textures for 3D models that are highly transferable and invariant to different camera views and lighting conditions. Our method makes use of neural rendering with explicit control over the model texture and background. We ensure transferability of the adversarial textures by employing an ensemble of robust and non-robust models. Our technique utilizes 3D models as a proxy to simulate closer to real-life conditions, in contrast to conventional use of 2D images for adversarial attacks. We show the efficacy of our method with extensive experiments.
Aafaq, Nayyer; Mian, Ajmal; Naveed Akhtar, Wei Liu; Shah, Mubarak
Dense Video Captioning with Early Linguistic Information Fusion Journal Article
In: IEEE Transactions on Multimedia, pp. 1-1, 2022.
Abstract | Tags: Adversarial Attacks | Links:
@article{nokey,
title = {Dense Video Captioning with Early Linguistic Information Fusion},
author = {Nayyer Aafaq and Ajmal Mian and Naveed Akhtar, Wei Liu and Mubarak Shah},
url = {https://www.crcv.ucf.edu/wp-content/uploads/2018/11/IEEE_TMM_Nayyer_Jan_2022_Final_Version_Manuscript.pdf},
doi = {10.1109/TMM.2022.3146005},
year = {2022},
date = {2022-01-25},
urldate = {2022-01-25},
journal = {IEEE Transactions on Multimedia},
pages = {1-1},
abstract = {Dense captioning methods generally detect events in videos first and then generate captions for the individual events. Events are localized solely based on the visual cues while ignoring the associated linguistic information and context. Whereas end-to-end learning may implicitly take guidance from language, these methods still fall short of the power of explicit modeling. In this paper, we propose a Visual-Semantic Embedding (ViSE) Framework that models the word(s)-context distributional properties over the entire semantic space and computes weights for all the n-grams such that higher weights are assigned to the more informative n-grams. The weights are accounted for in learning distributed representations of all the captions to construct a semantic space. To perform the contextualization
of visual information and the constructed semantic space in a supervised manner, we design Visual-Semantic Joint Modeling Network (VSJM-Net). The learned ViSE embeddings are then temporally encoded with a Hierarchical Descriptor Transformer (HDT). For caption generation, we exploit a transformer architecture to decode the input embeddings into natural language descriptions. Experiments on the large-scale ActivityNet Captions dataset and YouCook-II dataset demonstrate the efficacy of our method.
Index Terms—Dense video captioning, event localisation, language and vision, video captioning, context modeling.},
keywords = {Adversarial Attacks},
pubstate = {published},
tppubtype = {article}
}
Dense captioning methods generally detect events in videos first and then generate captions for the individual events. Events are localized solely based on the visual cues while ignoring the associated linguistic information and context. Whereas end-to-end learning may implicitly take guidance from language, these methods still fall short of the power of explicit modeling. In this paper, we propose a Visual-Semantic Embedding (ViSE) Framework that models the word(s)-context distributional properties over the entire semantic space and computes weights for all the n-grams such that higher weights are assigned to the more informative n-grams. The weights are accounted for in learning distributed representations of all the captions to construct a semantic space. To perform the contextualization
of visual information and the constructed semantic space in a supervised manner, we design Visual-Semantic Joint Modeling Network (VSJM-Net). The learned ViSE embeddings are then temporally encoded with a Hierarchical Descriptor Transformer (HDT). For caption generation, we exploit a transformer architecture to decode the input embeddings into natural language descriptions. Experiments on the large-scale ActivityNet Captions dataset and YouCook-II dataset demonstrate the efficacy of our method.
Index Terms—Dense video captioning, event localisation, language and vision, video captioning, context modeling.
of visual information and the constructed semantic space in a supervised manner, we design Visual-Semantic Joint Modeling Network (VSJM-Net). The learned ViSE embeddings are then temporally encoded with a Hierarchical Descriptor Transformer (HDT). For caption generation, we exploit a transformer architecture to decode the input embeddings into natural language descriptions. Experiments on the large-scale ActivityNet Captions dataset and YouCook-II dataset demonstrate the efficacy of our method.
Index Terms—Dense video captioning, event localisation, language and vision, video captioning, context modeling.
2021
Akhtar, Naveed; Mian, Ajmal; Kardan, Navid; Shah, Mubarak
Advances in Adversarial Attacks and Defenses in Computer Vision: A Survey Journal Article
In: IEEE Access, vol. 9, pp. 155161-155196, 2021.
Abstract | Tags: Adversarial Attacks | Links:
@article{Akhtar2021,
title = {Advances in Adversarial Attacks and Defenses in Computer Vision: A Survey},
author = {Naveed Akhtar and Ajmal Mian and Navid Kardan and Mubarak Shah},
editor = {IEEE Access},
url = {https://www.crcv.ucf.edu/wp-content/uploads/2018/11/Advances_in_Adversarial_Attacks_and_Defenses_in_Computer_Vision_A_Survey.pdf},
doi = {10.1109/ACCESS.2021.3127960},
year = {2021},
date = {2021-11-12},
urldate = {2021-11-12},
journal = {IEEE Access},
volume = {9},
pages = {155161-155196},
abstract = {Deep Learning is the most widely used tool in the contemporary field of computer vision. Its ability to accurately solve complex problems is employed in vision research to learn deep neural models for a variety of tasks, including security critical applications. However, it is now known that deep learning is vulnerable to adversarial attacks that can manipulate its predictions by introducing visually imperceptible perturbations in images and videos. Since the discovery of this phenomenon in 2013, it has attracted significant attention of researchers from multiple sub-fields of machine intelligence. In 2018, we published the first-ever review of the contributions made by the computer vision community in adversarial attacks on deep learning (and their defenses). Many of those contributions have inspired new directions in this area, which has matured significantly since witnessing the first generation methods. Hence, as a legacy sequel of our first literature survey, this review article focuses on the advances in this area since 2018. We thoroughly discuss the first generation attacks and comprehensively cover the modern attacks and their defenses appearing in the prestigious sources of computer vision and machine learning research. Besides offering the most comprehensive literature review of adversarial attacks and defenses to date, the article also provides concise definitions of technical terminologies for the non-experts. Finally, it discusses challenges and future outlook of this direction based on the literature since the advent of this research direction.},
keywords = {Adversarial Attacks},
pubstate = {published},
tppubtype = {article}
}
Deep Learning is the most widely used tool in the contemporary field of computer vision. Its ability to accurately solve complex problems is employed in vision research to learn deep neural models for a variety of tasks, including security critical applications. However, it is now known that deep learning is vulnerable to adversarial attacks that can manipulate its predictions by introducing visually imperceptible perturbations in images and videos. Since the discovery of this phenomenon in 2013, it has attracted significant attention of researchers from multiple sub-fields of machine intelligence. In 2018, we published the first-ever review of the contributions made by the computer vision community in adversarial attacks on deep learning (and their defenses). Many of those contributions have inspired new directions in this area, which has matured significantly since witnessing the first generation methods. Hence, as a legacy sequel of our first literature survey, this review article focuses on the advances in this area since 2018. We thoroughly discuss the first generation attacks and comprehensively cover the modern attacks and their defenses appearing in the prestigious sources of computer vision and machine learning research. Besides offering the most comprehensive literature review of adversarial attacks and defenses to date, the article also provides concise definitions of technical terminologies for the non-experts. Finally, it discusses challenges and future outlook of this direction based on the literature since the advent of this research direction.
2020
Zhang, Xiaoyu; Mian, Ajmal; Gupta, Rohit; Rahnavard, Nazanin; Shah, Mubarak
Cassandra: Detecting Trojaned Networks from Adversarial Perturbations Technical Report
no. arXiv:2007.14433, 2020.
Tags: Adversarial Attacks | Links:
@techreport{Zhang2020,
title = {Cassandra: Detecting Trojaned Networks from Adversarial Perturbations},
author = {Xiaoyu Zhang and Ajmal Mian and Rohit Gupta and Nazanin Rahnavard and Mubarak Shah},
url = {https://www.crcv.ucf.edu/wp-content/uploads/2020/08/Publications_Cassandra-Detecting-Trojaned-Networks-from-Adversarial-Perturbations.pdf},
year = {2020},
date = {2020-07-28},
urldate = {2020-07-28},
number = {arXiv:2007.14433},
keywords = {Adversarial Attacks},
pubstate = {published},
tppubtype = {techreport}
}
Edraki, Marzieh; Karim, Nazmul; Rahnavard, Nazanin; Mian, Ajmal; Shah, Mubarak
Odyssey: Creation, Analysis and Detection of Trojan Models Technical Report
no. arXiv:2007.08142, 2020.
Tags: Adversarial Attacks | Links:
@techreport{Edraki2020,
title = {Odyssey: Creation, Analysis and Detection of Trojan Models},
author = {Marzieh Edraki and Nazmul Karim and Nazanin Rahnavard and Ajmal Mian and Mubarak Shah
},
url = {https://www.crcv.ucf.edu/wp-content/uploads/2020/08/Publications_Odyssey-Creation-Analysis-and-Detection-of-Trojan-Models.pdf
https://www.crcv.ucf.edu/research/projects/odyssey-creation-analysis-and-detection-of-trojan-models/},
year = {2020},
date = {2020-07-16},
number = {arXiv:2007.08142},
keywords = {Adversarial Attacks},
pubstate = {published},
tppubtype = {techreport}
}