Deep learning architectures such as convolutional deep networks, residual networks and Transformers are at the forefront of automatically making sense of large volumes of data, e.g. considering the typical task of recognizing patterns in such data and classifying them to classes of a pre-determined class set. In the last few years, they brought tremendous gains in classification / recognition accuracy; at the same time, though, because of their complex multi-layer nonlinear structure and end-to-end learning strategies, they typically act as ‘‘black box’’ models that lack transparency. This is a barrier to adoption in application domains where explainability is important: it makes it difficult to convince both machine learning engineers and end-users in such critical application domains, including law and governance, to trust and employ such ‘‘black box’’ models.
Therefore, it is necessary to develop solutions that address the transparency challenge of deep neural networks. In response to this need, explainable artificial intelligence (XAI) is gaining momentum and is becoming a very active research direction. XAI focuses on developing explainable techniques that help the developers and users of AI models to comprehend, trust, and more efficiently manage them.
Explainable AI for visual classifiers in CRiTERIA
In CRiTERIA, we consider the image/video classification problem as a proxy for developing explainable AI techniques, both because images and videos are important to CRiTERIA and because of the abundance of useful data in existing datasets that are widely used in the research community, such as ImageNet. Our work was based on two important considerations. On the one hand, we should not introduce explainability at the expense of classifier accuracy or computational complexity; thus, we should be able to efficiently explain already trained and optimized deep learning models. And, on the other hand, we should develop a solution that is generally applicable to various deep learning architectures rather than tailored to a specific one, because deep learning architectures evolve at a quick pace, thus present-time architectures may soon become outdated.
In response to these considerations, we developed T-TAME: Transformer-compatible Trainable Attention Mechanism for Explanations. T-TAME is a post-hoc explainability method, i.e. it works with already-trained deep models and does not in any way degrade the accuracy of their decisions. It is also a fairly general methodology for explaining deep neural networks used in visual classification tasks: T-TAME’s architecture and training protocol can be easily applied to any convolutional, residual or Vision Transformer-like neural network, using a streamlined training approach. T-TAME also scores highly in terms of efficiency: after training, explanation maps can be computed in a single forward pass through the deep network and T-TAME’s explanation layers.
We tested T-TAME with three popular deep learning classifier architectures, VGG-16, ResNet-50, and ViT-B-16, trained on the ImageNet dataset, and we demonstrate improvements over existing state-of-the-art explainability methods. T-TAME achieves state-of-the-art performance, comparing favorably even to perturbation-based explainability techniques that are a few orders of magnitude more computationally expensive.
Example explanation map (right) for the decision of a ResNet-50 classifier to classify the input image (left) to the “padlock” class. Red-colored areas in the explanation map indicate where in the image the classifier “looked at” for making its decision.
Vasileios Mezaris
Vasileios Mezaris is a Research Director with the Information Technologies Institute (ITI) / Centre for Research and Technology Hellas (CERTH), Thessaloniki, Greece. He is the Head of the Intelligent Digital Transformation (IDT) Laboratory of ITI/CERTH, where he leads a group of researchers working on multimedia understanding and artificial intelligence. He holds a BSc and a PhD in Electrical and Computer Engineering, both from the Aristotle University of Thessaloniki.
References:
- M. Ntrougkas, N. Gkalelis, V. Mezaris, “T-TAME: Trainable Attention Mechanism for Explaining Convolutional Networks and Vision Transformers“, arXiv:2403.04523.
- M. Ntrougkas, N. Gkalelis, V. Mezaris, “TAME: Attention Mechanism Based Feature Fusion for Generating Explanation Maps of Convolutional Neural Networks“, Proc. IEEE Int. Symposium on Multimedia (ISM), Naples, Italy, pp. 58-65, Dec. 2022.
- I. Gkartzonika, N. Gkalelis, V. Mezaris, “Learning Visual Explanations for DCNN-Based Image Classifiers Using an Attention Mechanism“, Proc. ECCV 2022 Workshops, Springer LNCS vol. 13808, pp. 396-411, Oct. 2022.
All publications are available via the CRiTERIA publication portal.
Banner image by Timothy Muza on Unsplash.
