Performance Insights with eXplainable Model mONitoring

In the midst of the digital transformation era, Artificial Intelligence (AI) has been the technology most widely adopted by companies in Bizkaia over the last year. The percentage of entities using it has doubled compared to 2023, and investment in AI is expected to continue to increase. However, the use of AI-based tools at the production level (TRL9) entails a number of risks and challenges. Models that use these tools tend to perform worse when deployed in operational environments and degrade over time. In addition, these models operate in an opaque manner, commonly referred to as “black boxes,” whose functioning is not easily understood by humans, making them difficult to maintain. This project proposes the development of a real-time monitoring tool for AI models deployed in operational environments. This tool will use Explainable Artificial Intelligence (XAI) techniques, model drift detection, and adversarial attack detection to automatically detect possible model failures, whether due to natural or malicious causes, and generate reports that are understandable to humans. This information can be used by companies to identify and understand vulnerabilities in their product and activate the process of improving it.