IKUN, a project developing large multimodal models for quality assurance and operator support in Smart Industry

The IKUN project comes at a crucial time, when Artificial Intelligence (AI) is transforming strategic sectors thanks to advances in language models, computer vision, and content generation technologies such as GPT-4, LLaMA, DALL·E, and CLIP. However, integrating these into industrial environments still poses major challenges, from the need for large volumes of labeled data to interfaces poorly suited to the realities of the factory floor.

IKUN was created with the goal of adapting Large Multimodal Language Models (MLLMs) to industrial contexts, paving the way for smarter, more accessible, and more efficient solutions.

The project tackles several key technological challenges. These include the creation of multimodal industrial datasets, which are essential for adapting models to production environments; the development of MLLM adaptation techniques aimed at ensuring robust, explainable, and reliable systems; the generation of synthetic data (images and time series) to train advanced quality assurance systems and reduce the need for real data; and the design of conversational interfaces that allow operators to interact naturally with systems, whether through text, voice, or images.

IKUN develops solutions for various data types and enhances human-system interaction

With a practical and progressive approach, IKUN moves from defining pilots and collecting real data to validating models in industrial settings. Specific solutions for various data types are being developed, enhancing human-system interaction and ensuring usefulness and transferability in real operational conditions.

Vicomtech plays a central role in the IKUN project as the coordinating entity, leading overall project development and ensuring alignment among the partners. In addition to overseeing pilot progress, Vicomtech is actively involved in several of them, researching key new technologies for quality assurance and operator support. Its work spans from data collection and industrial corpus creation to training and validating multimodal models in real production environments, ensuring their applicability and robustness.

Expected results include the creation of new industrial datasets, generation of high-quality synthetic images and time series for quality assurance, the design of intelligent conversational assistants to support operators on production lines or with technical documentation, and validation of prototypes ready for transfer to the industrial ecosystem.

The IKUN consortium includes the technology centers VICOMTECH, TECNALIA, IKERLAN, TEKNIKER, and AZTERLAN; the University of the Basque Country (EHU); the business R&D unit IKOR TECHNOLOGY CENTER; and the intermediary agent IMH Campus. The project is funded by the ELKARTEK 2024 program of the Basque Government.