Coach Assistant via Projected and Tangible Interface



01.12.2017 - 30.11.2020


See CORDIS data

Older adults typically prefer living at their homes as long as possible. However, they often need to be institutionalized due to the age related problems. Homecare can benefit today from a range of existing technologies including smartphones, sensors, etc., however, their effectiveness is limited by the sense of fear and uncomfortableness that many older adults feel. Most current technologies, in fact, are often not designed for older adults, least of all for users with memory impairments. The derivative limitations become a major barrier, severely limiting use of technological assistance in a home environment. CAPTAIN proposes a “transparent” technology designed to turn the home of the older adult into a ubiquitous assistant specifically designed to compensate for their physical and memory impairments during their daily living. To do so, it leverages on a few state of the art technologies, as follows:
“Projected augmented reality” to project, through use of micro-projectors, contextualised (directly on walls, floor, tables, etc.) information and instructions on top of the real environment. Real-time 3D sensing technologies to comprehend the “indoor space” (nature and position of objects and actions of the persons) and to allow transforming it into a ubiquitous tangible interface. 3D sensing, together with speech analysis, will allow device-free comprehension of the environment and of the behaviour of the adults (including its changes). Non-invasive physiological and emotional data analysis from facial micro-expressions and human body pose thanks to scalable, robust, and accurate deep learning and artificial intelligence. The coach will leverage on a motivational engine to promote correct nutrition, physical activity, cognitive and physical training, risk avoidance, and social participation. To achieve this CAPTAIN will foster a truly user-centered co-design philosophy -based on constant involvement of older adult in the design, development, and testing stages.

CAPTAIN proposes to turn the home of the older adult into a ubiquitous assistant to compensate for their physical and memory impairments during their daily living, using natural user interfaces and a user adapted and evolving virtual coach.

Vicomtech has developed a virtual coach based on SMART goals. People can select goals aimed at improving their status in physical, cognitive, socializing and nutritional aspects of a catalog, developed by domain experts. Once selected, the coach proposes activities within these plans and supports the person to achieve them in a personalized and adaptive way. To do this it employs data-mining and visual analytics technologies. In addition to that, Vicomtech has worked on the design of the system software architecture and open APIs to facilitate the development of new services to third parties.      

Looking for support for your next project? Contact us, we are looking forward to helping you.


Parque Científico y Tecnológico de Gipuzkoa,
Paseo Mikeletegi 57,
20009 Donostia / San Sebastián (Spain)

+(34) 943 309 230

Edificio Ensanche,
Zabalgune Plaza 11,
48009 Bilbao (Spain)

close overlay