Cloud Large Scale Video Analysis

The aim of Cloud-LSVA is the creation of cloud platforms for the mass annotation of data, which can be applied to the automotive, cartography or other sectors. In particular, technologies such as cloud computing, Deep Learning, real-time inference, and 3D web visualisation have been investigated in the project.

During the project, specific tools have been created for the annotation of multi-sensor recordings captured with test vehicles equipped with cameras, Lidar sensors, DGPS and recording and execution systems. These tools are basically grouped into two categories: back-end tools and front-end tools. In the back-end category, work has been done with cloud technologies, such as Docker and Kubernetes, for the automation, distribution and optimisation of the execution of automatic computation tasks, such as, for example, the pre-annotation of images and point clouds with Deep Learning algorithms. A large part of the work has focused on the development of these algorithms, as well as the creation of pipelines which execute the algorithms in the proper order and distribute the computation in the available cloud resources. The front-end part has used web technologies to build web user interfaces with multiple video and point cloud sequence synchronous visualisation capacity (using WebGL), as well as temporal navigation and the utilisation of user tools to create, delete, modify and save annotations of different types (for example, bounding boxes, cuboids, 3D polylines, areas, etc.).

Vicomtech’s role has been administrative, innovative and technical leadership of the project. In this regard, Vicomtech has been the undisputed protagonist of the project, leading the integration of the different generated prototypes, of the demonstrations carried out with test vehicles (including Vicomtech’s autonomous vehicle), as well as the cloud computing capacities and Deep Learning algorithms optimised for execution on integrated real time platforms in the vehicle (like NVIDIA Drive PX 2 and Intel Halocreek).

At the conclusion of the project, Vicomtech has acquired a know-how and some prototype tools, making it one of the most experienced research sites in the growing field of semi-automatic annotation of multimedia content.

Cloud-LSVA is a European project, financed by the European Commission, in the framework H2020, led by Vicomtech, and which has a consortium of companies, universities and research centres (including outstanding actors in the automotive sector, such as Valeo, TomTom, and the computing sector, such as IBM and Intel). The project has been conducted over 36 months, from January 2016 to December 2018, and has reached a successful conclusion, exceeding the European Commission review.