Maintenance of large databases based on visual information using incremental learning

Collaborating partners: University of Ljubljana, Faculty of Computer and Information Science; DFG CONSULTING informacijski sistemi d.o.o.

Type of the research project: applied project, Slovenian Research Agency (project code: L2-6765)

Project duration: 1st July 2014 - 30th June 2017

Acronym: VILLarD (Visual Incremental Learning for Large Databases)


  • izr. prof. dr. Danijel Skočaj
  • doc. dr. Matej Kristan
  • univ. dipl. inž. Domen Tabernik
  • univ. dipl. inž. Matjaž Majnik
  • dr. Rok Mandeljc
  • mag. Peter Uršič
  • Domen Rački, dipl. inž.
  • mag. Tomaž Gvozdanović
  • univ. dipl. inž. Simon Jud
  • mag. Uroš Ranfl
  • dr. Rok Vezočnik
  • mag. Domen Smole
  • univ. dipl. inž. Marko Mahnič

Project overview

We live in the era of information abundance. However, rather than quantity, the central concern is becoming the quality and credibility of the acquired data. This is especially true for visual information databases. Although the field of computer vision has achieved significant progress recently, the methods for automatic image interpretation are still not reliable enough to be used for autonomous annotation and maintenance of image and video databases (e.g. databases of detected objects). On the other hand, manual annotation of video sequences with relevant objects is very time consuming, expensive, as well as tedious and therefore prone to errors.

In this project we aspire to combining two approaches: computer-based automation of image interpretation that is necessary for database maintenance as well as suitable introduction of a human verifier into the loop. Such combination is of central importance for developing a methodology suitable for semiautomatic maintenance of traffic signalization records, which is partially our project’s practical goal. Even the database of such records for only state roads in the Republic of Slovenia may contain more than 250.000 entries along with additional information. Automation is therefore crucial for continuous maintenance of such databases. The main goal of the project is to develop a framework for semi-supervised incremental learning as well as specific methods for visual learning and recognition that will increase the quality and efficiency of large visual information databases maintenance.

We approach the problem holistically. The problem of incremental learning in interaction with a human is addressed on a general meta level, at which we are determining a strategy for interactive learning that will give the best results in terms of learning success and recognition rate as well as the reduction of required manual interventions. To achieve this, are developing powerful visual learning methods at the base level. These methods will not use visual information alone as it is common with traditional computer vision systems. Instead, we are focused on considering also the on text and fusion of different kinds of available information (multimodal information fusion, temporal information fusion, geometry, etc.). We use different kinds of context (temporal and spatial as well as semantic context) to narrow down the search area in the images and improve recognition results. Besides that we are developing methods for learning and incremental updating of the context. At the end of the project we therefore expect significant scientific contributions to the field of incremental learning for object detection, context learning, as well as learning of optimal strategies of interactive incremental learning.

The methodology which we develop is also being applied to the case of maintenance of records of vertical and horizontal traffic signalization. This use case is very suitable for evaluation of developed algorithms, since the problem is very well defined, we have an abundance of multimodal data at our disposal, and at the same time enables efficient learning and use of contextual information. The development of proposed algorithms is also required for significant automation of the records maintenance process. We therefore also expect a significant contribution of our research towards improving efficiency of traffic signalization monitoring that would in the long run significantly reduce the cost of some elements of the traffic infrastructure.


Scientific output of our work within the project is described in these publications: Publications for the VILLarD project