Robust and real-time capable long term object tracking in video


Reliable tracking of objects like e.g. persons and cars throughout a video for a longer term is a problem which is still not solved today for real-world content. This is due to multiple reasons. Short term effects like occlusion, local changes of the object appearance (e.g. person turning around) or global effects (e.g. illumination variations due to flicker, lighting effects in music videos, background change) can cause the tracker to lose focus on the object and often the tracker is not able to recover again.

The focus of this work is to develop a reliable long term object tracker which combines and extends existing state-of-the art computer vision algorithms for short-term tracking, object detection & classification, image segmentation and (possibly) optical flow. By fusing the complementary information from the different building blocks, the long-term tracker should be much more robust against to fast changes of the object appearance and other issues. The tracker has to be real-time (or near real-time) capable for Full HD video content on a normal PC (with a GPU).

For the short-term tracking component, a state of the art method from a recent paper [1] will be employed. In order to make it real-time capable, the existing Matlab implementation has to be re-implemented in C++, and extensions have to be done in order to allow the efficient tracking of multiple objects simultaneously.

For object detection, classification and semantic segmentation DNN-based real-time capable approaches like YoloV2, SSD or Blitznet will be employed, whereas for optical flow TV-L1 or DIS flow are candidates.

[1] “Learning Background-Aware Correlation Filters for Visual Tracking”, Galoogahi et al, ICCV 2017

Required Skills

  • Good knowledge in image and video processing
  • Basics of machine learning
  • Software development:  good skills in C ++


As soon as possible


This master thesis is scheduled for approx. 6 months.

Academic Supervisor

JOANNEUM RESEARCH offers interested students cooperation with all Austrian but also European or non-European educational institutions and is willing to assist you looking for a supervisor, if necessary