Lightweight inference for deep learning based image/video analysis
Due to highly parallel computing infrastructures such as GPUs and the availability of large training data sets, deep convolutional neural networks have become a very effective and versatile tool for many image/video analysis problems such as detection or classification.
While it is in many applications feasible to perform training on centralized infrastructure, inference (i.e., applying the trained network) needs to be done on the client. Many of these deep learning approaches use frameworks that are well suited for training and deployment as a hosted service, but are too bulky and computationally heavy for deployment on client devices, in particular embedded and mobile ones. For example, deep neural networks may be used to recognise persons or objects in media content, that should be analysed on a local machine (e.g., for privacy reasons). Applications are documentation of content for media production or for handling user generated content contributions.
The work should address the following aspects around the deployment of deep neural networks (DNN) for image/video analysis tasks:
- Use of a lightweight and self-contained implementation of the inference engine.
- Use of an appropriate representation of the trained network to be interoperable between the frameworks used on the training and inference side.
- Compacting the representation of the network, using e.g. pruning of infinitesimal weights or compression of network structures
Compacting the representation of the network, using e.g. pruning of infinitesimal weights or compression of network structures
Solutions for these issues have been proposed in the scientific literature. The scope of the work is to assess their applicability to specific existing DNN based analysis services and to select, implement and test the most appropriate approach.
- Image analysis, image/video processing
- Basics of machine learning
- Software development: C ++, Python, MATLAB
As soon as possible
This master thesis is scheduled for approx. 6 months.
JOANNEUM RESEARCH offers interested students cooperation with all Austrian but also European or non-European educational institutions and is willing to assist you looking for a supervisor, if necessary.