One of the problems when using professional media archives in the domains of TV or movie production or offered on the internet is to find the interesting media segments among the vast amount of un-annotated content. SEMEDIA will develop methods and tools for annotating, tagging and searching of content in large, heterogeneous media collections. This will be based on state-of-the-art research in the areas semantic web, AI, content-based information retrieval and human computer interaction. The project’s results will support fast, semi-automatic annotation of large amounts of data at greatly reduced cost. Professional end user applications will provide efficient searching in large, distributed databases containing un-indexed media data. The project will develop methods and applications for:

• Navigating and searching efficiently in large media archives 
• Clustering of visually similar content, supporting queries like „find scenes similar to…“ 
• Finding scenes with a certain actor and all related date (3D geometry, lighting, camera & object motion) 
• Data structures and services supporting distributed access by multiple users yet preserving data security. 
• User interfaces providing a condensed overview on media content and feedback methods and using context for annotation support

The main focus of work of the Institute for Information Systems & Information Management will on clustering and visualizing large collections of sparsely annotated media.

See also: SEMEDIA Homepage

Automatic Summarisation of Rushes
In film and video production usually large amounts of raw material (“rushes”) are shot and only a small fraction of this material is used in the final edited content. The reason for shooting that amount of material is that the same scene is often shot from different camera positions and several alternative takes for each of them are recorded.

The result of this practice in production is that users dealing with rushes have to handle large amounts of audiovisual material which makes viewing and navigation difficult. Our work is motivated by two application areas where this problem exists. One is post-production of audiovisual content, where editors need to view and organize the material in order to select the best takes to be used (the ratio between the playtime of the rushes and that of the edited content is often 30:1). The other application area is documentation of audiovisual archives.

The dataset used is that of the TRECVID  video retrieval BBC rushes summarisation task 2007. The aim of the task is to create a short summary of a video (a so called video skim) that has only 4% of the length of the original, but contains all relevant content segments. Our approach mainly reduces redundancy by identifying repeated takes of the same scene and the selects relevant clips of the remaining segments. Below you find three examples of the skims produced by our system (click on the frames):

This work has been partly supported by the European Commission in the projects IP-RACINE, SEMEDIA and K-Space.