Warning: your browser seems to be quite old, or at least old enough that it does not support the minimal standard capabilities of today's browsers, which are used extensively on this site. Please upgrade to a browser that supports web standards, specifically one that supports minimal CSS1 capabilities, such as Netscape 6 or higher, IE 5 or higher, or Opera 5 or higher. Your browsing experience will be much better! Thanks.

The main objective of this action is to push forward the frontiers of current research on semantic analysis, inference and conceptualisation for high-level annotation and retrieval of digital audiovisual content.

To achieve this goal this action will bring together leading European research teams working on knowledge-assisted semantic analysis, unification, inference and conceptualisation for high-level recognition of digital content. In particular, the proposed integrative research will seek solutions to issues for which current approaches fail giving focused attention to two main aspects: Semantic learning and inference and multimodal analysis.

Semantic Learning and Inference: The goal is to develop automatic and semi-automatic (e.g. using relevance feedback) approaches to detect and recognize semantically meaningful scenes, objects and events present in the content. This process requires the association of low-level and mid-level automatically extracted features with higher-level semantic concepts.

It is envisioned to develop and test algorithms for:

• Fast uncompressed and compressed domain sound, audio and image feature extraction (based on MP3, MP4, MPEG-2/4)
• Integration of all available visual information (colour, texture, shape, motion).
• Efficient means for description of features using hierarchical or multiresolution descriptors.

Multimodal Based Retrieval Mechanisms: The aim is to develop retrieval mechanisms that fully exploit multimodal features and inferred concepts. Given that each concept will rely on many different features and associations, drawn from different input media sources, it is a significant research challenge to investigate how to combine these features and concepts to best answer a user's query.

As in any retrieval task, interface design for browsing, search and retrieval from large repositories of content using low-level features, higher-level semantics and user's relevance feed-back will be also considered.

Main Measurable and Achievable Outcome

The main measurable outcome of the envisaged research will be a modular software system for semantic driven annotation and retrieval. The actual annotation interface and search engine of this COST action should feature two main functionalities:

• Automatic audio-visual content annotation using concepts and levels of abstraction humans are familiar with.
• The ability to build up a knowledge base from past experience through user interaction (relevance feed-back) and adaptive learning.