Inhalt des Dokuments
Prof. Dr.-Ing. Thomas Wiegand
Leiter des Fachgebiets Bildkommunikation
Institutsleiter des Fraunhofer Heinrich-Hertz-Instituts
TUB: +49 30 314 28777
HHI: +49 30 31002 617
Mobil: +49 172 381 3814
Technical University Berlin
Sekr. EN 16
E-Mail: (beste Art, mich zu erreichen)
Gabriela Thiele: +49 30 31002 211
3D High-Efficiency Video Coding
We are developing 3D video codecs for 3D television, 3D cinema and 3D mobile applications, including novel coding tools for multi-view video and depth (MVD) compression, extending the high-efficiency video coding (HEVC) method.
Generic Formats for 3D Video
We are investigating generic formats for 3D television and 3D video applications to support a wide range of stereoscopic and autostereoscopic multi-view displays with a single format.
Advanced View Synthesis
We have developed view synthesis methods with robust rendering artifact reduction for multi-view display support from depth-enhanced multi-view video data, such as MVD.
Mobile Applications for 3D Video
We have developed coding and rendering solutions specifically for mobile 3D services, e.g. stereo video coding and monoscopic video plus depth coding with simplified view rendering to support the hardware limitations of mobile devices.
3D Computer Graphics Models and Free Viewpoint Video
We develop representations and coding methods for free-viewpoint video and 3D natural and synthetic scenes based on dynamic 3D geometry formats, such as animated meshes and multi-texture and -video representations in VRML or X3D.
Multi-View Video Coding
We have developed a coding method for multi-view video data based on MPEG4-AVC/H.264 by introducing inter-view prediction for neighboring camera views.
The combination of Dynamic Adaptive Streaming over HTTP (DASH) with layered codecs, such as SVC and MVC opens up a wide range of possibilities for a more improved usage of network resources.
Wavefronts for HEVC Parallelism
For improving processing performance of encoders and decoders, Wavefronts have been added as tool to the HEVC standard for allowing high-level parallelism suitable for multi and many core hardware architectures.
Improved DASH over LTE
A novel technique, which combines LTE features with knowledge about HTTP Streaming sessions for optimization of the wireless resources. The combined optimization yields an improved transmission of videos over cellular wireless systems which are based on LTE and LTE-Advanced.
Layer-Aware FEC (LA-FEC) constitutes a FEC scheme designed for transmission of layered media. LA-FEC generates FEC data across existing dependencies in the media stream and thereby improves the robustness without increasing the required bitrate.
P2P Group Communication
Application Layer Overlays, based on Peer-to-Peer techniques, allow for flexible and cost effective solutions for Group Communication, with a large number of participants that are consuming lots of network resources.
Cloud Computing for Real-time Communication
The Cloud allows for flexible and improved solutions for Real-time Multimedia communication, by enabling new delivery architectures. Furthermore, multiple network paths can be utilized to improve the quality of the service.
Multimedia Transmission Using SVC
Multimedia transmission is the key component for delivering media to mobile and fixed clients. Scalable Video Coding is a technique which brings improvements in different scenarios form conversational Internet services over, p2p streaming in ad-hoc networks up to multi-screen broadcast.
The combination of SVC, LA-FEC and Unequal Time Interleaving enables a new transmission scheme that allows fast zapping while providing robustness against long burst errors.
A collection of initial works on multimedia communication and transmission for 3G and DVB-T/DVB-H networks as well as joint source/channel coding.
Image & Video Coding
High Efficiency Video Coding
We have contributed significantly to the development of the basic architecture of the video coding layer of the upcoming High Efficiency Video Coding (HEVC) standard. Several new coding tools have been developed and successfully contributed as part of the novel specification.
HEVC Extension for 3D Video Coding
We are developing an extension of HEVC for 3D video coding that supports the coding of multiple views and associated depth data. It adds new coding tools to the HEVC design, which improve the compression capabilities for dependent video views and depth data.
We have contributed significant parts of the basic video coding layer of the first, basic version of H.264/MPEG-4 AVC. These parts include motion-compensated prediction, tree-structured picture partitioning, entropy coding and rate-distortion optimized encoder control.
Scalable Video Coding (SVC) Extension of H.264/MPEG-4 AVC
We have developed an extension of H.264/MPEG-4 AVC that provides the support of different types of scalability at a bitstream level with only a moderate increase in decoder complexity. This Scalable Video Coding (SVC) extension allows on-the-fly adaptation of bit rate, format, and power consumption as well as graceful degradation in lossy transmission environments.
Multiview Video Coding (MVC) Extension of H.264/MPEG-4 AVC
We have developed a simple, but very efficient approach for multiview video coding as an extension of H.264/MPEG-4 AVC. This extension does not include any changes of the basic decoding process and requires only a modification of the high-level syntax of H.264/MPEG-4 AVC.
Fidelity Range Extension (FRExt) of H.264/MPEG-4 AVC
We have developed and contributed a design for the support of 8x8 transforms and 8×8 intra prediction modes in H.264/MPEG-4 AVC. This basic architecture was adopted as the basis for the development of FRExt, in particular the High Profile of H.264/MPEG-4 AVC.
Rate-Distortion Optimized Encoding
We are very active in optimizing the encoder control for different video coding standards. The investigated encoder control concepts have become an integral part of the reference model and reference software for the video coding standards H.264/MPEG-4 AVC and HEVC.
Statistical Coding & Modeling
We are developing efficient methods for statistical modeling and coding with a particular focus on application in video coding. Our research includes the topics of fast binary arithmetic coding, context modeling, probability estimation, and binarization. In addition, we are investigating the concept of probability interval partitioning entropy (PIPE) codes as a low complex, but yet highly efficient variant of binary arithmetic coding.
Image & Video Understanding
Depth Image-Based Rendering with Advanced Texture Synthesis
Depth image-based rendering (DIBR) techniques are advanced tools in 3-D video (3DV) applications that are used to synthesize a number of additional views in a multiview-video-plus-depth (MVD) representation. One of the main obstacles in the DIBR technique lies in the disocclusion problem which results from the fact that a scene can only be observed from a set of original views. This can lead to missing information in the generated virtual views, especially in extrapolation scenarios. Our work describes novel algorithms that synthesize such uncovered textures.
Automatic Soccer Video Analysis
Fraunhofer HHI develops methods to automatically analyze soccer videos. These technologies enable faster and more precise retrieval of soccer content as well as more interactive experience when watching soccer matches.
2D and 2D+t Image Completion
Inpainting, Texture Synthesis, Image Reconstruction, Image Stuffing
We have developed efficient image and video completion methods, including novel 2D and 2D+t tools, e.g. fully automatic image inpainting with fast structure interpolation. Enhanced post-processing modules have been also developed to increase the quality of the final image or video quality.
Perception-oriented Video Coding
We have designed and developed new and efficient perception-oriented video coding frameworks. Our research activities in this area have shown that bit rate reductions of up to 41% can be reached, when compared at the same visual quality to a reference H.264/AVC codec. Some research is still required to achieve a robust performance for generic contents.
With more and more archives digitalizing image and video content in order to preserve cultural heritage, the amount of resources and time wasted due to diverse distortion sources during the process is ever increasing. Automated quality assessments systems within the digitalization process can quickly detect quality issues at both global and local level. The same technologies enable archives to better tag existing content whether it is suitable for repurposing and rank multiple copies of the same content based on quality-criteria.
Semantic Metadata Generation
We have developed technologies to automatically analyze visual content in images and videos, and assign to them, keywords which are representative of semantic concepts such as „beach“, “flower” or “landscape”, for efficient archiving and search in large multimedia collections.