VIDEO FORMATS

(MPEG-4)

A

SEMINAR REPORT

II

TABLE OF CONTENTS

PAGE NO ABSTRACT 1
1 INTRODUCTION 2 1.1 ABOUT MPEG-4 3

2 THE LAYER STRUCTURE FOR MPEG-4 4 TERMIAL

3 OVERALL SYSTEM ARCHITECTURE 6

4 CLIENT –SERVER MODEL 4.1 THE MPEG-4 SERVER 10 4.2 THE MPEG-4 CLIENT 13

5 APPLICATIONS OF MPEG-4 15

6 MPEG-4 ADDRESSES THE NEED FOR 16

7 REQURIMENTS FOR MPEG-4 VIDEO 18 STANDARD

8 CONCLISION 21

BIBLIOGRAPHY 22

APPENDIX – A POWER POINT SLIDES

III

ABSTRACT

The Multimedia Technology Research Center (MTrec) is one of the leading research centers in the world which was engaged in MPEG-4 Research. MPEG-4 is mainly targeted for interactive multimedia applications & became the international standard in 1998 .MPEG-4 makes it possible to construct content such as movie , song , or animations out of multimedia objects. MPEG-4 is the global multimedia standard, delivering professional quality audio and video streams over a wide range of bandwidths, from cell phone to broadband and beyond . MPEG-4 interactive client-server applications are expected to play an important role in online multimedia services.

1
1. Introduction

(The Moving Picture Experts Group (MPEG) is a working group under ISO/IEC in charge of the development of international standards for compression, decompression, processing and coded representation of moving pictures, audio and their combination.

( In August 1993 the MPEG group released the so-called MPEG-1 standard for “Coding of moving pictures and associated audio at up to about 1.5 Mbit/s” . It was mainly targeted for CD-ROM applications . [1]

( In 1990 MPEG started the so-called MPEG-2 standardization phase . The MPEG-2 standard addresses substantially higher quality for audio and video with video bit rates between 2 Mbits/s and 30 Mbits/s, primarily focusing on requirements for digital TV and HDTV applications .

( Anticipating the rapid convergence of telecommunications industries, computer and TV/film industries, the MPEG group officially initiated a new MPEG-4 standardization phase in 1994 - with the mandate to standardize algorithms for audio-visual coding in multimedia applications, allowing for interactivity, high compression and/or universal accessibility and portability of audio and video content .

( Bit rates targeted for the video standard are between 5-64 kbits/s for mobile applications and up to 2 Mbits/s for TV/film applications . 1. About MPEG-4 Most of the multimedia services consist of a single audio or natural 2D video stream . MPEG-4 which is an ISO/IEC standard , provides a broad framework for the joint description , compression ,storage, and transmission of natural and synthetic audio-visual data . It defines improved compression algorithms for audio and video signals, and efficient object – based representation of audio-video scenes. There are 3 main features of MPEG-4 that distinguish it from other technologies: object based nature, interactivity and a high degree of compression. MPEG-4 is different from MPEG-2 in a number of ways: 1. It is not designed to be either just a video or an audio specification. It's an entire multimedia protocol, with standards for how to stream video, how to synchronize multimedia, and how to manage different data types. 2. It doesn't treat these multimedia scenes as a single entity. Instead, it breaks the picture down further. The sequences can be segmented in objects, and the audio/video objects are then sent in independent streams .

2 The Layer Structure For MPEG-4 Terminal In MPEG-4 , audio-video objects are encoded separately into their own Elementary Streams (ES). The Scene Description (SD),also referred to as the Binary Format for Scene (BIFS),defines the spatio-temporal features of these objects in the final scene to be presented to the end user .Object Descriptors(ODs) are used to associate scene description components to the actual elementary streams that contain the corresponding coded media data. ODs carry information on the hierarchical relationships, locations and properties of ESs. The Command Descriptor Framework (CDF) , provides a means to associate commands with media objects in the SD.
The MPEG-4 standard defines a three layer structure for an MPEG-4 terminal : [2] 1. The Compression Layer 2. The Synchronization Layer 3. The Delivery Layer
1. The Compression Layer : The Compression Layer processes individual audio-video media streams and organizes them in Access Units(AU), the smallest elements that can be attributed individual timestamps. The compression layer can be made to react to the characteristics of a particular delivery layer such as the path –MTU or loss characteristics. 2. The Synchronization Layer : The Sync Layer primarily provides the synchronization between streams. Aus are here encapsulated in SL packets. In case that the AU is larger than the SL packet, it will be fragmented across multiple SL packets . The SL produces an SL- packetized stream i.e. sequences of SL packets. The SL-packets headers contain timing , sequencing and other information necessary to provide synchronization at the remote end. The packetized streams are then sent to the Delivery Layer.
3. The Delivery Layer : In the MPEG-4 standard, a delivery framework referred to as the Delivery Multimedia Integration Framework (DMIF) is specified at the interface between the MPEG-4 synchronization layer and the network layer. DMIF provides an abstraction between the core MPEG-4 system components and the retrieval methods .
Two levels of primitives are defined in DMIF. 1.One is for communication, between the application and the delivery layer to handle all the data and control flows.
2. The other one is used to handle all the message flows in the control plane between DMIF peers.

3 Overall System Architecture
The system architecture is shown in figure 1. It consist of 1. An MPEG-4 server ,which stores encoded multimedia objects and produces MPEG-4 content streams. 2. An MPEG-4 client, which serves as the platform for the composition of an MPEG-4 presentation as requested by the end user . 3. An IP network that will transport all the data between the server and the client. The essence of Mpeg-4 lies in its object oriented structure. Each object forms an independent entity that may or may not be linked to other object , spatially and temporally. The SD, ODs, the media objects, and the CDs are transmitted to the client through separate streams. Because of this the end user at the client side get the tremendous flexibility to interact with the multimedia presentation and manipulate the different media objects. End users can change the spatio-temporal relationships among media objects ,turn on or shut down media objects, or even specify different perceptual quality requirements for different media objects dependent upon the associated command descriptors for each object or group of objects. This results in more difficult and complicated session management and control architecture. The design targets a flexible session management scheme with efficient and adaptive encapsulation of data for Q0s provisioning. User interactivity consist of three levels of interactivity that correspond to what type of control is desired:
1. Presentation Level Interactivity : In which a user makes changes to the scene by controlling an individual object or group of objects . It also includes presentation creation .
2. Session Level Interactivity : In which a user controls the playback process of the presentation.
3. Local Level Interactivity: In which a user makes changes that can be taken care of locally , e. g ., changing the position of an object on the screen ,volume control etc.

[pic]
The server maintains a database or a list of available MPEG-4 content and provides WWW access to it. An end user at a remote client side retrieves information regarding the media objects the he/she is interested in, and composes a presentation based upon what is available and desired . The system operation , after the end user has completed the composition of presentation is summarized as follow: 1 The client requests a service of submitting the description of the presentation to the Data Controller (DC) at the server side.
2. The DC on the server side , controls the Encoder/ Producer module to generate the corresponding SD, ODs, CDs and other media streams based upon the presentation description information submitted by the end user at the client side . The DC then triggers the Session Controller (SC) on the server side to initiate a session.
3 The SC on the server side is responsible for session initiation , control and termination . it passes the stream information that is obtained from the DC to the Q0S Controller(QC) that manages in conjunction with the Packer , the creation of the corresponding transport channels with the appropriate Q0S provisions.
4. Messenger Module (MM) on the sever side, which handles the communication of control and signaling data, then signals to the client the initiation of the session and network resource allocation .The encapsulation formats and other information generated by the Packer when processing the “packing” of the SL- packetized streams are also signaled to the client to enable it to unpack the data.

5. The actual stream delivery commences after the client indicates that it is ready to receive and streams flow from the server to the client .After the decoding and composition procedures, the MPEG-4 presentation authored by the end user is rendered on his or her display .

4 Client–Server Model
4.1 The MPEG-4 Server
Upon receiving a new service request from a client , the MPEG-4 starts a thread for the client and setup a session with the client. The server maintains a list of sessions established with clients and a list of associated transport channels and their Q0S characteristics. Fig 2 shows the components of the MPEG-4 Server. The Encoder / Decoder compresses raw video sources in real time or reads out MPEG-4 content stored in MP4 files . The elementary streams produced by the Encoder/Producer are packetized by the SL-Packetizer . The SL -Packetizer adds SL –Packet headers to the AUs in the elementary streams to achieve intra-object stream synchronization . The headers contain the information such as decoding and composition time stamps ,clock references , padding indication , etc . The whole process is scheduled and controlled by the DC . The DC is responsible for several functions : 1. It responds to control messages that it gets from the client side DC . These messages include the description of the presentation composed by the user at the client side and the presentation level control commands issued by the remote client DC resulting from user interactions.
2. It communicates with the SC to initiate a session . It also sends SC the session update information as it receives user interactivity commands and makes the appropriate SD and OD changes.
3. It controls the Encoder/Producer and the SL-Packetizer to generate and packetize the contents as requested by the client .
4. It schedule audio-visual objects under resource constraints . With reference to the System Decoding Model , the AUs must at the client terminal before their decoding time . Efficient scheduling must be applied to meet this timing requirement and also satisfy the delay tolerances and delivery priorities of the different objects. [pic]

The SC is responsible for several functions :
1. When triggered by the DC for session initiation , it will coordinate with the QC to set-up and maintain the numerous transport channels associated with the SL packetized streams.
2. It maintains session state information and updates this whenever it receives changes from the DC resulting from user interactivity.
3. It responds to control messages sent to it by the client side SC. These massages include the VCR type commands that the user can use to control the session .

4.2 The MPEG-4 Client
The architectural design of the MPEG-4 client is based upon the MPEG-4 System Decoder Model (SDM) , which is defined to achieve media synchronization , buffer management , and timing , when reconstructing the compressed media data . Fig 3 illustrates the components of the MPEG-4 client . The SL Manager is responsible for binding the received ESs to decoding buffers. The SL-Depacketizer extracts the ESs received from the Unpacker and passes them to the associated decoding buffers . The corresponding decoders then decode the data in the decoding buffers and produce Composition Units (CUs) , which are then put into composition memories to be processed by the compositor . The User Event Handler module handles the user interactivity . It filters the user interactivity commands and passes the messages along to the DC and the SC for processing . The DC at the client side has the following responsibilities : 1. It controls the decoding and composition process . It collects all the necessary information , e.g. , the size of the decoding buffers which is specified in decoder configuration descriptors and signaled to the client via the OD , the appropriate decoding time and composition time which is indicated in the SL packet header , etc. , for the decoding process .
2. It also maintains the flow of control and data information , controls the creation of buffers and associates them with the corresponding decoders .
3. It relays user presentation level interactivity to the server side DC and processes both session level and local level interactivity to manage the data flows on the client terminal .

[pic]
The SC at the client side communicates with the SC at the server side exchanging session status information and session control data. The User Event Handler will trigger the SC when session level interactivity is detected . The SC then translates the user action into the appropriate session control command
.
5 APPLICATIONS OF MPEG-4
( MPEG-4 makes it possible to construct content such as a movie, song, or animation out of multimedia objects. That's done in Hollywood studios today using specialized equipment at a cost of hundreds of thousands of dollars .
( A final key difference is that MPEG-4 can handle slower data rates. Unlike the older approach, MPEG-4 can handle data rates ranging down to 5 Kbps and up to 4 Mbps. That means that it's possible to create data channels running over standard dial-up Internet connections that carry video and audio.
( The object orientation of MPEG-4 makes it easier to implement things like interactive television .
( Another possible use is in mobile applications, such as cell phones and pagers. Thanks to the ability to gracefully handle low bandwidths, MPEG-4 technology may be especially suited to the coming generation of Web-enabled phones. MPEG-4 needs only 128 Kbps bandwidth, half that demanded by MPEG-1, to provide CD-quality audio .

6 MPEG-4 ADDRESSES THE NEED FOR
( Universal accessibility and robustness in error prone environments ( Multimedia audio-visual data need to be transmitted and accessed in heterogeneous network environments, possibly under severe error conditions (e.g. mobile channels). Although the MPEG-4 standards will be network (physical-layer) independent in nature, the algorithms and tools for coding audio-visual data need to be designed with awareness of network peculiarities. [3]

( High interactive functionality ( Future Multimedia applications will call for extended interactive functionalities to assist the user's needs. In particular the flexible, highly interactive access to and manipulation of audio-visual data will be of prime importance. It is envisioned that - in addition to conventional playback of audio and video sequences - the user need to access "content" of audio-visual data to present and manipulate/store the data in a highly flexible way.

( Coding of natural and synthetic data ( Next generation graphics processors will enable Multimedia terminals to present both pixel based audio and video data together with synthetic audio/speech and video in a highly flexible way. MPEG-4 will assist the efficient and flexible will assist the efficient and flexible coding and representation of both natural (pixel based) as well as synthetic data. meaning a good quality of the reconstructed data, is required. Improved coding efficiency, in particular at very low .

( Compression efficiency ( For the storage and transmission of audio-visual data a high coding efficiency, meaning a good quality of the reconstructed data, is required. Improved coding efficiency, in particular at very low bit rates below 64 kbits/s, continues to be an important functionality to be supported by the MPEG-4 video standard.

7 REQUIREMENTS FOR THE MPEG-4 VIDEO STANDARD

|Functionality |MPEG-4 Video-Requirements |
|Content-Based Interactivity |
|Content-Based Manipulation and Bitstream Editing |Support for content-based manipulation and bitstream editing |
| |without the need for transcoding. |
|Hybrid Natural and Synthetic Data Coding |Support for combining synthetic scenes or objects with natural |
| |scenes or objects. |
| |The ability for compositing synthetic data with ordinary video, |
| |allowing for interactivity. |
|Improved Temporal Random Access |Provisions for efficient methods to randomly access, within a |
| |limited time and with fine resolution, parts, e.g. video frames or |
| |arbitrarily shaped image content from a video sequence. This |
| |includes 'conventional' random access at very low bit rates. |
|Compression |
|Improved Coding Efficiency |MPEG-4 Video shall provide subjectively better visual quality at |
| |comparable bit rates compared to existing or emerging standards. |
|Coding of Multiple Concurrent Data Streams |Provisions to code multiple views of a scene efficiently. For |
| |stereoscopic video applications, MPEG-4 shall allow the ability to |
| |exploit redundancy in multiple viewing points of the same scene, |
| |permitting joint coding solutions that allow compatibility with |
| |normal video as well as the ones without compatibility constraints.|
|Universal Access |
|Robustness in Error-Prone Environments |Provisions for error robustness capabilities to allow access to |
| |applications over a variety of wireless and wired networks and |
| |storage media. Sufficient error robustness shall be provided for |
| |low bit rate applications under severe error conditions (e.g. long |
| |error bursts). |
|Content-Based Scalability |MPEG-4 shall provide the ability to achieve scalability with fine |
| |granularity in content, quality (e.g. spatial and temporal |
| |resolution), and complexity. In MPEG-4, these scalabilities are |
| |especially intended to result in content-based scaling of visual |
| |information. |

8 CONCLUSION

For a transport infrastructure to support interactive multimedia presentations , which enable end users to choose available MPEG-4 media content to compose their own presentations , control the delivery of such media data and interact with the server to modify the presentation in real-time . The initial design and implementations of a transport infrastructure for an IP based network will support a client-server system which enables end user to: 1. Author their own MPEG-4 presentations 2. Control the delivery of the presentations and, 3. Interact with the systems to make changes to the presentations in real time. It is foreseen that MPEG-4 will be an important component of multimedia applications on IP-based networks in the future.

BIBLIOGRAPHY

1. Thomas Sikora ,”The MPEG-4 Video Standard Verification Model” , Affiliation Of Author , Heinrich-Hertz-Institute (HHI) for Communication Technology, Berlin, FRG. http://wwwam.hhi.de/mpeg-video/papers/sikora/final.htm .
2. Haining Liu, Xiaoping Wei and Magda El Zarki “ A Transport Infrastructure Supporting Real Time Interactive MPEG-4 Client-Server Applications over IP Networks”, Department of Information and Computer Science , University of California, IRvinc .
3. T. Sikora and L. Chiariglione “ MPEG-4 Video and its Potential for Future Multimedia Services” , Heinrich-Hertz-Institute (HHI), Einsteinufer 37, D-10587 Berlin, Germany. http:// wwwam.hhi.de/mpeg-video/papers/sikora/iscas.htm .
4. Lights, Camera ..… The Latest in Multimedia Technology By Hank Hogan
5. MPEG-4 : A Multimedia Standard for the Third Millenium , Part2 Stefano Battista bsoft Franco Casalino Ernst and Young Consultants , Claudio Lande CSELT.
6. Thomas Sikora ,”MPEG Video Webpage” , Affiliation Of Author , Heinrich-Hertz-Institute (HHI) for Communication Technology, Berlin, FRG.

22

-----------------------

To
MESSENGER

CONTROL FLOW

DATA FLOW

Structure of the MPEG-4 Client

FROM
UNPACKER

COM-POSI-TOR

USER
EVENT
HANDLER

TO /
FROM
MESSE-NGER

DATA
CONTROLLER

COMPOSITOR
BUFFER

COMPOSITOR
BUFFER

OD

SD GRAPH

OD DECODER

BIFS DECODER

MEDIA OBJECT DECODER

MEDIA OBJECT DECODER

MEDIA OBJECT
DECODING BUFFER

MEDIA OBJECT DECOING BUFFER

OD DECODING
BUFFER

BIFS DECODING
BUFFER

SL
DE-
PACKET-IZER

SL
MANAGER

SESSION
CONTROLLER

Fig 3

Fig 2

TO/ FROM Q0S CONYROLLER

TO / FROM MESSENGER

FROM MESSENGER

STRUCTURE OF THE MPEG-4 SERVER

TO PACKER

DATA FLOW

CONTROL FLOW

LOCAL MP4
FILES

RAW VIDEO
RESOURCES

SL-
PACKETIZER

ENCODING/
PRODUCER

SESSION
CONTROLLER

DATA
CONTROLLER

Client

Server

MPEG-4 APPLICMATION

DECODER/
ENCODER

CONTROL

DATA

CON- -TROL

DATA

MPEG-4 APPLICATION

DELIVERY

SYSTEM ARCHITECTURE

DELIVERY

IP
NETWORK

USER
EVENT
HANDLER

SL
MESSEN-
GER

DATA CONTRO-
LLER

SESSION
CONTRO-
LLER

SL-
DEPACKET-
IZER

UN-
PACKER

MESSENGER

SESSION
CONTRO
LLER

PACKET

Q0S
CONTRO-
LLER

MESSENGER

SL-
PACKET
IZER

ENCODER /
DECODER

DATA CON-
TROLLER

Fig 1