Contents lists available at ScienceDirect

Signal Processing: Image Communication

Signal Processing: Image Communication 28 (2013) 1255–1277

0923-59http://d

n CorrE-m

journal homepage: www.elsevier.com/locate/image

Parser description-based bitstream parser generationfor MPEG RMC framework

Hyungyu Kim a, Sowon Kim a, Seungwook Lee b, Euee S. Jang a,n

a Digital Media Laboratory, Department of Electronics and Computer Engineering, Hanyang University, 222 Wangsimni-ro,Seongdong-gu, Seoul, Republic of Koreab Electronics and Telecommunications Research Institute, 218 Gajeong-ro, Yuseong-gu, Daejeon, Republic of Korea

a r t i c l e i n f o

Available online 3 September 2013

Keywords:Reconfigurable media codingBitstream syntax descriptionBitstream parserScript languageRun-timeGeneric parser FU

65/$ - see front matter & 2013 Elsevier B.V.x.doi.org/10.1016/j.image.2013.08.011

esponding author. Tel.: þ82 2 2220 1086.ail address: esjang@hanyang.ac.kr (E.S. Jang)

a b s t r a c t

In this paper, we present a bitstream syntax description scheme that enables theautomatic generation of a bitstream parser in the reconfigurable media coding (RMC)framework. In the RMC framework, a standard video decoder can be defined with a set offunctional units (FUs) pre-defined in a tool library and a decoder description containingthe interconnections between the FUs and the bitstream parser description. The FUs in thetool library are codec-independent in that any FU can be reused. In contrast, the decoderdescription is codec-specific. In particular, the bitstream parser description is required forthe generation of the bitstream parser FU in the RMC framework. There have been severalworks on bitstream parser description and parser FU generation; however, conventionalmethods have mainly focused on built-in parser generation during the design time, ratherthan on the automatic generation of the parser FU in the run-time from the description.As a response to this problem, we propose a bitstream parser description format and arun-time parser generation mechanism based on a generic parser FU. The proposedlanguage aims to support run-time parser FU generation by describing three types ofinformation: control flow, external functions, and parser interfaces. The control flowdescription is described based on the finite state machine concept, making the translationof the BSD in run-time easy. Throughout the paper, we show how the proposed scheme isbetter suited for the MPEG RMC framework than the existing description formats. We alsodiscuss the implementation of the generic parser FU (GPFU) to show that the run-timeparser generation can be done efficiently with the proposed description format. Asimulation result using the generic parser FU based on the existing MPEG standards(MPEG-4 SP and SC3DMC) is provided as an existential proof.

& 2013 Elsevier B.V. All rights reserved.

1. Introduction

There are currently multiple video coding standards andvarious multimedia platforms. Not only are the number ofvideo coding standards ever increasing, but the number ofmultimedia platforms is also increasing. The implementationof standard video codecs becomes more costly because a

All rights reserved.

.

decoder design for a specific multimedia platform is seldomreusable for another platform without a major modification.

As an answer to such a problem, MPEG recently standar-dized the reconfigurable media coding (RMC) framework[1,2], which was formerly known as reconfigurable videocoding (RVC). The MPEG RMC framework permits modulardesign of tools used in standard video codecs (e.g., DCT,inverse quantization, motion compensation, etc.). A modulartool in the RMC framework is called a functional unit (FU).Each FU is carefully designed to have appropriate granularityand reusability with a clear description of the input and

H. Kim et al. / Signal Processing: Image Communication 28 (2013) 1255–12771256

output behavior. A collection of the FUs in the RMC frame-work is referred to as tool library. By selecting FUs from thetool library and configuring the modular FU network byinterconnecting them, a multimedia codec based on amodular design can be composed in the RMC framework.The configuration of a decoder is described in a documentcalled a decoder description.

All the FUs used to construct an FU network in the RMCframework are pre-defined in a tool library except for thebitstream parser FU. This exception is due to the fact thatthe parser FU is codec-dependent whereas the other FUsare generally codec-independent for reusability. Therefore,the parser FU is supposed to be generated from anexternally provided bitstream syntax description (BSD).BSD is part of the decoder description that containsdescriptions of the rules to parse a specific format ofbitstream (e.g., order of syntax elements, control flow, bitlength of each syntax element, VLD tables).

The information contained in the BSD is used by the parsergeneration mechanism to generate a bitstream parser FU.Although the parser FU should be generated per BSD in theRMC framework, the MPEG RMC framework currently appliesa built-in parser implementation approach. In the built-inapproach, the bitstream parser FU is generated in design-timeand then pre-defined in the tool library. Consequently, theconfigurability of the parser FUs in run-time is limited.Moreover, the generation mechanism of the built-in parserFU is not normatively defined in the MPEG RMC standard; inother words, there is no standard and interoperable way toprovide built-in parser FUs over different implementations ofthe RMC framework. From these observations, the currentdesign of the RMC framework with the built-in approachcannot fully support the MPEG RMC requirements [3] includ-ing the easy instantiation of the decoder—including the parserFU—from a decoder description.

The run-time configuration of the bitstream parser FUcan overcome the drawbacks of the built-in parserapproach. Allowing automatic generation of a bitstreamparser FU in run-time from a BSD has several benefits; forexample, the reusability of the bitstream parser design canbe enhanced, and the implementation burden to build thebitstream parser FU for a number of different implementa-tions of the RMC framework can be reduced. However, wefound that the existing methods, especially the BSD formatsin MPEG standards such as Flavor [4–6] or RVC-BSDL [7],are not sufficient to support the run-time parser generation.Some RMC framework-aware technical extensions arenecessary for Flavor to enable run-time parser generation,and the run-time parser generation mechanism based onRVC-BSDL is neither specified nor demonstrated in theMPEG RMC standard.

In this paper, we propose a new BSD format and a parsergeneration approach that are suited for run-time parsergeneration. The proposed BSD format, XML script-basedparser description language (XS-PDL), describes the bitstreamsyntax parsing process and the parser configuration con-structs in an XML-based format. The key feature of XS-PDLis its directly executable bitstream parsing control flowdescription based on the finite state machine (FSM) concept.We also propose a generic parser FU (GPFU) approach tocomplete the run-time parser generation mechanism together

with XS-PDL. The GPFU is a parser FU pre-defined in the toollibrary that receives and executes a given BSD in run-time,consequently allowing an intrinsic implementation of the run-time configurable bitstream parser while maintaining thevirtues of the built-in parser implementation approach.Through this paper, we discuss the design philosophy of theXS-PDL and the GPFU and the benefits compared to existingmethods. We also demonstrated the feasibility of the pro-posed methods using example XS-PDL BSDs and a GPFUimplementation.

This paper is organized as follows: in Section 2, wereview the existing BSD formats that can be found in MPEGstandards and the parser generation methods based onthem. We define four key functionalities of the BSD formatto support run-time parser generation and compare theexisting formats and methods according to them. Theproposed BSD format, XS-PDL, is described in Section 3,and example BSDs written in XS-PDL are briefly introducedin the following section. In Section 5, we describe theconcept of the GPFU as a parser generation mechanismfor XS-PDL BSD. The experimental results based on the realworld multimedia coding standards are shown in Section 6as an existential proof of the proposed methods. Finally, weconclude this paper in Section 7.

2. Overview of BSD formats and parser generation

2.1. Textual bitstream syntax specification in standards

In this section, we provide an overview of the existingbitstream syntax description formats officially adopted inMPEG standards. The description of bitstream syntax andsemantics is an indispensable part of video coding stan-dards for clear definition of the syntax elements and thedecoding process: since MPEG-1 [8], the explicit textualspecification of bitstream syntax (in a tabular form) andbitstream semantics has been introduced.

For explicit textual specification of the bitstream syntax,three key elements are described in tabular form: the nameof syntax elements (e.g., width/height of video frame,quantization parameters, DCT coefficients, motion vectors),the data type in the mnemonic (e.g., bit string, unsignedinteger, variable length code), and the bit length of syntaxelements (e.g., 13 bits for video frame width, 3–9 bits forquantization parameters). Because the bitstream syntaxparsing is an algorithmic process, the basic control state-ments (e.g., if, for, and while) are included in the table.

The explicit textual specification of the bitstream syn-tax alone is not complete in describing the bitstreamsyntax parsing process. In the bitstream semantics section,supplemental information in addition to the bitstreamsyntax specification is necessary including variable lengthcoding (VLC) tables, context tables, and additional expla-nations on syntax parsing.

An example of the textual bitstream specification inMPEG-4 Visual [9] is shown in Figs. 1 and 2. A simplifiedsnapshot of the macroblock bitstream syntax description ina tabular form is shown in Fig. 1. Each row in the table inFig. 1 is either a control (or a branch) statement or thedefinition of a syntax element. From this example, it can beshown that the tabular description is not enough to

Fig. 1. An example of the explicit textual bitstream specification in the MPEG-4 Visual standard.

H. Kim et al. / Signal Processing: Image Communication 28 (2013) 1255–1277 1257

describe the entire bitstream syntax parsing process. Thesemantic conversion process from an encoded syntax ele-ment to a meaningful semantic value should be combined tocomplete the bitstream syntax parsing process.

In Fig. 1, there is a syntax element called dquant, which isencoded with two bits. The two-bit representation of dquantneeds to be converted to a meaningful value according to thesemantic table shown in Fig. 2. The semantic conversionprocess of most of the syntax elements is relatively simplelike the dquant case; however, some syntax elements needto be converted into many semantic values as in the case ofmcbpc. From Fig. 1, mcbpc is a VLC codeword that isconverted to two semantic values (i.e., derived_mb_typeand cbpc) as shown in Fig. 2.

As illustrated in the aforementioned examples, this dualstructure of bitstream syntax and semantics description hasan advantage for human readability. However, it is difficult toautomatically translate the description into an executablebitstream syntax parsing process without human assistance.If there is any inconsistency or ambiguity (which may havebeen caused by a human error) in the textual specification, it

will be very difficult for the developers to detect and debugsuch problems. This is why it is recommended to refer to thereference video decoder to confirm the accuracy of thebitstream syntax and semantics description.

2.2. Flavor (MPEG SDL)

The formal language for audio–visual object representation(Flavor) [4,5], also known as the MPEG syntactic descriptionlanguage (SDL) [6], was one of the earliest approaches toprovide a formal description of the bitstream syntax in aprogram-like grammar. The language is designed to besuitable for automatic generation of the parser from a givenbitstream syntax description since the origin of the language,mkvlc, was a language scheme for efficient automatic initi-alization of the variable length coding table [10].

Using a syntax like Cþþ/Java, Flavor provides a formalspecification of the bitstream with several class declarations.In Flavor, each bitstream syntax element is represented bya parsable variable within a specific class. The declarationscheme of the parsable variable is an extension from the

Fig. 2. Textual bitstream semantics description in the MPEG-4 Visual standard. (a) Bitstream semantics and (b) VLD table referred by bitstream semantics.

H. Kim et al. / Signal Processing: Image Communication 28 (2013) 1255–12771258

standard variable declaration convention in Cþþ/Java. Addi-tionally, Flavor has two unique data structure declarationschemes, map and bac, for syntax elements using variablelength coding or binary arithmetic coding, respectively. Morecomplex algorithms can be integrated within the Flavorsource code with a verbatim code feature; this feature allowssome part of the Flavor description to be described usingCþþ or Java code.

The example in Figs. 3 and 4 shows how a bitstreamspecification including the semantic conversion processcan be described in Flavor. Fig. 3 shows a macroblocksyntax description of MPEG-4 Visual and can be comparedwith Figs. 1 and 2. In the class declaration in Fig. 3, theextraction process of the coded block pattern (cbp) frommcbpc and cbpy is described using Flavor code, while thesame process is literally described in Fig. 2. Fig. 4 showsother examples of the semantic description tools in Flavor.The constant variables can be used to enhance the humanreadability of the description, and the map declarationssubstitute the tables in Fig. 2 in machine-readable format.

Efforts to generate a bitstream parser from Flavor havebeen reported by several researchers. A Flavor translator (a.k.a. “Flavorc”) [11] is able to translate a BSD written in Flavorto a number of formats including source codes (e.g., Cþþor Java) and XML schema (e.g., XFlavor schema [12] orMPEG BSDL schema [13]). A key benefit of Flavor is the codegeneration capability from a BSD, which is a significantimprovement over the conventional textual specification.

2.3. RVC-BSDL

The bitstream syntax description language for reconfi-gurable video coding (RVC-BSDL) is developed to describethe bitstream syntax and semantics for MPEG RMC [1].RVC-BSDL is an XML-based language, which is an extendedversion of the MPEG bitstream syntax description lan-guage (BSDL) [14]. BSDL was standardized as a part of theMPEG-21 digital item adaptation (DIA) framework [15].RVC-BSDL is designed by extending BSDL since BSDL was

Fig. 3. An example of the Flavor description for MPEG-4 Visual.

H. Kim et al. / Signal Processing: Image Communication 28 (2013) 1255–1277 1259

the latest MPEG standard to describe the bitstream syntaxand semantics.

RVC-BSDL inherits many characteristics from BSDL suchas XML-schema-based description and parsing-specificconstructs. However, RVC-BSDL does not provide back-ward compatibility with BSDL because some BSDL con-structs are not supported in RVC-BSDL. This is to simplifythe description and to facilitate the development of bit-stream parsing applications using RVC-BSDL [7]. Newconstructs (e.g., data types and attributes) are introducedin RVC-BSDL to facilitate the interface between the parserand other FUs.

The hierarchy of the bitstream syntax structure in RVC-BSDL is described in an XML-schema-based representation.The native XML schema constructs such as xsd:sequence,xsd:choice, xsd:complexType, xsd:group can be usedto define the syntax structure. Control flow statements arepartially available: loop control and conditional statementscan be inserted using BSDL constructs (e.g., bs2:nOccurs,bs2:if, bs2:ifNext). Two RVC-specific constructs (i.e.,rvc:ext and rvc:port) are newly introduced in RVC-BSDL to specify the interfaces of the parser FU to otherdecoding FUs. The complex parsing processes that are difficult

to describe using RVC-BSDL can be externally processed byother FUs and integrated into the parsing process using theaforementioned elements.

Figs. 5 and 6 show examples of the bitstream syntaxdescription in RVC-BSDL, which describes the MPEG-4 Visualmacroblock hierarchy. The structure of the XML schema inFig. 5 follows the tabular standard specification in Fig. 1. FromFig. 5, it can be observed that the RVC-BSDL descriptionstructure still conforms to the original W3C recommendationfor XML schema. One key feature of RVC-BSDL is the use ofrvc:port attributes to describe the output ports from theparser FU. This attribute can be used to specify the VLDprocess when combined with the rvc:ext data type, asshown in Fig. 6. The VLD process is currently described as anexternal FU outside of RVC-BSDL; thus far, RVC-BSDL is onlycomplete for the bitstream syntax and parsing descriptionwhen external functions are defined.

The bitstream parser generation from a BSD written inRVC-BSDL is not clearly defined in the standard. Currently,the bitstream parser FUs in the MPEG RMC reference soft-ware (i.e., RMC Simulation Model (RMC)) are not derived fromBSD; instead, the parser FUs are manually implemented. Thetranslation process from RVC-BSDL to RVC-CAL [16] source

Fig. 4. Bitstream semantic description in Flavor.

H. Kim et al. / Signal Processing: Image Communication 28 (2013) 1255–12771260

code using the XML stylesheet transformation (XSLT) [17] hasbeen proposed [18,19]. However, no explicit parser generationmechanism is in use within the MPEG RMC framework.

2.4. Run-time parser generation from BSD

In the RMC framework, a key functionality of BSD is thegeneration of the bitstream syntax parser FU. The MPEGRMC framework currently adopts the built-in FU approach,which is illustrated in Fig. 7. In this approach, the bitstreamparser FUs are built during the design time based on pre-defined BSDs to make them available in the tool library.During the run-time, the built-in parser FUs are used in theMPEG RMC framework like other FUs to compose anexecutable decoder. This approach is feasible only whenthe syntax parsing process is already known during thedesign time. In the built-in approach, BSD is not alwaysnecessary because the bitstream syntax and parsing rulesare often well specified in the standard documents.

However, a serious drawback of the built-in approach isthat the automatic generation of a bitstream parser FU from anew BSD given in run-time is not possible. In the case of anew codec where only the bitstream syntax is changed whileall the other functional units are reused, a bitstream coded bythe codec cannot be decoded with a RMC decoder equippedonly with built-in parsers. Moreover, the reusability of thepre-defined parser FUs is limited because no modification orextension is possible. Differently from other FUs, the parser FUis codec-specific; in other words, it should be generated percodec. Even when the difference between the two codecs is

very marginal, the parser FU should be generated separatelyin the design time.

For example, the MPEG-4 advanced simple profile (ASP) isan extension from the MPEG-4 simple profile (SP). Thedifference between the two profiles is very marginal, asshown in Table 1 [20,21]. Table 1 shows that five new toolsare added in ASP, while all the other tools are reused from SP.From the syntax description viewpoint, the increase by thefive new tools is about 30% (i.e., 460 lines in SP to 602 lines inASP). In other words, an MPEG-4 ASP parser FU can begenerated during the run-time by extending an MPEG-4 SPparser FU, as shown in Fig. 8. However, such an extension isnot possible in the built-in approach because there is no run-time extensibility. Therefore, an MPEG-4 ASP decoder can beconfigured only when a new bitstream parser is available. Thiscan happen when a video coding standard is used with manyprofiles and levels, which is the usual case sinceMPEG-2. Suchlimited reusability and extensibility of the bitstream parser FUbased on the built-in approach will limit the efficient use ofthe RMC framework.

Another problem of the built-in approach is that it limitsthe interoperability of the parser FU design between thedifferent implementations of the RMC framework. Becauseimplementation of the RMC framework can take place invarious computing environments, how to perform the bit-stream parsing behavior based on the same BSD in each RMCframework implementation may be completely different. Insuch a case, it is very inefficient to prepare built-in parser FUs,which are generated in design-time, for every single type ofRMC framework implementation. A combination of the twomethods may solve this problem: the run-time generation ofthe parser FU and a platform-independent BSD format. Therun-time generation can decrease the implementation burdenby allowing the parser generation to take place only whenthere is actual need. The platform-independent BSD formatwill reduce the run-time resource consumption that may becaused by the run-time parser generation process.

In conclusion, to achieve the ultimate vision of the RMCframework, it is necessary to have a method to generatethe bitstream parser FU in run-time and a BSD format tosupport the run-time parser generation efficiently. Wefound that the existing BSD formats reviewed in thepreceding sections are not adequate to support the run-time parser generation because the following four keyfeatures are not sufficiently provided:

(1)

Control flow description: The bitstream syntax andparsing rules in BSD describes the process to decodea bitstream into semantically meaningful elements.Control flow description can be considered as anexecutable specification of the bitstream syntax andparsing rules, which can be interpreted or executeddirectly from BSD. It is beneficial to have control flowrepresented in a machine-readable format to supportthe run-time generation of parser FU. The control flowdescription should also be abstracted and concen-trated on describing the minimal and necessary func-tionalities of bitstream parsing.

(2)

Definition of external functions: Not all functionalities canbe described as a control flow in BSD because somecomplex parsing algorithms such as entropy decoding can

Fig. 6. MCBPC complexType declaration example in RVC-BSDL.

Fig. 5. An example of RVC-BSDL description for MPEG-4 Visual.

H. Kim et al. / Signal Processing: Image Communication 28 (2013) 1255–1277 1261

be better represented as predefined external functions.There are two main reasons to predefine such algorithms:first, such algorithms are very difficult to describe in asimple control flow description; second, such parsingrules are often shared over different codec standards,and therefore can be reused in different bitstream speci-fications. Within the external functions, the reusable partand the codec-dependent part coexist. For example, in thecase of variable length decoding (VLD), the table lookupalgorithm is reusable for a number of state-of-the-art

video coding standards, while the VLD tables used for theVLD operation are codec-dependent. For this reason, theexternal functions should be able to be parameterizedwith the codec-dependent parameters delivered withina BSD.

(3)

Definition of the parser interface: One unique featureof the parser FU in the RMC framework is to read aninput bitstream from an input port and to producesemantically decoded outputs to the correspondingoutput ports. This output data are used as data tokens

Fig. 7. Built-in approach for bitstream parser FUs.

Table 1Comparison between MPEG-4 SP and ASP functionalities.

Functionality SP ASP

I-VOP ✓ ✓

P-VOP ✓ ✓

AC/DC prediction ✓ ✓

4-MV mode ✓ ✓

Unrestricted MV ✓ ✓

GOB resynchronization ✓ ✓

Data partitioning ✓ ✓

Reversible VLC ✓ ✓

Short header ✓ ✓

B-VOP ✓

Method 1/Method 2 quantization ✓

Interlace ✓

Global motion compensation ✓

Quarter-pel motion compensation ✓

H. Kim et al. / Signal Processing: Image Communication 28 (2013) 1255–12771262

in an FU network in RMC. Therefore, the accuratedefinition of input and output ports is very critical toguarantee the functionality of the run-time-generatedparser FU in a given FU network. Accordingly, a BSDshould support a comprehensive list of the input andoutput ports from/to the parser FU.

(4)

Run-time compilation capability: It would be desirableto have a run-time compilation capability to ensurethe run-time bitstream parser generation is similar toconventional script languages such as Python, PHP, orECMAScript.

We summarize the comparison result of the existing BSDformats in MPEG standards with regard to the four keyfunctionalities in Table 2. As listed in the table, the existingBSD formats fell short in support run-time parser generation.In the case of the textual specification, even though thespecification provides a comprehensive explanation of thedecoding process, the specification cannot be used directly togenerate a parser FU in run-time due to the lack of machinereadability. On the other hand, Flavor can provide a machine-

readable BSD format based on Cþþ/Java, which allows forfeasible source code generation. However, some technicalextensions are still necessary to make Flavor feasible for therun-time parser generation in the RMC framework. RVC-BSDLprovides advanced features such as external functions andparser interfaces for the RMC framework; however, the run-time parser generation based on RVC-BSDL is still notsupported in the MPEG RMC.

3. XML script-based parser description language (XS-PDL)

3.1. Design philosophy

We propose a new language called XML script-basedparser description language (XS-PDL) as an alternative BSDformat to support run-time bitstream parser generation.XS-PDL is designed to fulfill the four key elements for therun-time bitstream parser generation explained in Section2.4. Two unique features of XS-PDL are (1) language todescribe a finite-state machine (FSM) model and (2) aXML-based description format consistent with the MPEGRMC decoder description.

(1)

Language to describe an FSM model: FSM is a popularmodeling methodology in parsing, translation, andcompilation that can support a minimized representa-tion of a given grammar as input data. The virtues ofFSM modeling are also valid for the description of thebitstream parsing process. The functional minimalismprovided by FSM modeling in a BSD format provideshigh interoperability for the parser FU implementationfrom BSD because of the abstract structure of FSM.

(2)

XML-based description format: XML is a well-known,wide-spread language to describe and exchange data inmany fields including multimedia metadata [15,22,23].In MPEG RMC, XML is used as a format for decoderdescription (e.g., functional unit network language(FNL) [1]); XS-PDL is also based on XML to support

Fig. 8. Differences between the MPEG-4 SP and MPEG-4 ASP bitstream syntax. (a) Video Object Plane, (b) Video Packet Header and (c) Macroblock.

H. Kim et al. / Signal Processing: Image Communication 28 (2013) 1255–1277 1263

XML compatibility as MPEG RMC. Another advantage ofthe XML-based format is that document validation canbe done with commonly available XML authoring tools.

structure is shown in Fig. 9. A BSD written in XS-PDL iscomprised of three parts: control flow description,

XS-PDL follows W3C XML grammar [24]; the overall

Table 2Comparison of the existing bitstream syntax description methods.

Textual specification Flavor RVC-BSDL

Design purpose Documentation Automatic parsing Automatic parsing� Content adaptation

(XFlavor/BFlavor)� Content

adaptation (BSDL)

(1) Control flow descriptionLanguage basis Human-readable text Cþþ/Java XML SchemaMachine-readability No Yes YesFlow control Pseudo-code Cþþ/Java-based BSDL attributesVariable handling None Cþþ/Java-based BSDL attribute, XPath 1.0

(2) Definition of external functionsDefinition method � Textual description

(e.g., narrativealgorithm explanation,flowcharts)

� Reference software

� Unique data type (map, bac)� Verbatim code feature

By communication withexternal FUs (rvc:port with rvc:ext)

Parameterization In bitstream semantics By Cþþ/Java-based syntax None; use alreadyparameterized external FUs storedin tool library

(3) Definition of parser interfaceInput port specification None None Only bitstream input and

automatically generatedfeedback ports

Output port specification None None Supported (rvc:port)Network-level parameter retrieval None None None

(4) Run-time compilation capabilityParser generation mechanism Manual implementation Code generation (flavorc) Code generation (BSDL2CALML)Run-time compile solution None None None

H. Kim et al. / Signal Processing: Image Communication 28 (2013) 1255–12771264

external function declaration, and parser interface declara-tion. The syntactic structure of XS-PDL will be explained inthe following sections. In addition, we provide a fullspecification and an XML schema for XS-PDL in [25].

3.2. Control flow description

Table 3 shows the formal description of XML elements forcontrol flow description in XS-PDL. The bitstream parsingcontrol flow is described as an FSM in XS-PDL. Therefore, acontrol flow description consists of one or more FSM elements(one main FSM and other sub-FSMs) that enables a hierarch-ical description of control flow. Each FSM (i.e., oFSM4) canbe represented by states where a state can be one of thefollowing four types: parsing state, external function call state,FSM call state, and branch state. Each state has the name

attribute as its identifier and contains a description of aspecific bitstream parsing behavior according to its type.It should be noted that the first FSM to be executed in thecontrol flow is the first FSM that appears in the control flowdescription in a sequential order. Similarly, the initial state ofan FSM is the first state that appears in the FSM.

3.2.1. Parsing stateParsing state (i.e., oparsing4) contains information on

how to parse a specific length of data from an input bit-stream. This information is described using the attributes of aparsing state declaration. As shown in Table 3, the attributes

in a parsing state can be used to describe a bitstream readingprocess with a specific bit length, look-ahead parsing option,byte alignment option, big- or little-endian option, and inputport designation. The bitstream syntax elements that arenecessarily to be set to a specific value (e.g., marker bitsshould always have value of 1) can be validated with theirexpected value defined by the expected attribute. A parsingstate can also be used to define semantic data that is notparsed from the bitstream by assigning a specific value usingthe value attribute. Additionally, a parsing state can be usedto designate the final state of the current FSM; if the final

attribute is set as true, the execution of the current FSM willbe terminated after the current state is processed.

The parsed data are stored and processed as variablesduring the bitstream parsing process. Each parsing state orexternal function call state generates a variable whose nameis the same as that of the state. In other words, it is possible toread some bits from the bitstream without declaring avariable by not assigning the name attribute to a parsing state.

Additional decoding operations can be added using theoprocess4 element in a parsing state. The additionaldecoding operations include mathematical, logical, andbinary operations to compute semantic values from theexisting variables. A oprocess4 element may containseveral ooperation4 elements and oif4 elements.The ooperation4 and oif4 elements can be labeledwith the name attribute. An ooperation4 element isan assignment operation of which the target attribute

Fig. 9. Language structure of XS-PDL.

H. Kim et al. / Signal Processing: Image Communication 28 (2013) 1255–1277 1265

defines the variable to be updated and the value attributedefines the operation to update the variable. A simpleC-style mathematical expression can be described in thevalue attribute. The oif4 element provides a branchcondition to control the operation flow. In the oif4element, the condition attribute defines the branchcondition to be examined. The value of the condition

attribute is described using the same expression syntaxwith the value attribute in the ooperation4 element.When the value is evaluated as true (i.e., non-zero), thenext operation to execute is the operation whose name isdesignated in the next attribute. Otherwise, the opera-tions are processed in a sequential order.

The ultimate role of a bitstream parser FU is to providesemantically decoded data to the rest of the FU network ina decoder. For each parsing state, the port output behaviorcan be described using the ooutput4 element. Theoutput port is designated in the dest attribute. Thedesignated output port must be declared in the parserinterface declaration (i.e., ointerface4). The valuewhich will be exported through the output port isdescribed in the value attribute. The value of the value

attribute can be a fixed value or a variable.

3.2.2. External function call stateThe external function call state (i.e., oplugin4) can

be used to call an external parsing function. The externalfunction to be called is described in the callee attribute,which can be selected from the list of the externalfunctions described in the external function declaration(i.e., oplugin_list4). Parameters can be assigned tothe external function using the oparameters4 ele-ment, which may consist of several oparameter4elements. Each oparameter4 element has a name

attribute as the identifier of the parameter which can beidentified by the external function to be called and the

value attribute as the value of the parameter. Like theparsing state, an external function call state generates avariable and stores the result of the external function callin the variable. Additional decoding operations and portoutput behavior can also be described in an externalfunction call state by the same XML syntax.

3.2.3. FSM call stateIn the control flow description, more than one FSMmay

appear, and an FSM can be called in another FSM as a sub-FSM using the FSM call state (i.e., oFSM_call4). Similarto the external function state, the FSM to be called isdescribed in the callee attribute. The value of thecallee attribute must be an FSM described in the currentcontrol flow description. An FSM call state can also haveadditional decoding operations and port output descrip-tion like the previously described state types. When theprocessing of the sub-FSM ends, an expression describedin the return attribute of the sub-FSM is evaluated andgiven as a return value to the FSM call state. Like theparsing state, an FSM call state generates a variable andstores the return value in the variable.

3.2.4. Branch stateEach state in the control flow description is described to

be run in a sequential order; however, the branch state(i.e., obranch4) allows a conditional jump to other states.A branch state may contain the branch condition described inthe condition attribute, and the corresponding destinationstate is defined in the next attribute. The branch condition isspecified using the same expression syntax with the value

attribute in the additional decoding operation description.A branch state may contain an additional decoding processdescribed in the oprocess4 element; the result of theadditional operation can be used in the branch condition.

Table 3Formal description of the XS-PDL control flow description syntax.

Element Attributes/Children/Notes

ocontrol4 A top-level element of control flow description in XS-PDL.Children: oFSM4*

oFSM4 FSM declaration element.name ¼ "FSM_name" The identifier of an FSM.return ¼ "expression" A return value to the parent FSM.Children: ostates4

ostates4 Container for the states.Children: (oparsing4|oplugin4|oFSM_call4|obranch4)*

oparsing4 A bitstream syntax element parsing state.name¼"state_name" The identifier of a state. (Also used as the name of a variable.)bits¼"expression" Bit length to be read from bitstream.source¼"input_port_name" Input port. (When there are two or more input ports)bytealign¼"(true|false)" Byte alignment option.peek¼"(true|false)" Look-ahead parsing option.expected¼"expression" Expected value. If the parsed value is not equals to this

attribute, an error will occur.endian¼"(big|little)" Endian of the parsed data.value¼"expression" Default value.final¼"(true|false)" A flag for final state of the current FSM.Children: {oprocess4, ooutput4*}

oplugin4 An external function (i.e., plugin) call state.name¼"state_name" Same with that of oparsing4 .callee¼"plugin_instance_name" An external function instance to be called.Children: (oparameters4|oprocess4|ooutput4)

Note: The formal description of the oparameters4 element is shown in Table 5.oFSM_call4 An FSM call state.

name¼"state_name" Same with that of oparsing4 .callee¼"FSM_name" An FSM to be called.Children: {oprocess4, ooutput4*}

obranch4 A branch state based on a conditional jump among states.name¼"state_name" Same with that of oparsing4 .condition¼"expression" Branch condition. If the evaluation result of the expression is true

(i.e., non-zero), a conditional jump will occur.next¼"operation_name" Next state as a jump destination.Children: oprocess4

oprocess4 Description of additional decoding operations.Children: (ooperation4|oif4)*

ooutput4 Description of a port output of a parsed data.dest¼"output_port_name" Output port.value¼"expression" A value or variable to be exported.

ooperation4 An operation for additional decoding process.name¼"operation_name" The identifier of a single operation. Can be used for branch

destination identifier by oif4 .target¼"variable" The name of a variable to be updated with value.value¼"expression" A simple calculation process.

oif4 A flow control statement within additional decoding process using a conditional jump among operations.name¼"operation_name" Same with that of ooperation4 .condition¼"expression" Branch condition. If the evaluation result of the expression is true

(i.e., non-zero), a conditional jump will occur.next¼"operation_name" Next operation as a jump destination.

H. Kim et al. / Signal Processing: Image Communication 28 (2013) 1255–12771266

3.2.5. Precedence in state executionThe bitstream parsing behaviors described in a state

follow the execution precedence rule as described inTable 4. In general, state-level behaviors such as bitstreamreading, external function calling, and sub-FSM callingprecede the additional decoding process and port outputbehavior. In the case of the conditional state transitiondescribed in a branch state, it is executed after theadditional decoding process to allow the variablesupdated in an additional decoding process to be used inthe branch condition evaluation. The final attribute toterminate the execution of the current FSM is executedafter the additional decoding process and the port outputbehavior.

3.3. External function declaration

The external functions that are used in the control flowdescription must be declared in the external functiondeclaration (i.e., oplugin_list4). Table 5 shows aformal description of the XML elements for the externalfunction declaration. A prototype of an external function isexpected to be available for the bitstream parser FU withan identifier string. The prototype function can be instan-ciated using the oinstance4 element to have anexternal function instance. A single function prototypemay be instanciated to more than one instance withdifferent parameters. The parameterization of a functioninstance is described by the oparameters4 element,

Table 4The processing precedence levels in a state description.

Level Parsing state Ext. function call state FSM call state Branch state

Element Attribute Element Attribute Element Attribute Element Attribute

1 oparsing4 source oparameter4 name oFSM_call4 callee ooperation4 value

2 oparsing4 peek oparameter4 value ooperation4 value ooperation4 target

3 oparsing4 bytealign oplugin4 callee ooperation4 target oif4 condition

4 oparsing4 bits oplugin4 name oif4 condition oif4 next

5 oparsing4 value ooperation4 value oif4 next obranch4 condition

6 oparsing4 expected ooperation4 target ooutput4 value

7 oparsing4 name oif4 condition

8 ooperation4 value oif4 next

9 ooperation4 target ooutput4 value

10 oif4 condition ooutput4 dest

11 oif4 next

12 ooutput4 value

13 ooutput4 dest

14 oparsing4 final

Table 5Formal description of the XS-PDL external function declaration syntax.

Element Attributes/Children/Notes

oplugin_list4 A top-level element of external function declaration in XS-PDL.Children: oinstance4*

oinstance4 Declaration of an external function instance.name¼"plugin_instance_name" The identifier of an instance.prototype¼"plugin_prototype_name" An external function prototype to be instanciated.Children: oparameters4

oparameters4 Container for the external function instanciation parameters.Children: oparameter4*

oparameter4 A parameter description element for an external function.name¼"plugin_parameter_name" The identifier of an instanciation parameter.value¼"(value | array_value)" Parameter value in string. Numeric or array values

also should be represented in a string form.

Table 6Formal description of the XS-PDL parser interface declaration syntax.

Element Attributes/Children/Notes

ointerface4 A top-level element of parser interface declaration in XS-PDL.Children: (oinput4|ooutput4)*

oinput4 Declaration of an input port for the bitstream parser FUname¼"input_port_name" The name of an input port.token_type¼"token_type" The token type identifier of an input port.

ooutput4 Declaration of an output port for the bitstream parser FUname¼"output_port_name" The name of an output port.token_type¼"token_type" The token type identifier of an output port.

H. Kim et al. / Signal Processing: Image Communication 28 (2013) 1255–1277 1267

which has the same description syntax with that of theexternal function call state in the control flow.

3.4. Parser interface declaration

The interface of the bitstream parser FU, in otherwords, the input and output ports of the parser shouldbe explicitly declared in a BSD. The formal description ofthe parser interface declaration is described in Table 6. Theinput and output ports are declared using the oinput4and ooutput4 elements, respectively. These two ele-ments share an identical attribute structure. The nameof the port must be specified in the name attribute, and

the data type of the port (e.g., token type in the MPEGRMC framework) may optionally be described in thetoken_type attribute for the debugging and validationpurpose at the FU network level.

4. Example XS-PDL descriptions

In this section, we present some example BSDs written inXS-PDL based on the conventional MPEG multimedia codingstandards: MPEG-4 Visual (MPEG-4) [9] and Scalable Com-plexity 3D Mesh Coding (SC3DMC) [26]. We chose thesestandards because the built-in implemented bitstream parserFUs based on the standards can be found in the MPEG RMC

H. Kim et al. / Signal Processing: Image Communication 28 (2013) 1255–12771268

framework [27,28]. In the following sections, we explain thehigh-level structure of the given BSD example and provideexample scripts describing some portion of the bitstreamsyntax parsing. The full version of the example BSD can befound in [25].

4.1. MPEG-4 SP

Table 7 shows the structure of the example BSD for MPEG-4 SP. The example BSD is 1080 lines long and contains 8 FSMsconsisting of 148 parsing states, 15 external function callstates, 25 FSM call states, and 95 branch states. Each FSM isdesigned to represent a specific bitstream hierarchy (e.g., videoobject layer, video object plane, macroblock). For the parserinterface, one input port for bitstream input and 12 outputports are declared. To maintain consistency with the currentparser implementation of the MPEG RMC framework [29], theerror resilience features (i.e., data partitioning and reversibleVLC) and short header support are not included in the currentversion of the example BSD for MPEG-4 SP. An excerpt fromthe example BSD is shown in Fig. 10: the excerpt describes themacroblock hierarchy of the MPEG-4 SP bitstream and can becompared to Fig. 1. The size of the example BSD is 42,950bytes, while the MPEG-4 SP BSD written in RVC-BSDL [30] is60,284 bytes.

Table 7Structure of the XS-PDL BSD for MPEG-4 SP.

Control flow (ISO/IE

oFSM name¼"VS"4 6.2.2oFSM name¼"VO"4

oFSM name¼"VOL"4 6.2.3oFSM name¼"VOP"4 6.2.5

oFSM name¼"MB"4 6.2.6oFSM name¼"MV"4 6.2.6.2

oFSM name¼"Block"4 6.2.7oFSM name¼"escapeDec"4 7.4.1.3

oFSM name¼"next_start_code"4 5.2.4

External functions

oinstance name¼"VLD_B_6" prototype¼"VLDPlugin"4oinstance name¼"VLD_B_7" prototype¼"VLDPlugin"4oinstance name¼"VLD_B_8" prototype¼"VLDPlugin"4oinstance name¼"VLD_B_12" prototype¼"VLDPlugin"4oinstance name¼"VLD_B_13" prototype¼"VLDPlugin"4oinstance name¼"VLD_B_14" prototype¼"VLDPlugin"4oinstance name¼"VLD_B_16" prototype¼"VLDPlugin"4oinstance name¼"VLD_B_17" prototype¼"VLDPlugin"4oinstance name¼"EOF" prototype¼"EOFPlugin"4

Parser interfaces

ooutput name¼"WIDTH"/4ooutput name¼"HEIGHT"/4ooutput name¼"ROUND"/4ooutput name¼"FCODE"/4ooutput name¼"ACPRED"/4ooutput name¼"FOURMV"/4ooutput name¼"VOPMODE"/4ooutput name¼"MOTION"/4ooutput name¼"ACCODED"/4ooutput name¼"QP"/4ooutput name¼"MV"/4ooutput name¼"B"/4

4.2. MPEG SC3DMC QBCR

Table 8 shows the structure of the example BSD forSD3DMC quantization based compact representation (QBCR)coding mode. The example BSD describes the bitstreamsyntax for the fixed length (FL) and bit-precision (BP)entropy decoding modes in the SC3DMC QBCR standard.The example BSD is 864 lines long and 34,541 bytes in size.The BSD contains two FSMs consisting of 153 parsing states,8 FSM call states, and 71 branch states. In Fig. 11, a part ofthe standard bitstream specification written in MPEG SDL(i.e., Flavor) is compared with the control flow description inXS-PDL source code. For bitstream input and port output,one input port and 130 output ports are declared. The outputports are designed to correspond with the currently pro-posed FU network for RGC in MPEG Reconfigurable GraphicsCoding (RGC) [28].

5. Generic parser FU approach

For the run-time compilation of a BSD written in XS-PDL,we propose a generic parser FU (GPFU) as a parser generationmechanism. Fig. 12 shows the concept of the GPFU approach,which can be compared to the built-in approach illustratedin Fig. 7. The GPFU is a configurable bitstream parser FU

C 14496-2:2004)

Visual Object Sequence and Visual Object

Video Object LayerVideo Object Plane and Video Plane with Short HeaderMacroblockMotion vector

BlockEscape code

Definition of next_start_code() function

(ISO/IEC 14496-2:2004)

Table B-6Table B-7Table B-8Table B-12Table B-13Table B-14Table B-16Table B-17(EOF check from the Ref. SW)

Fig. 10. An excerpt from the XS-PDL description for MPEG-4 SP.

Table 8Structure of the XS-PDL BSD for SC3DMC.

Control flow (ISO/IEC 14496-16:2009/FDAM 1:2010)

oFSM name¼"SC3DMCStreamHeader"4 5.9.2.2 SC3DMCStreamHeader classoFSM name¼"QBCRStreamData"4 5.9.2.4 QBCRStreamData class

Parser interfaces

oinput name¼"bitstream"/4ooutput name¼"CCW"/4ooutput name¼"SOLID"/4ooutput name¼"CONVEX"/4ooutput name¼"COLORPERVERTEX"/4ooutput name¼"NORMALPERVERTEX"/4ooutput name¼"OTHERATTRIBUTESPERVERTEX"/4ooutput name¼"ISTRIANGULARMESH"/4ooutput name¼"Number_Geo"/4ooutput name¼"Number_GeoIdx"/4ooutput name¼"Number_Color"/4(Total 130 output ports)

H. Kim et al. / Signal Processing: Image Communication 28 (2013) 1255–1277 1269

Fig. 11. Comparison between the SC3DMC bitstream specification and the XS-PDL description. (a) MPEG SDL/Flavor and (b)XS-PDL description.

H. Kim et al. / Signal Processing: Image Communication 28 (2013) 1255–12771270

pre-defined in the tool library of which the behavior is con-figured during run-time according to a BSD. The existingparser generation methods (for Flavor or RVC-BSDL) are

mainly focused on the source code generation. Different fromexisting methods, the GPFU approach provides a run-timeconfigurable parser FU on top of executable BSD. The basic

H. Kim et al. / Signal Processing: Image Communication 28 (2013) 1255–1277 1271

structure of the GPFU is depicted in Fig. 13. A GPFU can beimplemented with four parts: the BSD interpreter, scriptengine, plugin functions, and parser interface.

(1)

BSD interpreter: A BSD interpreter reads a BSD given inthe run-time and evokes a parser configuration pro-cess by distributing information extracted from theBSD to the script engine, plugin functions, and theparser interface. For example, a BSD written in XS-PDLcan be separated into three parts of description and

Fig. 13. Structure of the

Fig. 12. The concept of the GP

will be distributed to the script engine, the interface,and the plugin functions. To alleviate the implementa-tion burden, the conventionally available XML parserlibraries (e.g., MS-XML, Xerces, or TinyXML) can beused to process a BSD format based on XML.

(2)

Script engine: The script engine is the core of the GPFUbecause the run-time configurability of the bitstreamparser FU is provided by the script engine. The scriptengine directly executes the control flow descriptioncontained in the given BSD. The script engine can be

GPFU.

FU approach.

H. Kim et al. / Signal Processing: Image Communication 28 (2013) 1255–12771272

implemented based on the conventional run-timecompilation methodologies for script languages; forexample, run-time interpreter, Just-in-Time (JIT) com-piler, or virtual machine. To perform the bitstreamdecoding, the script engine should be able to commu-nicate with the input/output interfaces and the pluginfunctions in the GPFU.

(3)

Plugin functions: In the GPFU approach, the externalfunctions for the complex parsing functionalities areconsidered to be the plugin functions connected inthe GPFU. The plugin functions are instanciated by theplugin function declaration and then called by thecontrol flow in the script engine with a specific set ofparameters. Therefore, the plugin functions should bedesigned as prototype functions that can be instan-ciated with a set of parameters during the parserconfiguration process. Also, plugin functions shouldbe able to respond to a parameterized function callevoked by the script engine and return a value to thescript engine if necessary because the result of theplugin function call will be processed by the controlflow in the script engine.

(4)

Fig. 14. Flow chart of the overall GPFU configuration process.

Parser interface: The GPFU communicates with theRMC framework through the parser interface. As withother FUs in the RMC framework, the bitstream parserFU also has a number of input and output ports as itsinterface. Input ports provide the bitstream input, andthe output ports deliver the parsed, semanticallydecoded data to other FUs via the FU network for thedecoding process. A major difference in the parserinterface of the GPFU from the built-in parser FUs isthe configurability of the input and output ports sincethe input and output behavior of the GPFU is depen-dent on the bitstream syntax. Hence, the input andoutput ports should be able to be dynamically mana-ged in run-time according to the given BSD.

Fig. 15. Flow chart of the validation process.

5.1. Parser configuration process

To configure the GPFU as a parser conforming to anXS-PDL BSD, the BSD interpreter extracts the three parts ofthe description from the BSD. Then, the script engineperforms the bitstream parsing according to the controlflow description, the parser interface is reconfigured usingthe parser interface declaration, and the plugin functionsare parameterized by the external function declaration.More specifically, the parser reconfiguration process basedon a given BSD can be done using the following proce-dures: (1) validation, (2) configuration, and (3) execution,as shown in the top-level flow chart depicted in Fig. 14.

5.1.1. Validation processBefore configuring the behavior of the GPFU, a given

BSD can be validated to prevent unexpected errors in theBSD. The validation of the BSD includes not only thechecking of grammatical errors but also the functionalvalidity of the BSD, assuring that the BSD is suitable for theparser configuration in the current GPFU implementation.The flow chart of the BSD validation process is shown inFig. 15. Through the BSD validation process, it is possible to

Fig. 16. Flow chart of the configuration process.

H. Kim et al. / Signal Processing: Image Communication 28 (2013) 1255–1277 1273

early determine any error within BSD before configuringthe GPFU.

The syntactic correctness check for the BSD can be doneby XML validation tools using XML schema since XS-PDL isbased on XML. The XML schema-based validation cancheck both the XML syntax and XS-PDL syntax at thesame time. Once the BSD validation is finished withouterror, the BSD interpreter performs the functional valida-tion of the given BSD. The functional validation includestwo validation processes: the validation of the parserinterface and the validation of the external function list.The BSD interpreter validates the parser interface bycomparing the parser interface declaration in the BSD(i.e., ointerface4) and the actual input and outputports that are currently connected. The external functionlist (i.e., oplugin_list4) is compared with the pluginfunctions that are currently connected to the GPFU. Thefunctional validation process ensures that the bitstreamparsing behaviors are properly performed in the GPFU.

5.2. Configuration process

After the validation process, the GPFU configures itselfaccording to the given BSD. The flow chart of the config-uration process is shown in Fig. 16. During this process, theBSD interpreter extracts three parts of information fromthe given BSD and distributes them. The parser interface isestablished according to the parser interface declaration.The input and output ports of the GPFU are configured andconnected with the FU network in the RMC framework.Plugin function instances are prepared in the GPFU. Theplugin functions can be implemented in the GPFU asprototype functions; using the prototypes, the pluginfunction instances may be set up with the parametersdescribed in BSD and prepared for the function call fromthe script engine. The control flow is loaded to the scriptengine. The FSM structure is run-time compiled andtranslated into an intermediate form that can be directlyexecuted by the script engine.

5.3. Execution process

The execution process takes place after the configura-tion process. Fig. 17 shows the flow chart of the executionprocess. In this process, the script engine conducts bit-stream parsing by processing the FSM described in thecontrol flow description. The parsing behavior is deter-mined by the type of state; the expected behavior for eachtype of state is according to the XS-PDL specificationexplained in Section 3. The parsing process will be con-tinued until the control flow is terminated or the inputbitstream has reached the end (i.e., the end of the file(EOF)).

6. Experimental results

6.1. Test environment

We implemented the GPFU for the experiment toconfirm the feasibility of XS-PDL BSD. The implementationwas built using Cþþ in Microsoft Visual Studio 2010 and

the Microsoft Windows 7 operating system. The TinyXMLlibrary [31] was integrated with the implementation forthe XML parsing of XS-PDL BSD. A proprietary scriptengine which is similar to the conventional interpreterengines (e.g., ECMAScript interpreter) was designed toprocess the control flow description and contained withinthe GPFU implementation. To keep the design complexityof the script engine low, the functionality of the scriptengine was constrained to the bitstream parsing behaviors.Plugin functions were implemented in dynamic linklibrary (DLL) and connected to the GPFU through functioninterfaces. The GPFU implementation contained two exter-nal functions: one for the variable length decoding andone for the end-of-file (EOF) check for the input ports. Thetotal size of the GPFU implementation including the pluginfunctions was 152,064 bytes when compiled.

Using the GPFU implementation, we conducted bit-stream parsing experiments to evaluate the feasibility ofthe XS-PDL format and the GPFU concept. The experimentwas conducted using two example BSDs written in XS-PDL: MPEG-4 SP and SC3DMC QBCR, which are described

Fig. 17. Flow chart of the execution process.

H. Kim et al. / Signal Processing: Image Communication 28 (2013) 1255–12771274

in Section 4. To execute the GPFU implementation, weused a C/Cþþ based RMC framework environment calledthe Codec Configuration Simulation Model (CC-SiM) [32].The parsing experiment was conducted on a PC equippedwith Intel Core i7 2.13 GHz dual-core CPU, 6.00 GB RAM,and the 64-bit Microsoft Windows 7 operating system.

6.2. Bitstream parsing experiments

Table 9 shows the result of the bitstream parsingexperiment using MPEG-4 SP BSD. The experiment was

to compare the bitstream parsing performance of theGPFU with the MPEG reference software [33]. The bit-streams used for the experiment were selected from theMPEG-4 Visual conformance bitstreams [34]. In the experi-ment, the bitstream parsing time was measured from theGPFU and MPEG reference software. The bitstream parsingtime shown in Table 9 is an average of five consecutiveexecutions. In the case of the reference software, thebitstream parsing process is not separated from the videodecoding process; hence, we approximated the bitstreamparsing time as 40 percent of the decoding time, which is

Table 9The results of the MPEG-4 SP bitstream parsing experiment.

Bitstream Ref. SW GPFUΔTP (%)

Name File size(KB)

Image size(pixel)

Codedframes

Decodingtime (msa)

Parsingtimeb (ms)

Parsingspeed (fpsc)

Parsingtime (ms)

Parsingspeed (fps)

hit001.m4v 288 352�288 100 1696 678 147 1264 79 53.7hit002.m4v 85 352�288 100 1505 602 166 478 209 125.9hit003.m4v 441 352�288 100 1751 700 143 1821 55 38.5hit004.m4v 301 352�288 100 1526 610 164 1489 67 41.0hit005.m4v 201 352�288 100 1591 636 157 888 113 71.6hit007.m4v 444 352�288 100 1772 709 141 1770 56 40.0hit008.m4v 281 352�288 100 1498 599 167 1420 70 42.2hit009.m4v 422 352�288 100 1790 716 140 1739 57 41.2hit010.m4v 445 352�288 100 1755 702 142 1849 54 38.0jvc000.m4v 470 176�144 100 1260 504 198 1541 65 32.7jvc014.m4v 183 352�288 150 2149 859 175 700 214 122.7jvc015.m4v 59 352�288 150 2500 1000 150 780 192 128.2jvc016.m4v 469 352�288 150 2270 908 165 1856 81 48.9san000.m4v 87 352�288 50 856 342 146 602 83 56.9san001.m4v 159 352�288 100 1703 681 147 779 128 87.5san002.m4v 137 352�288 136 1822 729 187 706 193 103.1san013.m4v 157 352�288 150 2010 804 187 790 190 101.8san015.m4v 158 352�288 149 3061 1224 122 810 184 151.2san016.m4v 158 352�288 149 2248 899 166 833 179 107.9san017.m4v 157 352�288 150 1994 798 188 823 182 96.9san018.m4v 157 352�288 150 2198 879 171 831 180 105.8san020.m4v 158 352�288 149 2197 879 170 813 183 108.2(Average) 2300 920 155 1163 125 85.3

a Millisecond.b Estimated (40% of the decoding time).c Frames per second.

H. Kim et al. / Signal Processing: Image Communication 28 (2013) 1255–1277 1275

an estimate from various implementations. Then, wederived the speed of the bitstream parsing by the follow-ing calculation: SP¼F/TP, where SP is the bitstream parsingspeed in frames-per-second (fps), F is the frames in thebitstream, and TP is the time elapsed to parse the bitstreamin second. We also evaluated the performance of the GPFUby comparing the bitstream parsing time by ΔT¼TP(RSW)/TP(GPFU), where TP(RSW) is the approximate parsing time of thereference software and TP(GPFU) is the parsing time of theGPFU. A higher ΔT value means a better performance ofthe GPFU. From Table 9, it can be observed that SP of theGPFU ranges between 54 fps and 214 fps while the SP ofthe reference software ranges between 122 fps and198 fps. On average, the performance of the GPFU isevaluated as 85.3% of the reference software. The resultshows that the performance of the GPFU is comparablewith the reference software, providing a sufficient parsingefficiency for practical use. Also, it should be noted that theinitial delay for the configuration process was not morethan 10 ms for the test bitstreams; therefore, it wasnegligible in the parsing time evaluation.

Table 10 shows the result of the bitstream parsingexperiment using SC3DMC QBCR BSD. This experimentwas conducted in order to confirm the feasibility of XS-PDL and the GPFU is not limited to the video codingstandard. We provided 18 bitstreams coded by theSC3DMC reference software to the GPFU implementation,and we observed that the GPFU implementation hassuccessfully generated the parsed data via its output ports.The parsed data were compared with the data retrieved

from the reference software and confirmed as identical tothe reference software.

7. Conclusion

Throughout this paper, we have proposed a newmethod for BSD representation and a run-time parsergeneration mechanism. The new BSD format we proposed,XS-PDL, is a format to support four key features of BSDthat are necessary for the run-time parser generation. Thedescription feasibility of XS-PDL is tested with the bit-stream syntax of the conventional MPEG standards (i.e.,MPEG-4 SP and SC3DMC QBCR). Based on the XS-PDLformat, we designed a configurable bitstream parser FU,GPFU, as a run-time parser generation mechanism. Wediscussed how the GPFU can be configured by a given BSDand can be executed. The performance is evaluated by thebitstream parsing experiments and compared with theMPEG reference software.

The XS-PDL and the GPFU allow for a flexible codecdesign in the RMC framework by allowing run-time parsergeneration; we are continuing our research on the effi-ciency of the BSD format and the GPFU implementation. Afuture research topic could be the integration of the GPFUwith the MPEG RMC framework. As we mentioned pre-viously, the decoder description language in the MPEG RMCframework is based on the XML representation. Hence, itshould be possible to combine the existing decoder descrip-tion language (e.g., functional unit network language (FNL)[1]) and XS-PDL in a single XML document to make a

Table 10The results of the SC3DMC QBCR bitstream parsing experiment.

BitstreamGPFU

Name Model File size (KB) Verticesa Facesa BPd Parsing time (msb)

NPFLCROSS_f.s3d Cross_f 1 41 24 0e

DPBPCROSS_f.s3d Cross_f 1 41 24 ✓ 2XORBPCROSS_f.s3d Cross_f 1 41 24 ✓ 1NPFLHEX0_f.s3d HEX0_f 1 96 50 0e

DPBPHEX0_f.s3d HEX0_f 1 96 50 ✓ 2XORBPHEX0_f.s3d HEX0_f 1 96 50 ✓ 2NPFLAH64.s3d AH64 21 2709 1972 0e

DPBPAH64.s3d AH64 16 2709 1972 ✓ 55XORBPAH64.s3d AH64 16 2709 1972 ✓ 50NPFLchicken.s3d Chicken 54 5664 3030 5DPBPchicken.s3d Chicken 43 5664 3030 ✓ 145XORBPchicken.s3d Chicken 44 5664 3030 ✓ 133NPFLtroll.s3d Troll 163 4961c 9896c 3DPBPtroll.s3d Troll 118 4961c 9896c ✓ 370XORBPtroll.s3d Troll 121 4961c 9896c ✓ 322NPFLCRATERS_F.s3d Craters_F 1479 167,833 84,765 1DPBPCRATERS_F.s3d Craters_F 1334 167,833 84,765 ✓ 2961XORBPCRATERS_F.s3d Craters_F 1341 167,833 84,765 ✓ 2501

a For geometry.b Millisecond.c With the same number of vertices/faces for normal.d Bit-precision entropy coded.e Less than one millisecond.

H. Kim et al. / Signal Processing: Image Communication 28 (2013) 1255–12771276

complete decoder description written in XML. Such acombined decoder description would enable the run-timereconfiguration of the entire decoder including the modularFU network and the bitstream parser FU.

Another possible approach of the integration is translationbetween XS-PDL and other BSD formats, especially RVC-BSDL,which is the current standard BSD language in the MPEG RMCframework. For example, we observe that the descriptionsyntax of RVC-BSDL can be extended with additional func-tionalities to support the GPFU [35]. Based on such anextension, the translation process between the two languagescan be established; research on the translation capability maylead to seamless integration of XS-PDL and the GPFU withRVC-BSDL and the MPEG RMC framework.

Acknowledgments

This work was supported by the strategic technologydevelopment program of MCST/MKE/KEIT (KI001798,Development of Full 3D Reconstruction Technology forBroadcasting Communication Fusion).

References

[1] ISO/IEC JTC 1, Information Technology—MPEG Systems—Part 4:Codec Configuration Representation, ISO/IEC 23001-4:2011, 2011.

[2] ISO/IEC JTC 1, Information Technology—MPEG Video Technologies—Part 4: Video Tool Library, ISO/IEC 23002-4:2010, 2010.

[3] ISO/IEC JTC1/SC29/WG11, Reconfigurable Video Coding Require-ments v.4.0, MPEG-document ISO/IEC JTC1/SC29/WG11 N9585,Moving Picture Experts Group (MPEG), Antalya, Turkey, 2008.

[4] Alexandros Eleftheriadis, Flavor: a language for media representa-tion, in: Proceedings of ACM Multimedia 97 Conference, 1997.

[5] Flavor—Formal Language for Audio-Visual Object Representation,May 2012, ⟨http://flavor.sourceforge.net/⟩.

[6] ISO/IEC JTC 1, Information Technology—Coding of Audio–visualObjects—Part 1: Systems, ISO/IEC 14496-1:2010, 2010.

[7] Annex C. (normative) Specification of RVC-BSDL, ISO/IEC JTC 1,Information Technology—MPEG systems—Part 4: Codec Configura-tion Representation, ISO/IEC 23001-4:2011, 2011.

[8] ISO/IEC JTC 1, Information Technology—Coding of Moving Picturesand Associated Audio for Digital Storage Media at up to About1.5 Mbit/s—Part 2: Video, ISO/IEC 11172-2:1993, 1993.

[9] ISO/IEC JTC 1, Information Technology—Coding of Audio–visualObjects–Part 2: Visual, ISO/IEC 14496-2:2004, 2004.

[10] Yihan Fang, Alexandros Eleftheriadis, Automatic generation ofentropy coding programs using flavor, in: IEEE Second Workshopon Multimedia Signal Processing, 1998, pp. 341–346.

[11] Flavor—Download, May 2012, ⟨http://sourceforge.net/projects/flavor⟩.[12] Danny Hong, Alexandros Eleftheriadis, XFlavor: providing XML

features in media representation, Multimedia Tools and Applications39 (2008) 101–116.

[13] Wesley De Neve, Davy Van Deursen, Davy De Schrijver,Sam Lerouge, Koen De Wolf, Rik Van de Walle, BFlavor: a harmo-nized approach to media resource adaptation, inspired by MPEG-21 BSDL and XFlavor, Signal Processing: Image Communication 21(2006) 862–889.

[14] ISO/IEC JTC 1, Information Technology—MPEG Systems Technologies—Part 5: Bitstream Syntax Description Language (BSDL), ISO/IEC23001-5:2008, 2008.

[15] ISO/IEC JTC 1, Information Technology—Multimedia Framework(MPEG-21)—Part 7: Digital Item Adaptation, ISO/IEC 21000-7:2007,2007.

[16] Annex D. (normative) Specification of RVC-CAL Language, ISO/IECJTC 1, Information Technology—MPEG Systems—Part 4: Codec Con-figuration Representation, ISO/IEC 23001-4:2011, 2011.

[17] W3C, XSL Transformations (XSLT) Version 2.0, W3C Recommenda-tion 23 January 2007, ⟨http://www.w3.org/TR/xslt20/⟩.

[18] Joseph Thomas-Kerr, Jomn Janneck, Marco Mattavelli, Ian Burnettl,Christian Ritz, Reconfigurable media coding: self-describing multi-media bitstreams, in: IEEE Workshop on Signal Processing System,2007, pp. 319–324.

[19] Christophe Lucarz, Jonathan Piat, Marco Mattavelli, Automaticsynthesis of parsers and validation of bitstreams within the mpegreconfigurable video coding framework, Journal of Signal ProcessingSystems 63 (2011) 215–225.

H. Kim et al. / Signal Processing: Image Communication 28 (2013) 1255–1277 1277

[20] Table 9–1—Tools for Version 1 Visual Object Types, ISO/IEC JTC 1,Information technology—Coding of audio–visual objects—Part 2:Visual, ISO/IEC 14496-2:2004, 2004.

[21] Table 9–4—Tools for FGS Visual Object Types, ISO/IEC JTC 1, Informa-tion technology—Coding of Audio–visual objects—Part 2: Visual, ISO/IEC 14496-2:2004, 2004.

[22] ISO/IEC JTC 1, Information Technology—Multimedia ContentDescription Interface—Part 2: Description Definition Language,ISO/IEC 15938-2:2002, 2002.

[23] W3C, Synchronized Multimedia Integration Language (SMIL) 1.0Specification, W3C Recommendation 15 June 1998, ⟨http://www.w3.org/TR/REC-smil/⟩.

[24] W3C, Extensible Markup Language (XML) 1.0 (Fifth Edition),W3C Recommendation 26 November 2008, ⟨http://www.w3.org/TR/REC-xml/⟩.

[25] Digital Media Laboratory, XS-PDL Homepage, May 2012, ⟨http://dmlab.hanyang.ac.kr/xspdl⟩.

[26] ISO/IEC JTC1/SC29/WG11, ISO/IEC 14496-16:2009 FDAM1 ScalableComplexity 3D Mesh Compression, MPEG-document ISO/IEC JTC1/SC29/WG11 N11124, Moving Picture Experts Group (MPEG), Kyoto,Japan, 2010.

[27] ISO/IEC JTC 1, Video Tool Library Conformance and Reference Soft-ware, ISO/IEC 23002-4:2010/Amd 1:2011, 2011.

[28] Seungwook Lee, Ji Hyeong Lee, Hyerin Lim, Sinwook Lee, TaeheeLim, Bonki Koo, Euee S. Jang, Contribution of SC3DMC FUs for RGC

Framework, MPEG-document ISO/IEC JTC1/SC29/WG11 M21220,Moving Picture Experts Group (MPEG), Torino, Italy, 2011.

[29] MPEG SVN, Bitstream Syntax Parser FU for MPEG-4 SP in RVCSimulation Model (RSM), 2012 ⟨http://wg11.sc29.org/svn/repos/MPEG-C/RVC/trunk/Working_models/RVC/src/org/sc29/wg11/mpeg4/part2/parser/Algo_Synp_SP.cal⟩.

[30] MPEG SVN, RVC-BSDL Bitstream Syntax Description for MPEG-4 SP,2012 ⟨http://wg11.sc29.org/svn/repos/MPEG-C/RVC/trunk/Bitstream_Syntax_Descriptions/MPEG-4-SP/⟩.

[31] Lee Thomason, TinyXML Project Page, 2012, ⟨http://sourceforge.net/projects/tinyxml/⟩.

[32] Digital Media Laboratory, CC-SiM Homepage, 2012, ⟨http://dmlab.hanyang.ac.kr/ccsim⟩.

[33] MPEG SVN, MPEG-4 Reference Software Version 2.5 090419, 2012⟨http://wg11.sc29.org/svn/repos/MPEG-C/RVC/trunk/Reference_Software/microsoft-v2.5-090419-NTU_latest/⟩.

[34] ISO/IEC, Video Conformance Bitstreams, Publicly Available Stan-dards: ISO/IEC 14496-4:2004 Conformance Testing, 2012 ⟨http://standards.iso.org/ittf/PubliclyAvailableStandards/ISO_IEC_14496-4_2004_Conformance_Testing/video_conformance/⟩.

[35] Seungwook Lee, Hyungyu Kim, Sinwook Lee, Sowon Kim, Euee S.Jang, RGC CE1: Study on the Generic Parser FU, MPEG-documentISO/IEC JTC1/SC29/WG11 M25850, Moving Picture Experts Group(MPEG), Stockholm, Sweden, 2012.