1

Purpose • The intent of this module is to introduce you to the multimedia features and

functions of the i.MX31. You will learn about the Imagination PowerVR MBX-Lite hardware core, graphics rendering, video processing, video encoding, and video decoding.

Objectives • Identify the key features of PowerVR MBX-Lite.• Describe multimedia capabilities of the i.MX31.• Identify the features of the IPU.• Describe MPEG-4 video encoding.• Describe the role of H.264 decoding during video playback.

Content• 15 pages• 2 questions

Learning Time• 25 minutes

Module Introduction

The intent of this module is to introduce you to the multimedia features and functions of the i.MX31. You will learn about the Imagination PowerVR MBX-Lite hardware core, which provides high performance 3D graphics rendering for less power and bandwidth than many traditionally architected accelerators. You will also learn about video processing, video encoding, and video decoding. It should be noted that the i.MX31L does not have 2D/3D graphics acceleration; otherwise, unless specifically mentioned, all information in this module applies to both the i.MX31 and the i.MX31L.

Bec

ause

of a

n or

der f

rom

the

Uni

ted

Sta

tes

Inte

rnat

iona

l Tra

de C

omm

issi

on, B

GA

-pac

kage

d pr

oduc

t lin

es a

nd p

art n

umbe

rs in

dica

ted

here

cur

rent

ly

are

not a

vaila

ble

from

Fre

esca

le fo

r im

port

or s

ale

in th

e U

nite

d S

tate

s pr

ior t

o S

epte

mbe

r 201

0: i.

MX

31 P

rodu

ct F

amily

, i.M

X31

L P

rodu

ct F

amily

2

PowerVR MBX-LiteKey features of the MBX-Lite:

• Tile-based renderer– Allows lower bandwidth to system memory vs. traditional architectures– Allows high precision color and depth operations

• PowerVR Texture Compression (PVR-TC)3D performance:

• Up to 1 million triangles per second• 118 million pixels per second

•Per vertex fog •16-bit textures •32-bit textures •YUV video textures •Point, bilinear, trilinear and anisotropic filtering •Full range of blend modes

•Flat and Gouraud shading •Perspective texturing •Specular highlights •Two-layer multitexturing •32-bit Z support •Full tile blend buffer •Alpha test •Full-scene anti-aliasing

Standard Features

The MBX-Lite uses a tile-based rendering technique to achieve high performance while keeping power and bandwidth low. The MBX-Lite is also able to yield higher precision color and depth processing.

The MBX-Lite further reduces bandwidth and memory consumption by providing PowerVR Texture Compression (PVR-TC) texture compression. This reduces the size of textures to shrink the memory footprint of textures and the overall size of applications.

In addition to these attractive key features, the MBX-Lite supports up to 1 million triangles per second and 118 million pixels per second, allowing developers to create compelling 3D applications.

Lastly, MBX-Lite provides a host of standard 3D features to support industryAPIs and developers. Here you can see these 3D features.

Bec

ause

of a

n or

der f

rom

the

Uni

ted

Sta

tes

Inte

rnat

iona

l Tra

de C

omm

issi

on, B

GA

-pac

kage

d pr

oduc

t lin

es a

nd p

art n

umbe

rs in

dica

ted

here

cur

rent

ly

are

not a

vaila

ble

from

Fre

esca

le fo

r im

port

or s

ale

in th

e U

nite

d S

tate

s pr

ior t

o S

epte

mbe

r 201

0: i.

MX

31 P

rodu

ct F

amily

, i.M

X31

L P

rodu

ct F

amily

3

Tile-based Rendering

Traditional 3DRenderer

SystemMemory

MBX-Lite 3DRenderer on

i.MX31

SystemMemory

Tile On-chip

All 3D Data

Resulting Data

Low LatencyIntermediate Data

In tile-based rendering, the system divides the 3D data into blocks that refer to rectangular regions of the display. This division allows the rendering to occur in one region at a time and utilizes much fewer resources than if the whole screen were considered at one time.

In traditional 3D rendering systems, all 3D data was saved to system memory. The MBX-Lite uses a set of small on-chip buffers that replaces the large, fast buffers of the traditional 3D renderer. Due to the order of rendering, only the resulting rendered scene is written out to system memory, and the on-chip memory absorbs the intermediate accesses.

In addition, the deferred aspect of a tile-based approach allows the renderer to only read texture data that the end scene requires from the system memory. For the i.MX31 unified memory architecture, this results in lower system bandwidth usage and less power drain. The increased bandwidth and lower latency of the on-chip buffers allows the system to afford higher precision calculations than those available in traditional architectures. This results in more accurate color values and fewer depth-based artifacts.

Bec

ause

of a

n or

der f

rom

the

Uni

ted

Sta

tes

Inte

rnat

iona

l Tra

de C

omm

issi

on, B

GA

-pac

kage

d pr

oduc

t lin

es a

nd p

art n

umbe

rs in

dica

ted

here

cur

rent

ly

are

not a

vaila

ble

from

Fre

esca

le fo

r im

port

or s

ale

in th

e U

nite

d S

tate

s pr

ior t

o S

epte

mbe

r 201

0: i.

MX

31 P

rodu

ct F

amily

, i.M

X31

L P

rodu

ct F

amily

4

Graphics Partitioning

SceneManagement Lighting Geometry

Processing Rasterization

ARM11VFP

MBX-Lite

Display

IPU

To render a 3D image, the data must pass through a set of standard stages of processing. Let’s look at the hardware and software partitioning of these stages.

The ARM1136 is partitioned to handle the scene management, lighting, and geometry processing stages in software. These stages are accelerated by the vector floating point (VFP) unit on the processor. This eliminates the need to do costly floating point conversions and emulation.

The MBX-Lite 3D acceleration hardware handles the rasterization portion of the pipeline, which is traditionally the most bandwidth-intensive portion. This stage handles the interpolation of triangles, blending of colors, and occlusion checking. In addition, the tile partitioning is executed as a pre-processing step in hardware just prior to rasterization.

Lastly, the IPU handles the final compositing and display of the resulting 3D rendered image.

Bec

ause

of a

n or

der f

rom

the

Uni

ted

Sta

tes

Inte

rnat

iona

l Tra

de C

omm

issi

on, B

GA

-pac

kage

d pr

oduc

t lin

es a

nd p

art n

umbe

rs in

dica

ted

here

cur

rent

ly

are

not a

vaila

ble

from

Fre

esca

le fo

r im

port

or s

ale

in th

e U

nite

d S

tate

s pr

ior t

o S

epte

mbe

r 201

0: i.

MX

31 P

rodu

ct F

amily

, i.M

X31

L P

rodu

ct F

amily

5

Graphics Software APIs

•High level (scene-graph) based Java API•Available for i.MX31 JVM

•Low level graphics API•Microsoft mobile 3D API•Available only for WinCE 5.0 devices

•Low level graphics API•Open standard developed by the Khronos Group•Available for non-Microsoft platforms for i.MX31

M3G / JSR184Direct3D MobileOpenGL ES

Depending on the platform, the i.MX31 provides one of three application programmer interfaces for accessing the capabilities on the MBX-Lite.

OpenGL ES provides a low-level hardware abstraction API for native programming on most operating systems. Based on a subset of the desktop OpenGL, this API is an open, royalty-free standard developed by the Khronos Group.

Direct3D Mobile is also a low-level API for 3D graphics accelerators. Similar to Direct3D, version 8 for personal computers, Direct3D Mobile provides a comprehensive interface to 3D hardware for WinCE based platforms.

For Java-based platforms, M3G provides a higher level scene-graph interface for 3D accelerators. While commonly criticized for its floating-point usage, M3G excels on the i.MX31 due to the integrated VFP unit.

Bec

ause

of a

n or

der f

rom

the

Uni

ted

Sta

tes

Inte

rnat

iona

l Tra

de C

omm

issi

on, B

GA

-pac

kage

d pr

oduc

t lin

es a

nd p

art n

umbe

rs in

dica

ted

here

cur

rent

ly

are

not a

vaila

ble

from

Fre

esca

le fo

r im

port

or s

ale

in th

e U

nite

d S

tate

s pr

ior t

o S

epte

mbe

r 201

0: i.

MX

31 P

rodu

ct F

amily

, i.M

X31

L P

rodu

ct F

amily

6

Which of the following statements about the tile-based rendering scheme of the MBX-Lite are true? Click all that apply, and then click Done.

a. Tile-based rendering allows lower system bandwidth.

b. Tile-based rendering allows better scene management.

c. Tile-based rendering allows higher texture compression.

d. Tile-based rendering allows higher precision color operations.

Done

Question

Here is a question to check your understanding of the MBX-Lite.

Correct.

Tile-based rendering allows lower system bandwidth and higher precision color operations.

Bec

ause

of a

n or

der f

rom

the

Uni

ted

Sta

tes

Inte

rnat

iona

l Tra

de C

omm

issi

on, B

GA

-pac

kage

d pr

oduc

t lin

es a

nd p

art n

umbe

rs in

dica

ted

here

cur

rent

ly

are

not a

vaila

ble

from

Fre

esca

le fo

r im

port

or s

ale

in th

e U

nite

d S

tate

s pr

ior t

o S

epte

mbe

r 201

0: i.

MX

31 P

rodu

ct F

amily

, i.M

X31

L P

rodu

ct F

amily

7

Multimedia Capabilities

i.MX31i.MX31

MMC/SDIO

MS Pro

USB HS

ARM11ARM11 VFPVFP

IPUIPU MPEG-4MPEG-4

18bits

HDD

ATA

StereoDAC

StereoDAC WLANWLAN Base

BandBaseBand

Up to 37 Hours of Viewfinder

Operation

16 MegapixelsResolution

In Still Picture Capture

Up to 10 Hoursof Real-Time Video

Capture & EncodingVGA 30 fps

6 Hours (3 Full Movies)of MPEG-4

Decoding and PlaybackVGA 30 fps

2 Displays2 Sensors

TV Encoder

Up to 480 Mbps

SynchronizationSpeed

MMC card,Flash Card

SDIO, MS ProHDD

Up to 60 Hours of MP3 Playback 128

Kbps

The i.MX31 processor is optimized to support a variety of image and video applications. It offers power-efficient image and video processing, pre- and post-processing in hardware, simultaneous MPEG-4 Simple Profile (SP) video encoding and decoding, real-time video decode in advanced formats, and image capture of up to 30 megapixels per second. The video implementation in the i.MX31 processor is the result of a smart trade-off between performance and flexibility. With a VFP co-processor and L2 cache, the i.MX31 is designed for any wireless device running computationally-intensive multimedia applications such as digital video broadcast and videoconferencing. The i.MX31 has many multimedia highlights, including up to 60 hours of MP3 playback at 128 Kbps. It provides versatile connectivity to a variety of image sensors and display devices as well as many peripherals and expansion ports for devices such as MultiMedia Card™, Flash cards, the SDIOs, Memory Stick PRO, and HDDs. The synchronization speed is up to 480 Mbps.Image capture in the i.MX31 can reach up to 30 megapixels per second, supporting VGA at 30+ fps in real time, 3 megapixels at 10 fps, and 16 megapixels for still picture capture. The synchronization speed is up to 480 Mbps.Image and video processing is very power efficient in the i.MX31. In particular, pre- and post-processing is performed fully in hardware, and the viewfinder, with up to 37 hours of operation, does not involve the ARM CPU. The i.MX31 supports simultaneous MPEG-4 SP Video Encoding and Decoding with up to VGA at 30 fps and 3 Mbits per second. Encoding is accelerated in hardware (approximately 1300 MHz of equivalent ARM11 performance), and decoding is performed in software. Pre- and post-processing is performed fully in hardware, adding considerable processing power to the system (approximately 1200 MHz of equivalent ARM11 performance). Pre- and post-processing includes functions such as resizing, inversion, rotation, de-blocking, de-ringing, blending, and color space conversion.i.MX31 supports six hours of real-time video decoding and playback with VGA at 30 fps. Other features of MPEG-4 video decoding include hardware-accelerated Post-Filtering for MPEG-4 and hardware-accelerated In-Loop De-Blocking for H.264. The i.MX31 supports real-time video decode in the following advanced formats: MPEG-4 Simple Profile (SP), H.264, Windows Media Video™ (WMV), RealVideo™ (RV), MPEG2, and DiVX.Video conference calling is supported on the i.MX31 with up to VGA at 30 fps and 1 Mbps.

Bec

ause

of a

n or

der f

rom

the

Uni

ted

Sta

tes

Inte

rnat

iona

l Tra

de C

omm

issi

on, B

GA

-pac

kage

d pr

oduc

t lin

es a

nd p

art n

umbe

rs in

dica

ted

here

cur

rent

ly

are

not a

vaila

ble

from

Fre

esca

le fo

r im

port

or s

ale

in th

e U

nite

d S

tate

s pr

ior t

o S

epte

mbe

r 201

0: i.

MX

31 P

rodu

ct F

amily

, i.M

X31

L P

rodu

ct F

amily

8

Video Processing

Camera (Image Signal Processing)(or ARM11 SW)IPU in i.MX31MPEG-4 Encoder in i.MX31ARM11 SW

Image Sensor

Combiningwith Audio

Display

Post Filtering

Bayer

YUV

YUV

IPU

Viewfinder Window

RGB

Memory

CommunicationNetwork

Separationfrom Audio

Decompression

ImageConversion

FormatConversion

QualityEnhancement

ImageConversion

MPEG-4Encoder

Compression

Performed by:

Let’s examine the video processing chain and its implementation. Images are captured by a camera and input directly to the Image Processing Unit (IPU) via the sensor interface.

The IPU performs some very processing-intensive image manipulations, adding considerable processing power to the system: approximately 1200 MHz of equivalent ARM11 performance. The IPU includes all the functionality required for image processing and display management. It allows a camera preview function to be performed fully in hardware, allowing the CPU to be powered down in this stage. It performs post filtering for MPEG-4, including de-blocking and de-ringing, and it also performs in-loop de-blocking for H.264 as specified in this standard. Video and graphics can be combined, and transparency specified by a key color, global alpha value, or per-pixel alpha values interleaved with the pixel components.

With regards to image conversion, it provides a fully flexible resizing ratio essentially between any two resolutions. Pixel format conversion features include fully flexible conversion coefficients, color space, and color adjustments. Other IPU functions include filtering, 90, 180, and 270 degree rotation, and horizontal/vertical inversion.

The pre-processor is part of the IPU, and it resizes the data and performs color space conversion. The pre-processor can send data to a small viewfinder display, which provides visual feedback to the user to ensure that the desired data is being captured. The pre-processor then sends data to the MPEG-4 encoder, which performs data compression according to the MPEG-4 video standard. The encoded data can be stored to file or sent to a communication network for later retrieval and playback.

Later, when the user wants to view the recorded video, the encoded data is retrieved and passed through the MPEG-4 decoder, which decompresses the data. The decompressed data is then sent to the post-processing module for quality enhancement, image resizing, and color space conversion. The data is then viewable on a display such as an LCD or TV monitor.

Bec

ause

of a

n or

der f

rom

the

Uni

ted

Sta

tes

Inte

rnat

iona

l Tra

de C

omm

issi

on, B

GA

-pac

kage

d pr

oduc

t lin

es a

nd p

art n

umbe

rs in

dica

ted

here

cur

rent

ly

are

not a

vaila

ble

from

Fre

esca

le fo

r im

port

or s

ale

in th

e U

nite

d S

tate

s pr

ior t

o S

epte

mbe

r 201

0: i.

MX

31 P

rodu

ct F

amily

, i.M

X31

L P

rodu

ct F

amily

9

Video ProcessingPre/Post processing:

• Performed fully in hardware• Includes resizing, rotation and inversion, color conversion, de-blocking, de-ringing,

and blending with graphics

Encoding:• MPEG-4 SP (fully HW accelerated)

– High performance; up to VGA @ 30 fps; image quality not compromised– Very power efficient– CPU is totally free to perform other tasks

• Sufficient for most purposes:– MPEG-4 SP is used for video conferencing– MPEG-4 SP is supported by most video players

• Other standards are left to SW

Decoding:• Post-filtering (de-blocking and de-ringing) is HW accelerated, providing significant

acceleration.• For H.264, the most processing-intensive standard, the de-blocking filter is HW

accelerated.• Other standards are implemented in software, enabling full flexibility to support a

variety of algorithms and future extensions.• This is enabled by the powerful ARM11 MCU and multilevel cache system.

The i.MX31 has built in pre- and post- processing in hardware that includes all the functionality required for image processing and display management, including de-block, de-ring, color space conversion, independent horizontal and vertical resizing, blending of graphics and video planes, and rotation in parallel to video decoding.

For video encoding, MPEG-4 SP and the H.263 baseline formats are fully hardware accelerated, supporting resolutions up to VGA at 30 fps. This achieves a high degree of power efficiency and frees the CPU to perform other tasks. It is sufficient for most purposes, as video conferencing and most video players support MPEG-4 SP. Software performs the encoding for other video standards.

Based on a mixture of software and hardware, this implementation provides the greatest flexibility to support a variety of algorithms and future extensions. The advanced ARM11 instruction set and multilevel cache system optimizes software. For MPEG-4, IPU hardware accelerates the post-filtering (deblocking and deringing), which results in a 75 percent load reduction on the ARM11 core. For H.264 baseline format—the most processing-intensive format—hardware also performs the deblocking filter, which provides a 30 percent acceleration improvement.

The software does implement other standards, which enables full flexibility to support a variety of algorithms and future extensions. The powerful ARM11 processor (including its multi-level cache system) provides the flexibility to decode at a high rate any currently relevant formats (up to HVGA at 30 fps), as well as possible future extensions.

Bec

ause

of a

n or

der f

rom

the

Uni

ted

Sta

tes

Inte

rnat

iona

l Tra

de C

omm

issi

on, B

GA

-pac

kage

d pr

oduc

t lin

es a

nd p

art n

umbe

rs in

dica

ted

here

cur

rent

ly

are

not a

vaila

ble

from

Fre

esca

le fo

r im

port

or s

ale

in th

e U

nite

d S

tate

s pr

ior t

o S

epte

mbe

r 201

0: i.

MX

31 P

rodu

ct F

amily

, i.M

X31

L P

rodu

ct F

amily

10

IPU

Memory

Cameras

Displays

TV Encoder

GraphicsAccelerator

IPU

EM

I

ARM11CPU

i.MX31CPU Complex

MPEG-4 Encoder

As you saw earlier, the IPU is at the heart of the video processing chain. It offers an integrative approach, including all functionalities required for image processing and display management. The IPU supports connectivity to a wide range of external devices including cameras, displays, graphics accelerators, and TV encoders and decoders. To support all these devices, the IPU has a synchronous interface and an asynchronous interface. The synchronous interface is for transfer of display data in synchronization with the screen refresh cycle. This interface is for memory-less displays and TV encoders, and it also transfers video to smart displays that have a video port. The asynchronous interface is for random read/write access to the memory and registers of smart displays and graphics accelerators. The data bus is 18 bits wide (or less), and it can transfer pixels of up to 24-bit color depth.

The interface with cameras and TV decoders is much more systematic than the interface with displays and requires much less flexibility. The interface receives one data sample per bus cycle, with 8 to 16 bits per sample. There is one exception, a nibble mode, in which 8-bit samples are received through a 4-bit bus, each during two cycles. Synchronization signals (Vsync, Hsync) are either embedded in the data stream, following the BT.656 protocol, or transferred through dedicated pins. The main pixel formats are YUV (4:4:4 or 4:2:2) and RGB. Any other format, such as Bayer or JPEG, can be received as generic data, which is transferred without modification, to the system memory. B

ecau

se o

f an

orde

r fro

m th

e U

nite

d S

tate

s In

tern

atio

nal T

rade

Com

mis

sion

, BG

A-p

acka

ged

prod

uct l

ines

and

par

t num

bers

indi

cate

d he

re c

urre

ntly

ar

e no

t ava

ilabl

e fro

m F

rees

cale

for i

mpo

rt or

sal

e in

the

Uni

ted

Sta

tes

prio

r to

Sep

tem

ber 2

010:

i.M

X31

Pro

duct

Fam

ily, i

.MX

31L

Pro

duct

Fam

ily

11

IPU

Sensor Port

VideoProcessing

AHB Master

PortSystem Memory

Display Port

IPU

AHB Slave Port ARM11

Interface to:smart image sensorsraw image sensorscamera flash support

Interface to:• a smart/memory-less display• a TV encoder• a graphics accelerator

• Deblocking and deringing• Resizing• Color conversion• Combining with graphics• Inversion and rotation

IP Port

Synchronization & Control

The IPU is equipped with powerful control and synchronization capabilities to perform its tasks with minimal involvement of the ARM CPU. The integrated DMA controller (with two AHB master ports) allows autonomous access to system memory. An integrated display controller performs screen refresh of memory-less displays.A page-flip double buffering mechanism synchronizes read and write accesses to the system memory to avoid tearing. The IPU also offers internal synchronization.

Here you can see the layout of the IPU. The sensor port provides interface to smart image sensors, raw image sensors, and camera flash support. Video processing provides deblocking and deringing, resizing, color conversion, combining with graphics, and inversion and rotation. The display port provides interface to a smart/memory-less display, a TV encoder, and a graphics accelerator.

With the ARM platform powered down, the IPU performs the following activities completely autonomously: screen refresh of a memory-less display, periodic update of the display buffer in a smart display, and display of a viewfinder window. When the system is idle, the user may want to display on the screen a changing image such as an animation or a running message. In i.MX31, this can be performed automatically. The CPU stores in system memory all the data to be displayed, and the IPU performs the periodic display update without further CPU intervention.

Integration, combined with internal synchronization, avoids unnecessary access to system memory, so it reduces the load on the memory bus and power consumption. In particular, input from a smart sensor (in YUV or RGB pixel formats) can be processed on the fly before being stored in system memory, and output to a smart display can be processed on the fly while being read from system memory. In some cases, input from a sensor can be sent directly to a display without passing through system memory at all.

The integrative approach enables efficient hardware design in which the hardware is reused whenever possible for different applications. For example, the DMA controller is used for video capture, image processing and data transfer to display. In addition, the image conversion hardware is used both for captured video (from camera) and for video playback (from memory).

Bec

ause

of a

n or

der f

rom

the

Uni

ted

Sta

tes

Inte

rnat

iona

l Tra

de C

omm

issi

on, B

GA

-pac

kage

d pr

oduc

t lin

es a

nd p

art n

umbe

rs in

dica

ted

here

cur

rent

ly

are

not a

vaila

ble

from

Fre

esca

le fo

r im

port

or s

ale

in th

e U

nite

d S

tate

s pr

ior t

o S

epte

mbe

r 201

0: i.

MX

31 P

rodu

ct F

amily

, i.M

X31

L P

rodu

ct F

amily

12

Question

A

B

C

Label the components in the IPU diagram below to show that you recognize the function of each. Drag the letters from the left to the corresponding positions on the right. Click “Done” when you are finished.

Sensor Port

VideoProcessing

AHBMasterPort

Display Port

IPU

AHB Slave Port

Interface to smart image sensors, raw image sensors, camera flash support

Interface to a smart/memory-less display, a TV encoder, a graphics accelerator

Autonomous access to system memory

IP Port

Synchronization & Control

A

C

B

Let’s review the functions of the components of the IPU.

Correct.

The sensor port is the interface to smart image sensors, raw image sensors, and camera flash support. The two AHB Master Ports are for autonomous access to system memory, and the display port is the interface to a smart/memory-less display, a TV encoder, and a graphics accelerator.

Bec

ause

of a

n or

der f

rom

the

Uni

ted

Sta

tes

Inte

rnat

iona

l Tra

de C

omm

issi

on, B

GA

-pac

kage

d pr

oduc

t lin

es a

nd p

art n

umbe

rs in

dica

ted

here

cur

rent

ly

are

not a

vaila

ble

from

Fre

esca

le fo

r im

port

or s

ale

in th

e U

nite

d S

tate

s pr

ior t

o S

epte

mbe

r 201

0: i.

MX

31 P

rodu

ct F

amily

, i.M

X31

L P

rodu

ct F

amily

13

MPEG-4 Encoding in Hardware

Memory

Camera

IPU

MPEG-4 Encoder

EMIi.MX31

Display

DisplayDouble Buffer

GraphicsOverlay

Video InputDouble Buffer

ReferenceFrame Buffer

MPEG-4 Stream

VLC-EncodedFrame

Encoder Processing:•Motion estimation, DCT& quantization•Inverse quantization, IDCT& motion compensation•Scan, run-length coding& Huffman coding•Rate control

ARM Processing:MPEG-4 stream forming

IPU Processing:•For compression:de-interleaving•For display (viewfinder):color conversion,combining with graphics•For both (independently):resizing, inversion, rotation

ARM11 CPU

Here you can see how data flows for video capturing using MPEG-4 encoding. IPU processing takes care of de-interleaving for compression; color conversion and combining with graphics for display (viewfinder); and resizing, inversion, and rotation for both compression and display (independently). Next, the encoder processes motion estimation, discrete cosine transform (DCT) and quantization, inverse quantization, inverse DCT (IDCT) and motion compensation, scan, run-length coding and Huffman coding, and rate control. Finally, the ARM takes care of MPEG-4 stream forming.

The video encoding hardware accelerator of the i.MX31 processor supports MPEG-4 SP (all levels) and H.263 baseline and enables pixel rates up to VGA at 30 fps and compressed bit rate up to 4 Mbps. This adds up to 1300 MHz of equivalent ARM11 performance. Two methods can detect that the encoding of one frame is finished: either poll the register 1 or catch the interrupt signal (IP Indigo IF).

The VGA MPEG-4 encoder in the i.MX31 has motion estimation capabilities with a motion vector length up to 32 pixels. VGA MPEG-4 encoding also includes error resilience tools as defined in the MPEG-4 standard. Additional features of the VGA MPEG-4 encoder include pre-processing for picture smoothing using a low-pass filter and camera movement stabilization, both of which are patented technologies.

Bec

ause

of a

n or

der f

rom

the

Uni

ted

Sta

tes

Inte

rnat

iona

l Tra

de C

omm

issi

on, B

GA

-pac

kage

d pr

oduc

t lin

es a

nd p

art n

umbe

rs in

dica

ted

here

cur

rent

ly

are

not a

vaila

ble

from

Fre

esca

le fo

r im

port

or s

ale

in th

e U

nite

d S

tate

s pr

ior t

o S

epte

mbe

r 201

0: i.

MX

31 P

rodu

ct F

amily

, i.M

X31

L P

rodu

ct F

amily

14

Video Playback: H.264

Memory

IPU

EMIi.MX

Display

DisplayDouble Buffer

GraphicsOverlay

Video OutputDouble Buffer

ReferenceFrame Buffer

H.264 Stream

IPU Processing:In-loop de-blocking,resizing, color conversion,combining with graphics,Inversion, rotation

ARM Processing:Decodingexcept in-loop deblocking

For in-loop deblocking

For post-processing

ARM11 CPU

Here you can see the data flow of video playback using H.264 decoding. ARM processing takes care of decoding except in-loop deblocking. IPU processing takes care of in-loop de-blocking, resizing, color conversion, combining with graphics, inversion, and rotation.

Bec

ause

of a

n or

der f

rom

the

Uni

ted

Sta

tes

Inte

rnat

iona

l Tra

de C

omm

issi

on, B

GA

-pac

kage

d pr

oduc

t lin

es a

nd p

art n

umbe

rs in

dica

ted

here

cur

rent

ly

are

not a

vaila

ble

from

Fre

esca

le fo

r im

port

or s

ale

in th

e U

nite

d S

tate

s pr

ior t

o S

epte

mbe

r 201

0: i.

MX

31 P

rodu

ct F

amily

, i.M

X31

L P

rodu

ct F

amily

15

Module Summary

• Imagination PowerVR MBX-Lite – High performance 3D graphics– Less power and bandwidth than traditional architectures

• Three graphic software APIs:– OpenGL ES– Direct3D Mobile– M3G/JSR184

• i.MX31 processor multimedia capabilities– Power-efficient image and video processing– Simultaneous MPEG-4 SP video encoding and decoding– Real-time video decode in advanced formats– Image capture of up to 30 megapixels per second

• IPU

In this module, you learned about the features and functions of the of the Imagination PowerVR MBX-Lite hardware core, which provides high performance 3D graphics for less power and bandwidth than many traditionally architected accelerators. You also learned about the three graphic software APIs: OpenGL ES, Direct3D Mobile, and M3G/JSR184. Next you examined the multimedia capabilities of the i.MX31 processor, which include power-efficient image and video processing, simultaneous MPEG-4 SP video encoding and decoding, real-time video decode in advanced formats, and image capture of up to 30 megapixels per second. Finally, you learned about the features of the IPU.

Bec

ause

of a

n or

der f

rom

the

Uni

ted

Sta

tes

Inte

rnat

iona

l Tra

de C

omm

issi

on, B

GA

-pac

kage

d pr

oduc

t lin

es a

nd p

art n

umbe

rs in

dica

ted

here

cur

rent

ly

are

not a

vaila

ble

from

Fre

esca

le fo

r im

port

or s

ale

in th

e U

nite

d S

tate

s pr

ior t

o S

epte

mbe

r 201

0: i.

MX

31 P

rodu

ct F

amily

, i.M

X31

L P

rodu

ct F

amily