Multimedia in Every Mobile

Peter Chiang

Nov, ‘04

A statement about the launching of SMedia

q Color mobile-phone occupy over 60% mobile-phone market in 2004

q Camera phone occupy over 50% of color mobile-phone in 2004q Multimedia become default function of mainstream handheld

device in 2005q Enhanced capability drives data bandwidth and carrier revenueq Multimedia offers handset differentiation opportunityq Consumers will pay for compelling games and contentq SMedia delivers key enabling technologies for multimedia

handheld devices

Founded: 2004 AugInitial Capital: USD 6 Million (approximately)Major Shareholder: UMCEmployees: 60 (approximately)Headquarters: Taiwan/ Hsin-ChuOperating Sites: S&M, Service: Taiwan/ Taipei

SMedia at a Glance

Core Competence

R&D- 100% own IP for core multimedia features- First full hardwired solution of low-power graphics, streaming,

imaging integration in the world- Initial/Endorse industry standard via participating these

committee/organization

Service and Support- Long-term relationship and timely support to customers with time-to-

market product support- Close relationship with alliance partners to synchronize in latest

technology development and industry specifications.

Imaging

VideoGraphics

Audio

uP

LowPower

Innovation

Integraion

Core Value

Integrated core technologies into real innovation

Deliver the valuable innovation to customers completely

Deep service

Customer

Satisfaction

>Real-time feedback >>Efficient logistics >

> Knowledge transfer>> Total solution >

Note:SiS GPU department spun off at Q2’03. So there is no correct statistic data after Q3’03. SiS GPU department combined Trident Inc. as a new company – XGi, which became the top 3 GPU suppliers, and SiS is still the biggest shareholder.

About SiS Multimedia Score

Desktop Graphics Market Share Q2'03

Intel35%ATI

13% Nvidia31%

SiS10%

Matrox1%

VIA/S39%

Others1%

Source: Jon Peddie ResearchDiscrete Desktop Market Share Q2'03

Matrox3%

ATI28%

SiS4%

Nvidia65%

IGC Desktop Market Share Q2'03

ATI<1%

Intel64%

SiS16%

VIA/S317%

Nvidia3%

Glamo 3360

Polyphonic

Smart message

Multimedia

Inte

ract

ive

Native 2D Gaming

Instant Messaging

SMS EMS MMS

E-mail

Video Conference

On-line Gaming

Video Broadcast

Native 3D Gaming

3D MMI

WAP Broadcast

Still CameraVideo

Camcorder

4 Development Directions for Feature Phone: Multimedia, Interactive, User-friendly, Personalization

Mobile Phone (Handheld device) Trend

Inverse Screen Size Law

The need for advanced graphicsSmaller screens need more advanced graphics processing per pixel

To make best use of restricted screen real-estate

Advanced graphics techniques provided by state-of-the-art APIsAnti-aliasing, multi-level compositing, rich media, 3D

Interactive3D Gaming Content

Video Processing and synchronization

Perspective windows maximize screen real-estate

Anti-aliased text to makesmall text extremely legible

Translucent Menus don’t obscure lower

layers

Asynchronous SRAM

VideoCapture,Image

Processor

JPEG,MPEG4Codec

2D/3DGraphics

LCDController

Host BusController

Memory Controller

LCDCOMSSensor

Host CPU

MMCController MMC

Asynchronous SRAM

VideoCapture,Image

Processor

JPEG,MPEG4Codec

2D/3DGraphics

LCDController

Host BusController

Memory Controller

LCDCCD/CMOSSensor

Host CPU

MMCController MMC

§ SMedia is integrating the interactive multimedia visual solution for handheld devices to enable new applications that enrich more enjoyment and quality to life.

SMedia Solution

Camera Interface

2D EngineLCD

Interface3D Engine

JPEG decode JPEG encode

MPEG decode MPEG encode

LCD

BBMemory

Power ManagementBattery

RF

Antenna

Melody &Audio Amplifier

(MP3/MIDI… )Speaker &

Vibrator

Lens Sensor

Stacked SRAM

Multimedia Mobile Phone Evolution

Possible Architecture 1

JPEG codec

MPEG4 codec

2D engine

3D engine

Cam

era sensor interface

LCD

interface

RAM

Lens SensorImaging DSP

YUV LCD

Mem

(CPU interface)

Traditional multimedia phone architecture

B.B.

JPEG codec

MPEG4 codec

2D engine

3D engine

CM

OS

sensor interface

LCD

interfaceLens Sensor

LCD

Newer multimedia phone architecture

Imaging DSP

Raw RGB

RGB

B.B.

Possible Architecture 2

RAM MemAdvantage:•Save total system cost, theoretically•From graphic view, it can perform much better quality

Disadvantage:•More bandwidth usage of stack pSRAM for LCD display

Advantage:•Save total system cost, theoretically

Disadvantage:•Need extra effort for calibration•Lack flexibility

Geometry performance with lighting§ Glamo 3D Hardware : minimum 100k

polygons/sec (15 times of the StrongARM200Mhz with 160Mhz FPU)

Rendering performance with texturing§ Glamo 3D Hardware : 5M pixels/sec (80 times

of the StrongARM 200Mhz with 160Mhz FPU)

For Performance

ISCAS 2002 IEEE International Symposium on Circuits and Systems Paper : 2490

Reference data:

a) QVGA 320x240x2 overdraw x30fps/(5x5/2pixels/triangle)=369Kpoly/s

b) QVGA 320x240x2 overdraw x30fps=4.6Mpixels/s

Max 1280k polygons/sec

Max 33M pixels/sec

Why H/W Graphic Accelerator

Why H/W Graphic AcceleratorFor Power Consumption

§ Software solution (executed by CPU)§ Define every picture§ Clean & Redraw every movement on display§ Execute complex instructions to compute simple movement§ Saturation accessing memory

§ Hardware solution§ Only define some components:

§ Object oriented operation to execute a simple instruction to do complex action

Comparison of Performance & PowerPerformance: 4.6 times speed (200x200pixel BLTs x 30 times)

25ms 115ms

PDA with PXA250 + Graphic Accelerator

PDA with PXA250 only

Only initial (around 10ms) need higher power

Power consumption: 85% less power (see cyan-blue portion)

Outstanding MPEG4 CodecFull hard-wired solution à better performance + less power consumption

§ vs E company’s half H/W + half SW

Full support MPEG4 simple profile à better quality & efficiency§ Based on the VM 18 & MPEG-4 reference code:

§ A modified mode decision algorithm for deciding the MB mode. Can achieve over 0.2dB improvement for complex sequence and 1.0dB in scene change.

§ A new rate control algorithm can remain high quality when scene changed.§ Full search in search range

§ I & P frame support§ P frame will have over 5X performance/quality than I frame

§ Special “Scaling-down”process as a partial low-pass filter to reduce the blocking & ringing

§ Rate-control support to meet HVS(Human Visual System) behavior§ Human eyes are more sensitive on low-complexity MB(Macro Block), rate –control

can reduce blocking impact§ vs A company’s H.263 only à Just one of MPEG-4 mode (short-head mode)§ vs A company’s “No rate control”§ vs S & A companies’I frame support only§ vs S & A companies’blocking & ringing result

Ringing & Blocking Example

Why H/W Camera/CamcoderDramatics Power Saving: around 10X~

0

50

100

150

200

250

300

350

400

450

MPEG4 Encode(CIF@30fps)

JPEG Encode(Continous encode

1.3M)

Preview(1.3M@15fps; 60fpsLCD update)

ARM9 core

Glamo core

Technology of Power-saving§ Clock§ Low power PLL for 32KHz clock input (standard RTC clock for the standby usage

of mobile-phone)

§ Active Graphics§ Drawing and rendering more power efficient on dedicated hardware than on

Baseband§ SMedia solution spends less time drawing power by her decades of graphic

experience(SiS experience)

§ Image capture & video recording§ Page-mode processing to minimize memory accessing times which is the major

power consume

§ Screen Refresh§ Display refresh isolated from system & Baseband§ Partial Display make the efficient power saving for whole system

§ ASIC Design for Power§ Stack pSRAM, not SDRAM§ Auto switch-off non-active function§ Low-power process

Average Power can be < 1mW per 1MHz

Glamo Measured Core Power Summary

Best Solution (1)§ Economical system solution§ Add ISP to reduce camera module cost – save around $0.5 for VGA, around $0.5~ $1

for 1.3M model§ Add LCD controller w/ frame buffer to reduce LCM cost – save around $4+ for 176x220

model; save around $10+ for 240x320 model

§ Short developing period§ Strong FAE support with completed documents§ Sample code supply for application support§ Strategy alliance with Game/Middleware/LCD/CMOS/BB vendors to give a total

solution choice

§ Small power-consumption system solution§ With power-saving design and process (~85% less power than one processor solution)§ The most advanced process support by UMC group

§ Follow main stream standard§ Compatible with most of I/F of BB, LCD, and CMOS/CCD sensor§ Follow 3GPP video stream standard§ Follow Java & OpenGL_ES graphics standard

Best Solution (2)§ Fancy features§ Full 2D & 3D graphic acceleration§ Newest MPEG4 (up to CIF 30fps), JPEG codec for imaging & video application§ Fancy imaging processing of digital camera – 16X Continuous Zoom, Flash

synchronization control, Emboss/Sepia/Mono/Negative/Color-Filer effect

§ Best performance§ Best performance of 2D/3D (80X than strong-ARM simulation)§ Full hardwired architecture make the most efficient imaging processing§ Full hardware MPEG4 acceleration make the fastest video compression

§ Flexible architecture§ Support most of CMOS sensor w/ programmable compensation àYUV & Bayer

Pattern input§ Support most of TFT LCD à CPU I/F, RGB I/F§ Brightness Statistics Table output to reserve the flexibility for manufacturers to

make own AE/AF…§ Support the rotation/mirror of Camera & LCD & 2D object to make your design

more varied§ Support extra Java H/W accelerator SIP solution

Strategy Partner

GlamocoProcessor

Driver

SMedia API

CMOS/CCD

S companyO companyB companyP companyA companyM(F) company Baseband

I companyM companyA company

LCD

S companyS2 companyC companyT companyA companyS3 company

SRAM

S companyN companyS2 company

Graphics AP

U company

M companyT company

OS

GameG companyB companyI company

I companyH companyX companyF companyS company

Middleware & authoring tool

Java accelerator

N company

Features & Specification

Feature List (1)§ Host Interface & Clock

§ Support 16bits SRAM-like bus§ Support most of B.B./CPU chip (16 bit 80/68 series)§ Input Clock: 32.768KHz or 13MHz

§ JPEG Engine § Fully Compliant with JPEG ISO/IEC 10918 Baseline standard§ JPEG encoding and decoding

§ Display Interface§ Display Resolution: Max. 512x512 75Hz 16/18bpp (RGB565/RB666)§ Support CPU I/F, RGB I/F, LCD I/F§ Support C-STN and TFT of most of LCD module venders§ Display 90o/180o/270o Rotate§ Support 8/9/16/18 bit Output Interface§ Gamma correction§ Partial Display at Power Saving Mode. § (LCM should support)

Feature List (2)

§ MPEG-4 Engine§ Full compliance with ISO/IEC 1446-2 (MPEG4) video simple profile level 0,1,2,3§ Full H/W acceleration § MPEG4/H.263 Encoder and Decoder§ CIF/QCIF 30 FPS§ Provide audio and video synchronization scheme

§ High Performance 2D Accelerator§ Full compliant with J2ME MIDP 2.0 2D requirement§ Built-in an 1T pipelined 16bit BitBlt graphics engine

§ ROP3, Rectangle fill, Font expansion, Line-drawing§ Transparent BitBlt with source and destination keys§ Alpha blended Bitblt, Clipping

§ Support maximum fill rate: minimum 33M pixels per second

Feature List (3)

§ High Performance 3D Accelerator§ Full compliance with OpenGL ES v 1.0 and OpenGL ES v 1.1(proposal7)§ Full compliance with JSR-000184 Mobile 3D Graphics API for J2METM

§ Built-in pipelined 3D graphics engine § Nearest/Bi-linear Perspective Corrected Texture Mapping§ Texture transparency, blending, wrapping, mirroring and clamping§ 1/2/4/8/16-bit texture format§ Support palette texture and compressed texture§ 16-bit Z-buffer, triangle setup engine § Fog, alpha blending and specula effects§ Alpha blended Bitblt, Clipping, dithering

§ Extra 8 lighting support§ Support maximum fill rate: 33M pixels per second§ Support polygon rate: 150~1280K polygons per second

Feature List (4)§ Video Interface

§ Max. Image Resolution : 3 M§ Sensor Interface supports : programmable image size

§ up to 3 Million-pixel CCD/CMOS sensors.

§ Supports Input Formats§ Bayer (Raw Data)§ YUV 4:2:2

§ Gamma Correction by programmable look-up table.§ Lens Shading Correction.§ Bad Pixel Compensation.§ Black Level Compensation.§ Support Flashlight function.§ Preview/MPEG Frame Rate Control.§ Independent Size Scale-down of Vertical or Horizontal direction.§ Support 180o Rotate & Mirror

§ Software Support§ Nucleus, Smartphone, Linux, Symbian

§ Process and Package§ UMC 0.15um low power process§ 3.3~1.8V I/O VDD and 1.5V core VDD supply§ BGA 8mm x 8mm 140 ball 0.6mm ball-pitch

S/W Architecture

Game-related module

Hardware (Processor + Glamo coProcessor)

RTOS Core SMedia Driver SuitSMedia Driver Suit

OS-related Graphics API SMedia 2D/3D API (OpenGL ES)

KVM (CLDC 1.1)

MIDP 2.0 JSR184

3D Graphics Middleware

2D/3D Native Application (C/C++)2D/3D Midlet

ES

CS

MPGlamo Product Roadmap

First customer sample- 3360 8/E/2004

Nucleus driver 9/M/2004

Application-note release 9/M/2004

EV kit (1) 9/M/2004

SDK (Drivers, APIs, Tools, Sample programs) 9/E/2004

EV kit (2) 10/E/2004

Smartphone 2003 driver (1) 10/E/2004

Smartphone 2003 driver (2) 11/E/2004

Linux driver 12/M/2004

Mass production – 3360 12/M/2004

Symbian driver 2/E/2005

Glamo 3360 Schedule

3360J : CS: 12/E MP: 2/E, 20053260 : CS: 12/M MP: 1/E, 20056330 : CS: 12/E MP: 2/M, 2005

Thank you