Upload
suniltomar04
View
602
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Citation preview
1
LinuxDevelopment Centre
Displays in Mobile Devices
OSI Days 2011, Bangalore
Archit Taneja ([email protected])
Sumit Semwal ([email protected])
2
LinuxDevelopment Centre
Agenda• Historical look at mobile devices
– Non-phone mobile devices• Types of display panels• Display Subsystems
– In PC World
– SoC World
– PC vs SoC
– Example Data Flow• Mobile Display Interface Standards• Interesting Display Use Cases• Example of SoC DSS:
– OMAP2+ Display Subsystem
– Current software design – DSS2 framework• Linux Kernel Display Frameworks• What Next?• Q & A
LinuxDevelopment Centre
Under CC License from http://xkcd.com/732/
Nokia 1011First mass producedGSM phoneMonochrome [1]
One of the first color screen phonesSony Ericsson T68i101x80, 256color[2]
All trademarks and copyrights are properties of and attributed to their respective companiesSources: in references
Google Nexus OneAMOLED: 480x800, 3.7”
Google Galaxy NexusSuper AMOLED plus720x1280; 4.65”
Apple iPhone 4“Retina” display960 x 640; 3.5” [5]
LG Optimus 3D3D Stereoscopic LCD; 800x480; 4.3” [4]
Nokia 9000 Communicator: First 'business phone'Monochrome display, Wide640x200: [3]
A Historical Look at Mobile Displays
LinuxDevelopment Centre
Device Positioning Screen size
Resolution Technology
Samsung Galaxy Note
Tablet Smartphone
5.3” 1280 x 800 HD sAMOLED
Dell PhoneTablet
5”7”10.1”
800 x 480800 x 4801366 x 768
TFT LCDCapacitativeTFT
Amazon Kindle 3
eBook Reader 6” 600 x 800 eInk, 16 level grayscale
B&N Nook eBook Reader 6” 800 x 600 eInk, 16 level grayscale
Archos PMPTabletTablet
7”8”10.1”
800 x 4801024 x 7681024 x 600
TFT
Apple iPad Tablet 9.7” 1024 x 768 IPS LED
Samsung Galaxy Tab
Tablet 7”8.9”10.1”
1024 x 6001280 x 8001280 x 800
TFT
'non-phone' Mobile Displays
All trademarks and copyrights are properties of and attributed to their respective companiesSources: in references
Only an indicative list; certainly not exhaustive!
LinuxDevelopment Centre
All trademarks and copyrights are properties of and attributed to their respective companiesSources: in references
Types of Display Panels• Retina – Apple iPhone 4
– In Plane Switching (IPS) LCD– Highest density: 326 ppi (3.5” screen, 960x640)
• AMOLED (Active Matrix Organic LED)– No Backlight needed
• Higher contrast ratios, more power efficient.– Pen Tile arrangement – RGBG arrangement of sub-pixels, where G sub-
pixel is shared by 2 pixels. So the display has only 2 sub-pixels per real pixel.
• Super AMOLED– Same as AMOLED, but touchscreen is integrated onto the same glass, thus
increasing visibility on the screen.
LinuxDevelopment Centre
All trademarks and copyrights are properties of and attributed to their respective companiesSources: in references
Types of Display Panels• Super AMOLED Plus
– Integrated touchscreen, and no Pen tile arrangement – much brighter, thinner and more efficient.
– regular RGB, so 50% more subtiles, hence lesser resolution.
• eInk Panel– Based on actual electronic ink that is coated on some ultra-thin plastic.– eInk contains microcapsules filled with ink, and pigmented, charged chips.– Each microcapsule acts as a pixel; when an electric charge is applied, either
the light or dark pigmented chip will 'come on top', thus 'switching' the pixel on or off.
LinuxDevelopment Centre
Display Subsystems: PC World
GPU Display I/F
Display(s)
HDMI
Display Port
LVDS
Analog Video
Video Decoders
Display Port
2D/3D Accelerators
Overlay /Blending
Scene informationor
Encoded frames
Interconnect
Shared/Dedicated Memory
Pixel Gen 1
Pixel Gen 2
Pixel Gen M
Display Subsystem
LinuxDevelopment Centre
Generic SoC Hardware Blocks
interconnect
MPU GPUVideo
AcceleratorDisplay Sub
System
DSP Imaging OtherPeripherals
EMIF
Display(s)
MemoryOnly display related blocks are shown; there are many other blocks too, which are out of scope for this discussion.
LinuxDevelopment Centre
Display Subsystem in SoCs
Plane 1
Plane 2
Plane 3
Plane N
Compositor 1 &
Pixel Gen
Serial Interface
(MIPI DSI)
HDMI/Display Port
GPU composed framesor
Decoded Video frames
Interconnect
Overlay
Parallel Interface
(MIPI DPI)
Remote framebuffer
Interface(MIPI DBI)
Display I/F
Display(s)
Display Subsystem
Compositor 2 &
Pixel Gen
Compositor M &
Pixel Gen
LinuxDevelopment Centre
Display Subsystems: PC vs SoC• Differences:
– PC: GPUs are generally responsible for composing different windows/layers to get one single frame. Display Subsystem only fetches and display this composed frame .
– SoC: Display Subsystem is 'also' responsible for composing layers generated by the GPU and Video accelerators.
– PC: Display Subsystem may only fetch frame data from the dedicated memory for the graphics card/controller.
– SoC: Display Subsystem fetches frame data from the main memory connected to the SoC.
• Reasons:– Legacy: GPUs and Video Accelerators have existed in SoCs as separate entities for a while. Making it
important for the Display Subsystems to compose the output of each into one frame.
– Area: SoCs (and mobile devices) can't afford dedicated memory for GPUs.
– IP reuse: Different Vendors of GPUs and Display Subsytems in SoCs, easier to interface with a standard interconnect.
– Power: Overlaying different layers is more optimal for SoCs in some cases.
LinuxDevelopment Centre
Socinterconnect
MPU GPU Video Accelerator
Display SubSystem
Display
Memory
Only display related blocks are shown; there are many other blocks too, which are out of scope for this discussion.
Example data flow
1. Commands to draw UI
1. Video frames to decode
2.Done
2. Done
2.2.
3. Display frame on panel
3.
Data path
Control path
LinuxDevelopment Centre
MIPI- Display Working Group
Primary (fixed) panels: - Display Serial Interface (DSI):
- serializes all pixel data, commands, and events.- low power(SLVS)- Works with Smart / Dumb Panels.
- Display Parallel Interface (DPI):- Legacy parallel interface.- Lots of lanes needed for pixel data and control events.- used with Dumb Panels.
- Display Bus Interface (DBI):- Parallel Interface for smart panels.
Pluggable panels:- HDMI- Display Port- SDTV / Composite- DVI
All trademarks and copyrights are properties of and attributed to their respective companiesSources: in references
Mobile Display Interface Standards
14
LinuxDevelopment Centre
Interesting Display Use cases• Device Dock
• Dynamic configurability, parallel usage– Switch output paths, using multiple paths simultaneously
• Handle audio interfacing additionally– HDMI support
– DSI support in future
• Support for 'virtual' pipelines – when we run out of video pipelines– Co-use of 2D/3D graphics accelerator.
• Multi-display configurations– Clone, Virtual.
● Display security
15
LinuxDevelopment Centre
OMAP2+ DSS hardware: Overview
RFBI
DSI2 encoder
VENC HDMI
RFBI
SDTV
SecondaryDSI
Primary DSI
DisplayPanels
DPI
GFX
VID1
VID2
VID3
DSI1 encoder
DPI
HDMI encoder
TVmgr
LCD1mgr
TVmgr
LCD2mgr
DISPC
DisplayInterfaces
DSS hardware
OMAP2,3,4OMAP2,3,4OMAP3,4OMAP3,4OMAP4OMAP4
● Gfx, vid1-vid3 are overlays, which color-convert, scale each frame
● Managers can compose, overlay multiple streams into one.
●Not all paths are shown: All overlays connect to all overlay managers.
16
LinuxDevelopment Centre
Current software design – DSS2 framework
UI / GraphicsUserspace appUI / Graphics
Userspace appVideo playbackUserspace appVideo playbackUserspace app
sysfs/dev/fb0/dev/fb1
/dev/video1/dev/video2
Framebuffer APIs V4L2 APIs
Control
omapfb driver omap_vout driver
DSS2 DSS2 gfx vid1 vid2 vid3
lcd1-mgr lcd2-mgr tv-mgr
HDMI TV LCD panelsDisplayhardware
Kernel space
User space
OMAP DSS hardware
Paneldrivers
LinuxDevelopment Centre
Linux Kernel Display Frameworks• Frame Buffer
– Oldest of them all - first public release in 1999.
– Linux Kernel Abstraction for graphics hardware – represents framebuffer of some video hardware, so the software doesn't need to know the low-level (hardware) stuff.
– allows emulate text console without hardware text-mode support.
– is a possible graphic output method for X server; independent of video adapter hardware and drivers.
– allows 'direct' graphics programs without X; popular in embedded world [GTK+, Qt Extended are some examples]
– No inherent window or buffer management
• DirectFB [6]
– Specifically designed for embedded systems.
– Is a thin library providing hardware graphics acceleration.
– Integrated windowing system with support for translucent windows and multiple display layers.
– Input device handling.
– Is a complete hardware abstraction layer, with software fallbacks to every graphics operation not supported by an underlying hardware.
– Provides for X-less graphical systems: gtk+, LinuxTV, HP WebOS to name a few.
LinuxDevelopment Centre
Linux Kernel Display Frameworks• Video For Linux (V4L)
– Kernel interface for streaming capture and output drivers.
– Provides framework for buffer management and streaming – very relevant for streaming kind of use-cases [video record, playback, TV, even analog radio! :)]
– Buffers are tied to each device, and allocated before streaming starts.
• Direct Rendering Manager (DRM)– The modern Linux (and *BSD) display driver framework
– The kernel component of DRI (Direct Rendering Infrastructure)
– But not just DRI:• Hotplug• XRandR/KMS• EDID parsing
– Kernel Mode Setting (KMS)• Setting screen resolution/timings• Hotplug, and retrieving EDID• Multi-display configurations
– Buffers are independent of overlays
– Security inherent
LinuxDevelopment Centre
What Next?
- HDMI 3D
- eInk Color (Triton) [7]
20
LinuxDevelopment Centre
Q and A?
LinuxDevelopment Centre
References3430 public Technical Reference Manual
4430 public Technical Reference Manual
[1]: http://en.wikipedia.org/wiki/List_of_Nokia_products
[2]: http://www.mobiles.co.uk/mobile-phone-history-uk.html
[3]: http://royal.pingdom.com/2010/02/08/15-fantastic-firsts-on-the-internet/
[4]: http://www.lg.com/in/mobile-phones/all-phones/LG-smart-phones-P920.jsp
[5]: http://gigaom.com
[6]: http://directfb.org
[7]: http://www.eink.com/display_products_triton.html
22
LinuxDevelopment Centre
Thank You!
23
LinuxDevelopment Centre
Backup
24
LinuxDevelopment Centre
Acronyms• SoC: System on Chip
• DSI: Display Serial Interface: a MIPI standard interface for serial display panels.
• DPI: Display Pixel Interface: a MIPI standard interface for parallel display panels.
• RFBI: Remote Frame Buffer Interface: a MIPI standard interface for parallel display panels supporting remote frame buffer mechanism.
• HDMI: High Definition Multimedia Interface: a standard compact AV interface to transmit uncompressed digital data.
• VENC: Video ENCoder: standard interface to display content on standard definition TVs, using TV formats like NTSC / PAL.
25
LinuxDevelopment Centre
Features of OMAP2+ DSS
OMAP3/4 support ARGB color formats.OMAP4 additionally has native support for NV12, configurable Z-order.
Overlay perspective
GFX overlay supports CLUT/gamma table, anti-flicker, replication logic – primarily meant for UI.
VID overlays support Multiple color formats(RGB, YUV, NV12 based on OMAP version), colorspace conversion, independent horizontal and vertical polyphase filter scaling, and VC-1 range mapping.
WB overlay, in mem-to-mem mode: colorspace conversion, rescaling, compositing processes.Capture mode: allows composited encoded pixel data to be stored back into memory.
Overlay perspectiveEach overlay supports: overlaying [fixed-order for OMAP2/3, z-order per pipeline for OMAP4], transparency color-keys [source and destination] and global and pixel alpha blending [OMAP3/4], Gamma correction, Sync buffers.
LCD overlays additionally support Active / Passive matrix dithering, color phase rotation.
* Read more about Virtual Rotated Frame Buffer (VRFB) and TILER in the 3430 public TRM and 4430 public TRM respectively
Rotation(per overlay)
Via VRFB* Via VRFB* Via TILER*
Feature OMAP2430 OMAP3430 OMAP4430
GFX overlay 1 1 1
VID overlay 2 2 3
WB overlay - - 1
LCD overlay manager
1 1 2
TV overlay manager
1 1 1