Android UI Subsystem Design

Bin (Pierr) Chenhttp://pierrchen.blogspot.com

Pattern’s Perspective

Android View System

Libgui

SufaceFlinger

HWC EGL

ComponentsApps

3D

Perspective

How Why

Combined

Libgui

SufaceFlinger+ How/Why

Pattern

Design Patternis a general reusable solution to acommonly occurringproblem

Observer/Publish-Subscribe

UML

A oo way of saying callback in c.

Examples You subscribe to an RSS to get notification for new article Controller subscribe to View for Event

MediaPlayer notify user the status of current player ConsumerBase subscribe to BufferQueue for enqueue

event SurfaceFlinger subscribe to HWC for hotplug/vsync

In China, we need a ________ to visit YouTube.

Proxy - UML

Examples Sp implementation - Smart Reference Binder - Remote Proxy

Bp/Bn

Bp/Bn Example

Sushi shop

Simplest form

put getproducer consumer

Not that simple, Actually

put/get usually in different thread->Need synchronization->Need define wait/wake policy for empty/full

Evolution Evolves

(Resource)Queue Evolves Producer Evolves Consumer

Questions? Isn’t put and get enough to describe the model? Why we need dequeue? is dequeue an consumer action or producer

action? dequeue acquire

enqueue release

Sushi shop, example

Each Plate cost $1000 So for each customer they only provide you

3 Plate You have to return the used Plate to the trail

so that servant can make next sushi.

+Resource Pool

dequeue acquire

enqueue release

Resource State

free dequeued

queuedacquired

dequeue

queue

cancel

acquire

release

Participants店员 - Producer - SurfaceTextureClient/ANativeWindow顾客 - Consumer - SurfaceTexture/FramebufferService 转盘 - Resource Pool - BufferQueue盘子 - Resource - Graphic Buffer寿司 - Resource Content - Content of Graphic Buffer

When to consume? +Observer

dequeue acquire

enqueue release

Observer

Scheduled consumer

dequeue acquire

enqueue release

Observer SchedulerMessage

trigger

Producer in another Process+ Proxy

BQdequeue acquire

enqueue release

Observer SchedulerMessage

Proxy

trigger

+Sync-er

When a dequeue a resource but actually you can not use it immediately. You have to wait until that resource is free.

Then, why not dequeue block?

BQdequeue acquire

enqueue release

Observer SchedulerMessage

Proxy

trigger

sync-er

Sync-er

SurfaceFlingerVSYNC

BQdequeue acquire

enqueue release

Observer SchedulerMessage

Proxy

surfaceTextureClient

BufferQueue

SurfaceTexture SF main threadtrigger

fence

new fence

trigger

Surface INVALIDATE1. SurfaceTexturClient::queue -> BufferQueeu::QueueBuffer - >

2. SurfaceTexture got notified -> Layer::onFrameQueuedLayer-> SignalLayerUpdate ->

3. requestNextVsync

4. VSYNC come -> dispatchMessageInvalidate

5. SF main thread -> handleMessageInvalidate -> acquireBuffer->createIMGKHR-> postMessageRefresh

1.Producer thru proxy

2.Listener get notified

3.Schedule Consumer

4.It is time to consume! (thru ActiveObject)

5.Consume

Connecting Producer/ConsumerYou can understand Compositor

SurfaceTexture

SurfaceTexture

FrameBufferSurface

REFRASH happened after INVALIDATE

REFRESH1. Draw each layer’s latest onscreen buffer as Texture to FBS -

prepare

2. eglSwapBuffer for FBS1. EGL queue current buffer - produce2. dequeue next buffer of FBS

3. FBS’s listener update fb handle - listener notified

4. HWC commit will display the updated fb handle - consume

INVALIDATE and REFRESH

More

Thread Pool

Goal:Increase throughput and Improve responsiveness

ServerSocket socket = new ServerSocket();

While(true){

Socket con = socket.accept();

handleRequest(con);

}

Sequential execution Thread– Per -Task

service

clientclient

clientclient

clientclient

Thread pool Pattern is where a number of threads are

created to perform tasks submitted to task queue

client

Task submission Task execution

Submit task

retrieve result

Let’s return to Producer/Consumer a while…

Task Queue is a Producer/Consumer , multiply Consumer, one or more Producer

BufferQueue is really a Resource Pool

Binder Server

Those Binder Thread will submit the task to SurfaceFlinger main

thread.

Which is Achieved though….

Active Object

The active object design pattern

decouples

method execution from

method invocation

for objects that each reside in their own thread of

control.

Why not Passive Object

Object called in the client thread, result in more complex synchronization

Schedule/MessageQueue sequential the request to servant object

Active Object:Participants

SF main thread Message queue

Client

postMessage

surfaceFlinger

Active Object:Used in SurfaceFlinger

How message are submitted from client thread to server thread?

Command PatternPurpose:Decouple the Invoker and the ReceiverThe command can be queue, transferred, stored and/or execute at different time.

Command Pattern

SF in Action

SF main thread

SF MessageCreateSurfaceClient

Combined:

Thread Pool

+

Command

+

Active Object

+

Proxy

sp<ISurface> Client::createSurface( ISurfaceComposerClient::surface_data_t* params, const String8& name, uint32_t w, uint32_t h, PixelFormat format, uint32_t flags){ class MessageCreateLayer : public MessageBase { public: MessageCreateLayer(SurfaceFlinger* flinger, ISurfaceComposerClient::surface_data_t* params, const String8& name, Client* client, uint32_t w, uint32_t h, PixelFormat format, uint32_t flags) : flinger(flinger), params(params), client(client), name(name), w(w), h(h), format(format), flags(flags) { } sp<ISurface> getResult() const { return result; } virtual bool handler() { result = flinger->createLayer(params, name, client, w, h, format, flags); return true; } };  sp<MessageBase> msg = new MessageCreateLayer(mFlinger.get(), params, name, this, w, h, format, flags); mFlinger->postMessageSync(msg);

return static_cast<MessageCreateLayer*>( msg.get() )->getResult();} 

Architecture

Integrated,

Connected Patterns

Recap Observer Proxy Producer/Consumer Thread Pool Command Pattern Active Object

BnSurfaceTexture

SF Main Thread

BufferQueue

SurfaceTextureClient

FBSBufferQueue

invalidate

invalidate

refresh

SurfaceComposerClient Client

SurfaceFlinger

Proxy Thread Pool

Producer/Consumer

Command

Active Object

Observer

Android View System

Libgui

SufaceFlinger

HWC EGL

Apps

3D

Android UI Framework: View ,Usage & Patterns

Android App Development 101

Android SurfaceFlinger: Architecture explained by User case

Android UI:Patterns Perspective

Other Topics

Android Graphic Integration