When I learn more about Android's graphics system, and do more work about how to use CPU/GPU in more parallelized way to improve the graphics performance in Android, I start to think that there are actually some big design mistakes in Android graphics system, especially the rendering architecture in the client side.Some mistakes have been solved after 3.x, especially above 4.1, but others can never be solved due to the compatible reason. As developers, we need to know how the Android graphics system work, how to utilize the new features Android 3.x and 4.x provided, and how to do the optimization and overcome the shortage of Android.
- 1.Why your Android Apps Suck (Roger)email@example.com/roger2yi
2. 6/20/2013 Roger, UCAbout Mehttp://weibo.com/roger2yi 3. 6/20/2013 Roger, UCWhy I wrote this article? When I learn more about Androids graphicssystem, and do more work about how to useCPU/GPU in more parallelized way toimprove the graphics performance in Android I start to think that there are actually some bigdesign mistakes in Android graphics system,especially the rendering architecture in theclient side 4. 6/20/2013 Roger, UC Some mistakes have been solved after 3.x,especially above 4.1, but others can never besolved due to the compatible reason As developers, we need to know how theAndroid graphics system work, how to utilize thenew features Android 3.x and 4.x provided, andhow to do the optimization and overcome theshortage of Android 5. 6/20/2013 Roger, UCDirect Rendering In the very beginning, Androids client/serverarchitecture of its graphics system choose thedirect rendering mode Android team develop a lightweight WindowCompositor - SurfaceFlinger by themself, itresponse to create native windows, allocategraphics buffers and composite the windowsurface to display (via framebuffer) 6. 6/20/2013 Roger, UC In client side, apps (through Android runtime)can use CPU to access the graphics buffer ofnative window directly (through Skia), or useGPU to manipulate it indirectly (through GLES) This direct rendeing mode is more appropriatethan the traditional X in desktop Linux 7. 6/20/2013 Roger, UC 8. 6/20/2013 Roger, UCWindow Composite in ServerSide SurfaceFlinger use GLES1 to do the windowcomposition (via texture operation) in the verybeginning, this cause two issues Chip may have more efficient way (DisplayController) to do the window composition thanuse GLES When the client side also use GLES to renderthe window surface, it may competes with theserver about the GPU resources Just mention, when GPU do not support GLES1, Android has a built-inSW version GLES1 stack, and it can use copybit HAL module providedby chip vendor to accelerated the texture operation 9. 6/20/2013 Roger, UCiOS do not needWindow Composite!!! It only has one visible window at one time, sothe app can render to framebuffer directly Advantages Save memory usage Reduce GPU usage Save memory bandwidth Disadvantages Only one visible app at screen Do not support 3rd SIP or lanucher forsecurity 10. 6/20/2013 Roger, UC So, after 3.0, Android introduce hwcomposerHAL module to solve these issues, also abandonthe old copybit module Chip vendor can through the implementation ofhwcomposer module to declare they can do allof or part of windows composition bythemselves, usually with dedicated hardwareand always more efficient than use GLES 11. 6/20/2013 Roger, UC When all windows composited by hwcomposer,and keep SurfaceFlinger idle Reduce GPU usage Save memory bandwidth Improve performance and save energy In theory, Android will has same performancewith iOS when it has only one visible window 12. 6/20/2013 Roger, UC Also, after 4.1, hwcomposer can provide thevsync signal of the display, so that Android cansync three things together the windows composition in server side the rendering of window surface in client side the refresh of display 13. 6/20/2013 Roger, UC 14. 6/20/2013 Roger, UCRendering Architecture in ClientSide The most mistake Android team make is therendering architecture of its GUI framework It do not has a layer rendering architecture (orcalled scene graph in some GUI fw) It do not has a dedicated rendering thread torender the window surface Its rendering only use CPU until 3.0 The first one is partially support after 3.0, thethird is support after 3.0, but the second problemcan never be solved... 15. 6/20/2013 Roger, UCCompare to iOS Every View has a Layer as its backing store, appcan create more Layers for better performance Views content will be drew into its Layer, aslong as the content do not changed, the View donot need to draw itself again iOS do a lot of things to avoid change thecontent of a View, many many properties of aView can be changed without affect to itscontent, such as background, border, opacity,position, transformation, even the geometry!!! 16. 6/20/2013 Roger, UC The composition of Layers done by anotherdedicated rendering thread, it always use GPUto draw each Layer to the window surface The main thread only reponse to handle touchevent, relayout the Views, draw Views contentinto its Layer when necessary, etc... So the main thread only use CPU and therendering thread use GPU mostly, and I thinkthere will be just a few synchronization betweenthese two threads, and they can run concurrentlyin most of time 17. 6/20/2013 Roger, UC But in Android, the main thread need to doeverything, such as handle touch events,relayout views, dequeue/enqueue graphicsbuffer, draw views content to window surface,etc... And it only use CPU before 3.0!!! Even the position of a View just change onepixel, Android need to redraw its content alongwith the contents of other Views overlapped, andthe content drawing is very expensive for CPU!!! 18. 6/20/2013 Roger, UCThe Improvements A lot improvements have been made after 3.0 toovercome the shortage of previous version Android 3.0 introduce a new hwui module, and itcan use GPU to accelerated the drawing ofViews content, it create a hw acceleratedCanvas to replace the old sw Canvas, the newCanvas use OpenGL ES as its backend insteadof use SkCanvas from Skia 19. 6/20/2013 Roger, UC Android 3.0 also introduce the DisplayListmechanism, DisplayList can be considered as a2D drawing commands buffer, every View hasits own DisplayList , and when its onDrawmethod called, all drawing commands issuethrough the input Canvas actually store into itsown DisplayList 20. 6/20/2013 Roger, UC When every DisplayList are ready, Android willdraw all the DisplayLists, it actually turn the 2Ddrawing commands into GLES commands touse GPU to render the window surface So the rendering of View Hierarchy actuallyseparated into two steps, generate ViewsDisplayList, and then draw the DisplayLists, andthe second one use GPU mostly 21. 6/20/2013 Roger, UC When app invalidate a View, Android need toregenerate its DisplayList, but the overlappedViews DisplayList can keep untouched Also, Android 4.1 introduce DisplayListproperties, DisplayList now can has manyproperties such as opacity, transformation, etc...,and the changed of some properties of View willjust cause changed of corresponding propertiesof its DisplayList and need not to regenerate it These two improvements can save some timesby avoid regenerate DisplayLists unnecessary 22. 6/20/2013 Roger, UC Although Android can never has a layerrendering architecture, it actually introduce someLayer support after 3.0, a View can has a Layeras its backing store The so called HW Layer actually back by a FBO,if the content of View is too complicated andunlikely to change in the future, use Layer mayhelp Also, when a View is animating (but content donot change), cache itself and its parent withLayers may also help 23. 6/20/2013 Roger, UC But use Layer with caution, because it increasethe memory usage, if your want to use Layersduring animation, your may need to releasethem when the animation is finish, Android 4.2provide new animation API to help you aboutthat Also, because Android use GLES to draw thecontent of View, so most Views drawing will befast enough, and the use of Layer may beunnecessary 24. 6/20/2013 Roger, UC Android 4.0 introduce a new type of native window -SurfaceTextureClient (back by a SurfaceTexture) and itsJava wrapper TextureView, app can create and own thiskind of native window and response to its composition If the content of View is too complicated and continue tochange, TextureView will be very helpful, app can useanother thread to generate the content of TextureViewand notify the main thread to update, and main threadcan use the TextureView as a normal texture and draw itdirectly on the window of current Activity 25. 6/20/2013 Roger, UC Android 4.1 make the touch event handling andui drawing sync with the vsync signal of display,it also use triple buffers to avoid block the mainthread too often because it need to wait theSurfaceFlinger to release the buffer, andSurfaceFlinger will always sync with vsync signalof display The OpenGL Renderer for hw acceleratedCanvas is continue be improved and becomefaster, especially for complicated shape drawing 26. 6/20/2013 Roger, UCBut... But Android can never has a dedicatedrendering thread... Although the drawing is much faster than before,and keep the changed of everything as little aspossible during animating, it still need to sharethe 16ms interval with other jobs in main threadto achieve 60 fps 27. 6/20/2013 Roger, UCSo... So, as developer, we need to utilize theimprovements in higher version Android asmuch as possible Turn on the GPU acceleration switch aboveAndroid 3.0 Use the new Animation Framework for youranimation Use Layer and TextureView when necessary etc... 28. 6/20/2013 Roger, UC And avoid to block the main thread as much as possible, that means If your handle the touch events too long, do it in another thread If your need to load a file from sdcard or read data fromdatabase, do it in another thread If your need to decode a bitmap, do it in another thread If your Views content is too complicated, use Layer, if it continueto change, use TextureView and render it in another thread Even your can use another standalone window (such asSurfaceView) as a overlay and render it in another thread etc... 29. 6/20/2013 Roger, UCGolden Rules for ButterGraphics Whatever you can do in another thread, then doit in another thread Whatever you must do in main thread, then do itfast Always profiling, it is your most dear friend 30. 6/20/2013 Roger, UCAll you need to do is keep the loopof main thread under 16ms interval,and every frame will be perfect! 31. 6/20/2013 Roger, UCThe Last Word When I finish this article, what make