Upload
hamilton-turner
View
1.819
Download
8
Embed Size (px)
Citation preview
CONFIDENTIAL. ALL RIGHTS RESERVED.Optio is a subsidiary of Allied Minds, an innovative U.S. science and technology development and commercialization company. Operating since 2006, Allied Minds forms, funds, manages and builds products and businesses based on innovative technologies developed at leading U.S. universities and federal research institutions. Allied Minds serves as a
diversified holding company that supports its businesses and product development with capital, central management and shared services. More information about the Boston-based company can be found at www.alliedminds.com.
Hamilton TurnerOct 2015
1
Fast-paced Introduction to Android Internals
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
$ whoami
2
@hamiltont@hamiltont
• Director of Malware Research, OptioLabs– Machine Learning, Data Science, Penetration
Testing, and Malware Prevention• Doctorate in Computer Engineering
– Mobile & Cloud Computing, Distributed Software Engineering, Security & Privacy
• Heavy contributor to open source software• Android framework hacker for ~4 years
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
$ who…are we
3
and now a brief message from our sponsors
OptioLabs– Mobile security company founded in 2011– Caters to Government, Enterprise, and Mobile Carriers/Manufacturers– Founders Dr. Charles Clancey, Dr. Jules White, Dr. Brian Dougherty– Offices in Baltimore & Nashville– Our OptioCore product increases Android security and enables legal compliance with
numerous regulations– Recent news:
• AMD and OptioLabs partner to increase enterprise security• Rugged Meets Secure: OptioCore Powers Sonim XP7
www.optiolabs.com
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
$ who…are we
4
We are hiring!
Mobile Developer (Application & Framework)Server Developer (Backend & Frontend)DevOps
Send resumes to our VP of Engineering
www.optiolabs.com
Andy [email protected]
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Slide Set EULA• Android is…..
– A capital-B Big project– A complex project– Changing rapidly– Different on each physical product
• This presentation is…..– not comprehensive– probably not fully correct– already outdated
• Go explore yourself!• Please let me know when you find something wrong with this slide deck
5
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
What does it mean to be ‘Android’?• Android is a (loosely) protected term!
– AOSP is all the open source code– Stock ROM is what manufacturer ships– Vanilla is very close to AOSP– Android is ROM that passed Android Compatibility
• Android Compatibility– Different for each version of Android– 3 main parts
• Compatibility Test Suite (automated tests)• Compatibility Verifier (human tests)• Compatibility Definition Doc (manual)
• Android + Google Mobile Services (GMS)– Google Play! Maps! YouTube! Gmail!– All these are not open source and require a license agreement with Google if you want to sell a device that comes with these
https://source.android.com/compatibility/index.html
6
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Is Android really Open Source?~90% of development is behind closed doors
– Only Google&partners see source code– Only Google&partners accept (or ignore/deny) patches
• Google is both the KeyMaster and the Architect– Very few public discussions occur on AOSP design principles– No public roadmap of features– However….
• Google accepts huge responsibility for standardization & long term stability of platform• Major contributions to upstream Linux kernel• Huge money influx for PR releases, OEM standardization efforts, etc
“Open source is different than a community-driven project. Android is light on the community-driven side and heavy on the open source. Everything we do ends up in the open source repository.” - Andy Rubin, Cofounder of Android Inc, SVP@Google until 2013
See https://source.android.com/source/code-lines.html#about-private-code-lines
7
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Super basic legality of Android• Pre-note: I am not a lawyer. This slide is extremely simplified for explanation purposes and there is therefore incorrect by
design• Google is very careful to minimize “threats” to parties interested in modifying AOSP
– Actively avoids using components that require ‘derived works’ to be open sourced• E.g. heavy modders are very familiar with the busybox install process ;-)• Why Android doesn’t come with the well-tested, POSIX correct glibc?
– Actively avoids using components that *might, possibly* require licensing fees or agreements• Apache Harmony vs Oracle (sun) Java – see https://en.wikipedia.org/wiki/Oracle_America,_Inc._v._Google,_Inc.
• When modifying Android, you should take a strong interest in the software licenses!– Smartphone IP is extremely valuable and heavily protected – see https://en.wikipedia.org/wiki/Smartphone_patent_wars
8
Component LicenseAndroid applications Up to developer
GSM App suiteAOSP External libraries Apache 2 BSD, misc
Android Kernel GPLv2Bootloader ???
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Android
• Often explained using the stack on the right
9
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Android
• Or these…
10http://blogs.freescale.com/mcus/2010/05/android-makes-the-move-to-power-architecture-technology/http://www.techdesignforums.com/edasource/images/68/esd1004_mentor1_large.jpg
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Android, Simplified
11http://blogs.freescale.com/mcus/2010/05/android-makes-the-move-to-power-architecture-technology/http://www.techdesignforums.com/edasource/images/68/esd1004_mentor1_large.jpg
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Android• The running software stack is critically important to understand
– However, the build system is quite obtuse– And documentation is sparse & scattered– One of your big challenges will be mapping running code to
• Source code• Android filesystem
• Always be on the lookout for the ‘bigger picture’– All 5 of these are representations of ‘Android’– Of the docs you can find, 90% will be on the software stack– 10% will be on the source code– The rest is up to you
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Filesystem of Android device• Roughly derived from standard linux filesystem
– See https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard– Uses ramdisk, /sbin, /sys, /dev (sometimes)
• Android adds multiple root folders – /data, /system, /cache– These are basically an Android filesystem Hierarchy
• Build system enforces this consistent hierarchy– Many of the FHS folders are symlinks
• E.g. /etc /system/etc– Just enough of a linux filesystem to run a kernel and some helpful functions
• Heavy use of partitions– Reliable update mechanisms are easier with multiple bootable partitions– Updates can be made smaller
13
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Android Filesystem Partitions• System
– Android OS– Preinstalled APKs– Shared libraries– Executables
• Userdata– User data, user apps– App-specific data folders– Backups– Dalvik caches– Crash information– Encrypted and/or persistent system data
• Cache– Simple writable, always trashable partition
• Recovery, Boot– Not first-class citizens while AOSP is running!
14
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Android Filesystem Partitions
• Adding main symlinks– /etc, /tombstones,
/vendor• Note: Some phones
have distinct vendor.img partition
15
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Android Filesystem Partitions• Interactions with Kernel• /dev is tmpfs
– Android uses udev (ueventd) to fill /dev
• /acct is used by cgroups– CPU, Memory, DiskIO accounting
and limiting• /proc and /sys are standard linux
mechanisms for userspace and kernel interaction
16
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Useful tooling: Android shell• Android uses MirBSD Korn shell (since 4.0
ICS)• Utilities
– Gnu core utilities? GPL!– Busybox? GPL!
• Android toolbox? BSD– See `ls –l /system/bin`– Emulates ~75 standard tools (ls, rm, rmdir, top,
sleep)– See AOSP_ROOT/system/core/toolbox for
source
17
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Physical Android Architecture
• Most Android phones have two processors– Mobile app processor– Mobile baseband processor– Communication using interrupts– Which one ‘leads’?– Frequently on same Soc
• Sensors/USB/Screen/modem– Sometimes on SoC, sometimes connected externally
18
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Physical Android Hardware: Sensors• General problem: sensors normally require proprietary code
– To be business-friendly, AOSP tries very hard to avoid GPL-ing someone’s sensor code
– This unfortunately means that hardware access is *not* done using one standard mechanism
• Access mechanisms:– /dev nodes (display, lights)– Shared library objects (touch, audio)– Sockets– Android HAL
• Android specifies header files, manufacturer provides *.so files– D-Bus (until recently)
19
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Acquiring Android Source Code• Google has comprehensive docs
– https://source.android.com/source/downloading.html• AOSP uses git for version control
– Git repositories become huge given too much time– AOSP has a lot of pieces (~450 different git repos)– Working with ~450 git repositories is a nightmare….
• Repo– Google tool to automate working across many git repos– Enable branch/fetch/status/etc in multiple git repositories with one command
$ repo init -u https://android.googlesource.com/platform/manifest -b android-4.0.1_r1
See https://source.android.com/source/build-numbers.html#source-code-tags-and-builds
20
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Acquiring Android Source Code• In repo terms, each git repo is called a project• A project has
– A remote (e.g. url)– A path– A name
• Repo will clone your projects into ‘detached HEAD’– aka anonymous branch– You should use `repo start <project name>` to do a
git branch• Repo has a concept of project groups
– Note the ‘notdefault’ group
See https://gerrit.googlesource.com/git-repo/+/master/docs/manifest-format.txt
21
<manifest> <remote name="aosp“ fetch="https://android. googlesource.com" />
<project path="packages/apps/Mms“ name="platform/packages/apps/Mms" /> … … <project path="prebuilts/eclipse-build-deps" name="platform/prebuilts/eclipse-build-deps" groups="notdefault,eclipse" /> </manifest>
$ cat .repo/manifest.xml
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Uploading Android Source Code• Doing code reviews across ~450 git repos is a pain!• Enter ‘gerrit’
– Google developed a code review website that is specific to AOSP modifications
– Extremely structured process– Can be self-hosted– Integrated with ‘repo upload’
• Note: Your dev team does not have to use gerrit– You can use git directly to push code– Not quite as painless as ‘repo upload’, but you don’t
have to setup/maintain/use gerrit
22
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Navigating Android Source Code
23
abi C++ support for ABIart Android ART runtime, compiler, dex2oat, etcbionic Android C standard librarybootable Android Recovery image (no bootloader)build Makefile-based AOSP build systemdalvik Dalvik VM source code, dex format spec, libdexdevelopers Source code examplesdevelopment Resources for setting up dev environments. Config files, small scripts, USB driversdevice Device-specific files and binariesdocs Doxyfile for source.android.comexternal External projects used by AOSPframeworks Android core componentshardware Android Hardware Abstraction Layer and hardware supportkernel Precompiled kernel images with Android modificationslibcore All java.* packages from Object.java down, as supplied by Apache Harmonylibnativehelper Android wrapper to make JNI easier to work withndk Native Development Kit source code & tools (for using C/C++ from APKs)out Build output directory. make clean ~= rm -rf outpackages Stock Android apps, Input methods, content providers, etcpdk Platform Development Kit for OEMs to test upcoming AOSP versionsprebuilts Binary (vs source) tools. SDK, qemu for emulator, clang, etcsdk Software Development Kit – eclipse, ADT, SdkManager, etcsystem Native components of Android that run on top of Linuxtools Compilation and IDE tools – gradle files, Eclipse add-ons, Studio add-onsvendor OEM chipset binaries and device definitions
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Navigating Android Source Code
24
abi C++ support for ABIart Android ART runtime, compiler, dex2oat, etcbionic Android C standard librarybootable Android Recovery image (no bootloader)build Makefile-based AOSP build systemdalvik Dalvik VM source code, dex format spec, libdexdevelopers Source code examplesdevelopment Resources for setting up dev environments. Config files, small scripts, USB driversdevice Device-specific files and binariesdocs Doxyfile for source.android.comexternal External projects used by AOSPframeworks Android core componentshardware Android Hardware Abstraction Layer and hardware supportkernel Precompiled kernel images with Android modificationslibcore All java.* packages from Object.java down, as supplied by Apache Harmonylibnativehelper Android wrapper to make JNI easier to work withndk Native Development Kit source code & tools (for using C/C++ from APKs)out Build output directory. make clean ~= rm -rf outpackages Stock Android apps, Input methods, content providers, etcpdk Platform Development Kit for OEMs to test upcoming AOSP versionsprebuilts Binary (vs source) tools. SDK, qemu for emulator, clang, etcsdk Software Development Kit – eclipse, ADT, SdkManager, etcsystem Native components of Android that run on top of Linuxtools Compilation and IDE tools – gradle files, Eclipse add-ons, Studio add-onsvendor OEM chipset binaries and device definitions
You want to work with AOSP in any capacity
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Navigating Android Source Code
25
abi C++ support for ABIart Android ART runtime, compiler, dex2oat, etcbionic Android C standard librarybootable Android Recovery image (no bootloader)build Makefile-based AOSP build systemdalvik Dalvik VM source code, dex format spec, libdexdevelopers Source code examplesdevelopment Resources for setting up dev environments. Config files, small scripts, USB driversdevice Device-specific files and binariesdocs Doxyfile for source.android.comexternal External projects used by AOSPframeworks Android core componentshardware Android Hardware Abstraction Layer and hardware supportkernel Precompiled kernel images with Android modificationslibcore All java.* packages from Object.java down, as supplied by Apache Harmonylibnativehelper Android wrapper to make JNI easier to work withndk Native Development Kit source code & tools (for using C/C++ from APKs)out Build output directory. make clean ~= rm -rf outpackages Stock Android apps, Input methods, content providers, etcpdk Platform Development Kit for OEMs to test upcoming AOSP versionsprebuilts Binary (vs source) tools. SDK, qemu for emulator, clang, etcsdk Software Development Kit – eclipse, ADT, SdkManager, etcsystem Native components of Android that run on top of Linuxtools Compilation and IDE tools – gradle files, Eclipse add-ons, Studio add-onsvendor OEM chipset binaries and device definitions
You want to port AOSP to a really-small IoT kind of device
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Navigating Android Source Code
26
abi C++ support for ABIart Android ART runtime, compiler, dex2oat, etcbionic Android C standard librarybootable Android Recovery image (no bootloader)build Makefile-based AOSP build systemdalvik Dalvik VM source code, dex format spec, libdexdevelopers Source code examplesdevelopment Resources for setting up dev environments. Config files, small scripts, USB driversdevice Device-specific files and binariesdocs Doxyfile for source.android.comexternal External projects used by AOSPframeworks Android core componentshardware Android Hardware Abstraction Layer and hardware supportkernel Precompiled kernel images with Android modificationslibcore All java.* packages from Object.java down, as supplied by Apache Harmonylibnativehelper Android wrapper to make JNI easier to work withndk Native Development Kit source code & tools (for using C/C++ from APKs)out Build output directory. make clean ~= rm -rf outpackages Stock Android apps, Input methods, content providers, etcpdk Platform Development Kit for OEMs to test upcoming AOSP versionsprebuilts Binary (vs source) tools. SDK, qemu for emulator, clang, etcsdk Software Development Kit – eclipse, ADT, SdkManager, etcsystem Native components of Android that run on top of Linuxtools Compilation and IDE tools – gradle files, Eclipse add-ons, Studio add-onsvendor OEM chipset binaries and device definitions
You want to modify what it means to be ‘Dalvik’
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Navigating Android Source Code
27
abi C++ support for ABIart Android ART runtime, compiler, dex2oat, etcbionic Android C standard librarybootable Android Recovery image (no bootloader)build Makefile-based AOSP build systemdalvik Dalvik VM source code, dex format spec, libdexdevelopers Source code examplesdevelopment Resources for setting up dev environments. Config files, small scripts, USB driversdevice Device-specific files and binariesdocs Doxyfile for source.android.comexternal External projects used by AOSPframeworks Android core componentshardware Android Hardware Abstraction Layer and hardware supportkernel Precompiled kernel images with Android modificationslibcore All java.* packages from Object.java down, as supplied by Apache Harmonylibnativehelper Android wrapper to make JNI easier to work withndk Native Development Kit source code & tools (for using C/C++ from APKs)out Build output directory. make clean ~= rm -rf outpackages Stock Android apps, Input methods, content providers, etcpdk Platform Development Kit for OEMs to test upcoming AOSP versionsprebuilts Binary (vs source) tools. SDK, qemu for emulator, clang, etcsdk Software Development Kit – eclipse, ADT, SdkManager, etcsystem Native components of Android that run on top of Linuxtools Compilation and IDE tools – gradle files, Eclipse add-ons, Studio add-onsvendor OEM chipset binaries and device definitions
You want to add a new system service to Android
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Navigating Android Source Code
28
abi C++ support for ABIart Android ART runtime, compiler, dex2oat, etcbionic Android C standard librarybootable Android Recovery image (no bootloader)build Makefile-based AOSP build systemdalvik Dalvik VM source code, dex format spec, libdexdevelopers Source code examplesdevelopment Resources for setting up dev environments. Config files, small scripts, USB driversdevice Device-specific files and binariesdocs Doxyfile for source.android.comexternal External projects used by AOSPframeworks Android core componentshardware Android Hardware Abstraction Layer and hardware supportkernel Precompiled kernel images with Android modificationslibcore All java.* packages from Object.java down, as supplied by Apache Harmonylibnativehelper Android wrapper to make JNI easier to work withndk Native Development Kit source code & tools (for using C/C++ from APKs)out Build output directory. make clean ~= rm -rf outpackages Stock Android apps, Input methods, content providers, etcpdk Platform Development Kit for OEMs to test upcoming AOSP versionsprebuilts Binary (vs source) tools. SDK, qemu for emulator, clang, etcsdk Software Development Kit – eclipse, ADT, SdkManager, etcsystem Native components of Android that run on top of Linuxtools Compilation and IDE tools – gradle files, Eclipse add-ons, Studio add-onsvendor OEM chipset binaries and device definitions
You want to build a custom ROM to is AOSP with different images
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Navigating Android Source Code
29
abi C++ support for ABIart Android ART runtime, compiler, dex2oat, etcbionic Android C standard librarybootable Android Recovery image (no bootloader)build Makefile-based AOSP build systemdalvik Dalvik VM source code, dex format spec, libdexdevelopers Source code examplesdevelopment Resources for setting up dev environments. Config files, small scripts, USB driversdevice Device-specific files and binariesdocs Doxyfile for source.android.comexternal External projects used by AOSPframeworks Android core componentshardware Android Hardware Abstraction Layer and hardware supportkernel Precompiled kernel images with Android modificationslibcore All java.* packages from Object.java down, as supplied by Apache Harmonylibnativehelper Android wrapper to make JNI easier to work withndk Native Development Kit source code & tools (for using C/C++ from APKs)out Build output directory. make clean ~= rm -rf outpackages Stock Android apps, Input methods, content providers, etcpdk Platform Development Kit for OEMs to test upcoming AOSP versionsprebuilts Binary (vs source) tools. SDK, qemu for emulator, clang, etcsdk Software Development Kit – eclipse, ADT, SdkManager, etcsystem Native components of Android that run on top of Linuxtools Compilation and IDE tools – gradle files, Eclipse add-ons, Studio add-onsvendor OEM chipset binaries and device definitions
You want to use a different Bluetooth stack
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Navigating Android Source Code
• godir is your friend!– It’s find and cd rolled into one command
30
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Building Android Source Code
• Use make!
• I don’t know why that needed an entire slide…
31
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Building Android Source Code1. Setup computer
See https://source.android.com/source/building.html2. Setup your shell
$ source build/envsetup.sh 3. Choose compile target
$ lunch aosp_arm-eng4. Compile – full compile takes hours
$ make5. Run emulator (or skip this and run on device instead)
$ emulator6. Flash device
$ adb reboot-bootloader $ fastboot –w flashall
32
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Building Android Source Code1. Setup computer
See https://source.android.com/source/building.html2. Setup your shell
$ source build/envsetup.sh 3. Choose compile target
$ lunch aosp_arm-eng4. Compile – full compile takes hours
$ make5. Run emulator (or skip this and run on device instead)
$ emulator6. Flash device
$ adb reboot-bootloader $ fastboot –w flashall
33
What is this?
How does this work?
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Android Build System – High Level• AOSP has ~40k lines of makefiles under /build and /device
– Important to grok the main points before diving in there!
• Comparisons to standard projects using make– Uses ‘Android.mk’, not ‘Makefile’– Heavy usage of ANDROID_* environment variables– All output (even intermediate) goes into ‘out’ folder
• rm –rf out/– Smallest buildable unit is a ‘module’
• Similar to standard make targets, but there are 1000s of these• While many ‘modules’ are named statically, even more are created dynamically!
– For example, every module “foo” automatically gets an accompanying module “foo-clean”
– Non-recursive build system• See http://aegis.sourceforge.net/auug97.pdf• See build/core/build-system.html
– You never call gcc yourself! Multiple ‘build templates’ exist• Android application, shared library, static library, standalone executable, java library, prebuilt• Build templates enforce the Android filesystem layout• Accept “parameters” in the form of make variables (will default parameters if these variables are not set)
34
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Android Build System – High LevelTo build a module:
• Setup expected variables:LOCAL_PATH := $(call my-dir)LOCAL_MODULE_NAME := some_example
• Then include the proper build template:include $(BUILD_JAVA_LIBRARY)
The build template will read the variables you set, compile as expected (in this case, as expected for a java library), and place the intermediates and final output in the correct locations inside the out/ folder
35
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Android Build System – High Level
• Build system includes lots of ‘dark magic’• Not recommended to just start writing an Android.mk
– Find one that does what you want and modify it– Or you’ll have forgotten a variable
• Look into build/core/ to find makefiles for each build template– All options listed at build/core/config.mk line ~75– Missing variables are often set to default values
36
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
What is lunch?• AOSP has many compile options
– Set options with env variables or buildspec.mk
– envsetup.sh defines functions to help set options using env variables
– One helper is ‘lunch’– “I’m having JellyBeans for lunch”
• Just calling lunch presents a menu of the form:
$TARGET_PRODUCT-$TARGET_BUILD_VARIANT
37
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Android Build Targets• Target product ~= device + configuration
– E.g. nexus 6 (aka shamu)– E.g. en keyboard/apps/tts– Device name is arbitrary
• Example products– aosp_arm: AOSP default locale on Emulator– full_fugu: All locales on Nexus Player– aosp_shamu: AOSP default on Nexus 6– See https://source.android.com/source/build-
numbers.html• To build your own, take a look at the different
AndroidProducts.mk and vendorsetup.sh scripts– See https://source.android.com/source/add-device.html
38
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Android Build Variants & Type• Target_build_variant determines the default apps and settings
– user: Include stuff plan user would want– userdebug: Include user stuff, and a few items/settings to make
debugging easier– eng: Include userdebug stuff, user stuff, and item/settings an engineer
might use• Target_build_type
– You can set this to release or debug to change the settings of ConfigBuildFlags.DEBUG
– This is used to enable/disable some extra logging in core system services
39
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Building Android Source Code1. Setup computer
See https://source.android.com/source/building.html2. Setup your shell
$ source build/envsetup.sh 3. Choose compile target
$ lunch aosp_arm-eng4. Compile – full compile takes hours
$ make5. Run emulator (or skip this and run on device instead)
$ emulator6. Flash device
$ adb reboot-bootloader $ fastboot –w flashall
40
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Modifying a Single Module1. Modify code
$ gedit frameworks/base/core/java/com/android/internal/os/ZygoteInit.java2. Compile that module
Note: findmakefile function is helpful to discover the LOCAL_MODULE_NAME $ make framework
3. Rebuild the system.img file using special snod module $ make snod
4. Reflash $ adb reboot-bootloader $ fastboot –w flashall
41
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Android Bootup (Roughly) Explained
42
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Android init.rc• Android uses it’s own language for the init files
– Easy to understand– I recommend you read init.rc
• Main primitives– service: names a command string execution. Distinct from other uses of word
‘service’• Start with ‘start <servicename>’• Can be grouped into ‘classes’• Start class with ‘class_start <classname>’
– action: on some trigger, do this• Predefined actions
– early-init, init, early-fs, fs, early-boot, boot• User-defined action: property setting changed
– trigger: something happened• ‘on <trigger>’ defines an action to react• ‘trigger <triggername>’ fires the trigger
• Location of init.rc files– In running system (e.g. in ramdisk) : /init.*rc– In build system: system/core/rootdir/*.rc and /device
43
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Android Properties• OS-wide key-value properties
– Not the same as Java properties– Not environment variables– Follow a hierarchy of naming
• Use getprop and setprop tools– Note: There is actually a socket file, but its tricky to
access• Setprop can be used to trigger actions in init.rc!
– Start/stop native code– Set logging levels– Change properties that persist across boot
44
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Android Properties• Where do properties originally get set?!
– Filesystem (/default.prop, /system/build.prop)– Init.rc files– Java code – SystemProperties.set– C++ code – property_set
• Some properties are controlled by UID/GID permissions– E.g. ril.* properties can only be set by RADIO user
• Special cases– ro.* is read-only. These can only be set once (normally at boot)!– persist.* properties are written to the filesystem and persist across reboots– ctl.* properties are not actually stored into the properties map. They enable you to start or stop an init.rc ‘service’ using the
properties system• E.g. ‘ctl.start foobar’ would start service ‘foobar’
45
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
What’s a zygote?• At a native level, binary is called app_process• Encapsulates starting the Dalvik VM• Eases the use of JNI• Loads all Java libraries into itself• Is capable of being forked• Drastically speeds up creation of new, ready-
to-go processes pre-loaded with the Dalvik VM and java libraries
46
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Android System Server• Note that only one bit of Android talks to
the Hardware & Kernel• This is system server
– Multiplexes access to single-user systems like the display
– Does most of the complex book-keeping– Accepts remote messages from apps and
acts on their behalf (e.g. send SMS)• Most ‘Android System Services’ run as a
thread inside the system_server process– Although a number run as standalone apps
with the persistent=true flag
47
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Main Android Services• Some biggies:
– WifiService– ActivityManagerService– PowerService– PackageManagerService– LocationManagerService
• There are about ~90 more– Individually they are not too bad– Look in source code under frameworks/base/services– Don’t start with ActivityManager ;-)
• Developers access these using Context.getSystemService• How does this all happen?
48
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Android IPC Architecture
“In the Android platform, the binder is used for nearly everything thathappens across processes in the core platform.” - Dianne Hackborn, Android Framework Engineer @ Google - https://lkml.org/lkml/2009/6/25/3
I highly recommend: https://newcircle.com/s/post/1340/deep_dive_into_android_ipc_binder_framework_at_andevcon_iv
49
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Android IPC Basics
• Android app calls into android.jar• In many places, android.jar wants
to call some core framework service– Proxy software design pattern!
• To send data through the Binder driver, you have to write data into a string– Marshalling/unmarshalling
50
https://i.stack.imgur.com/EkzyV.png
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Android IPC Basics
• Android app calls into android.jar• In many places, android.jar wants
to call some core framework service– Proxy software design pattern!
• To send data through the Binder driver, you have to write data into a string– Marshalling/unmarshalling
51
https://i.stack.imgur.com/EkzyV.png
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Android IPC Basics
• AIDL files are used to generate the Java source code for Proxy– Exactly the same as you can
do in an Android application!• Very useful to read
generated Proxy & Stub files
52
https://i.stack.imgur.com/EkzyV.png
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Android: Files to Read
• Some source code files are important enough to actually read…
• Zygote, ZygoteInit• SystemServer• Init.rc
57
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
How is Android not Linux?• ueventd vs udev
– Check for devices showing up and populate /dev• Kernel changes!
– Android Out of Memory (OOM) killer – Wakelocks – Binder RPC / IPC– Android’s Anonymous Shared Memory (ashmem)– Android Alarm (on top oc Linux’s Real time clock RTC)– Android’s ring logger mechamisms– Custom init.rc language– Active work to upstream most of these changes
• https://events.linuxfoundation.org/images/stories/slides/abs2013_stultz.pdf• Android Upstreaming group - https://wiki.linaro.org/WorkingGroups/Kernel/AndroidUpstreaming
• C standard library– Bionic vs glibc. Seems this is mainly for licensing reasons
• User management (e.g. uid/gid mapping)– Linux uses UID/GID. Android uses these for apps, and manages users in the AOSP code
• Hardware approach– Linux mainly uses /dev for hardware exposure– Android uses mostly shared object files
60
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
Resources• Embedded Android
– Really excellent for everything below the AOSP java code• New Android Book –
– Good for Android AOSP code – E.g. ‘How does the PowerManager code work’
• Exploring SDK Add-ons– Very interesting talk covering some of the interaction between Google and OEMs– https://www.youtube.com/watch?v=1TY5rqyAyp4
• Deep Dive into Android IPC/Binder Framework at AnDevCon IV– Binder is critical to Android, more understanding == better!– https://newcircle.com/s/post/1349/andevcon_iv_video_presentations_android_developers– https://
newcircle.com/s/post/1340/deep_dive_into_android_ipc_binder_framework_at_andevcon_iv
• Binder IPC talk– http://www.dre.vanderbilt.edu/~schmidt/cs282/PDFs/android-binder-ipc.pdf
61
CONFIDENTIAL. ALL RIGHTS RESERVED.Oct 2015
$ who…are we
62
We are hiring!
Mobile Developer (Application & Framework)Server Developer (Backend & Frontend)DevOps
Send resumes to our VP of Engineering
www.optiolabs.com
Andy [email protected]