How to build a minimal GNU/Linux embedded system from scratch · How to build a minimal GNU/Linux embedded system. . . from scratch! ... GNU/Linux GNU software ... (tftp server) Board

Mirko Damiani

How to build a minimalGNU/Linux embedded system

. . . from scratch!

Better Embedded ConferenceFirenze, Sept. 24-25, 2012

2 / 35

Mirko Damiani – [email protected]

Terms of Interest● GNU/Linux

● GNU software● Linux kernel

● Minimal● Size and number of utilities● Complexity of scripts

● Embedded system● Custom boards● Low resources

3 / 35


Prerequisites● Entry level

● GNU/Linux kernel● Hardware schematics● Operating systems● Computer architectures

● Intermediate level● GNU/Linux utilities● SW development

4 / 35


Reference board● LeopardBoard 365

● DM365 SoC (Texas Instruments)● ARM processor

● Linux support● TI evaluation board (DaVinci)● LeopardBoard differs from TI evaluation board● Board Support Package available

5 / 35


Overview● Board flashing

● Strategy

● System startup● Bootloader

● Kernel● How to load and start it

● Root filesystem● How to create it

6 / 35

Board flashing

● Common methods● Flashing strategy● Rootfs archive● Read-only rootfs● Flashing on the fly

7 / 35


Common methods● Utilities to program the NAND flash

● SD/MMC card● USB (memory stick or protocol)● Serial port

● Custom boards may need a specificflashing strategy

● SD/MMC or USB may be missing● Flashing utilities might not work● Serial port is almost always present

8 / 35


Flashing strategy● Let's suppose we are able to flash

at least the bootloader

● We load the entire system in RAM● Kernel is always loaded in RAM● Rootfs is passed as an initramfs archive● Kernel extracts its initramfs in RAM when it boots up

● System is flashed for permanent setup● Linux starts from NAND flash

9 / 35


Flashing strategy● Small components

● Kernel is less than 4MB● Rootfs fits in 32MB (unpacked)

● No different configurations● The same system is used both for

flashing and production● Software development is easier

● Need more space?● Minimal rootfs is able to mount

other NAND partitions

PC(tftp server)

Board

NAND

10 / 35


Rootfs archive● The same rootfs must be saved in 2 different formats

● Initramfs (gzipped cpio archive)● SquashFS image (read-only archive)

● Different kernel parameters, for instance:● Initramfs

rdinit=/sbin/init initrd=0x82000000,0x82a190● SquashFS

init=/sbin/init root=/dev/mtdblock3 rootfstype=squashfs

11 / 35


Read-only rootfs● Still need to have some R/W folders

● /etc, /tmp, /var, /dev

● Move contents in RAM and remount from there● mount -n -t tmpfs none /mnt/rwfs -o size=4096k● mkdir -p /mnt/rwfs/etc● cp -a /etc/* /mnt/rwfs/etc● mount -n --bind /mnt/rwfs/etc /etc

● Now we don't have write errors anymore● Read-only filesystems are fast

12 / 35


Flashing on the fly● Kernel and rootfs are downloaded via tftp server again● It's not convenient to save files in RAM,

since we might not have enough room

● Both files are directly flashed,without storing them in RAM

RAM NAND

NAND

13 / 35

System startup

● DM365 as reference● Boot overview● Boot loading stages● Debug at early stages

14 / 35


Boot overview● Board initialization

● Low level, hardware related

● Bootloader execution● Persistent configuration (kernel params)

● Kernel loading● Operating system and devices setup

● System startup● /sbin/init and other scripts

bootloader

kernel

rootfs

15 / 35


Bootloader stages (DM365)Power on

RBL(1st stage)

UBL(2nd stage)

U-Boot(3rd stage)

● ROM boot loader (RBL)● ARM subsystem initialization

● User boot loader (UBL)● External devices initialization

● U-Boot● Kernel loading

16 / 35



RBL(1st stage)

UBL(2nd stage)

U-Boot(3rd stage)

NAND

UART

USB

Get UBL

Store in IRAM(32KB)

17 / 35



RBL(1st stage)

UBL(2nd stage)

U-Boot(3rd stage)

Get U-Boot

NAND

UART

USB

Store in RAM(128MB)

18 / 35



RBL(1st stage)

UBL(2nd stage)

U-Boot(3rd stage)

Get Kernel

Run Linux

Scripts

Parameters

Environment

19 / 35


RBL NAND loading (DM365)● Loading algorithm

● NAND memories are organized in sectors and pages

● RBL searches for a valid block● In case of read errors,

next block is searched fora valid UBL (up to #24)

● Robustness● Multiple UBL / U-Boot pairs

may be saved in NAND● Flashing utilities do this for you

20 / 35


Debug at early stages● U-Boot also provides an UBL for DaVinci boards

● TI UBL may be replaced with U-Boot UBL● UBL sets the ARM processor frequency,

so i might want to change that values

● Very hard to debug● Commons methods (printf, debugger, gdb, valgrind...)

are not available● Boards simply don't boot,

without any evidence of errors

● A useful approach: exploit GPIO pins

21 / 35


LeopardBoard GPIO● Access some registers (memory mapped I/O)

● Setup pin multiplexing● Select GPIO direction● Drive GPIO as output high or low

22 / 35

Kernel

● Board Support Packages● Configuration and build● Loading and parameters● Patches● NAND partitioning

23 / 35


Board Support Packages● Kernel and drivers are patched to work with the

specified development board● We must understand how they works,

e.g. a camera driver would use video4linux● Custom boards may differ from development boards,

so additional SW patches are required

● Ad-hoc drivers are not easy to write from scratch, e.g a camera driver we would support only a specific sensor and very few operation modes

● Minimal drivers lead to an higher code maintenance

24 / 35


Configuration and build● Compilation

● Download from www.kernel.org● Toolchain: Sourcery Codebench Lite

● Select the correct system and board, e.g.:● ARM system type● DaVinci 365 based system

● Disable all which is not needed● Vanilla

● We have a mainstream kernel● Easy to maintain (update)● Recent version

25 / 35


Kernel loading● Loading is performed

by the bootloader● Arguments are passed

to the kernel● Kernel sets up its devices● When booting is completed,

/sbin/init is called(from within the rootfs)

● /sbin/init is responsible forbringing up the rest of thesystem (the userland)

args

bootloader

/sbin/initpid = 1

26 / 35


Kernel patches● DaVinci RTC on LeopardBoard 365

● Driver is tested on DM360 EVM● LeopardBoard has a different schematic

● Mux pins and drive some GPIOs

27 / 35


Kernel parameters● Serial console device and options

● console=ttyS0,115200n8

● LeopardBoard 365● MAC address changes at every reboot● We can save it on U-Boot environment,

and pass it to the Linux kernel– davinci_emac.macaddr=AA:BB:CC:DD:EE:FF– davinci_emac.macaddr=${ethaddr}

28 / 35


NAND partitioning● Bootloader is

never touched

● Environment holdskernel arguments

● This layout may bestatically writtenin kernel sources,or passed througharguments

bootloader

environment

kernel

rootfs

30 blocks

256 KB

4 MB

32 MB

29 / 35

Root filesystem

● Busybox● Accessing bootloader environment● Building the root filesystem

30 / 35


Busybox● Provides almost a complete environment

for small systems● Tiny version of common utilities (fewer options)

● Single executable● Size optimization● Static or dynamic library linking● Symlinks to busybox executable

– e.g: /bin/ls -> /bin/busybox● Default configuration

● Static: 2MB● Dynamic: 936KB

31 / 35


Busybox● Packages

● Coreutils, Shells, Editors, Network, …● Login files

– /etc/passwd, /etc/shadow, /etc/groups, …● System logs

– syslogd, klogd● ...

● Init program (/sbin/init)● Reads /etc/inittab● Runs /etc/rc.d/rcS● Other scripts, /etc/rc.d/init.d/network, ...

32 / 35


Bootloader environment● Recall: boot env is stored into the 2nd partition● Example: U-Boot

● Provides some tools: fw_printenv, fw_setenv

● Generally speaking, it's possible toaccess them from userspace.

● Within our flashing procedure,IP address of tftp servermay be read from thebootloader environment

● System might not save at allkernel and rootfs into NAND,while being still configurable

192.168.10.101

33 / 35


Building the root filesystem

foldersstructure

toolchainlibraries

busybox

kernelmodules

skeletonfiles

rootfs

cleanbinaries

Let's Talkoffice

web

e-mail / jabber+39 055 3984627

[email protected]

www.develer.com

Creative Commons Attribution 3.0 Unported

License

Documents

How to build a minimal GNU/Linux embedded system from scratch · How to build a minimal GNU/Linux embedded system. . . from scratch! ... GNU/Linux GNU software ... (tftp server) Board