1

Embedded Linux/ Debian with ARM64 Platform

Harder Better Faster Stronger

SZ LINszlin@cs.nctu.edu.tw

Date: 2016/11/17

2

SZLIN (林上智 )➔ Debian Developer➔ Industrial Grade Linux Distribution➔ Blog: https://szlin.me

About Me

3

Trends

That’s really what ARM has to look at when defining a new architecture. That is the nature of our business, we need to look a long way forward, and plan.

Why ARMv8 instead of using ARMv7+ LPAE [11]

4

5

ARMv7 -> ARMv8 [2]

32-bit ARMv7-A compatibility

Cryptography support

Dual Instruction set architecture

Improved support for virtualization

6

AArch32 (compat) Support [8]Support ARMv7 Linux EABI for compat tasks • Different set of system calls (unistd32.h) • Compat user structures • No SWP instruction, no unaligned LDM/STM access Supports both ARM and Thumb-2 32-bit user tasks Supports 32-bit ptrace Address space limited to 4GB Emulated vectors page • ARM Linux EABI expects helper routines in the vectors page accessible by user tasks

7

AArch32 (compat) Support

arch/arm64/kernel/ptrace.c

arch/arm64/include/asm/unistd32.h

arch/arm64/include/asm/unistd.h

8

ARMv8-A Designed for Efficiency [2]

Enhancement Why it Matters

64-bit architecture Efficient access to large datasets

Increased number and size of general purpose registers

Gains in performance and code efficiency

Double the number and size of NEON registers

Enhanced capacity of multimedia engine

Cryptography support Over 10x software encryption performance New security models for consumer and enterprise

9

ARMv8-A Family [3]Core Instruction cycle DMIPS/MHz Processor

Architecture big.LITTLE role

Cortex-A32 In-order execution 2.0 ~ 2.1 32 Unknown

Cortex-A35 In-order execution 1.78 64 little

Cortex-A53In-order execution 2.24 64 little

Cortex-A57 Out-of-order execution 4.6 64 big

Cortex-A72 Out-of-order execution 4.72 64 big

Cortex-A73 Out-of-order execution 4.8 64 big

10

The ARM Registers Comparison [4]AArch32 AArch64 Remarks

32-bit General purpose registers r0 - r14 w0 - w30 aliased to lower half of each 64-bit register in AArch64

64-bit General purpose registers - x0 - x30

Program counter r15 aka. pc PC not directly accessible in AArch64

Zero register - x31 / w31 aka. xzr / wzr

in most instructions, except as a base register for address generation

Stack pointer r13 SP encoded as x31 on address generation

Procedure call link register r14 x30

11

ARMv7 name

AArch64/ARMv8 name Remarks

Monitor mode EL3 highest exception level, mostly for firmware

HYP mode EL2exception level for hypervisors like Xen (or parts of KVM)

SVC mode EL1 the Linux kernel is running in this

USR mode EL0 for unprivileged userland

The ARM Execution Mode [4][8]

12

Memory Layout [15]

AARCH64 It allows up to 4 levels of translation tables with a 4KB page size and up to 3 levels with a 64KB page size.

13

Memory Layout [9][15]

32 bit address space: • 4GB of virtual addresses • 1MPages • Each entry is 4 bytes (a 32 bit physical address) • Page size : 4kb

14

Memory Layout [14][15]

AArch64 Linux uses either 3 levels or 4 levels of translation tables with the 4KB page configuration, allowing 39-bit (512GB) or 48-bit (256TB) virtual addresses, respectively, for both user and kernel.

64 bit address space• 16 exabytes(1018) of virtual address space • 4PetaPages (1015)

• Entry is 8 bytes • Page size : 4kb/ 64kb

15

Memory Layout [14][15]

16

Memory Layout [14]With 64KB pages, only 2 levels of translation tables, allowing 42-bit (4TB) virtual address, are used but the memory layout is the same.

17

The ARM64 with Linux Kernel [5]

First patch in Kernel 3.7 Without real hardware

First ARM64 SoC supported in Kernel 3.11 2013/6

Second ARM64 SoC supported in Kernel 3.18 2014/12

Different assembly code/ ABI between ARM32/64

AArch64 is the name for the 64-bit ARM architecture

18

The ARM64 with Linux Kernel [5]

Device tree arch/arm64/boot/dts/<vendor>/

Compressed Kernel arch/arm64/boot/Image.gz

No more “arch/arm64/mach-<soc>” [7] No more C files, all SoC features should be handled by drivers Or it can go “drivers/soc” if it cannot fit an existing drivers

No more uImage [7] Incompatible with multiplatform kernels

Switch to bootz command in bootloader

19

The ARM64 with Debian [6]

First release in Debian 8

AARCH64 Toolchain in Debian Package: crossbuild-essential-arm64

aarch64-linux-gnu-gcc

Build ARM64 Debian root filesytem via debootstrap

20

Hands on experience Build Debian on AARCH64

21

The QorIQ® LS1043A reference design board [13]

22

The QorIQ® LS1043A reference design board [13]

23

Cross Tool chain in Debian

24

Build Kernel 4.1…with Errorroot@szlin:/home/szlin/work/Kernel/moxa_linux/ls1043a# make -j30 CHK include/config/kernel.release CHK include/generated/uapi/linux/version.h HOSTCC scripts/kallsyms CC scripts/mod/empty.o CC scripts/mod/devicetable-offsets.s MKELF scripts/mod/elfconfig.h HOSTCC scripts/mod/modpost.o HOSTCC scripts/mod/sumversion.oIn file included from include/linux/compiler.h:54:0, from include/uapi/linux/stddef.h:1, from include/linux/stddef.h:4, from ./include/uapi/linux/posix_types.h:4, from include/uapi/linux/types.h:13, from include/linux/types.h:5, from include/linux/mod_devicetable.h:11, from scripts/mod/devicetable-offsets.c:2:include/linux/compiler-gcc.h:121:30: fatal error: linux/compiler-gcc6.h: No such file or directory #include gcc_header(__GNUC__) ^

25

Build Kernel 4.1 – Patch with GCC6 issue

26

Build Kernel 4.1 – LZ4 is not existed in ARM 64LZ4 is existed in ARM Kernel 4.1

But not in ARM 64… it needs patch !

27

Build uImage – Failed in ARM64 [17]> Bringing in uImage as a target on ARM might have been a mistake, since> now people want to bring in new targets for whatever boot format they> happen to want. So it's a good idea to keep arm64 from going down the> same path and stick to the native, raw, formats.

mkimage -A arm64 -O linux -T kernel -C gzip -a 0x80080000 -e 0x80080000 -n 4.1.0-ltsi-rt+ -d arch/arm64/boot/Image.gz uImage

No, there should be no uImage target on arm64. Newer u-boots canalready boot an ARM zImage (through the bootz command), and should beenhanced as needed to boot a vmlinux if needed.

However, we can still make uImage manually

28

Configure system account

Configure pure root filesystem

4321

Have fun with Debian

Steps to Build Your Debian ARM

Pre-download pure root filesystem

29

Configure system account

Configure pure root filesystem

4321

Have fun with Debian

Steps to Build Your Debian ARM

Pre-download pure root filesystem

30

Pre-download pure root filesystem

1. Build pure Debian root filesystem via “debootstrap” command

debootstrap --arch=arm64 --foreign --keyring /usr/share/keyrings/debian-archive-keyring.gpg --exclude=debfoster jessie *DIRECTORY*

Note:“man debootstrap” to get further information

2. Download ARM64 “sudo” package https://packages.debian.org/jessie/sudo

3. Put “sudo” package into *DIRECTORY*/var/cache/apt/archives/

DEMO: https://github.com/szlin/Pure_Debian_aarch64.git

31

Configure system account

Configure pure root filesystem

4321

Have fun with Debian

Steps to Build Your Debian ARM

Pre-download pure root filesystem

32

Linux Kernel and device tree

Debian root filesystem

Tiny root filesystem

Non-volatile memory

setenv bootargs "console=ttyS0,115200 root=/dev/mmcblk0p3 rw rootdelay=5 earlycon=uart8250,0x21c0500,115200"fatload mmc 0 0xa0000000 uImagefatload mmc 0 0x90000000 fsl-ls1043a-rdb.dtbbootm 0xa0000000 - 0x90000000

Set root = tiny root filesystem in first boot up

33

Boot up the device via manual command

34

ln -sf /dev/null /dev/tty3ln -sf /dev/null /dev/tty4ln -sf /dev/null /dev/tty2

Mount Debian partition and chroot to debian

35

Start to configure Debian rootfilesystem in second stage

Unnecessary files will removed afterwards

36

Configure system account

Configure pure root filesystem

4321

Have fun with Debian

Steps to Build Your Debian ARM

Pre-download pure root filesystem

37

Create “sudo” account

DEMO: https://github.com/szlin/Debian_aarch64.git

38

Configure system account

Configure pure root filesystem

4321

Have fun with Debian

Steps to Build Your Debian ARM

Pre-download pure root filesystem

39

Linux Kernel and device tree

Debian root filesystem

Tiny root filesystem

Non-volatile memory

setenv bootargs "console=ttyS0,115200 root=/dev/mmcblk0p2 rw rootdelay=5 earlycon=uart8250,0x21c0500,115200"fatload mmc 0 0xa0000000 uImagefatload mmc 0 0x90000000 fsl-ls1043a-rdb.dtbbootm 0xa0000000 - 0x90000000

Set root = Debian root filesystem in bootloader

40

Login and check “sudo” mechanism

41

ARM 64 Programming

42

ARM 64 Programming [16]What is the different between 32 and 64 bits in C

43

ARM 64 Programming [16]Pre-defined macros relating to ARMv8 compilation

44

ARM 64 Programming [16]Bit manipulation operations

Any issue?

45

ARM 64 Programming [16]Bit manipulation operations

“1” has int type…

46

ARM 64 Programming [16]Indexes

Any issue?

47

ARM 64 Programming [16]Indexes

size_t: 64 bitunsigned int: 32 bit

Infinite loop

48

ARM 64 Programming [16]Pointers and ints

Is p = q?

49

ARM 64 Programming [16]Pointers and ints

1. For ARM32 p==q2. For ARM64 p!=q

50

ARM 64 Programming [16]Structure padding

What is the size of structure in C ?

51

ARM 64 Programming [16]Structure padding

1. For ARM32int a : 4long b: 4int c: 4Total = 16 bytes2. For ARM64int a : 4long b: 8int c: 4Total = 20 bytes

52

ARM 64 Programming [16]Structure padding

1. For ARM64int a : 4long b: 8int c: 4Total = 20 bytes

For double-word aligned, it needs to introduce four bytes of padding between the end of the first ”int” and the ”long”

53

Thank you

54

References1. AArch64 planninghttps://www.einval.com/~steve/talks/Debconf12-aarch642. ARMv8-Ahttp://media.corporate-ir.net/media_files/IROL/19/197211/ARMv8-A%20IR%20webcast%2024_03_2014.pdf3. Comparison of ARMv8-A coreshttps://en.wikipedia.org/wiki/Comparison_of_ARMv8-A_cores4. Arm64http://linux-sunxi.org/Arm645. ARM64 SoC checklisthttp://events.linuxfoundation.org/sites/events/files/slides/clement-arm64-soc-checklist.pdf6. ARM64Porthttps://wiki.debian.org/Arm64Port7. AArch64 kernel image decompressionhttp://lists.infradead.org/pipermail/linux-arm-kernel/2014-January/225354.html8. Linux on AArch64 ARM 64-bit Architecture https://events.linuxfoundation.org/images/stories/pdf/lcna_co2012_marinas.pdf9. Migrating code from ARM to ARM64https://www.linuxplumbersconf.org/2014/ocw/system/presentations/2343/original/08%20-%20Migrating%20code%20from%20ARM%20to%20ARM64.pdf

55

References10. ARM64 vs ARM32 -- What's different for Linux programmers http://edn.com/design/systems-design/4440662/ARM64-vs-ARM32-What-s-different-for-Linux-programmers11. ARMv8https://www.arm.com/files/downloads/ARMv8_white_paper_v5.pdf12. Debianhttps://www.debian.org13. LS1043a – RDBhttp://www.nxp.com/products/microcontrollers-and-processors/arm-processors/qoriq-arm-processors/qoriq-ls1043a-reference-design-board:LS1043A-RDB14. Memory Layout on AArch64 Linux https://www.kernel.org/doc/Documentation/arm64/memory.txt15. Virtual memoryhttps://cseweb.ucsd.edu/classes/wi11/cse141/Slides/19_VirtualMemory.key.pdf

56

References16. Porting to 64-bit ARM http://malideveloper.arm.com/downloads/Porting%20to%20ARM%2064-bit.pdf17. AArch64 kernel image decompressionhttp://lists.infradead.org/pipermail/linux-arm-kernel/2014-January/225354.html18. arm64: enable more compressed Image formatshttps://patchwork.kernel.org/patch/6810841/19. Fix-the-compile-issue-under-gcc6 https://raw.githubusercontent.com/Freescale/meta-freescale-3rdparty/master/recipes-kernel/linux/linux-variscite-4.1.15/Fix-the-compile-issue-under-gcc6.patch