FreeBSD ARM64: Porting on a new board - BSDCan · I Upstream guy in Linux for DTS and U-Boot....

FreeBSD ARM64: Porting on a new board

Emmanuel Vadotmanu@FreeBSD.org

BSDCanOttawa Canada

June 8 – 9, 2018

Who am I ?

I ARM Kernel Hacker for 2 and half year

I Self proclaimed maintainer for Allwinner SoCs (and nowRockChip)

I Self proclaimed DTS Maintainer in FreeBSD

I U-Boot Maintainer

I Upstream guy in Linux for DTS and U-Boot

Agenda

I ARM/SoC/SBC

I Bootloader

I Serial

I First kernel boot

I Clocks and Resets

I Clock API

What is an SoC ?

I SoC == System On Chip

I ARM does not manufacture processor

I SoC vendor buys IP from ARM for the core

I Sometimes they also buy IP from other companies

I An SoC integrates a processor and peripherals

Single Board Computer

I SBC = Single Board Computer

I Generaly from another company than the SoC one

I Integrates SoC and other chips (PMU, PHY etc ...)

I Also adds GPIOs, SD/MMC, ethernet connectors etc ...

Pine64 Rock64

I RK3328 based SBC

I 1Gbps Ethernet

I USB3

I eMMC socket

I ...

I Donated by Pine64, Thank you TL Lim

Why Porting ?

I It’s fun

I You learn a lot

I New arm/arm64 boards every month or soHaving FreeBSD working on it expand our market

I Porting to a new arch is hard, new SoC not that much

Why Porting ?

I It’s fun

I You learn a lot

I New arm/arm64 boards every month or soHaving FreeBSD working on it expand our market

I Porting to a new arch is hard, new SoC not that much

Why Porting ?

I It’s fun

I You learn a lot

I New arm/arm64 boards every month or soHaving FreeBSD working on it expand our market

I Porting to a new arch is hard, new SoC not that much

Why Porting ?

I It’s fun

I You learn a lot

I New arm/arm64 boards every month or soHaving FreeBSD working on it expand our market

I Porting to a new arch is hard, new SoC not that much

Bootloader requirement

I EFI aware

I FIT Image

I AArch64 Linux Image

I Convert kernel to kernel.bin

I But you want EFI aware

Bootloader requirement

I EFI aware

I FIT Image

I AArch64 Linux Image

I Convert kernel to kernel.bin

I But you want EFI aware

Bootloader requirement

I EFI aware

I FIT Image

I AArch64 Linux Image

I Convert kernel to kernel.bin

I But you want EFI aware

Bootloader requirement

I EFI aware

I FIT Image

I AArch64 Linux Image

I Convert kernel to kernel.bin

I But you want EFI aware

Bootloader requirement

I EFI aware

I FIT Image

I AArch64 Linux Image

I Convert kernel to kernel.bin

I But you want EFI aware

U-Boot life

I Mainline release every two months

I SoC Vendor pick a release or a random commit

I Stay on it and patch it

I SBC Vendor patch the SoC Vendor one

I Linux Distribution patch the SBC Vendor one

U-Boot life

I Mainline release every two months

I SoC Vendor pick a release or a random commit

I Stay on it and patch it

I SBC Vendor patch the SoC Vendor one

I Linux Distribution patch the SBC Vendor one

U-Boot life

I Mainline release every two months

I SoC Vendor pick a release or a random commit

I Stay on it and patch it

I SBC Vendor patch the SoC Vendor one

I Linux Distribution patch the SBC Vendor one

U-Boot life

I Mainline release every two months

I SoC Vendor pick a release or a random commit

I Stay on it and patch it

I SBC Vendor patch the SoC Vendor one

I Linux Distribution patch the SBC Vendor one

U-Boot life

I Mainline release every two months

I SoC Vendor pick a release or a random commit

I Stay on it and patch it

I SBC Vendor patch the SoC Vendor one

I Linux Distribution patch the SBC Vendor one

U-Boot life at RockChip

I Mainline release every two months

I SoC Vendor pick a release or a random commitRockchip fork is based on 2017.09

I Stay on it and patch itThey do upstream some patches to mainline

I SBC Vendor patch the SoC Vendor oneFew patches but none upstreamed

I Linux Distribution patch the SBC Vendor onesomething like 100 patches, none upstreamed

U-Boot life at RockChip

I Mainline release every two months

I SoC Vendor pick a release or a random commitRockchip fork is based on 2017.09

I Stay on it and patch itThey do upstream some patches to mainline

I SBC Vendor patch the SoC Vendor oneFew patches but none upstreamed

I Linux Distribution patch the SBC Vendor onesomething like 100 patches, none upstreamed

U-Boot life at RockChip

I Mainline release every two months

I SoC Vendor pick a release or a random commitRockchip fork is based on 2017.09

I Stay on it and patch itThey do upstream some patches to mainline

I SBC Vendor patch the SoC Vendor oneFew patches but none upstreamed

I Linux Distribution patch the SBC Vendor onesomething like 100 patches, none upstreamed

U-Boot life at RockChip

I Mainline release every two months

I SoC Vendor pick a release or a random commitRockchip fork is based on 2017.09

I Stay on it and patch itThey do upstream some patches to mainline

I SBC Vendor patch the SoC Vendor oneFew patches but none upstreamed

I Linux Distribution patch the SBC Vendor onesomething like 100 patches, none upstreamed

U-Boot life at RockChip

I Mainline release every two months

I SoC Vendor pick a release or a random commitRockchip fork is based on 2017.09

I Stay on it and patch itThey do upstream some patches to mainline

I SBC Vendor patch the SoC Vendor oneFew patches but none upstreamed

I Linux Distribution patch the SBC Vendor onesomething like 100 patches, none upstreamed

U-Boot for Rock64

I Using vendor u-boot first

I Decided to use the ’community’ build by ayufan

I Produce an sd card image for spi flash burning

I And it supports tftpboot

I andreast@FreeBSD.Org updated to recent u-boot

U-Boot for Rock64

I Using vendor u-boot first

I Decided to use the ’community’ build by ayufan

I Produce an sd card image for spi flash burning

I And it supports tftpboot

I andreast@FreeBSD.Org updated to recent u-boot

U-Boot for Rock64

I Using vendor u-boot first

I Decided to use the ’community’ build by ayufan

I Produce an sd card image for spi flash burning

I And it supports tftpboot

I andreast@FreeBSD.Org updated to recent u-boot

U-Boot for Rock64

I Using vendor u-boot first

I Decided to use the ’community’ build by ayufan

I Produce an sd card image for spi flash burning

I And it supports tftpboot

I andreast@FreeBSD.Org updated to recent u-boot

U-Boot for Rock64

I Using vendor u-boot first

I Decided to use the ’community’ build by ayufan

I Produce an sd card image for spi flash burning

I And it supports tftpboot

I andreast@FreeBSD.Org updated to recent u-boot

Serial

I FreeBSD Kernel try to resolve the /chosen/std{in,out}

I Fallback on dtb node named serial0

I Node need it’s status to be != disabled

I There is a good chance that the uart controller is alreadysupported

Serial

I FreeBSD Kernel try to resolve the /chosen/std{in,out}I Fallback on dtb node named serial0

I Node need it’s status to be != disabled

I There is a good chance that the uart controller is alreadysupported

Serial

I FreeBSD Kernel try to resolve the /chosen/std{in,out}I Fallback on dtb node named serial0

I Node need it’s status to be != disabled

I There is a good chance that the uart controller is alreadysupported

Serial

I FreeBSD Kernel try to resolve the /chosen/std{in,out}I Fallback on dtb node named serial0

I Node need it’s status to be != disabled

I There is a good chance that the uart controller is alreadysupported

First Boot

I uart + loader.efi = kernel booting

I using a mfsroot can be handy

First Boot

I uart + loader.efi = kernel booting

I using a mfsroot can be handy

Device Driver

I Now you can write device drivers !!!

I Well no, you need clocks and resets support first

Device Driver

I Now you can write device drivers !!!

I Well no, you need clocks and resets support first

Clocks

I 24Mhz oscilator on the board

I SoC derive some PLLs based on it

I Peripherals clocks are derived from PLLs

I Peripherals clocks can choose between multiples parent

I Each SoCs is different

I Most of the time Vendors reuse the clock models betweenSoCs

Clocks

I 24Mhz oscilator on the board

I SoC derive some PLLs based on it

I Peripherals clocks are derived from PLLs

I Peripherals clocks can choose between multiples parent

I Each SoCs is different

I Most of the time Vendors reuse the clock models betweenSoCs

Clocks

I 24Mhz oscilator on the board

I SoC derive some PLLs based on it

I Peripherals clocks are derived from PLLs

I Peripherals clocks can choose between multiples parent

I Each SoCs is different

I Most of the time Vendors reuse the clock models betweenSoCs

Clocks

I 24Mhz oscilator on the board

I SoC derive some PLLs based on it

I Peripherals clocks are derived from PLLs

I Peripherals clocks can choose between multiples parent

I Each SoCs is different

I Most of the time Vendors reuse the clock models betweenSoCs

Clocks

I 24Mhz oscilator on the board

I SoC derive some PLLs based on it

I Peripherals clocks are derived from PLLs

I Peripherals clocks can choose between multiples parent

I Each SoCs is different

I Most of the time Vendors reuse the clock models betweenSoCs

Clocks

I 24Mhz oscilator on the board

I SoC derive some PLLs based on it

I Peripherals clocks are derived from PLLs

I Peripherals clocks can choose between multiples parent

I Each SoCs is different

I Most of the time Vendors reuse the clock models betweenSoCs

Clocks

I 24Mhz oscilator on the board

I SoC derive some PLLs based on it

I Peripherals clocks are derived from PLLs

I Peripherals clocks can choose between multiples parent

I Each SoCs is different

I Most of the time Vendors reuse the clock models betweenSoCs

Resets

I Active/Deactivate the peripheral

I Usually just a bit in one register

Resets

I Active/Deactivate the peripheral

I Usually just a bit in one register

How to manage clocks

I Calling socname clock blah(uint64 t freq, bool enable)

I It means a lot of if/else in driver codeNo generic way to manage clocks and clock/parentrelationshipNo code reuse between SoCs (or just a little)

I Right way is to use the clock api

How to manage clocks

I Calling socname clock blah(uint64 t freq, bool enable)

I It means a lot of if/else in driver codeNo generic way to manage clocks and clock/parentrelationshipNo code reuse between SoCs (or just a little)

I Right way is to use the clock api

How to manage clocks

I Calling socname clock blah(uint64 t freq, bool enable)

I It means a lot of if/else in driver codeNo generic way to manage clocks and clock/parentrelationshipNo code reuse between SoCs (or just a little)

I Right way is to use the clock api

Clock API

I First appeared in FreeBSD 11, work done bymmel@FreeBSD.Org

I Used for Nvidia Tegra, Allwinner and RockChip SoCs

I Clock driver registers clocks

I Driver can enable/disable/change frequency of clock in aSoCs independant way.

I Sadly no man pages

Clock API

I First appeared in FreeBSD 11, work done bymmel@FreeBSD.Org

I Used for Nvidia Tegra, Allwinner and RockChip SoCs

I Clock driver registers clocks

I Driver can enable/disable/change frequency of clock in aSoCs independant way.

I Sadly no man pages

Clock API

I First appeared in FreeBSD 11, work done bymmel@FreeBSD.Org

I Used for Nvidia Tegra, Allwinner and RockChip SoCs

I Clock driver registers clocks

I Driver can enable/disable/change frequency of clock in aSoCs independant way.

I Sadly no man pages

Clock API

I First appeared in FreeBSD 11, work done bymmel@FreeBSD.Org

I Used for Nvidia Tegra, Allwinner and RockChip SoCs

I Clock driver registers clocks

I Driver can enable/disable/change frequency of clock in aSoCs independant way.

I Sadly no man pages

Clock API

I First appeared in FreeBSD 11, work done bymmel@FreeBSD.Org

I Used for Nvidia Tegra, Allwinner and RockChip SoCs

I Clock driver registers clocks

I Driver can enable/disable/change frequency of clock in aSoCs independant way.

I Sadly no man pages

Clock API - Clock type

I basic clocks type exists

I clk fixed : Either a fixed frequency or child of another clock+ multiplier or divider

I clk div : Support fractional divider or divider table

I clk mux : Simple multiple parent clock

I All SoCs specific clock needs to be created

Clock API - Clock type

I basic clocks type exists

I clk fixed : Either a fixed frequency or child of another clock+ multiplier or divider

I clk div : Support fractional divider or divider table

I clk mux : Simple multiple parent clock

I All SoCs specific clock needs to be created

Clock API - Clock type

I basic clocks type exists

I clk fixed : Either a fixed frequency or child of another clock+ multiplier or divider

I clk div : Support fractional divider or divider table

I clk mux : Simple multiple parent clock

I All SoCs specific clock needs to be created

Clock API - Clock type

I basic clocks type exists

I clk fixed : Either a fixed frequency or child of another clock+ multiplier or divider

I clk div : Support fractional divider or divider table

I clk mux : Simple multiple parent clock

I All SoCs specific clock needs to be created

Clock API - Clock type

I basic clocks type exists

I clk fixed : Either a fixed frequency or child of another clock+ multiplier or divider

I clk div : Support fractional divider or divider table

I clk mux : Simple multiple parent clock

I All SoCs specific clock needs to be created

Clock API - Create a clock type

I Subclass the clknode class (See clknode if.m)

I clknode init is called during the clock registrationShould init the parent(s)

I clknode setgate Enable/Disable the clock

I clknode setmux Switch parent

I clknode recalc Refresh the cached value of the current clockfrequency

I clknode setfreq Change the frequency of the clock

Clock API - Create a clock type

I Subclass the clknode class (See clknode if.m)

I clknode init is called during the clock registrationShould init the parent(s)

I clknode setgate Enable/Disable the clock

I clknode setmux Switch parent

I clknode recalc Refresh the cached value of the current clockfrequency

I clknode setfreq Change the frequency of the clock

Clock API - Create a clock type

I Subclass the clknode class (See clknode if.m)

I clknode init is called during the clock registrationShould init the parent(s)

I clknode setgate Enable/Disable the clock

I clknode setmux Switch parent

I clknode recalc Refresh the cached value of the current clockfrequency

I clknode setfreq Change the frequency of the clock

Clock API - Create a clock type

I Subclass the clknode class (See clknode if.m)

I clknode init is called during the clock registrationShould init the parent(s)

I clknode setgate Enable/Disable the clock

I clknode setmux Switch parent

I clknode recalc Refresh the cached value of the current clockfrequency

I clknode setfreq Change the frequency of the clock

Clock API - Create a clock type

I Subclass the clknode class (See clknode if.m)

I clknode init is called during the clock registrationShould init the parent(s)

I clknode setgate Enable/Disable the clock

I clknode setmux Switch parent

I clknode recalc Refresh the cached value of the current clockfrequency

I clknode setfreq Change the frequency of the clock

Clock API - Create a clock type

I Subclass the clknode class (See clknode if.m)

I clknode init is called during the clock registrationShould init the parent(s)

I clknode setgate Enable/Disable the clock

I clknode setmux Switch parent

I clknode recalc Refresh the cached value of the current clockfrequency

I clknode setfreq Change the frequency of the clock

Clock API - Create a Clock Unit Driver

I 1) Device create a clock domain with clkdom create

I 2) Create a clknode with clknode create

I 3) Register the clknode with clknode register

I Repeat 2 and 3 for every clock on the SoC and finalize theclock domain with clkdom finit

I Use clk set assigned to parse the ’assigned-clock’ properties

Clock API - Create a Clock Unit Driver

I 1) Device create a clock domain with clkdom create

I 2) Create a clknode with clknode create

I 3) Register the clknode with clknode register

I Repeat 2 and 3 for every clock on the SoC and finalize theclock domain with clkdom finit

I Use clk set assigned to parse the ’assigned-clock’ properties

Clock API - Create a Clock Unit Driver

I 1) Device create a clock domain with clkdom create

I 2) Create a clknode with clknode create

I 3) Register the clknode with clknode register

I Repeat 2 and 3 for every clock on the SoC and finalize theclock domain with clkdom finit

I Use clk set assigned to parse the ’assigned-clock’ properties

Clock API - Create a Clock Unit Driver

I 1) Device create a clock domain with clkdom create

I 2) Create a clknode with clknode create

I 3) Register the clknode with clknode register

I Repeat 2 and 3 for every clock on the SoC and finalize theclock domain with clkdom finit

I Use clk set assigned to parse the ’assigned-clock’ properties

Clock API - Create a Clock Unit Driver

I 1) Device create a clock domain with clkdom create

I 2) Create a clknode with clknode create

I 3) Register the clknode with clknode register

I Repeat 2 and 3 for every clock on the SoC and finalize theclock domain with clkdom finit

I Use clk set assigned to parse the ’assigned-clock’ properties

assigned-clock example (1)

assigned-clocks =

<&cru DCLK_LCDC>, <&cru SCLK_PDM>,

<&cru SCLK_RTC32K>, <&cru SCLK_UART0>,

<&cru SCLK_UART1>, <&cru SCLK_UART2>,

<&cru ACLK_BUS_PRE>, <&cru ACLK_PERI_PRE>,

<&cru ACLK_VIO_PRE>, <&cru ACLK_RGA_PRE>,

<&cru ACLK_VOP_PRE>, <&cru ACLK_RKVDEC_PRE>,

<&cru ACLK_RKVENC>, <&cru ACLK_VPU_PRE>,

<&cru SCLK_VDEC_CABAC>, <&cru SCLK_VDEC_CORE>,

<&cru SCLK_VENC_CORE>, <&cru SCLK_VENC_DSP>,

<&cru SCLK_SDIO>, <&cru SCLK_TSP>,

<&cru SCLK_WIFI>, <&cru ARMCLK>,

<&cru PLL_GPLL>, <&cru PLL_CPLL>,

<&cru ACLK_BUS_PRE>, <&cru HCLK_BUS_PRE>,

<&cru PCLK_BUS_PRE>, <&cru ACLK_PERI_PRE>,

<&cru HCLK_PERI>, <&cru PCLK_PERI>,

<&cru SCLK_RTC32K>;

assigned-clock example (2)

assigned-clock-parents =

<&cru HDMIPHY>, <&cru PLL_APLL>,

<&cru PLL_GPLL>, <&xin24m>,

<&xin24m>, <&xin24m>;

assigned-clock-rates =

<0>, <61440000>,

<0>, <24000000>,

<24000000>, <24000000>,

<15000000>, <15000000>,

<100000000>, <100000000>,

<100000000>, <100000000>,

<50000000>, <100000000>,

<100000000>, <100000000>,

<50000000>, <50000000>,

<50000000>, <50000000>,

<24000000>, <600000000>,

<491520000>, <1200000000>,

<150000000>, <75000000>,

<75000000>, <150000000>,

<75000000>, <75000000>,

<32768>;

Clock API - Create a Reset provider

I Usually same device as the clock unit

I Two DEVMETHODs hwreset assert and hwreset is asserted

I Register as a reset provider with hwreset register ofw provider

Clock API - Create a Reset provider

I Usually same device as the clock unit

I Two DEVMETHODs hwreset assert and hwreset is asserted

I Register as a reset provider with hwreset register ofw provider

Clock API - Create a Reset provider

I Usually same device as the clock unit

I Two DEVMETHODs hwreset assert and hwreset is asserted

I Register as a reset provider with hwreset register ofw provider

Clock API - Driver usage

I Clocks for devices are standardized

I Device driver get the clock using clk get by ofw nameor clk get by ofw index

I Enable/Disable using clk enable,disable,stop

I Set/Get frequency using clk set,get freq

I Free the clock using clk release

Clock API - Driver usage

I Clocks for devices are standardized

I Device driver get the clock using clk get by ofw nameor clk get by ofw index

I Enable/Disable using clk enable,disable,stop

I Set/Get frequency using clk set,get freq

I Free the clock using clk release

Clock API - Driver usage

I Clocks for devices are standardized

I Device driver get the clock using clk get by ofw nameor clk get by ofw index

I Enable/Disable using clk enable,disable,stop

I Set/Get frequency using clk set,get freq

I Free the clock using clk release

Clock API - Driver usage

I Clocks for devices are standardized

I Device driver get the clock using clk get by ofw nameor clk get by ofw index

I Enable/Disable using clk enable,disable,stop

I Set/Get frequency using clk set,get freq

I Free the clock using clk release

Clock API - Driver usage

I Clocks for devices are standardized

I Device driver get the clock using clk get by ofw nameor clk get by ofw index

I Enable/Disable using clk enable,disable,stop

I Set/Get frequency using clk set,get freq

I Free the clock using clk release

Clock API - Advices

I Starts we a few clocks - Mandatory PLLs, AHB clocks etc ...Also start with no set freq method

I Use clkdom dump after clkdom finit under boot verbose

I Use the hw.clock sysctl

I Make sure that your clock is really working and that it is not abootloader leftover

Clock API - Advices

I Starts we a few clocks - Mandatory PLLs, AHB clocks etc ...Also start with no set freq method

I Use clkdom dump after clkdom finit under boot verbose

I Use the hw.clock sysctl

I Make sure that your clock is really working and that it is not abootloader leftover

Clock API - Advices

I Starts we a few clocks - Mandatory PLLs, AHB clocks etc ...Also start with no set freq method

I Use clkdom dump after clkdom finit under boot verbose

I Use the hw.clock sysctl

I Make sure that your clock is really working and that it is not abootloader leftover

Clock API - Advices

I Starts we a few clocks - Mandatory PLLs, AHB clocks etc ...Also start with no set freq method

I Use clkdom dump after clkdom finit under boot verbose

I Use the hw.clock sysctl

I Make sure that your clock is really working and that it is not abootloader leftover

Device Driver

I Now you can write device drivers !!!

I Check if a driver already exists in the treeSome Vendor often use a common IP as a base

I Beware of docs, sometimes you need to read linux drivers ...

Device Driver

I Now you can write device drivers !!!

I Check if a driver already exists in the treeSome Vendor often use a common IP as a base

I Beware of docs, sometimes you need to read linux drivers ...

Device Driver

I Now you can write device drivers !!!

I Check if a driver already exists in the treeSome Vendor often use a common IP as a base

I Beware of docs, sometimes you need to read linux drivers ...

Questions ?Emmanuel Vadotmanu@freebsd.org

Twitter: @manuvadotFreelance contractor available for work