Hog tutorialDavide Cieri (MPI Munich)

10. December 2019 - L0MDT Trigger Firmware Workshop

Introduction● L0MDT firmware repository based on Hog framework○ Coordinate collaborative firmware development

○ Guarantee firmware synthesis and result reproducibility

○ Traceability of binary files

○ Exploiting Gitlab-CI for automatic synthesis and implementation when a Git MR is opened

● git clone --recurse-submodules ssh://git@gitlab.cern.ch:7999/atlas-tdaq-phase2-l0mdt-electronics/l0mdt-hdl-design.git

● Recent Hog talk at last TDAQ Week

Repository methodology

● Use of temporary feature branches:○ Developers create new temporary branch from master/devel○ Merge request to master/devel activate Gitlab-CI pipeline

● Projects can be run locally (CreateProject script)

● Repository structure follows Hog prescriptions

○ Special folder for constraints, list files, xmls, VHDL top files, Xilinx IP

Repository methodology

Repository methodology

Create Vivado Project

● Vivado projects are not pushed to the repository

● Create project with ./Hog/CreateProject.sh <proj_name>

● Run properties specified under Top/proj/proj.tcl (e.g. here)

● Project must be recreated if list files (Top/proj/list) are modified

● Hog scripts can be used to run synthesis/implementation from command line ○ ./Hog/LaunchSynthesis.sh <proj_name>○ ./Hog/LaunchImplementation.sh <proj_name>○ ./Hog/LaunchWriteBitstream.sh <proj_name>

● Projects created in VivadoProject folder with all run logs

Gitlab-CI implementation● Gitlab-CI added to the repository

● Created a CERN service account for the repo l0mdtel

● Gitlab runner run on 4 CERN Openstack machines○ 14 CPUs -> 14 parallel pipeline jobs with the current settings○ If you want to add new machine, look here○ Each machine has at least 50GB to build the project○ Vivado 2019.2 installed on /afs/cern.ch/work/l/l0mdtel/Xilinx

● So far running the pipeline till the post_route○ no pin constraints = no bitstream○ gitlab-ci.yml

Gitlab-CI implementation

● Every merge request activates the CI pipeline○ New commits to MR create a new pipeline○ First step, merge master branch into MR○ Evaluate version from most recent tag (v0.0.1)○ Tag new beta tag○ Approve merge request -> human can now merge it

● Ensure that mods have not broken any part of the firmware

● Bitstream files and run reports stored in cernbox

Hands On

● Let’s try to setup a Hog repository from scratch using a simple vivado project○ https://gitlab.cern.ch/atlas-tdaq-phase2-l0mdt-electronics/hog_tutorial_workshop

● Very simple project, targeting a Digilent Basys 3 board (Artix-7)

● Project contains 1 top_module, 1 submodule, 1 IP, 1 testbench and 1 constraint file

● Let’s try to on our VM ssh -XY l0mdtel@mdt-dcieri-01.cern.ch

cd mdt_tutorial

mkdir your_name && cd your_name

Hands On

● Project files located in

● Clone the repository and create a new branch

/home/l0mdtel/mdt_tutorial/tutorial_files

git clone --recurse-submodules ssh://git@gitlab.cern.ch:7999/atlas-tdaq-phase2-l0mdt-electronics/hog_tutorial_workshop.git

cd hog_tutorial_workshop && git checkout -b your_name

● Create the library (example) and Top folder

mkdir -p example/src example/sim example/xdc Top/example/list

Hands On● Copy source files to the respective folders

cp ../../tutorial_files/project_1/project_1.srcs/sources_1/imports/adder.vhd example/src

cp ../../tutorial_files/project_1/project_1.srcs/sources_1/imports/top_example.vhd Top/example/

cp ../../tutorial_files/project_1/project_1.srcs/constrs_1/imports/Basys3.xdc example/xdc

cp ../../tutorial_files/project_1/project_1.srcs/sim_1/imports/tb_example.vhd example/sim .,.

cp.

Hands On● Design contains also an IP core (FIFO)

mkdir -p IP/fifo_generator_1

cp ../../tutorial_files/project_1/project_1.srcs/sources_1/ip/fifo_generator_1/fifo_generator_1.xci IP/fifo_generator_1/

cp ../../tutorial_files/project_1/project_1.srcs/sources_1/ip/fifo_generator_1/fifo_generator_1.xml IP/fifo_generator_1/

Hands On● Now we need to create the list files

printf "../../example/src/adder.vhd\n../../IP/fifo_generator_1/fifo_generator_1.xci" > "Top/example/list/example.src"

printf "../../example/xdc/Basys3.xdc" > "Top/example/list/example.con"

printf "../../example/sim/tb_example.vhd" > "Top/example/list/example.sim"

Hands On● Finally we need a .tcl script for our project. A simple one is

############# modify these to match project ################

set bin_file 1

set use_questa_simulator 0

### FPGA and Vivado strategies and flows

set FPGA xc7a35tcpg236-1

set SYNTH_STRATEGY "Flow_AreaOptimized_High"

set SYNTH_FLOW "Vivado Synthesis 2018"

set IMPL_STRATEGY "Performance_ExplorePostRoutePhysOpt"

set IMPL_FLOW "Vivado Implementation 2018"

set DESIGN "[file rootname [file tail [info script]]]"

set path_repo "[file normalize [file dirname [info script]]]/../../"

source $path_repo/Hog/Tcl/create-project.tcl

Hands On● Create the file under Top/example/example.tcl

● In the tcl you can specify your strategies and the FPGA

● Have a look here: https://gitlab.cern.ch/atlas-tdaq-phase2-l0mdt-electronics/l0mdt-hdl-design/blob/devel/Top/profpga_duo_test/profpga_duo_test.tcl

● Now we are ready to create our project and to launch the synthesis

● CreateProject creates a vivado project under VivadoProjects/proj_name that can be opened with the gui

./Hog/CreateProject.sh example

./Hog/LaunchSynthesis.sh example

Hands On● Now we can launch also the implementation

● All the files must be on gitlab! Otherwise Hog cannot retrieve the SHA

git add .

git commit -a -m "my commit message"

git push --set-upstream origin my_branch

● Launch again the synthesis, it should work now

● You can check the synthesis log here:

VivadoProject/example/example.runs/synth_1/runme.log

Hands On● You can now launch the implementation

./Hog/LaunchImplementation.sh example

● If it meets timing (check the log under impl_1/runme.log), you can create the bistream file

./Hog/LaunchWriteBitstream.sh example

● Now we are ready to submit our Merge Request

Hands On● Let’s commit our changes

git commit -a -m "my changes"

git push --set-upstream origin my_branch

git commit -a -m "my changes" && git push

● Go to https://gitlab.cern.ch/atlas-tdaq-phase2-l0mdt-electronics/hog_tutorial_workshop/merge_requests and open your merge request

● This should start the Gitlab-CI pipeline○ The ci has been already configured for this design (see .gitlab-ci.yml)

● If the pipeline is successful, project maintainer can merge branch into master

Some final remarks● There are some work going on to integrate HLS into HOG○ For the moment one has to create its custom IP core and add to the

repository● If you add a file to your lists, you have to recreate the project○ Don’t forget to upload the new file to gitlab too

● Gitlab-CI job saves job reports and bit files on cernbox○ https://cernbox.cern.ch/index.php/s/LKnUoG9AjnrvgCA○ Each pipeline job creates a beta tag for tracking

Some final remarks● Hog supports also git submodules

○ Simply add your submodule to the repo via

git submodule add git_submodule_url

● Submodule files you wish to be used in your project must be specified in list/my_submodule.sub

● Hog supports also external libraries that cannot be uploaded to gitlab (e.g. profpga)○ Those must be specified in list/lib_name.ext (see here)○ HOG_EXTERNAL_PATH specifies the absolute path where the external path are located on the virtual

machine. You need to export the variable also in your local machine to create the project○ The list file must contain the file path relative to HOG_EXTERNAL_PATH and the SHA of each file○ Hog checks that each file has the correct SHA, if not sends a CRITICAL_WARNING○ If you update the file, remember also to update the SHA in the list file

Summary● Hog and Gitlab-Ci integrated in our l0mdt-hdl-design repository

● Direct contact with Hog creator (Francesco Gonnella)

● Personally active in the development of Hog

● Next step is to include external proprietary library (aka prodesign), without breaking copyright laws

● HLS integration also foreseen

● Gitlab-runner running on 4 CERN Openstack machines

Thank you for listening,any questions?

Davide Cieri (MPI Munich) - davide.cieri@cern.ch