Upload
bruno-cornec
View
80
Download
0
Embed Size (px)
Citation preview
© 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
Talk Title HereAuthor Name, Company
Using Containers and Continuous Packaging to Build Native Fossology PackagesSpeakersBruno Cornec ([email protected]), Michael C. Jaeger ([email protected])
© 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
Overview: Contents
1. Introduction FOSSologyWhat is FOSSology
2. MotivationWhat FOSSology needs
3. Introduction Project BuilderThe ProjectBuilder Project
4. Build Native Fossology PackagesTo get container running in the continuous build
5. ConclusionWhere to see it
2 Page 2
© 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
The Problem Actually
Distributing open source software requires to
∙ Provide licenses of involved software
∙ Provide copyright statements of involved authors
∙ Provide disclaimers
∙ … and much more
You know these examples
Page 4
© 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
It is about finding licenses
∙ License texts
∙ References to licenses
∙ Written texts explaining licensing
∙ License relevant statements
Finding Licenses
Page 5
© 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
What is FOSSology?
A Web server application for license and copyright compliance of software components.
FOSSology Projecthttps://www.fossology.org/
∙ Published first in 2008, GPL-2.0
∙ 2015: Linux Foundation collaboration project
∙ Web server based and command line interfaces
∙ Scanning agents searching for license and copyright relevant hits (and more …)
∙ A multi-user / multi-tenant Web UI for review organizing clearing job
FOSSology Developmenthttps://www.github.com/fossology/fossology
▪ Standard Web application stack:▪ Linux, Apache 2, PostgreSQL, PHP,
▪ Web-based UI in PHP, but scannerswritten in C / C++
▪ Two ways to interact: ▪ Web user interface▪ Command line utilities
Page 6
© 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
How does FOSSology work?
See more details the Basic Workflow Description: https://www.fossology.org/get-started/basic-workflow
▪ Upload an open source package to the server▪ Select scan agents that analyze the software
▪ Review what scanners have found▪ Review license occurrences and correct findings if necessary
▪ Generate report output▪ For example list of licenses or SPDX
Upload OSS Package
Review and Adjust (“Clearing”)
Generate
Page 7
© 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
What is the point of FOSSology?
See more details the Basic Workflow Description: https://www.fossology.org/get-started/basic-workflow
▪ Upload an open source package to the server▪ Select scan agents that analyze the software
▪ Review what scanners have found▪ Review license occurrences and correct findings if necessary
▪ Generate report output▪ For example list of licenses or SPDX
Upload OSS Package
Review and Adjust (“Clearing”)
Generate
Page 8
Page 9© 2016-2017 Siemens AG, Linux Foundation - CC-BY-SA 4.0Open Source Summit Europe 2017
Using FOSSology with this Example
∙ It is natural that an OSS project reuses available https://github.com/fossology/fossology
∙ Likely OSS from other projects is found
∙ For example, FOSSology will find 25 other licensing relevant text occurrences in Apache thrift
Open Source and Reuse
9 Page 9
© 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
The Problem Actually
∙ ~ “creating binaries for linux is difficult” (starts at 5:40)
∙ https://www.youtube.com/watch?v=qHGTs1NSB1s
∙ Many linux distros with own package universe
∙ Different distros and different versions of these
∙ E.g. Packages dependencies on debian 8 change with debian 9
∙ Even within Debian 8 postgresql changes ...
See Linus Torvalds
Page 12
© 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
FOSSology Demand
∙ Debian, Ubuntu, CentOS and Fedora
∙ To efficiently build packages for these
∙ = efficiently means not to have manual step for each distro
∙ also means dealing with specificities of each distro/version(dependencies, availability of packages, …)
Support (at least) a basic set of Linux Distros
Page 13
© 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
Technically
∙ Different Distros required (and their versions)
∙ Integration in the CI
∙ State-of-the-art: Docker
∙ Support of two main package building formats: RPM and Deb
It is about building Linux packages
Page 14
© 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
Introduction to
Project-Builder.org
© 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
Project-Builder.org goal
“Make upstream projects life easier with regards to packaging their software”
Page 16
© 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
Project-Builder.org goal
“Make mylife easier with regards to packaging my software”
Page 17
© 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
Benefits from Continuous Packaging
● Packaging should be a project concern as well as coding, testing, installing, .... especially for smaller projects
● Packaging as your only way of delivery (not a dream)● Minimal overhead, maximum benefit:● Consistancy and reproduceability for devs and users● Distribution & deployment server integration,● Consistency with distribution and avoids dependecy hell for consumers● Packaging as a marketing activity for the upstream project. Easy way to extend
your user base, and improve your community relationship and is a “competitive advantage”.
● New mantra: “Package early, package always”● THE SOLUTION IS INDEED CONTINUOUS PACKAGING (whatever the tool)
Page 18
© 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
Project-Builder.org goals
● VCS agnostic: no VCS but guys it's 21st century now, SVN, CVS, Mercurial, GIT and GIT/SVN, SVK....
● OS agnostic: Linux: RPM, deb, ebuild, slack based, ... 150+ distro tuples made and counting – repositories for yum, urpmi, apt. Solaris pkg.
● Build environment agnostic: local, VM (QEMU, KVM...), VE (Docker, chroot, rpmbootstrap, rinse, mock, debootstrap...), RM (build farm)
● No project impact: preserves the md5sum of the delivered upstream sources. Can be completely external to the upstream project.
● Avoids duplication of code and metadata● THE SOLUTION IS INDEED CONTINUOUS PACKAGING (with project-builder.org !)
Page 19
© 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
Project-Builder.org architecture
Page 20
© 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
Build Native Fossology Packages
Demonstration !!
© 2017 HPE, Siemens - CC-BY-SA 4.0Open Source Summit Europe 2017
Problem encountered
● fossology build issues● project-builder.org bugs● composer phar !● build infrastructure● introduction in CI toolset
Page 22