Embed Size (px)
Citation preview
Использование maven для сборки больших модульных C++ проектов на примере Odin Service Automation
Михаил Кочеров
Sr. Software Developer @ Odin / Parallels
Agenda
It’s all about “last mile” – how to effectively build a product from
your C++ code
1. Product description
2. Old build technology
3. Maven overview
4. Maven for C++ in OSA
5. Adoption costs & conclusion
1
Odin Service Automation: Purpose and Functions
http://www.odin.com/products/automation/product-overview/
Odin Service Automation – platform to automate cloud services delivery:
– Shared web hosting (Apache, IIS)
– VPS hosting (Virtuozzo)
– Mail hosting (Exchange, Qmail)
– APS-enabled: Office 365, Lync, etc.
• Automates full business cycle of cloud services delivery: online-store,
order placement, payments processing, service provisioning, upsales,
billing
• Users – Cloud Service Providers managing Data Centers
2
Odin Service Automation: Traits
• Started in 1999, more than >12 major releases
• > 300 of devs / QA / PM in MSK and NSK
• Multiplatform:
– Windows: MS Windows Server 2008, 2012
– Linux: CentOS / RHEL 5, 6, 7
• Distributed deployment: affects testing
• Multilingual: C++, C, Java, C#, Python
3
Odin Service Automation: Structure
4
Apache
IIS
chief
SaaS
VPS
Exchange
C++ / JAVA
PLATFORM
… - C++ service controller
- communication via CORBA
CORBA
5
Technology to build distributed programs in a language
agnostic way
APS SC
Apache IDL file
C++ Client stub
Apache SC
C++ Server stub
call
generate generate
- C++ CORBA derivatives: .h, .cpp, .inl, …
Odin Service Automation: Source Code
6
Lang. What is written Files count
LOC
C/C++ PlatformLibrariesService controllers
5300* 1.5M
Java PlatformUI
7100 1M
C# Windows Provisioning Engine
830 325K
Python Upgrade utilities 300 60K
TOTAL 13530 2.9M
80
10
10
Build Time Consumers (%)
C++ Java Other
Factors which Affects Build System
• Multiplatform – different compilers and tools
• Rich dependencies between service controllers –
correct build order of service controllers is req-d
• Large – small overheads starts to matter
• Distributed deployment – need expensive resources to
check if build is valid (BVT test), not available for devs
• Many people – many commits which could break the
build, ideally need a build for each commit in order to
test it7
Old Build System Overview
8
• BSD make (pmake) on Linux, MSBuild on
Windows
• Harness scripts on bash + python + perl
• C++ distributed compilation: Incredibuild / Distcc
• ccache
Flaws of An Old Build System, p. I
9
1) High “cost of ownership”: hand written makefiles, no
frontend (cmake or qmake):.for _t in all build package rpm install clean cleanup
${_t}: ${BUILD:S,^,${_t}-,}
.endfor
…
_path-${_c}= ${_srcs-${_c}:H:u:S,^,${.CURDIR}/,}
2) “Monolithic” nature of a build – problem in one module
blocks whole build
Flaws of An Old Build System, p. II
10
3) Bad incremental builds
– Non-reliable
– Slow
Frequent CLEAN
builds
Windows Linux
other 20 18
c++ 75 58
7558
20
18
0
10
20
30
40
50
60
70
80
90
100
Min
ute
s
Clean Build Duration (old tech.)
Build Process
11
1. Read makefiles
2. Build product dep. graph
3. Recurse for each component according to dep. graph
build.mkbuild.master.mk
build.submake.mkcomponents.mk
components.sc.mk…
Submake?
2. Build component dep. graph
3. Build component
+-
Make: Curse of the Recurse, p. I
12
a.cpp b.cpp c.cpp d.cpp g.cppf.cppe.cpp
l.cpp
j.cppi.cpp
a.h b.h c.h d.h g.hf.he.h
l.h
j.h
h.hk.h
i.h
SC1 SC2 SC…n
AllSubmake 1
Submake 2 Submake n
Make: Curse of the Recurse, p. II
13
GIGO principle: Garbage In, Garbage Out
• Poorly defined C++ projects boundaries – no isolation:
implicit dependencies between projects by referencing
files from several global pools
• Aggravated by the use of auto-generated C++ CORBA
derivatives
• Manual tweaking of a build order
Make: Slowness Reasons
14
• Expensive submakes: each submakes analyzes the same
things as its peers and its master
• Too much overhead on externals programs: C++ auto deps,
files copying, etc.
OSA Build Technology Requirements
15
• SUPPORT MULTIPROJECT CONFIGURATION WITH
COMPONENTS ISOLATION
• FAST COMPILATION OF C++
• FAST AND RELIABLE INCREMENTAL BUILDS
• Break “monolithic” nature of builds - modularization
• Lower “Cost of ownership”: get rid of scripts “zoo”, easy
customization of a build process (C++ CORBA derivatives
generation, EDL-processing, PPM packaging, etc.)
What is an Apache Maven?
16
• Build automation tool
• De-facto – industry standard for managing Java projects
• 60% of Java developers use Maven
• More that 100 000 Open Source Projects is managed by
Maven
http://maven.apache.org
Why Apache Maven? p. I
17
1) Inherent support of multiproject configuration
2) Dependency management and versioning
• Solves common dep. management problems: transitive
dependencies, cycles, different versioning arbitration, scoping
• Inherent support of artifact repositories: local, remote
• Sophisticated artifacts versioning: coordinates { groupId,
artifactId, version, packaging, classifier }
Why Apache Maven? p. II
18
3) Customization through
“Build Lifecycle” concept
• Predefined set of lifecycles:
default, clean, site
• Lifecycle – set of phases.
• Concrete actions performed on
each phase is customized via
goals
• Goals are executed via plugins
Phase Goal
process-resources
resources:resources
compile compiler:compile
process-test-resources
resources:testResources
test-compilecompiler:testCompile
test surefire:test
package jar:jar
install install:install
deploy deploy:deploy
Why Apache Maven? p. III
19
4) Spare costs through “Convention over
Configuration” principle
• Convention of source files layout
• Unified list of plugins used during build
• Build lifecycle
The idea: need to specify only exceptions,
everything else is by default
pom.xml
20
Make vs Maven
21
A B
c d e
All
f
B
e f
Make Maven
PROJECT
DepA
DepB
DepC
MAVENREPOSITORY
OSA “CELL” Concept
22
CELL – unified, (deps, lin/win, assembly) isolated C++
maven project for OSA which consumes and contributes
C++ artifacts (proxies) via maven bus
1.CONSUMEPROXIES
2. BUILD
3. CONTRIBUTE
PROXIES
Other cells:C++ CORBA derivatives
C++ headersC++ obj filesC++ libraries
This cell:C++ CORBA derivatives
C++ headersC++ obj filesC++ libraries
Cells Dependency Graph
23
cell
Apache
cell
BrandingManager
cell
pem_clientcell
chiefcell
libservices
cell
ConfigurationManager
EDL
EDLCORBA-FULL
CORBA-FULL,
CORBA-SDK,
HEADER,
LINKAGE
CORBA-FULL,
CORBA-SDK,
HEADER,
LINKAGE
EDL
CORBA-CLIENT,
EDL
CORBA-FULL,
CORBA-SDK,
HEADER,
LINKAGE
EDL CORBA-FULL,
CORBA-SDK,
HEADER,
LINKAGE
EDL
Cells Isolation via Proxies
24
cell
Apache
cell
ConfigurationManager
No C++ cross
component
dependencies
chief
proxies
C++ Compilation on Maven
25
Plugin Pros Cons
NAR maven pluginhttp://maven-nar.github.io/
Power-full Over-engineeredLimits use of classifiers
native-maven-pluginhttp://www.mojohaus.org/maven-native/native-maven-plugin/
“Good enough” for OSA Development abandoned
Dep. Graph
Basics of native-maven-plugin
26
1. Initialize
2. Analyze
C++ sourcesInclude dirs.
Lib dirs.obj fileslibraries
Any changes?
3. Rebuild
+
-
Compiler opts.Linker opts.Other opts.
A B
c d e
target
f
B
e f
PADependencyImplementor, native-maven-plugin and
Proxies
27
MAVEN LOCAL REPOSITORY (~/.m2)
CELLs’ proxies(zipped)
native-maven-plugin
UNZIPPED PROXIES- Headers- Obj. files- Inl files
- C++ CORBA derivatives
2. Unzip if
missing to a
shared
location
3. Feed files
locations
(for compile & link)
1. Cell
dependencies:
list of proxies
PADependencyImplementor
native-maven-plugin and Proxies
28
g++ -g -pipe -fPIC -DPIC -Wall -W -fno-strict-aliasing -Werror -Wno-parentheses -D_REENTRANT
...
-I/root/.m2/_pa_proxy/com-parallels-poa-sh/apache-assembly-PADEP-CORBA-FULL-RHEL6/6.1.0.2727
-I/root/.m2/_pa_proxy/com-parallels-poa-sh/iis-assembly-PADEP-CORBA-FULL-RHEL6/6.1.0.2727
-I/root/.m2/_pa_proxy/com-parallels-poa-peripheralBus/libservices-assembly-PADEP-CORBA-FULL-
RHEL6/6.1.0.2752
-I/root/.m2/_pa_proxy/com-parallels-poa-vps/vpsmanager-assembly-PADEP-CORBA-FULL-RHEL6/6.1.0.3930 -
I/root/.m2/_pa_proxy/com-parallels-poa-sh/configurationmanager-assembly-PADEP-CORBA-FULL-
RHEL6/6.1.0.2727
-I/root/.m2/_pa_proxy/com-parallels-pa/pacryptoapi-PADEP-HEADER-RHEL6/0.3.2.006
...
-o lin/target/objs/Tasks.o -c cpp/Tasks.cpp -DSINGLETON_MODULE=SAAS
C++ Compilation Tuning
29
Project Linux Maven Profile Window Maven Profile
cppProject g++, -g, -pipe, … cl.exe, /FD, /EHsc, …
cppExeProject <Lin linker opts> <Win linker opts>
cppDynamicProject -shared, … /DLL, …
cppStaticProject ar, … /LIB, …
-
C++ Build Lifecycle
30
Phase Plugin Task
initialize native-maven-plugin Read project configuration
PADependencyImplementor
Prepare proxies, pass it to native-maven-plugins
Bin2CProcessor Convert binaries (e.g. png) to a C-code
generate-sources
EDLProcessor Process .edl files which contain error description
IDLProcessor Process CORBA interface definition files .idl
precompile native-maven-plugin Generate precompiled headers (.pch)
compile native-maven-plugin Compile & link
package DebugStripper Extract debug info from compiled files to a separate ones
OSA Modularity
31
platform{ cells }
peripheralBus{ cells }sh
{ cells }
vps{ cells } dedicated
{ cells }
…{ cells }
mc{ cells }
OSA{ cells }
BENEFITS• Extra fast OSA
build• No failure
propagation• Custom builds
OSA Build Duration: make-based vs maven-based
32
95
75
20,00
11
0
10
20
30
40
50
60
70
80
90
100
Win32 Linux
Min
ute
s
Platforms
Make
Clean Incremental
39
25
5,50 40
10
20
30
40
50
60
70
80
90
100
Win32 Linux
Min
ute
s
Platforms
Maven
Clean Incremental
~63% reduction
for clean build
~68% reduction
for incr. build
Factors of a Build Speed-up
33
1) Modularity: extensive reuse of prebuild things
2) Much quicker dependency analysis: scope of
search is narrower
3) No expensive subprograms: C++ autodeps,
files ops., etc.
Factors of Incremental Build Reliability
34
1) Code Isolation
2) Code Isolation
3) … and Code Isolation one more time
There is no difference between clean build and
incremental – there will be exactly the same
environment for each C++ cell.
Maven for C++ in OSA: adoption costs
35
• Converted on the course of 2 OSA releases without stopping
dev. cycle
• Converted >300 projects
• Took 2 years to complete by 2 devs
• Labor cost: 4 m/y
REALLY WORTH ITS COST
Generalized Build Scheme
37
Windows• выкачать дерево
исходников
• скомпилировать
• загрузить на
collector
Сборка вместе всех артефакты от
всех платформ – оформление билда
Загрузка оформленного билда на
deliver сервер
Linux 1• выкачать дерево
исходников
• скомпилировать
• загрузить на
collector +distcc+IncrediBuild
Linux …• выкачать дерево
исходников
• скомпилировать
• загрузить на
collector
Linux n• выкачать дерево
исходников
• скомпилировать
• загрузить на
collector+distcc +distcc
