Dealing with large code bases. cd ams meetup

[email protected] | @sadovnikov

Dealing with Large Code BasesContinuous Delivery Amsterdam

August 2016


Viktor Sadovnikov

● 1985 - first keyboard touch ● 1988 - first line of code● 1993 - first computer at home● 1994 - first paid line of code● 2001 - first code to test another code● 2005 - first build server run● 2007 - first automated deployments

2

@sadovnikov


Common Build Project

3



Initial Scope

4

Source Repository From CVS to SVN

Build Tool From mix of Maven, Ant and Eclipse to Maven;Dependencies from Nexus

Artifacts Repository From “something” to Nexus; deployable from Jenkins only;

Build Server From “something” to Jenkins;Automate creation of builds jobs

Migration Done by projects themselves or (mainly stale) by us;Extensive migration checklist



“Bonus” Scope

5

Platform Moving builds from Windows to Linux

Upgrades Upgrading JDK during migration

Packaging 2 tailored packaging types

Deployment Number of applications shared dependencies at runtime



Volumes

6

Maven modules (artifacts)

3.500 (trunks only): 1.760 jars, 250 wars

External dependencies 2.200 libraries

Build jobs +/- 4.000 jobs (including branches)

Jenkins installations 12 instances; later moved to 4 masters with pool of slaves


Challenges and Lack of Tooling

7

Tooling for Software

Development


#1. Finding the Code

● Finding code of a dependency in SVN to check available tags (versions), to find out correct group and artifact IDs, to contact developers;

● Enquiries about existing projects

8

● FishEye from Atlassian… works only on SVN● SonarQube from SonarSource… finds everything

Solutions:


#2. Changing Shared Libraries

● Can I stop compiling for JDK 5?● How much effort will my change cause?● Do they use correct (after migration) and recent versions?● What is used? How is used?

9

● NoneSolutions:


#3. Conflicts of Versions

● Direct dependency does not match transitive dependency● What needs to be released before I can use changed shared library?● Why do I get “entire world” as my compile dependency?● (WRONG) Why do my builds pick-up old SNAPSHOT?

10

● Manual execution of Maven Dependency plugin (-Dverbose)● Parsing the results● On level of modules, not projects

Solutions:


#4. Finding the Builds

● 12 Jenkins instances and 4.000 build jobs● Hard requirement of every project migration

11

● grep on build job configuration files… on all 12 servers ● Ctrl+F on Jenkins “all” page… unless the builds are renamed● Referencing builds in pom.xml… gets obsolete quickly

Solutions:


development environment Cross examiner

12


Evolution of deCrex

deCrex - Development Environment Cross Examiner - scanner and analyzer of source repositories and build servers

13

main() to generate JSON.

Running from IntelliJ

Daily triggered from Jenkins to publish JSON

An intranet application,

running internally

Publically available

installation to scan OSS

Started simple... ...and could not stop

https://ossa.jv-ration.com


#1. Finding the Code

● Search for Maven Modules and builds

● Based on IDs, names, descriptions

14


#2. Changing Shared Libraries

● List of direct dependees● Versions direct

dependees depend on● Quick access to details

of dependees

15


#3. Conflicts of Versions

● Build Jobs you project depends on

● 7 Releases to push changes in “Spring Data Build” to “Spring Boot Build”

● There are cycle dependencies among builds

16


#4. Finding the Builds

● From modules to builds● From builds to modules

17


Dealing with Code: OSS approach in Enterprises

18


Dealing with Code: many vs. mono

19


Two Approaches

20

OSS Approach. Many-repos

● Every logically complete unit (project) is in its own repo with its own release cycle

● Build tools resolve dependencies using artifact repositories

● Most familiar approach, heavily used in OSS projects

Monolithic Repos. Monorepos

● Single repository with all code● No version constraints for

internal code● All internal code is a part of the

build● Used by companies like

Google, Facebook, Deis.io, Twitter, Salesforce...


Many-repos Approach

● Clean boundaries between projects

● Easier reusable in other contexts code

● Fine-tuned access control● Lighter, faster builds

21

● Refactoring becomes an incompatible change

● Diamond dependencies and interdependencies in internal code

● Loss of atomicity of commits● Temptation to pick a version and

“stabilize” - stagnate● Not fixing bugs in “not my code”● Growth of obsolete and deprecated

code


Monorepos Approach

● Refactoring enabler: changing APIs, removing obsolete code

● Atomic commits for large changes● No switching between repos● Collaboration among teams● Less management (CI and repo)● Code Management: search,

moving files around

22

● Ownership of the code is detached from repo authorization

● Implementation problems: pulling too much, mixed commit logs, etc

● Fully distributed SCM is problematic

● Latency and requirements of builds


Recognitions of Many-Repos Shortcomings● Semantic Versioning

contract between me and you: since I can't know what you're doing, I promise not to do certain things with my library so that you can use it with some confidence

● GIT Submodules and Subtrees“you want to be able to treat the two projects as separate yet still be able to use one from within the other”

● GitLabIssues, Milestones and Merge Requests combined on group level

● @depricated● Spring Boot

79 modules under one aggregator

23


deCrex - Wrapper for Many-repos

Enables use of many-repos with clean project boundaries and code ownership, lighter builds and distributed SCM by adding

● insights on dependees of every project, ● reporting on

○ stagnated dependencies and interdependencies in internal code,○ projects without dependees - either a “delivery” or an obsolete code,

● Build Plan to ease CI setup,● searching through “monorepo”

24


Upcoming Changes● Ease of installation on premises● UI improvements● Extending support for build tools (Gradle) and servers

(Travis CI), SCM (puur GIT)

25

● Configuring “Monorepo members”● Managing “ownership” of the code● Notifying code owners about stagnated dependencies

● Interested in scanning your code?


Q&Ahttps://ossa.jv-ration.com

26

@sadovnikov