Everything-as-code. A polyglot journey

Everything-as-codeA polyglot journey.

| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 1

About me

Mario-Leander ReimerChief Technologist, QAware [email protected]

twitter://@LeanderReimerhttp://github.com/lreimerhttp://speakerdeck.com/lreimerhttp://www.qaware.de



Which language or technology do real programmers use?


My #FirstSevenLanguages

• Pascal

• Basic

• C / C++

• Assembler

• PHP

• Java

• C#


My #LastSevenLanguages

• Java

• Groovy

• TypeScript

• Ruby

• Kotlin

• Scala

• Python





There is no unanimous opinion ...

• http://spectrum.ieee.org/computing/software/the-2015-top-ten-programming-languages

• http://spectrum.ieee.org/computing/software/the-2016-top-programming-languages

• https://www.sitepoint.com/whats-best-programming-language-learn-2015/


There is no best language!

Every language is strong in a specific domain.


Real programmers are polyglot!


The IDE is our workbench.


open fun everythingAsCode() : Boolean {

everytingIsMadeFromCode() && everythingIsMadeByCode()

}

val softwareIndustrialization = everythingAsCode()


Definition of Software Industrialization

• This has nothing to do with cheap labor!

• Automation of repetitive and laborious tasks

• Better software quality through standardized, streamlined tool chain

• Well integrated tool chain leads to a higher productivity and happiness of your team

• Better cost efficiency and competitiveness


Quest for a polyglot project archetype

• Which languages are used for specific domains in our projects?

• Which tools are used for Setup, Build, Code, Test, CI, Infrastructure, Documentation?

• What are the dos and don'ts of using a specific language or technology?

+ some wishful greenfield thinking!


The polyglot journey begins.


SEU-as-code


Lightweight Developer Provisioning

• Use a build tool for the automated creation and update of a software development environment

• Software packages are expressed as dependencies

• Gradle tasks and Groovy are used instead of shell scripting

• Everything is version controlled just like ordinary source code


plugins { id 'de.qaware.seu.as.code.base' version '2.4.0' }

import static de.qaware.seu.as.code.plugins.base.Platform.isMac

seuAsCode { seuHome = { if (isMac()) '/Volumes/Everything-as-code' else 'Y:' } projectName = 'Everything-as-code'}

dependencies { // list of software dependencies ... software 'org.groovy-lang:groovy:2.4.7' software 'org.scala-lang:scala:2.11.8' software 'org.jruby:jruby:9.1.4.0'}


Build-as-code


Why Gradle beats Maven.

• Very flexible. Gradle can build everything.

• Polyglot builds are supported easily.

• Succinct build scripts. Default conventions over configuration.

• Incremental builds, reduced build times.

• New features: Kotlin support, Composite Builds, ...

• Frequent releases. Mature and stable.


apply plugin: 'application'apply plugin: 'war'apply plugin: 'kotlin'apply plugin: 'groovy'

repositories { jcenter() }

dependencies { providedCompile 'fish.payara.extras:payara-micro:4.1.1.163' // and many more ...}

task everythingAsCode() << { println 'Everything-as-code @JavaOne2016.'}


Main-as-code


There is nothing wrong with Java as primary language!


But Kotlin is a serious alternative worth considering

as primary language.


But why Kotlin? And not Scala, Clojure, ...

• Easy to learn for a Java developer.

• Null Safety!

• Loads of other useful language features.

• Small library size.

• Good IDE support.

• Wishful greenfield thinking.


@JsonIgnoreProperties(ignoreUnknown = true)data class Book(val title: String, val isbn: String, val author: String) { }

@ApplicationScopedopen class Bookshelf { private val books = listOf(Book("The Hitchhiker's Guide to the Galaxy", "0345391802")) open fun byIsbn(isbn: String): Book? = books.find { it.isbn == isbn }}

@Path("books")@Produces(MediaType.APPLICATION_JSON)open class BookResource @Inject constructor(private val bookshelf: Bookshelf) { @GET @Path("/{isbn}") open fun byIsbn(@PathParam("isbn") isbn: String): Response { val book = bookshelf.byIsbn(isbn) return if (book != null) Response.ok(book).build() else Response.status(Status.NOT_FOUND).build() }}

@ApplicationPath("api")class BookstoreAPI : Application() { override fun getClasses() = hashSetOf(JacksonFeature::class.java, BookResource::class.java)}


Test-as-code


Groovy and Spock for general testing

class BookshelfSpec extends Specification { @Subject def bookshelf = new Bookshelf()

@Unroll def "Find book #title by ISBN #isbn"() { when: 'we search a book by ISBN' def book = bookshelf.byIsbn(isbn)

then: 'the title and author are correct' book?.title == title book?.author == author

where: isbn || title | author "0345391802" || "The Hitchhiker's Guide to the Galaxy" | "Douglas Adams" "0345391829" || "Life, the Universe and Everything" | "Douglas Adams" }}


Scala and Gatling for load testing

class BooksPerformanceTest extends Simulation { val conf = http.baseURL("http://localhost:18080").acceptHeader("application/json")

val feeder = csv("books.csv").random

val scn = scenario("Book Search") .exec(http("Get all books").get("/api/books")) .during(30 seconds) { feed(feeder) .exec(http("Get book by title ${Title}").get("/api/books?title=${Title}")) .pause(1 second) .exec(http("Get book with ISBN ${ISBN}").get("/api/books/${ISBN}")) }

setUp(scn.inject(atOnceUsers(10), rampUsers(50) over (30 seconds))) .assertions(global.responseTime.max.lessThan(5000)) .protocols(conf)}


Pipeline-as-code


Use Gradle and Docker to start Jenkins

task createJenkinsHome() { mkdir("${System.getProperty('user.home')}/Jenkins")}

task createJenkins(type: Exec, group: 'jenkinsci', dependsOn: createJenkinsHome, description: 'Create Jenkins') { commandLine 'docker', 'run', '--name', 'jenkinsci', '-p', '8080:8080', '-p', '50000:50000', '-v', "${System.getProperty('user.home')}/Jenkins:/var/jenkins_home", 'jenkinsci/jenkins'}

task startJenkins(type: Exec, group: 'jenkinsci', description: 'Start Jenkins') { commandLine 'docker', 'start', 'jenkinsci'}

task stopJenkins(type: Exec, group: 'jenkinsci', description: 'Stop Jenkins') { commandLine 'docker', 'stop', 'jenkinsci'}


Add Jenkinsfile and define pipeline#!/usr/bin/env groovy

node {

stage 'Checkout SCM'

checkout scm

stage 'Build/Analyse/Test'

sh './gradlew clean build'

archiveUnitTestResults()

archiveDistributions()

stage 'Dockerize'

sh './gradlew buildDockerImage'

stage 'Generate Documentation'

sh './gradlew asciidoctor'

}


Infrastructure-as-code


Docker, Docker, Docker, ...

FROM qaware-oss-docker-registry.bintray.io/base/debian8-jre8MAINTAINER M.-Leander Reimer <[email protected]>

RUN mkdir -p /appADD build/distributions/everything-as-code-1.0.0.tar /app

WORKDIR /app/everything-as-code-1.0.0RUN chmod 755 bin/everything-as-code

EXPOSE 18080

CMD ./bin/everything-as-code


Vagrant and Ruby for local VM setup

require 'yaml'

$setup = <<SCRIPT

sudo apt-add-repository ppa:ansible/ansible

sudo apt-get update

sudo apt-get install -y ansible sshpass

SCRIPT

Vagrant.configure("2") do |config|

config.vm.box = "ubuntu/trusty32"

settings = YAML.load_file 'src/vagrant/vagrant.yml'

config.vm.provider "virtualbox" do |vb|

vb.name = settings['vm']['name']

vb.gui = false

vb.memory = "512"

end

config.vm.provision "shell", inline: $setup

end


Provisioning with Ansible (and Python)

---

# file: jenkinsci.yml

- hosts: jenkinsci

remote_user: root

tasks:

- debug: msg="Creating a Jenkins pipeline job on {{ inventory_hostname }}"

- jenkins_job:

name: Everything-as-code Pipeline

config: "{{ lookup('file', 'templates/pipeline-job.xml') }}"

url: "http://{{ inventory_hostname }}"

user: admin

password: admin


Cluster orchestration with K8S---apiVersion: extensions/v1beta1kind: Deploymentmetadata: name: everything-as-codespec: replicas: 1 template: metadata: labels: tier: backend spec: containers: - name: everything-as-code image: "qaware-oss-docker-registry.bintray.io/lreimer/everything-as-code:1.0.0" ports: - containerPort: 18080 env: - name: PORT value: 18080


Cluster orchestration with DC/OS{ "id": "everything-as-code", "instances": 1, "cpus": 0.25, "mem": 256, "container": { "type": "DOCKER", "docker": { "image": "qaware-oss-docker-registry.bintray.io/lreimer/everything-as-code:1.0.0", "network": "BRIDGE", "portMappings": [ { "hostPort": 18080, "containerPort": 18080, "protocol": "tcp", "servicePort": 18080 } ] } }, "env": { "PORT": 18080 }}


Documentation-as-code


Yes, we need documentation!

• And no, the source code is not enough.

• Writing technical docs with Word is ! " #

• Documentation should be located next to the source code: change code, change docs.

• It should be easy, quick and fun to write.

• Support for code, images, UML diagrams, ...


AsciidoctorJ and Gradle to the rescue

plugins { id "org.asciidoctor.convert" version "1.5.3" }

asciidoctorj { version = '1.5.4.1' }

asciidoctor {

sourceDir 'src/docs/architecture'

resources {

from('src/docs/architecture') {

include 'images/**/*.png'

include 'images/**/*.jpg'

}

}

backends 'html5'

options doctype: 'article'

attributes 'source-highlighter': 'coderay'

}


// Example architecture documentation using arc42 (https://arc42.github.io):imagesdir: ./images

= image:qaware-logo.png[QAware GmbH,2016] Everything-as-code:toc-title: Table of Contents:toc:

[[section-introduction-and-goals]]== Introduction and Goals

The introduction to the architecture documentation should list the driving forcesthat software architects must consider in their decisions.

=== Requirements Overview

=== Quality Goals

=== Stakeholders

<<<<include::02_architecture_constraints.adoc[]

// further includes for the remaining sections


Presentation-as-code


These slides are written in Markdown.

---## [fit] These slides are written in Markdown.

- This is for real programmers! :smiley:- Several open source projects available- Use HTML and JavaScript alternatively.

---


What's the takeaway from this

journey?


Beware of the abstractions!


Everyone needs to do his homework.

• Developers: be polyglot, keep learning!

• Architects: choose the right language or tool for the job!

• Project Managers: give your techies freedom!

• Universities: teach polyglotism!


Have some fun and create your own polyglot project

archetype.


Q & A
