OpenShift-Build-Pipelines

Build ► Test ► Run!

Tobias Schneck

Software Engineer @ Loodse GmbH

Testautomatisierung @ ConSol Software GmbH (bis Juli 18)

+

Agenda

• OpenShift CI/CD Pipeline Concept

• Build up some Pipelines!• Stage I: CI-Build of the Artefact

• Stage II: Automated Testing

− Citrus Integration Testing

− Sakuli E2E Testing

• Stage III: Artefact Deployment

• Review

• Alternatives

• Links

OpenShift CI/CD Pipeline Concept

OpenShift-Build-Pipelines: Build -> Test -> Run! Folie 4

Prozess Development ► QA Stage

OP

S

Code

Build & Unit-Test

Deploy in Dev

Container Image

ContainerRegistry

POD

AUTOMATIC

MANUEL ODER AUTOMATIC

POD

Deploy in QA► Test

OpenShift-Build-Pipelines: Build -> Test -> Run! Folie 5

Prozess QA ► Production Stage

O

PS

Deploy in Production►Smoke-Test

Container Image from QA Stage

MANUEL

OpenShift-Build-Pipelines: Build -> Test -> Run! Folie 6

OpenShift Core Components

CI / CD Build Pipeline?

Jenkins Integration

+ Jenkins Template+ OpenShift Jenkins Client Plugin+ OpenShift Jenkins Sync Plugin

OpenShift-Build-Pipelines: Build -> Test -> Run! Folie 7

OpenShift Regular Builds (traditional)

Execution Node

OpenShift Builder Pod

OpenShift Master

Scheduler

OpenShift Internal Registry

Image Stream(Project/Global)

Source-2-Image Build

Docker native Build

build.yml Dockerfile

Image Artifcat

Builder Image Base Image

OpenShift-Build-Pipelines: Build -> Test -> Run! Folie 8

OpenShift CI Pipeline Build

Execution Node

OpenShift Builder Pod

OpenShift Master

Scheduler

OpenShift Internal Registry

Image Stream(Project/Global)

Image Artifcat

Jenkins Image

Maven Image

…

Jenkinsfile

Jenkins Pipeline Build

OpenShiftRegular Build

OpenShift-Build-Pipelines: Build -> Test -> Run! Folie 9

● Uses OpenShift CLI: oc○ Automatically authorized○ Use complete CLI possibilities○ Huge flexibility

● Uses OpenShift Jenkins Client Plugin○ Based on Jenkins kubernetes-plugin○ Groovy wrapper for OpenShift API○ Enables dynamic slave images

- built-in images: base, maven, nodejs- custom images

OpenShift Build & Deploy with Jenkinsfile

Jenkins Image

Jenkinsfile

Maven Image

Node Image

…

OpenShift-Build-Pipelines: Build -> Test -> Run! Folie 10

OpenShift Custom Jenkins Slave

Jenkinsfile

podTemplate(label: "citrus", cloud: "openshift", inheritFrom: "maven", containers: [ containerTemplate(name: "jnlp", image: "toschneck/citrus-jenkins-slave", ) ]) { node('citrus') { sh "echo execute oc citrus build" checkout scm sh "mvn install" junit 'citrus-tests/target/citrus-reports/**/*.xml' archiveArtifacts "citrus-tests/target/citrus-*/**/*" } }

Used by

citrusframework/citrus-docker-images PR #3

FROM consol/citrus:2.7.5

### sourced are copied from:# https://github.com/openshift/jenkins/tree/master/slave-base# https://github.com/openshift/jenkins/tree/master/slave-maven

# Copy the jenkins-slave entrypointADD contrib/bin/* /usr/local/bin/

# Run the Jenkins JNLP clientENTRYPOINT ["/usr/local/bin/run-jnlp-client"]

Build up a Build Pipeline!

OpenShift-Build-Pipelines: Build -> Test -> Run! Folie 12

Bakery App

OpenShift-Build-Pipelines: Build -> Test -> Run! Folie 13

Bakery App

OpenShift-Build-Pipelines: Build -> Test -> Run! Folie 14

Transition to OpenShift

Goals:

• Don’t change the Sourcecode of the Apps

• Also don’t change the Testcode

• Fully Automated Pipeline until Production

• All in OpenShift (CI, Build, Test, Deployment)

• Jenkins v1 -> v2

Stage I: CI-Build of the Artefact

OpenShift-Build-Pipelines: Build -> Test -> Run! Folie 16

CI Build ► Docker Image

build.pipeline.yml Jenkinsfile.dev

Maven Image

Dockerfiles

OpenShift Regular Build

Docker Image Artifact

DEMOtoschneck/openshift-example-bakery-ci-pipeline

Stage II: Automated Testing

Citrus Integration Testing

OpenShift-Build-Pipelines: Build -> Test -> Run! Folie 20

Mission ► Integration Tests

System under Test

SystemUnder Test

Verträge

Stammdaten

Dienstleister

Shipping

JMS

XML

FTP

CSV

Webclient

Kunde

Agent / Techniker

Mobile, PDA

Http

REST

Http

SOAP

OpenShift-Build-Pipelines: Build -> Test -> Run! Folie 21

Mission ► Integration Tests

SystemUnder Test

Stammdaten

Dienstleister

JMS

XML

FTP

CSV

Kunde

Agent / Techniker

Http

REST

Http

SOAP

Automated API Interface Tests

OpenShift-Build-Pipelines: Build -> Test -> Run! Folie 22

Use Case: Testing of Middleware Integration / ESB / API

Citrus Features

• Ready to use components− Client & Server Implementation

− Application Server Deployments

− Container Images

• Message transports− Http REST, SOAP, JMS, FTP, TCP/IP, Mail, RMI, SSH, ...

• Data formats− XML, Json, PlainText, CSV, XSD, WSDL

• Test DSL− XML & Java

citrusframework/citrus

OpenShift-Build-Pipelines: Build -> Test -> Run! Folie 23

more Citrus

• Citrus Admin UI • Easy Testsuite Management

• Reporting

• Citrus Simulator• Complex Messaging Workflow Simulation

• Predefined Interface Packages: WBCI, WITA

OpenShift-Build-Pipelines: Build -> Test -> Run! Folie 24

QA ► Citrus API Integration Tests

Citrus Slave Image

build.pipeline.yml Jenkinsfile.qa

DEMOtoschneck/openshift-example-bakery-ci-pipeline

E2E Testing

OpenShift-Build-Pipelines: Build -> Test -> Run! Folie 27

Sakuli End-to-End Use Cases

OpenShift-Build-Pipelines: Build -> Test -> Run! Folie 28

Motivation

• Founded February 2014 Open Source (Apache)

• Goals: − Combine a web and screen-based testing in ONE tool

− Use test results in CI- and monitoring systems

− Platform independent: (Linux/Windows/MacOS)

• Application tests from a End User Perspective− Functionality (business critical path)

− Performance (e2e loading time)

OpenShift-Build-Pipelines: Build -> Test -> Run! Folie 29

Component 1: Sahi / Selenium

Web testing tool (sahi.co.in, seleniumhq.org) method based DOM access:

_assertContainsText ("Logged in as: Sakuli", _div("user_field"));

_click(_span("Loaded Run Tabels"));

_assertExists(_table("cross_table_fixed"));

_assertExists(_cell(“testing allowed", _rightOf(_span("Name")), _under(_cell("Action")));

<table> <tbody> <tr> <td>Alice</td> <td>Bob</td> </tr> </tbody></table>

Webserverlocalhost:9999

{js}

OpenShift-Build-Pipelines: Build -> Test -> Run! Folie 30

Component 2: Sikuli

Visual automation tool (sikuli.org)

image identification, mouse & keyboard interaction:

screen.find("sap_ok").click();

screen.find("sap_ok").right(40).click().type("2223");

var bubble = new Region().waitForImage("bubble.png", 20);

bubble.dragAndDropTo(bubble.left(35)).highlight();

OpenShift-Build-Pipelines: Build -> Test -> Run! Folie 31

Benefits

• Integrated API for both Tools - one TestScope

• Step by Step Code (JavaScript, Java)

• Automatic Screenshots on any Errors

• Cloud-ready Docker Images for Kubernets/OpenShift

• Sakuli UI for Maintenance, Execution and Reporting (beta)

• "Forwarder" Modules:

− Monitoring (Gearman, Icinga, CheckMK)

− CI (Jenkins, Travis CI, Drone ...)

− Databases (MySQL/MariaDB, ...)

OpenShift-Build-Pipelines: Build -> Test -> Run! Folie 32

Monitoring Integration

• Ready-to-use setup in Kubernetes/OpenShift

• Combinable with „traditional“ monitoring checks

• Performance graphs

• Cron scheduled check execution

• Error screenshots

• Mail / Chat Notification

• Live watching possible

OpenShift-Build-Pipelines: Build -> Test -> Run! Folie 33

Sakuli End-2-End Testing Container

ConSol/docker-headless-vnc-container

OpenShift-Build-Pipelines: Build -> Test -> Run! Folie 34

QA ► Tested Docker Image

Docker Image Artifcat

Citrus Slave Image Sakuli Image

build.pipeline.yml Jenkinsfile.qa

DEMOtoschneck/openshift-example-bakery-ci-pipeline

OpenShift-Build-Pipelines: Build -> Test -> Run! Folie 36

Review

• Jenkins wurde nicht desingend für Container/PodsPlugins, Startup, Java Agents, lokales Filesystem, Weiterentwicklung

• Migration zu Jenkinsfiles

• Viele Wrapper-Scripte, um flexibel zu sein

• Verteiltes CI-Design erfordert ein Umdenken

• CI-Daten schwerer zugänglich Logs, Screenshots, Artifakte

• Technologie-Mix

• Slave-Image intransparent -> do it your selfBase Image: github.com/openshift/jenkins/slave-base

• Verteilte Dokumentation OpenShift, Kubernetes, Jenkins, Jenkins-Plugins, Docker, …

OpenShift-Build-Pipelines: Build -> Test -> Run! Folie 37

Review

• Skalierung übernimmt der Cluster

• "oc CLI client" anstatt Openshift-Jenkins-Plugin► Pipeline modular test/entwickelbar

• Parametrisierte YAML Templates für OpenShift

• Infrastructur as Code

• Neue Stages sind einfach zu erstellen

• Deployment wird automatisch mitgetestet

• Secrets durch Cluster-Admins verwaltbar

• Builds Steuerbar von CLI und UI

• Erweiterbarkeit durch OpenSource

OpenShift-Build-Pipelines: Build -> Test -> Run! Folie 38

Alternative Ansätze

hasura/gitkube

⇨ Gitkube• “git push” based Workflow on remote Git• Simple: Kombination von bash Scripts

⇨ Skaffold: • Einfacher CI/CD Workflow• 2 Stages: Local & Remote

⇨ JenkinsX• Vordefinierter Workflow für CI/CD in Kubernetes• kein neuer Jenkins -> Wrapper• GitHub only

⇨ DroneCI / KubeCI• Container first und einfache Konzept• Nutzbar für gebaute Images• Layered Build Concept für Pods (in Entwicklung)

GoogleContainerTools/skaffold

kubeciio drone/drone

jenkins-x/jx

KubeCI● Drone Kubernetes Runtime● Plugins:

○ kubectl○ helm○ img

(daemon-less build)○ … more will follow

pipeline: test: image: python:2-slim commands: - python test/wait-for-it.py

build-docker: image: plugins/docker repo: toschneck/wait-for-it dockerfile: Dockerfile



WIP

OpenShift-Build-Pipelines: Build -> Test -> Run! Folie 40

Links

toschneck/openshift-example-bakery-ci-pipeline

citrusframework/citrus www.citrusframework.org

ConSol/sakuli www.sakuli.org

@toschneck

@citrus_test

@sakuli_e2e

Questions?

Thank you Stuttgart!Tobias Schneck, Loodse@toschneck