© 2017 Nokia1

Petros Nanouris, Loukas Leontopoulos, Thalia Ikonomou, Marina Kesoglou, Giota Eftaxia, Panos Karantzias, Dimitris Fotiadis

30/3/2019

Augmented intelligence case studies for DevOps

Nokia Internal Use

© 2017 Nokia2

Presentation overview (Duration 20 min)

Nokia Internal Use

Short intro to DevOps

• Problem – symptoms• What is DevOps• The three pillars of DevOps

Conclusions - Summary

• Benefits• Next steps

Augmented intelligence solutions• Monitoring• Test strategy selection and• Intelligent Resource Manager (IRM)• Software Quality Evaluation Platform

Case study: Software Quality Evaluation Platform

• Solution design• Quality analytics lab• Use case

© 2017 Nokia3

• Defects into production.

• Late diagnosis.

• Problems appear in some environments only.

• Blame shifting/finger pointing.

• Long delays.

• “Manual errors”.

• Releases slip/fail.

• Delivering a service is too slow and error-prone.

• Internal friction points.

• Delay=loss

• IT is frequently the bottleneck.

Problem Symptoms

Nokia Internal Use

© 2017 Nokia4

What is DevOps?

Nokia Internal Use

Culture Automation Measurement Sharing

Product Dev

QA

Ops

Concept Cash

That is …

© 2017 Nokia5 Nokia Internal Use

Infrastructure automation Continuous delivery

DevOps three pillars

Reliability engineering

© 2017 Nokia6

DevOps augmented intelligence projects

Nokia Internal Use

Test log analysis to identify abnormal

behavior and produce metrics

Intelligentselection & weighting

of test cases

Nokia Internal Use

CI Orchestrator

StagingDevelopment Build

Software QualityEvaluation

Guidelines for quality improvement

SW Quality Evaluation platform

Test Strategy Analysis &

Optimization

ContinuousSystem Monitoring

Anomaly DetectionSelf Healing

GTC

IRM

Request Environment

provision

Intelligent Resource Manager

DPC

Quality Left shift

OptimizationTime reduction

Static Analysis

Testing dynamic analysis

Optimize HW utilization

Maximize availabilityGreen computing

© 2017 Nokia7

FEATURE EXTRACTION & ENGINEERING

Variety of quality metric data sources (e.g. static

analyzers, source control systems etc.)

Metrics per code fragment

Software quality evaluation platform case study

Nokia Internal Use

✓ Software Quality Evaluation Platform is a collection of prediction models.

✓ Models can be created on demand.

✓ Every instance of the code is analyzed.

✓ Feedback is in turn provided to the user.

✓ Prioritization of improvement actions.

RANDOM FOREST

TRAIN A REGRESSION MODEL DASHBOARD

SUPPORT VECTOR MACHINE

DEEP NEURAL NETWORKS

© 2017 Nokia8

Is static analysis adequate for continuous quality evaluation and improvement?

Nokia Internal Use

Static analysis

Dynamic Analysis

Test results & log analysis

Architecture -design analysis

Data persistence quality analysis

and the list continues …

Data report

Data report

Data report

Data report

Quality lab (platform) creating analytics

Data report

Data report

Data report

Data report

Prediction and quality evaluation models

Decision makingVisualization,

feedback, action

Data sources Reporting

Data report

Continuous quality evaluation and improvement loop

© 2017 Nokia9

Common quality metric extraction and collection (Feature Extraction)

Nokia Internal Use

C/C++

C#Java

Python

Technology agnostic

unused

bad-practice

code history

security

pitfall

code smell

convention

Testresults

suspicious

clumsy

Userexperience

design

bug

confusing

brain-overload

test coverage

unpredictable

Lock-in

Common quality metric extraction and collection

Various quality KPI areas

Code fragment

Metric 1 Metric 2 … Metric N

Data Data Data Data

Data Data Data Data

Transform extracted data into a suitable form for data analysis and machine

learning purposes

Data persistence layer

Map quality metrics to specific lines of code

© 2017 Nokia10

Graph and document query languagesQuality reporting and analytics

Nokia Internal Use

Abstract Syntax Trees

Code Dependency Graphs

• Is there a complete set of software quality KPIs out there?

✓ We need a platform (lab) used to create novel, custom and more efficient quality KPIs & analytics.

Basis for creating new quality metrics - features

Data

Analytics

Quality Lab

✓ Cypher.

✓ Mongo DB document querylanguage.

✓ Native Java API to Neo4j.

© 2017 Nokia11

Visualization of code quality metrics

Nokia Internal Use

Module

File

Class

Method

Nodes: code fragments (Classes. Methods, etc.)

Colour intensity and node size corresponds to depicted metric value (heat graph-map)

Code as a graph representation

Scale selection Cluster selection (neighborhoods or logical clusters that share common

characteristics)

Graph clusteringFocusing on data relationships

© 2017 Nokia12

Software EpidemiologyQuality reporting and analytics

Nokia Internal Use

How a deadly virus could spread around the world. The world is a metaphor for the product.

✓ “Source of infection” areas ✓ Risk assessment to add new code ✓ Prioritization and planning

© 2017 Nokia13

Case study: Software fault proneness analyticsQuality reporting and analytics

Nokia Internal Use

Fault proximity: defines how far in terms of change is a code fragment revision from an actual bug correction.

Fault dissemination: defines how much a change in a particular code fragment affects also its neighbouring codefragments.

Change proneness: defines how prone is a code fragment to change and estimates the amount of change.

Relative change proneness =1

NREVISIONS, Mean velocity of change =

ΔΧ

NREVISIONS

Exposure: defines how much a code fragment is affected by others in recent history, and estimates a future trend.

Exposure = σ𝟏

𝒅𝒔𝒑𝒂𝒄𝒆 𝒅𝒕𝒊𝒎𝒆

© 2017 Nokia14

Prediction & consulting (a use case)

Nokia Internal Use

Classification

Regression

Clustering

Machine Learningmodule

Predictionmodel

Training on Data

How long before our code gets a ticket correction?How prone is our code to change?How much is it going to change? How much a code fragment change affects the rest of the system?How much a code fragment is exposed to the neighbourhood’s bad quality code? … and so on …Do you have the analytic (target variable)? Create your custom model and use it.

The developer retrieves the current quality state of the code.

The developer Changes the code.

The model evaluates the new quality state of the code.

The model provides specific guidelines (at detail level) to avoid

quality compromisation.

The developer follows the guidelines and reassesses the code with the

support of the model.

Data

© 2017 Nokia15

Risks

• Data acquisition.

• Availability of test cases.

• Test suite running time

✓ Testing optimization module of DevOps DSS could mitigate these impediments.

Summary

Nokia Internal Use

Benefits

✓ User experience.

✓ Effectiveness.

✓ Continuous quality evaluation and improvement.

✓ State of the art data analytics and prediction models.

Next steps

➢ Performance evaluation.

➢ Dashboard.

➢ Feedback mechanism.

➢ Integrate additional quality data.

➢ Complete pilot and demo.

© 2017 Nokia16 Nokia Internal Use

Many thanks for your attention!Please feel free to ask questions