Software Architectures, Week 5 - Advanced Architectures

Week 5

Advanced Architectures 

COAST architecture: http://isr.uci.edu/projects/coast/about.html

Data Silos• Typical in many 'small' data architectures.

• Data are persisted in different databases with imposed borders.

• They are created due to organizational boundaries and divergent goals between individuals or teams.

Data Silos (cont)

A silo breaker, the data lake 

Data Lake Architecture

Credit: EMC

Data Lake Architecture

Credit: Zaloni

Data Lake vs Data Warehouse

Credit: Martin Fowlerr

Data Lake Architecture, Batch Processing

Credit: EMC

Data Lake Architecture, Stream Processing

https://www.confluent.io/blog/stream-data-platform-1/

Data Lake Architecture, Stream Processing (cont.)

https://www.confluent.io/blog/stream-data-platform-1/

Data Lake Architecture, Stream Processing (cont.)

https://www.confluent.io/blog/stream-data-platform-1/

What happens when we want to combine batch and stream processing?

λ!• Invented by Nathaniel Marz, it is intended to combine the pros of both worlds.

• Processing is separated but persistence is common.

• It introduces more complexity.

• Many more moving parts.

• Start getting out of favour in the last couple of years.

Lambda Architecture (cont.)

Lambda Architecture (cont.)

Lambda Architecture for Web Apps

Credit: Amazon

How far we can go in reducing elements?

Serverless Architecture• Do we still need servers? • - Seems not.

• Do we need storage clusters? • - Seems not.

• Do we need developers?• - Seems yes.

Serverless Architecture (cont.)

Credit: Amazon

Serverless Architecture (cont.)• Serverless systems don't need to be limited to web services or

contellations of microservices.

• They can also be used for Continuous Deployment (CI), Continuous Delivery (CD) or monitoring.

Since we mentioned Serverless Microservices 

• Functionality of Lambda functions can be grouped together and form serverless microservices (logically).

• Same principles regarding communication and separation of concerns apply here.

• Operations become much, much simpler and cheaper.

Serverless Microservices (cont.)

Credit: Amazon

Serverless Continuous Delivery• Jenkins is not needed any more.

• There is no limiation in compiling/building capacity (except your credit cards).

• Highly available (no more dead Jenkins workers).

• No risk of over or under-provisioning.

• No OS to maintain or worry about.

https://stelligent.com/2016/03/17/serverless-delivery-architecture-part-1/

Serverless Continuous Delivery (cont.)

https://stelligent.com/2016/03/17/serverless-delivery-architecture-part-1/

Bottom line...

https://stelligent.com/2016/03/17/serverless-delivery-architecture-part-1/

COAST Architecture• COmputAtional State Transfer (COAST).

• The principal goal is the construction of Internet-scale decentralized applications.

• Computation exchange is the bilateral exchange of computations among peers (code to data).

• A PoC has been implemented in Racket.

http://isr.uci.edu/projects/coast/about.html

COAST Architecture (cont.)

http://isr.uci.edu/projects/coast/about.html

Zeta Architecture• An enterprise-scale architecture aiming to meet always rising

expectations.

• Inspired by Google's top-level architecture.

• There are seven pluggable components of the architecture, and all of them must work together.

https://www.oreilly.com/ideas/zeta-architecture-hexagon-is-the-new-circle

Zeta Architecture in Google (cont.)

https://www.oreilly.com/ideas/zeta-architecture-hexagon-is-the-new-circle

Poor man's Zeta (cont.)

https://www.oreilly.com/ideas/zeta-architecture-hexagon-is-the-new-circle

Zeta Architecture (cont.)• Data protection schemes, backing up data, recovering from failures,

load balancing, and even running multiple versions of software are simplified with Zeta.

• These properties are offered as services and are available to every guest application.

• Resource allocation is optimized and results in lower HW requirements.

https://www.oreilly.com/ideas/zeta-architecture-hexagon-is-the-new-circle

Web  Service before Zeta

https://www.oreilly.com/ideas/zeta-architecture-hexagon-is-the-new-circle

Web  Service after Zeta

https://www.oreilly.com/ideas/zeta-architecture-hexagon-is-the-new-circle

Unikernels• Unikernels are specialised, single-address-space machine images

constructed by using library operating systems.

• They are built by compiling high-level languages directly into specialised machine images that run directly on a hypervisor.

• Unikernels benefits are improved security, smaller footprints, more optimisation and faster boot times.

http://unikernel.org

Unikernels (cont.)

http://unikernel.org

ClickOS, exercise in lightness• "A minimalistic, tailor-made, virtualized operating system to run Click-based

middleboxes."

• Small (6MB)

• Boot quickly (in about 30 milliseconds)

• Quick response time (45 microseconds)

• Targeting C++

http://cnp.neclab.eu/clickos/

MirageOS, Docker's successor?• The most widely used Unikernel system.

• In production as a microservice platform.

• Unikernel Systems has been acquired by Docker, making it an inherent part of the Docker ecosystem.

• Used in serverless, non-AWS deployment scenarios.

https://mirage.io

Looking in the (not so deep) future

https://mirage.io

Self-organizing Architectures• Self-adaptive software architecture.

• Autonomously exploring all possible architectures that can be used to realise a given software system.

• Monitoring that system in execution in terms of its performance and its operating environment.

• Identifying the optimal architecture for each set of operating environment conditions that are encountered

http://ieeexplore.ieee.org/document/7573144

Train your server 

https://www.wired.com/2016/05/the-end-of-code/

Deepcoder Learning to Write Programs

https://arxiv.org/abs/1611.01989

• A Cambridge Project.

• Deep Neural network which is not performing function aproximation but assembling simple pieces of code which approximate the function.

• It can solve the simplest problems on programming competition websites.

Deep Learning Architecture – High Level

http://www.slideshare.net/odsc/arno-candel-scalabledatascienceanddeeplearningwithh2oodscboston2015-48909623

Deep Learning Architecture – Neural Net

http://www.slideshare.net/odsc/arno-candel-scalabledatascienceanddeeplearningwithh2oodscboston2015-48909623

Deep Learning Architecture – Infrastructure

Thank you!