CSC301 WINTER 2022WEEK 4 - NOSQL GRAPH DBS. SOFTWARE ARCHITECTURE.

PLANNING AND PRIORITIZING.

Ilir Dema

University of Toronto

Feb 1-2, 2022

WHAT IS NOSQL?

INTRO TO NOSQL• The growth of Web raised the need for larger, more

scalable storage solutions.• a variety of key-value storage solutions were designed for

better availability, simple querying, and horizontal scaling.• This new kind of data store became more and more robust,

offering many of the features of the relational databases.• Different storage design patterns emerged, including

key-value storage, column storage, object storage, and themost popular one, document storage.

RELATIONAL VS DOCUMENT-ORIENTED DB

• In a common relational database, data is stored in differenttables, often connected using a primary to foreign keyrelation.

• A program will later reconstruct the model using variousSQL statements to arrange the data in some kind ofhierarchical object representation.

• Document-oriented databases handle data differently.• Instead of using tables, they store hierarchical documents

in standard formats, such as JSON and XML.

RELATIONAL DB EXAMPLE

blog post model - data stored in different tables:

DOCUMENT-ORIENTED DB EXAMPLE

• In a document-based database, the blog post will be storedcompletely as a single document that can later be queried.

• For instance, in a database that stores documents in aJSON format, the blog post document would probably looklike the following code snippet:

This model will allow faster read operations since yourapplication won’t have to rebuild the objects with every read.

WHAT IS NEO4J?

A GRAPH DATABASE

• Uses graphs to store and process the data• Data is organized into nodes and relationships• Properties are stored in either nodes or relationships• Recently, Neo4j and Google Cloud have teamed up to

deliver Neo4j for Google Cloud, the Neo4j graph databasedelivered as a Google Cloud Platform (GCP) nativeservice.

• Neo4j’s native data manipulation language is Cypher.

GRAPHS

HOW ABOUT DATA?

HOW ABOUT DATA?

COMPARE TO SQL

TRANSITION TO GRPAHS

MODELING WITH GRAPHS

NEO4J IS FULLY ACID COMPLIANT ...

ACID• ATOMIC: The whole transaction or nothing• CONSISTENT: Upon completion of a transaction, the db is

structurally sound• ISOLATION: Transactions appear to apply in isolation from

one another• DURABLE: Once a transaction is complete, it persists,

even in case of various failures

WELCOME TO CYPHER

NOT JUST A QUERY LANGUAGE

• Declarative, readable, expressive• Made for CRUD on graphs• Based on patterns• Interacts safely with the remote database using a binary

protocol called Bolt

PROPERTY GRAPHS

A PROPERTY GRAPH HAS

• Nodes (:PERSON)• have properties ({name: ”Donald”})

• Relationships [:WORKS_WITH]• also have properties ({company: ”Bluecat”})

AN EXAMPLE OF CREATECREATE

(: PERSON {name:"Donald"})-[:WORKS_WITH {company: "Bluecat"}]->

(: PERSON {name: "Jasvir"})

CYPHER WORKS BASED ON PATTERNS

WHO WORKS WITH JASVIR AT BLUECAT?MATCH(p1: PERSON)

-[:WORKS_WITH {company:"Bluecat"}]->(:PERSON {name:"Jasvir"})

RETURNp1

SOFTWARE DESIGN

TONY HOARE:There are two ways of constructing a software design: One wayis to make it so simple that there are obviously no deficiencies,and the other way is to make it so complicated that there are noobvious deficiencies.

LEVELS OF DESIGN

• Architectural design (also: high-level design)• architecture - the overall structure: main modules and their

connections• design that covers the main use-cases of the system• addresses the main non-functional requirements (e.g.,

throughput, reliability)• hard to change

• Detailed design (also: low-level design)• the inner structure of the main modules• may take the target programming language into account• detailed enough to be implemented in the programming

language

SOFTWARE ARCHITECTURE

DEFINITION

A software architecture is a description of the subsystems andcomponents of a software system and the relationshipsbetween them. Subsystems and components are typicallyspecified in different views to show the relevant functional andnonfunctional properties of a software system. The softwarearchitecture of a system is an artifact. It is the result of thesoftware development activity.

Buschmann et al., Pattern-Oriented Software Architecture, ASystem of Patterns

THE MVC PATTERN

• Model:• Contains Data Objects• Encapsulates the application state• Responds to state queries/updates• Exposes application functionality

• View:• Renders the model (i.e. the screen representation of the

data).• Note: The user is not necessarily a human. For example,

programs want to view the data using some text format (e.g.XML, or JSON)

• Sends user input to the Controller• Controller:

• Defines application behavior• Maps user actions to Model updates• Controls the flow of the application.• Defines the way the user interface reacts to the user input.

THE MVC AS ARCHITECTURAL PATTERN

MVC INTEGRATES A FEW DESIGN PATTERNS

• Model uses Observer to keep views and controllersupdated on the latest state changes

• View and Controller implement Strategy pattern. Controlleris the behavior of the View and can be easily exchangedwith another controller if you want different behaviour.

• View uses Composite pattern to manage the componentsof the display.

THE 3-TIERED ARCHITECTURE

WHAT DOES THREE TIERED MEAN?• The presentation tier is the front end layer in the 3-tier

system and consists of the user interface.• The application tier contains the functional business logic

which drives an application’s core capabilities.• The data tier comprises of the database/data storage

system and data access layer.

THE 3-TIERED ARCHITECTURE

WHERE DOES IT DIFFER FROM MVC?• MVC and 3-tier architecture are topologically different.• Conceptually the three-tier architecture is linear. MVC

architecture is triangular: the view sends updates to thecontroller, the controller updates the model, and the viewgets updated directly from the model.

• A fundamental rule in a three tier architecture is the clienttier never communicates directly with the data tier.

MICROSERVICES ARCHITECTURE

WHAT DOES MICROSERVICES MEAN?• Develop a single application as a suite of small services• Each running separately and communicating via HTTP.• These services are independently and automatically

deployable.• They may use different programming languages and use

different data storage technologies.

RELEASE PLANNING

• During the release planning meeting the following thingsare established:

• Major release goals• Release plan• Potential sprint goals• Completion date

• As each sprint progresses the burndown of story pointsmeasure the velocity of work, which can be used todetermine progress and adapt the plan as we go

SPRINT PLANNING

• The team decides (reviews) velocity - how many storypoints will they do in this sprint.

• Most priority stories from the product backlog are selected,filling up the velocity.

• team never overcommits!• The tasks from each selected story is broken down to build

the sprint backlog• Meeting may include additional domain experts (not part of

the team) to help answer any questions and aid in timeestimations.

• Implementation details are discussed• Product owner must be present to answer any questions

related to the design

TASK PLANNING

• Tasks are estimated in hours• Estimation is an ideal time (without interruptions / problems)

• After all tasks have been estimated the hours are totaledup and compared against the remaining hours in the sprintbacklog

• If there is room, the team picks more stuff from productbacklog and updates the velocity.

• All planning decisions are recorded on the tracker.

TRACKING PROGRESS

• Information about progress, impediments and sprintbacklog of tasks needs to be readily available

• How close a team is to achieving their goals is alsoimportant

• Scrum employs a number of practices for tracking thisinformation:

• Task cards• Burndown charts• Task boards• War rooms

BURNDOWN CHART

Source: Wikipedia

TASKBOARD

Source:https://manifesto.co.uk/agile-concepts-scrum-task-board/

DAILY SCRUM MEETINGS

• 15 minute meeting that everyone must attend• No sitting down, team stands in a circle and answers the

following questions: What have I done since the lastmeeting?

• What am I going to accomplish between now and the nextmeeting?

• What are the problems or impediments that are slowing medown?

• It is NOT for solving problems - the Scrum Master mustensure that all side conversations are kept to a minimum

• Solving problems happens throughout the rest of the day• Can be evolved to meet a specific team’s requirements, but

the purpose must remain the same (status, commitment,improvement)

SPRINT REVIEWS

• Occur on the last day of the sprint• Team and stakeholders come together to play the game

and discuss the work accomplished• Product owner accepts or declines the results of the sprint• If a feature is declined, the owner will decide if it is returned

to the backlog or simply dropped• Honesty is crucial• Cannot discourage criticism simply because a lot of work

was put in

WHAT IS DEPENDENCY INJECTION?

DEFINITION

The definition of dependency injection was first given by MartinFowler in the blog post Inversion of Control Containers and theDependency Injection pattern.

The dependency injection is an Enterprise Design Pattern,which aim is to

separate the responsibility of resolving object dependencyfrom its behaviour.

NOTE:An Enterprise Design Pattern is a Design Pattern used inenterprise applications.

FOWLER’S EXAMPLE

EXERCISE

• Try to write (on paper) a constructor for MovieLister.• Identify the broken design principles ...

THE MOVIELISTER CONSTRUCTOR

public class MovieLister {private MovieFinder finder;public MovieLister() {

// This statement tightly coupled (1)// the two classes because the class has a// direct reference to a particular// implementation of MovieFinder interface (2)this.finder = new MovieFinderImpl();

}// ...

HOW DID WE GET IN TROUBLE?

• Clearly, in the previous slide we used a long venerateddesign principle:

• Favor composition over inheritance• However letting a class to explicitly create an instance of

another class tightly coupled the two implementations,increasing the dependency between them.

FRED BROOKS: NO SILVER BULLET

There is no single development, in either technology ormanagement technique, which by itself promises even oneorder-of-magnitude improvement within a decade inproductivity, in reliability, in simplicity.

LET’S LOOK FOR HELP!

• Well we know a design pattern used to make objects.• The factory pattern!• Its intent is:

• creates objects without exposing the instantiation logic tothe client.

• refers to the newly created object through a commoninterface

UML FOR THE FACTORY PATTERN

LET’S APPLY IT...

Notice the dependency graph is directed acyclic (DAG).

HOUSTON, WE HAVE A PROBLEM!

• Now MovieLister has a dependency with theimplementation of MovieFinderFactory.

• Notice factory is implemented as a Singleton ...• Moreover, we certainly have more than a single

dependency in a class, which leads to a plethora offactories rising.

• And dependencies can have dependencies, and so on.• We get a chain of growing dependencies.• There are two possibilities:

• The chain blows up in finite time.• The chain stabilizes (i.e. the nightmare comes to a limit).

MARTIN FOWLER’S SOLUTION

Take the factory idea to the limit and we create a module that isresponsible to resolve dependency among classes.

DEPENDENCY INJECTOR

• The Injector module is called dependency injector and it isto all effects a container.

• Applying the Inversion of Control pattern, the injector ownsthe life cycle of all objects defined under its scope.

• The only things we still miss are the following:• A way to signal to the injector that a class has a certain

dependency• A way to instruct (or configure) the injector to resolve

dependencies

INTERLUDE: JAVA BEANS

• Since the definition of the Java programming language, theaim of the Sun Microsystem was standardization.

• The Java Bean Specification stated that a java bean musthave:

• a default constructor• a couple of getter and setter methods for each attribute it

owns.• Having defined a standard way of creating objects, the Sun

built a plethora of standards and interfaces on it, that arethe basis of the JEE specification.

CONSTRUCTOR INJECTION VERSUS SETTER INJECTION

• Since dependencies are nothing more then objectsattributes (more or less), the injector has to be instructed toknow how to fulfill these attributes.

• In Java there are three ways to set attributes of an object,which are:

• Using the proper constructor• Using setters after the object was build using the default

constructor• Using Reflection mechanisms

• Once you have selected your preferred way, you willannotate the corresponding statement with the @Injectannotation.

• The annotation and its behaviour are defined inside a JSR.Dependency injection is so important in the Javaecosystem that there are two dedicated Java SpecificationRequests (JSRs), i.e. JSR-330 and JSR-299.

CONSTRUCTOR DEPENDENCY INJECTION

• If you want the injector to use the constructor to inject thedependency of a class, annotate the constructor with@Inject.

• Every time you will ask the injector an instance of aMovieLister, it will know that it has to use theconstructor annotated with the @Inject annotation tobuild the object.

• So, the dependency are identified as the annotatedconstructor parameters.

public class MovieLister {private MovieFinder finder;

@Injectpublic MovieLister(MovieFinder finder) {

this.finder = finder;}

}

SETTER DEPENDENCY INJECTION

• To instruct the injector to use setter methods to create anobject, annotate the setter methods with the @Injectannotation.

public class MovieLister {private MovieFinder finder;

// The injector first uses the default// constructor to build an empty objectpublic MovieLister() {}// Then, the injector uses annotated setter// methods to resolve dependencies@Injectpublic void setFinder(MovieFinder finder) {

this.finder = finder;}

}

THE INJECTOR

• There are many implementations in the Java ecosystem ofdependency injectors. Each implementation differs fromeach others essentially for these features:

• How the injector is configured to find the beans it has tomanage

• How it resolves the dependencies DAG• How it maintains the instances of managed beans.

THE INJECTOR

• Some injectors used in practice are the following:• Google Guice: Guice is a lightweight dependency injection

framework that uses Java configuration, implements bothtypes of injection (constructor and setter injection) andmaintains managed instances with aMap<Class<?>, Object>

• Spring Injector: It is not quite lightweight as Guice, but it isfully integrated with the whole Spring ecosystem. Itimplements all types of injection; it provides an XML style ofconfiguration and a Java style. Differently from Guice, thecontainer is a Map<String, Object>, that associateseach managed instance with a label. It also support autodiscovery of beans through classpath scanning.

WHAT IS DAGGER-2?

• Dagger-2 is a fast and lightweight dependency injectionframework.

• It is implemented through an external component whichprovides instances of objects (or dependencies) needed byother objects.

• In particular, the injection happens at run-time or atcompile-time.

• Run-time DI is usually based on reflection which is simplerto use but slower at run-time. An example of a run-time DIframework is Spring.

• Compile-time DI, on the other hand, is based on codegeneration. This means that all the heavy-weightoperations are performed during compilation. Compile-timeDI adds complexity but generally performs faster.

• Dagger 2 falls into this category.

EXAMPLE

In order to use Dagger in a project, we’ll need to add thedagger dependency to our pom.xml:

MAVEN CONFIGURATION

<dependency><groupId>com.google.dagger</groupId><artifactId>dagger</artifactId><version>2.16</version>

</dependency>

Note: Eclipse needs be configured as follows:• Install m2e-apt• Window -> Preferences -> Maven -> Annotation

Processing: Select "Automatically configure JDT APT"

EXAMPLE

Also need to include the Dagger compiler used to convert ourannotated classes into the code used for the injections:

MAVEN CONFIGURATION

<plugin><groupId>org.apache.maven.plugins</groupId><artifactId>maven-compiler-plugin</artifactId><version>3.6.1</version><configuration><annotationProcessorPaths>

<path><groupId>com.google.dagger</groupId>

<artifactId>dagger-compiler</artifactId><version>2.16</version>

</path></annotationProcessorPaths>

</configuration></plugin>

THE CAR EXAMPLE

• Dagger uses the standard JSR-330 annotations in manyplaces, one being @Inject.

• Since Dagger doesn’t support injection on private fields,we’ll go for constructor injection.

BUILD A CAR BY INJECTING ITS COMPONENTS

public class Car {private Engine engine;private Brand brand;

@Injectpublic Car(Engine engine, Brand brand) {

this.engine = engine;this.brand = brand;

}// getters and setters

}

CODE NEEDED TO PERFORM THE INJECTION

• Next, we’ll implement the code to perform the injection.More specifically, we’ll create:

• a module, which is a class that provides or builds theobjects’ dependencies, and

• a component, which is an interface used to generate theinjector

• Complex projects may contain multiple modules andcomponents but since we’re dealing with a very basicprogram, one of each is enough.

CREATING A MODULE

A Module class is annotaed with the @Module annotation,indicating that it can make dependencies available to thecontainer. Then, we need to add the @Provides annotation onmethods that construct our dependencies:

A MODULE EXAMPLE

@Modulepublic class VehiclesModule {

@Providespublic Engine provideEngine() {

return new Engine();}@Provides@Singletonpublic Brand provideBrand() {

return new Brand("CSCC01");}

}

COMPONENTS

• Components are essentially the glue that holds everythingtogether.

• They are a way of telling Dagger what dependenciesshould be bundled together and made available to a giveninstance so they can be used.

• They provide a way for a class to request dependenciesbeing injected through their @Inject annotation.

COMPONENT EXAMPLE

Moving on, we’re going to create our component interface. Thisis the class that will generate Car instances, injectingdependencies provided by VehiclesModule.Simply put, we need a method signature that returns a Car andwe need to mark the class with the @Component annotation.Notice how we pas our module class as an argument to the@Component annotation. If we didn’t do that, Dagger wouldn’tknow how to build the car’s dependencies.

A COMPONENT EXAMPLE

@Singleton@Component(modules = VehiclesModule.class)public interface VehiclesComponent {

Car buildCar();}