CSCC01 FALL 2021

NoSQL Neo4j Software Architecture Planning Prioritizing the Product Backlog

CSCC01 FALL 2021WEEK 3 - NOSQL GRAPH DBS. SOFTWARE ARCHITECTURE.

PLANNING AND PRIORITIZING.

Ilir Dema

University of Toronto

Sep 22-24, 2021


WHAT IS NOSQL?

INTRO TO NOSQL• The growth of Web raised the need for larger, more

scalable storage solutions.• a variety of key-value storage solutions were designed for

better availability, simple querying, and horizontal scaling.• This new kind of data store became more and more robust,

offering many of the features of the relational databases.• Different storage design patterns emerged, including

key-value storage, column storage, object storage, and themost popular one, document storage.


RELATIONAL VS DOCUMENT-ORIENTED DB

• In a common relational database, data is stored in differenttables, often connected using a primary to foreign keyrelation.

• A program will later reconstruct the model using variousSQL statements to arrange the data in some kind ofhierarchical object representation.

• Document-oriented databases handle data differently.• Instead of using tables, they store hierarchical documents

in standard formats, such as JSON and XML.


RELATIONAL DB EXAMPLE

blog post model - data stored in different tables:


DOCUMENT-ORIENTED DB EXAMPLE

• In a document-based database, the blog post will be storedcompletely as a single document that can later be queried.

• For instance, in a database that stores documents in aJSON format, the blog post document would probably looklike the following code snippet:

This model will allow faster read operations since yourapplication won’t have to rebuild the objects with every read.


WHAT IS NEO4J?

A GRAPH DATABASE

• Uses graphs to store and process the data• Data is organized into nodes and relationships• Properties are stored in either nodes or relationships• Recently, Neo4j and Google Cloud have teamed up to

deliver Neo4j for Google Cloud, the Neo4j graph databasedelivered as a Google Cloud Platform (GCP) nativeservice.

• Neo4j’s native data manipulation language is Cypher.


GRAPHS


HOW ABOUT DATA?


HOW ABOUT DATA?


COMPARE TO SQL


TRANSITION TO GRPAHS


MODELING WITH GRAPHS


NEO4J IS FULLY ACID COMPLIANT ...

ACID• ATOMIC: The whole transaction or nothing• CONSISTENT: Upon completion of a transaction, the db is

structurally sound• ISOLATION: Transactions appear to apply in isolation from

one another• DURABLE: Once a transaction is complete, it persists,

even in case of various failures


WELCOME TO CYPHER

NOT JUST A QUERY LANGUAGE

• Declarative, readable, expressive• Made for CRUD on graphs• Based on patterns• Interacts safely with the remote database using a binary

protocol called Bolt


PROPERTY GRAPHS

A PROPERTY GRAPH HAS

• Nodes (:PERSON)• have properties ({name: ”Donald”})

• Relationships [:WORKS_WITH]• also have properties ({company: ”Bluecat”})

AN EXAMPLE OF CREATECREATE

(: PERSON {name:"Donald"})-[:WORKS_WITH {company: "Bluecat"}]->

(: PERSON {name: "Jasvir"})


CYPHER WORKS BASED ON PATTERNS

WHO WORKS WITH JASVIR AT BLUECAT?MATCH(p1: PERSON)

-[:WORKS_WITH {company:"Bluecat"}]->(:PERSON {name:"Jasvir"})

RETURNp1


SOFTWARE DESIGN

TONY HOARE:There are two ways of constructing a software design: One wayis to make it so simple that there are obviously no deficiencies,and the other way is to make it so complicated that there are noobvious deficiencies.


LEVELS OF DESIGN

• Architectural design (also: high-level design)• architecture - the overall structure: main modules and their

connections• design that covers the main use-cases of the system• addresses the main non-functional requirements (e.g.,

throughput, reliability)• hard to change

• Detailed design (also: low-level design)• the inner structure of the main modules• may take the target programming language into account• detailed enough to be implemented in the programming

language


SOFTWARE ARCHITECTURE

DEFINITION

A software architecture is a description of the subsystems andcomponents of a software system and the relationshipsbetween them. Subsystems and components are typicallyspecified in different views to show the relevant functional andnonfunctional properties of a software system. The softwarearchitecture of a system is an artifact. It is the result of thesoftware development activity.

Buschmann et al., Pattern-Oriented Software Architecture, ASystem of Patterns


THE MVC PATTERN

• Model:• Contains Data Objects• Encapsulates the application state• Responds to state queries/updates• Exposes application functionality

• View:• Renders the model (i.e. the screen representation of the

data).• Note: The user is not necessarily a human. For example,

programs want to view the data using some text format (e.g.XML, or JSON)

• Sends user input to the Controller• Controller:

• Defines application behavior• Maps user actions to Model updates• Controls the flow of the application.• Defines the way the user interface reacts to the user input.


THE MVC AS ARCHITECTURAL PATTERN

MVC INTEGRATES A FEW DESIGN PATTERNS

• Model uses Observer to keep views and controllersupdated on the latest state changes

• View and Controller implement Strategy pattern. Controlleris the behavior of the View and can be easily exchangedwith another controller if you want different behaviour.

• View uses Composite pattern to manage the componentsof the display.


THE 3-TIERED ARCHITECTURE

WHAT DOES THREE TIERED MEAN?• The presentation tier is the front end layer in the 3-tier

system and consists of the user interface.• The application tier contains the functional business logic

which drives an application’s core capabilities.• The data tier comprises of the database/data storage

system and data access layer.


THE 3-TIERED ARCHITECTURE

WHERE DOES IT DIFFER FROM MVC?• MVC and 3-tier architecture are topologically different.• Conceptually the three-tier architecture is linear. MVC

architecture is triangular: the view sends updates to thecontroller, the controller updates the model, and the viewgets updated directly from the model.

• A fundamental rule in a three tier architecture is the clienttier never communicates directly with the data tier.


MICROSERVICES ARCHITECTURE

WHAT DOES MICROSERVICES MEAN?• Develop a single application as a suite of small services• Each running separately and communicating via HTTP.• These services are independently and automatically

deployable.• They may use different programming languages and use

different data storage technologies.


RELEASE PLANNING

• During the release planning meeting the following thingsare established:

• Major release goals• Release plan• Potential sprint goals• Completion date

• As each sprint progresses the burndown of story pointsmeasure the velocity of work, which can be used todetermine progress and adapt the plan as we go


SPRINT PLANNING

• The team decides (reviews) velocity - how many storypoints will they do in this sprint.

• Most priority stories from the product backlog are selected,filling up the velocity.

• team never overcommits!• The tasks from each selected story is broken down to build

the sprint backlog• Meeting may include additional domain experts (not part of

the team) to help answer any questions and aid in timeestimations.

• Implementation details are discussed• Product owner must be present to answer any questions

related to the design


TASK PLANNING

• Tasks are estimated in hours• Estimation is an ideal time (without interruptions / problems)

• After all tasks have been estimated the hours are totaledup and compared against the remaining hours in the sprintbacklog

• If there is room, the team picks more stuff from productbacklog and updates the velocity.

• All planning decisions are recorded on the tracker.


TRACKING PROGRESS

• Information about progress, impediments and sprintbacklog of tasks needs to be readily available

• How close a team is to achieving their goals is alsoimportant

• Scrum employs a number of practices for tracking thisinformation:

• Task cards• Burndown charts• Task boards• War rooms


BURNDOWN CHART

Source: Wikipedia


TASKBOARD

Source:https://manifesto.co.uk/agile-concepts-scrum-task-board/


DAILY SCRUM MEETINGS

• 15 minute meeting that everyone must attend• No sitting down, team stands in a circle and answers the

following questions: What have I done since the lastmeeting?

• What am I going to accomplish between now and the nextmeeting?

• What are the problems or impediments that are slowing medown?

• It is NOT for solving problems - the Scrum Master mustensure that all side conversations are kept to a minimum

• Solving problems happens throughout the rest of the day• Can be evolved to meet a specific team’s requirements, but

the purpose must remain the same (status, commitment,improvement)


SPRINT REVIEWS

• Occur on the last day of the sprint• Team and stakeholders come together to play the game

and discuss the work accomplished• Product owner accepts or declines the results of the sprint• If a feature is declined, the owner will decide if it is returned

to the backlog or simply dropped• Honesty is crucial• Cannot discourage criticism simply because a lot of work

was put in


HOW TO PRIORITIZE?

• Prior to each sprint, the product owner is allowed tochange the priorities in the product backlog

• Decisions can be based on the following criteria:• Value: what value does the story add to the player buying

the game, helps maximize ROI• Cost: some features may prove too costly to implement and

affect ROI• Risk: uncertainty about value/cost• Knowledge: if the product owner doesn’t have enough

information about feature to do a proper estimate they canintroduce a spike to explore it and get more info


KANO MODEL


KANO MODEL

• Developed by Noriaka Kano in the 1970s and 1980s whilestudying quality control and customer satisfaction.

• Basic features that a user expects to be there and work willnever score highly on satisfaction, but can take inordinateamounts of effort to build and maintain.

• At the opposite end of the spectrum are features thatdelight the user. These score very highly on satisfactionand in many cases may not take as much investment.Small incremental improvements here have an outsizedimpact on customer satisfaction.

• Satisfiers: requirements that customers are not expecting,but the more of them that are included, the happier thecustomer.


PRIORITIZING DESIRABILITY

SELECTING FEATURES

• If you have a choice of two things and cant decide, takeboth.

- Gregory Corso• When you come to a fork in the road, take it.- Yogi Berra


ASSESSING THEMES ON THE KANO MODEL

• Kano proposed determining the category of a feature byasking two questions:

• how the user would feel if the feature were present in theproduct and

• how the user would feel if it were absent.• The answers to these questions are on the same five-point

scale:1 I like it that way2 I expect it to be that way3 I am neutral4 I can live with it that way5 I dislike it that way


EXAMPLE

THE SWIMSTATS WEBSITE

• SwimStats is a hypothetical website for swimmers andswim coaches.

• SwimStats will be sold as a service to competitiveage-group, school, and college swim teams.

• Coaches will use it to keep track of their roster ofswimmers, organize workouts, and prepare for meets

• Swimmers will use the site to see meet results, check theirpersonal records, and track improvements over time.

• Officials at swim meets will enter results into the system.


THE SWIMSTATS WEBSITE

Source: Cohn, Agile Estimating and Planning.


KANO MODEL FOR SWIMSTATS

• Suppose we are contemplating three new features:• The ability to see a graph of a swimmer’s times in an event

over the past season• The ability for swimmers to post autobiographical profiles• The ability for any registered site member to upload photos

• To determine which type of feature this is, we would surveyprospective users asking them:

• If you can graph a swimmer’s times in an event over thepast season, how do you feel?

• If you cannot graph a swimmer’s times in an event over thepast season, how do you feel?

• If swimmers can post autobiographical profiles, how do youfeel?

• If swimmers cannot post autobiographical profiles, how doyou feel?

• If you can upload photos, how do you feel?• If you cannot upload photos, how do you feel?


SAMPLE ANSWERS



MANAGING CONFLICTING ANSWERS



DISTRIBUTION OF RESULTS FROM SURVEYING USERS.



VALUE/COST: A PROXY FOR ROI

• A working definition for ROI is Value/Cost.• Need to have a relative measure of value and cost, w.r.t

other features.• Why?

• Easy to understand and calculate• It makes apparent economical sense

• We have a proxy for effort - story points.• What is a proxy for value?


WHAT ELSE DO WE NEED TO ACCOUNT FOR?

• First, we need to quantify value, and numbers can be readfrom the Kano graph.

• Next, need to consider the cost of delay. How much relativevalue do we lose if we deliver x-feature after y-feature?

• Last, but not least, need to consider how much do wereduce risk and uncertainty.

• If two user stories have same ROI, the one with highestCoD must be prioritized first.


COST OF DELAY

DEFINE COST OF DELAY

• What is cost of delay?• The impact of time on value (or rephrased: Impact of time

on the outcomes we hope to achieve)• It combines urgency and value.

• Cost of delay = value x urgency• Urgency: Describes the development of value over a given

timeframe


URGENCY: SHORT BENEFIT HORIZON.

Source: http://www.ontheagilepath.net/


URGENCY: LONG LIFE BUT SMALLER PEAK DUE TO

LATE ENTRY.



URGENCY: LONG LIFE , WITH PEAK NOT AFFECTED BY

DELAY.



COMPUTING COST OF DELAY.



EXAMPLE

Feature Effort Value ROI CoD Priority1 1 3 3 3 12 3 6 2 4 23 10 20 2 3 3

Documents

CSCC01 FALL 2021