27
#MDBlocal Just in Time Validation with JSON Schema

Data Governance with JSON Schema

  • Upload
    mongodb

  • View
    68

  • Download
    2

Embed Size (px)

Citation preview

Page 1: Data Governance with JSON Schema

#MDBlocal

Just in Time Validation with

JSON Schema

Page 2: Data Governance with JSON Schema

#MDBlocal

Shannon Bradshaw

VP of Developer Education,

Snowboarding Enthusiast, Father

MongoDB

@shannonbradshaw

Page 3: Data Governance with JSON Schema

#MDBlocal

$jsonSchema

Page 4: Data Governance with JSON Schema

#MDBlocal

Flexible

SchemaRigid

Schema

Your App

Page 5: Data Governance with JSON Schema

#MDBlocal

Flexible

SchemaRigid

Schema

Your App

risk

success!

regulation

policy

politics

Page 6: Data Governance with JSON Schema

#MDBlocal

Flexible

SchemaRigid

Schema

Your App

risk

success!

regulation

policy

politics

Production

Concerns

Page 7: Data Governance with JSON Schema

#MDBlocal

Page 8: Data Governance with JSON Schema

#MDBlocal

{

_id: 34556,

item: "Flannel T-shirt",

price: NumberDecimal("27.99"),

quantity: NumberInt(1)

color: "green"

}

Simple Schema to Start

Page 9: Data Governance with JSON Schema

#MDBlocal

Document Validation (v3.2)Use query filters to define schema constraints. E.g.,

db.createCollection("orders", {

validator: {

item: {$type: "string"},

price: {$type: "decimal"},

quantity: {$type: "int"},

color: {$in: ["red", "green", "blue"]}

}

});

(ex1 - ex6)

Page 10: Data Governance with JSON Schema

#MDBlocal

JSON Schema (v3.6)

● JSON Schema Draft 04

● Is becoming an industry standard

● Widely used; lots of community support

(ex5)

Page 11: Data Governance with JSON Schema

#MDBlocal

Flexible

SchemaRigid

Schema

Our App

Page 12: Data Governance with JSON Schema

#MDBlocal

(ex6 - ex8b)

Let's make our validator a little

more strict

Page 13: Data Governance with JSON Schema

#MDBlocal

Good for Starter Examples

{

_id: 34556,

item: "Flannel T-shirt",

price: NumberDecimal("27.99"),

quantity: NumberLong(1),

color: "green"

}

{

_id: 34556,

total: NumberDecimal("141.00"),

VAT: NumberDecimal("0.20"),

totalWithVAT: NumberDecimal("169.20"),

lineitems: [

{ sku: "MDBTS001",

name: "Flannel T-shirt",

quantity: 10,

unit_price: NumberDecimal("9.00") },

{ sku: "MDBTS002",

quantity: 5,

unit_price: NumberDecimal("10.00")}]

}

More Realistic

Page 14: Data Governance with JSON Schema

#MDBlocal

Good for Starter Examples

{

_id: 34556,

item: "Flannel T-shirt",

price: NumberDecimal("27.99"),

quantity: NumberLong(1),

color: "green"

}

{

_id: 34556,

total: NumberDecimal("141.00"),

VAT: NumberDecimal("0.20"),

totalWithVAT: NumberDecimal("169.20"),

lineitems: [

{ sku: "MDBTS001",

name: "Flannel T-shirt",

quantity: 10,

unit_price: NumberDecimal("9.00") },

{ sku: "MDBTS002",

quantity: 5,

unit_price: NumberDecimal("10.00")}]

}

More Realistic

Page 15: Data Governance with JSON Schema

#MDBlocal

Our New Orders Schema

● Orders must contain a lineitems array of 1 to 10 documents

● Documents embedded in lineitems have fields for:

○ sku (required)

○ name (optional)

○ unit_price (required)

○ quantity (required)

● Value of quantity must be an integer

● (Must also contain fields: total, VAT, totalWithVAT, but we'll get to that.)

Page 16: Data Governance with JSON Schema

#MDBlocal

Let's validate the line item documents

(embedded in an array)

(ex9 - 10b)

Page 17: Data Governance with JSON Schema

#MDBlocal

Now Let's Deal with Totals and VAT

The most recent order we looked at actually contains a few errors:● The totalWithVAT value should be 141 * 1.20 = 169.2

● The total value should be the sum of each line item sub-total

● I.e., (10 * 9) + (10 * 5) = 140

Can we enforce that totalWithVAT and total are correct using validation?

Page 18: Data Governance with JSON Schema

#MDBlocal

Yes we can!

● MongoDB 3.6 -- more expressive query syntax

● Can express complex business rules in validations

● E.g, compute and compare multiple values at runtime

Page 19: Data Governance with JSON Schema

#MDBlocal

$expr

Check that totalWithVAT == total * (1+VAT):

$expr: {

$eq: [

"$totalWithVAT",

{$multiply: [

"$total",

{$sum: [1, "$VAT"]}

]}

]

}

Let's add this as a validation rule alongside our JSON Schema validation ()

Page 20: Data Governance with JSON Schema

#MDBlocal

Now let's make sure the total is the

sum of the line item subtotals

Page 21: Data Governance with JSON Schema

#MDBlocal

Ensure total equals sum of line items$expr: {

$eq: [

"$total",

{$sum: {

$map: {

"input": "$lineitems",

"as": "item",

"in": {

"$multiply": [

"$$item.quantity",

"$$item.unit_price"

] } } }} ] }

Page 22: Data Governance with JSON Schema

#MDBlocal

All together now

(ex11 - 14c)

Page 23: Data Governance with JSON Schema

#MDBlocal

Flexible

SchemaRigid

Schema

Our App

Page 24: Data Governance with JSON Schema

#MDBlocal

Flexible

SchemaRigid

Schema

risk

success!

regulation

policy

politics

Our App

Page 25: Data Governance with JSON Schema

#MDBlocal

Flexible

SchemaRigid

Schema

risk

success!

regulation

policy

politics

Our App

Production

Concerns

Page 26: Data Governance with JSON Schema

#MDBlocal

Special thanks to Raphael Londner

Page 27: Data Governance with JSON Schema

#MDBlocal

Learn More

● What's New Guide

● MongoDB Documentation

○ Schema Validation

○ Expressive Query Syntax

● Free MongoDB 3.6 Online Course from MongoDB University