Final presentationwherecamp2019.geoit.org/wp-content/uploads/2019/10/6... · 2019-11-15 · v3:...

Final presentation

wherecamp berlin 2019

1

Design Patterns of GeoData Driven Apps

Shoaib Burqhttps://geografia.com.au

http://deeplearningbox.com

WhereCamp 2019 (Berlin)

Complexity in Data Driven Apps

3

greater number of data sources

more stakeholders

more applications

bigger datasets

4

5

6

7

Client Name

v1 (2017) - Traditional Geo Applications

8

v1: Challenges

9

No Quality Checks

Poor Schema Enforcement

Time Consuming

Not Scalable for Growth

10

11

v2: add workflow system

12

v2: add workflow systemAdd S3 to Store

Raw Data Trigger Workflow

(AWS lambda)

13

v2: add workflow systemAdd S3 to Store

Raw Data Trigger Workflow

(AWS lambda)

https://docs.prefect.io/core

14

Forecasting with Machine / Deep Learning

v3: Doing Data Science

15

v3: Doing Data Science

16

Workstation with GPUs to train ML &

DL Models ————————— deeplearningbox.com —————————

v3: Doing Data Science

17

Workstation with GPUs to build Deep Learning Models

Competing for Resources with

applicatioonManual Deployment

of ML Models

v4: ML Model Tracking + Deployments

18

v4: PostgreSQL read only replicas

19

v4: MLFlow + RDMBS scaling

20

v4: MLFlow + RDMBS scaling

21

Fix: Competing for ResourcesFix: ML

Experiment Track & Deploy

Futureexpected demands from customers & team

22

FutureInteractive Big Data Queries via Hadoop + Presto

23

FutureReal-Time events integration (e.g. social media feed)

24

Thank you! Questions?

Shoaib Burqhttps://geografia.com.au | http://

deeplearningbox.com

25