Webinar: Serverless Architectures with AWS Lambda and MongoDB Atlas

Serverless Architectures with AWS Lambda and MongoDB Atlas

Sig NarváezSr. Solutions Architectsig@mongodb.com @SigNarvaez

Serverless?• Landscape• Use cases

Going Serverless• What changes?• Considerations• MongoDB Atlas

AWS & MongoDB Atlas• Simple API for Customer Single View• Lambda & API Gateway• MongoDB Atlas & Compass• Postman

Agenda

Serverless

Big Iron Commodity Hardware

Virtualized

Containers

Functions

Where will my code run?

Serverless Frameworks and Platforms

https://github.com/serverless/serverless

https://www.zappa.io/

Chalice (awslabs)https://github.com/awslabs/chalice

Frameworks for Cloudproviders

On-Prem PaaS – now offering FaaS

1. Cloud services have matured

2. BaaS “SaaS-ification”

3. API’s are the glue

4. Containers – now per function

5. SysOps DevOps NoOps Less Ops, More Engineering

5 factors fuelling Serverless Computing

https://www.forbes.com/sites/janakirammsv/2016/02/28/five-factors-that-are-fueling-serverless-computing-part-1

Thoughtworks Technology Radar

Scheduled Jobs• Sequencing / Orchestration (AWS Steps?)

Data Quality• Trigger Identify Pass to function

Micro or Nano services• Clicks or Taps

Event and IoT processing• Don’t worry about scaling App Servers

Lightweight API’s• Focus of today!

Good fit for Serverless?

Serverless

Microservices

Before and after

Container-based Microservice

Payments Service

Product Catalog Service Shopping

CartService

Domainshttps://www.mongodb.com/blog/post/serverless-architectures-the-evolution-of-cloud-computing

Serverless Microservice

CommandQueryResponsibilitySegregation

Think about:Fine or course grainedShared logicStart-up time!

PackagingDeploymentVersioning

CQRS pattern on Serverless Microservices

GETAPI

PUT PATCH POST DELETE …

API

API Key

API Key

Lambda Function(s)

Lambda Function(s)

Code

Code

Lambda Function(s) VPC

Peering

Shape• Person• Insurance Policies

• Shape changes per policy type• Addresses

Operations via API• GET Customers with soon-to-expire

policies, within a geo radius• GET Customers / by SSN, id, etc.• PATCH Update basic contact info

(cell, email, …)

Customer Single View - Insurance Industry (hypothetical)

High-level architecture of a single view platform

MongoDB Atlas &

AWSBuild it!

Required MongoDB Services – Atlas!

mgeneratejs• https://

github.com/rueckstiess/mgeneratejs• npm install -g mgeneratejs• Create template – generate data• Upload to Atlas via mongoimport

• Hint: get connection string from Atlas UI!• Browse with Compass

Generate dataset

Template (InsuranceC360_Customers.json)

mgeneratejs -n 100 InsuranceC360_Customers.json | mongoimport --host ”YOUR ATLAS CLUSTER" --numInsertionWorkers 4--db WebinarCustomerSingleView --collection Customers --authenticationDatabase admin --ssl --username YOURUSER --password YOURPASSWORD

IAM• Role with Lambda execute

policies

VPC• VPC• Security Groups – traffic rules• Internet Gateway – outside

communication• VPC Peering Connection -

Route Table

Required AWS Services

Lambda• VPC, Security Group and IAM

role• Develop inline or upload

deployment package (.zip)• Use MongoDB Driver –

connect with MongoDB Atlas

API Gateway• API definition• API Keys & Usage Plans• Resources and HTTP Methods• Map Routes to Lambda

functions

VPC

MongoDB Atlas• Provision a Cluster – M10+ need an assigned AWS region for VPC peer• Same AWS region – (I will use us-west-2)• Initiate VPC peer with AWS

AWS VPC• Accept incoming Peering Connection• Update Route Table

EC2• Install MongoDB• Test connection from the MongoDB Shell to ensure VPC Peer is working• Optional but highly recommended – ensure VPC Peering is working before

proceeding to Lambda

MongoDB Atlas peered with your AWS VPC

VPC Peering

Atlas AWS

Verify VPC Peer works

Security Group

Peering Connections

Lambda

Role with lambda permissions (IAM)

Code packagingfrom __future__ import print_function

import jsonimport pymongo

print('Loading function')print(’=== CONNECTING TO MONGODB ATLAS ===')connstr = ”ENTER YOUR MONGODB ATLAS CONNECTION HERE"MONGOCLIENT = pymongo.MongoClient(connstr, readPreference=’secondaryPreferred’)

def GET_lambda_handler(event, context):

    … implement GET logic

def POST_lambda_handler(event, context):

    … implement POST logic http://docs.aws.amazon.com/lambda/latest/dg/lambda-python-how-to-create-deployment-package.html

Lambda functions

Upload & configure function

The handler function

The role with lambda permissions

The VPC (peered with Atlas)

The security group that allows traffic

At least 2 subnets

API Gateway

Read API – GET /api/v1/customers

CUD API - PATCH /api/v1/customers

Deploying the API

Access and throttling via API Keys

Test!

Test with Postman

Load test too!

AWS CloudWatch

Connections and containers …..

http://docs.aws.amazon.com/lambda/latest/dg/lambda-introduction.html

… AWS Lambda maintains the container for some time in anticipation of another Lambda function invocation. … the service freezes the container after a function completes, and thaws the container for reuse. If AWS Lambda chooses to reuse the container, this has the following implications:

- Any declarations in your Lambda function code (outside the handler code, see Programming Model) remains initialized, providing additional optimization when the function is invoked again. For example, if your Lambda function establishes a database connection, instead of reestablishing the connection, the original connection is used in subsequent invocations. You can add logic in your code to check if a connection already exists before creating one.

MongoDB Atlas Monitoring and Alerts

MongoDB Compass

Done!…

But what about?

Scaling?Scaling Lambda

No user intervention required - Default safety throttle of 100 concurrent executions per account per region.

Functions invoked synchronously throw 429 error code. Functions invoked asynchronously can absorb reasonable bursts for approx. 15-30 minutes. If exhausted, consider using Simple Queue Service (SQS) or Simple Notification Service (SNS) as the Dead Letter Queue (DLQ).

Read more at https://aws.amazon.com/lambda/faqs/

Scaling MongoDB Atlas

On-DemandZero downtimeUpscale/Downscale:• Instance size• Storage size• IOPS• Replication

factor.

Pricing?Lambda Costs

Cost depends on requests (per million), request time, memory (GB) allocated to each function.

First 1 million requests per month free - $0.20 per 1 million requests thereafter. $0.00001667 for every GB-second used.

Additional AWS services imply cost (e.g. API Gateway, …)

Read more at https://aws.amazon.com/lambda/pricing/

MongoDB Atlas Costs

Cost depends on instance size, storage, iops, replication factor and backup retention.

M0 free – great for you (no VPC peering, use IP whitelist)M10 starts at $0.08/hr – great for team DevM30 starts at $0.54.hr – great for Production

Read more at https://www.mongodb.com/cloud/atlas/pricing

Connections to MongoDB Atlas

Encrypt using AWS KMS – see this blog post: https://www.mongodb.com/blog/post/serverless-development-with-nodejs-aws-lambda-mongodb-atlas

Container freeze & recycle?• Connection outside lambda function

helps• On scale new containers, new

connections• Ok if API is used in bursts, but

maybe not ok if used seldomly

• If not?

Others?Local development? Lambda emulators

• python-lambda-local at https://pypi.python.org/pypi/python-lambda-local

• lambda-local (node.js) at https://www.npmjs.com/package/lambda-local

Serverless frameworks – evaluate them! … F500’s are!• Serverless Framework• Zappa• Chalice• More! - https://thenewstack.io

/tns-guide-serverless-technologies-best-frameworks-platforms-tools/

Façade Serverless Functions – logic querying backend API

Backend Traditional stateful layer - CRUD API to Data Stores

Would this be a Serverless

Architecture ??

Customer Single View - Insurance Industry (hypothetical)

High-level architecture of a single view platform

Stateful API Service Layer

Serverless Architectures with AWS Lambda and MongoDB Atlas

Q&AUse code "Sig" for 25% off!Parties of 3+ get addtl 25%

Sig NarváezSr. Solutions Architectsig@mongodb.com @SigNarvaez