What's Cooking in the Cloud?

Kasper Løvborg Jensen Leafcastle Labs | Aarhus University

What’s Cooking in the Cloud? Expanding the Menu for App Developers

Kasper Løvborg Jensen


Chief Wizard, Founder Leafcastle Labs Associate Professor Aarhus University School of Engineering Organizer Google Developer Group Aarhus

@kasperlj [email protected]


GCP NEXT ‘16 Conference

•  Machine Learning for all! •  Cloud Vision for all! •  Cloud Speech API for all (soon)! •  StackDriver (works with AWS)! •  …lots more!


Outline

•  Why cloud? •  Mobile backends – Experiences – Patterns

•  New powerful Tools/APIs – Machine Learning – Speech API – Cloud Vision

•  Demo: TA-800



Why I am Excited about Cloud / GCP

•  Backends for mobile/wearable/web apps –  Low/No Maintenance –  Scalability –  Reliability –  Security

•  APIs / Services for applications –  Integrate/synch with existing Services –  Add powerful features/functionality through APIs

•  Great for startups –  Business models and focus on creating value –  Cost scales with growth –  Speed of iteration and experiments

Kasper Løvborg Jensen Leafcastle Labs | Aarhus University 7


In My Steps




GCP Scalability


App Backend Needs

•  Storage •  Computing •  Network / serving •  Cloud Messaging •  Analytics •  Tools •  …


Google Cloud Platform

Source:GCPNEXT2016keynote


Mobile Backends?

Source:Google,Android,Apple,Chrome


Mobile Backends “Building a backend service for a mobile app is similar to building a web-based service, with some additional requirements:

•  Limit on-device data storage •  Synchronize data across multiple devices •  Handle the offline case gracefully •  Send notifications and messages •  Minimize battery drain”

hAps://cloud.google.com/soluEons/mobile/mobile-app-backend-services


Backend Patterns

M.C.Escher


App Engine w. EndPoints

Modifiedfrom:hAps://cloud.google.com/soluEons/mobile/mobile-app-backend-services


Pure Firebase



Firebase + App Engine



Firebase + MVMs | Containers



Machine Learning

•  Machine Learning •  Vision API •  Speech API •  Translate API

Source:Google


FROM THIS


TO THIS


Source:www.tensorflow.org


But wait, isn’t Machine Learning EVIL?


If by evil, you mean AWESOME!


Cloud Speech APi


Speech API

•  Solves really hard problems! – Natural language recognition – Open vocabulary – Noisy environments – 80+ Different languages

hAps://cloud.google.com/speech/


Speech Horror Story ca. 2005 •  Distributed speech recognition in

app as part of my PhD – On PDA running Windows Mobile

with feature extraction through custom DLL (c/c++)

–  Live stream features to Sphinx IV DSR on Tomcat server that could only handle one connection/request at a time due to memory requirements

– Months of building/training speech, grammar and acoustic models



Live or Batch?

•  Batch – Upload audio sample(s) and get

results when recognition complete

•  Live – Stream audio to server and

receive partial results as they become available

hAps://cloud.google.com/speech/


Cloud Vision API


Cloud Vision API •  Solves really hard problems! – Extract and label objects in arbitrary image

without context – Can be anything – Find details like facial features, tilt and

“emotions“ –  Lighting conditions – Blurry pictures – Vast amount of cameras and settings – …


Case Study

•  Problem: Students falling asleep during lectures

•  Solution: Need person some sort of contraption to detect students falling asleep and then wake them up

“Never send a man to do a machine’s job”

-Gandalf


TA-800

•  IF we wanted to build a Terminator as a teaching assistant – Running Android – Powered by Google Cloud / Skynet

•  Detect and “terminate” inactive students during lectures – High precision Nerf gatling gun


TA-800 Hardware

•  Nexus 6 or 6p •  ATLAS prototype •  Nerf minigun


TA-800 Behavior Protocol

While(true){ – Detect situation

•  Identify features and classify place/situation

– Acquire potential targets •  Detect if sleeping

– TERMINATE Wake up targets – Report to master

}


Architecture Master TA-800


Demo


TA-800 Hack/Demo •  Image processing based on Cloud Vision API

example app https://github.com/GoogleCloudPlatform/

cloud-vision/tree/master/android/CloudVision •  Firebase integration based on Firebase chat

example https://github.com/firebase/AndroidChat

•  Total dev time:~6 hours (50%+ time on UI)

•  Will be on GitHub:


Cloud Vision API

hAps://cloud.google.com/vision/


Using Cloud Vision API

•  Specify request (JSON) –  Image

•  Content/source – Features

•  Type •  Max results

•  Handle result (JSON) •  Android: use CloudVision API –  com.google.apis:google-api-services-vision:v1-rev2-1.21.0 !


Cloud Vision Detection Types DETECTION_TYPES = [ ! 'TYPE_UNSPECIFIED', ! 'FACE_DETECTION', ! 'LANDMARK_DETECTION', ! 'LOGO_DETECTION', ! 'LABEL_DETECTION', ! 'TEXT_DETECTION', ! 'SAFE_SEARCH_DETECTION', !] !


Rich Response Data as JSON •  Labels

–  Things/situations in the image •  Faces

–  Faces, location in picture –  Happy, angry, sad, surprised –  Face “landmarks”: eyes, ears, … –  Tilt, …

•  Logos –  Known brands and logos

•  Landmarks –  “Eiffel Tower”, “AROS museum”, … –  Also including e.g. location metadata

•  Texts –  OCR, finding text in the image


Your turn!

GCP: https://cloud.google.com/

List of (most) products: https://cloud.google.com/products/

Playground: https://cloud-playground.appspot.com/playground/

Launcher: https://cloud.google.com/launcher/

Firebase: https://www.firebase.com/


THANK YOU! Questions?

@kasperlj

[email protected]

Technology

What's Cooking in the Cloud?