Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
Enhancing the UmbracoEnhancing the Umbracoexperience using machineexperience using machine
learninglearning
Steve TempleTechnical Director
About meAbout meTechnical Director and coowner of Gibe
Umbraco MVPUmbraco Certified Master
Organiser of Umbraco Spark conference
Will be using AI and Machine Learning interchangeablyApproaching Machine Learning as a developer
I am not a data scientistHave been playing with Azure ML since it came outFirst experiments were predicting horse racing results
UmbracoSpark Developer Conference
Bristol6th March 2020
Point of the talkPoint of the talk
No �nished out of the boxNo �nished out of the boxsolutionssolutions
Intention is to spark ideas and inspire packagesThere is a massive learning curve to some of these
But for some solutions the hard work has been done an it's up to usto figure out how to use it
Machine Learning toolsMachine Learning tools
On the shoulders of giantsOn the shoulders of giants
Services available from amongst others:
Microsoft Azure, AWS,Microsoft Azure, AWS,Google, IBMGoogle, IBM
What can you useWhat can you useMachine Learning for?Machine Learning for?
ClassificationPrediction
Anomaly detection
How can we use these to solve problems inHow can we use these to solve problems inUmbraco sites?Umbraco sites?
Classi�cationClassi�cationCategorising & tagging mediaCategorising & tagging media
or contentor content
Problem to solveProblem to solveHow can we use AI to improveHow can we use AI to improve
Umbraco's media section?Umbraco's media section?
1. No standard location for media2. Doesn't scale to 10,000s of images 3. Hard to find the exact image you want
The Media TreeThe Media TreeDoes a tree make sense for Media?
Has some downsides:Needs more information on each image/video/file
Editors would have to tag/categorise all media including existingmedia
Could we searchCould we searchinstead?instead?
Tagging of mediaTagging of mediaUse Azure Vision API to autoUse Azure Vision API to auto
tag/categorise/detect featurestag/categorise/detect featuresin imagesin images
DEMO
Great demoGreat demoBut rubbish IRLBut rubbish IRL
Custom Vision APICustom Vision APIWe can train our own modelWe can train our own modelwith our own tags which wewith our own tags which we
want to detectwant to detect
DEMO
What other APIsWhat other APIscould we use?could we use?
Focal point?Video analysis?Image cropper?Visual search?
What about content?What about content?Auto tagging / subject identification
Automated translation
Optical Character Recognition (OCR)https://our.umbraco.com/packages/backofficeextensions/translationmanager/
https://our.umbraco.com/packages/backofficeextensions/ocrcontentapp/
PredictionPredictionRecommending other contentRecommending other content
Problem to solveProblem to solveHow can we use AI toHow can we use AI to
recommend content on a site?recommend content on a site?
Azure Machine Learning StudioAzure Machine Learning Studio
Azure Machine Learning StudioAzure Machine Learning Studio
Azure Machine Learning ServiceAzure Machine Learning Service
Replaces Azure ML StudioCan use VS Code or Jupyter Notebooks
Azure DatabricksAzure Databricks
Very powerfulVery complicated!
Works with Azure ML Services
Solutions/ExamplesSolutions/Examples
https://gallery.azure.ai
RecommendationsRecommendationsSolutionSolution
Prebuilt solution template that deploys Azure resources preconfigured for performing recommendations
Needs usage information and catalog information to train the
model
https://github.com/Microsoft/ProductRecommendations
Recommendations solutionRecommendations solution
Needs to be supplied with 2 models to createrecommendations
UsageUsageModelsModels
How users are using the site
CatalogCatalogModelsModels
Product or content information
Con�guring and usingCon�guring and usingthe API the API
REST API with Swagger UIREST API with Swagger UI1. Create a trained model2. Make requests to the API for recommendations
DEMO
Suggesting media to editorsYou might also like / related content
Estimated time to readPersonalisation
...
How about?How about?
Anomaly DetectionAnomaly DetectionMaking sites more secure andMaking sites more secure and
easier for the userseasier for the users
Problem to solveProblem to solveCan we use AI to makeCan we use AI to make
user/member logins simpleruser/member logins simplerwhile being more securewhile being more secure
LoginLoginCould we serve different user orCould we serve different user or
member login experiencesmember login experiencesbased on probability of the userbased on probability of the user
being legitimatebeing legitimate
Factors we mightFactors we mightconsiderconsider
IP AddressCountryRegionTime
Browser / cookie recognisedPrevious logins
Recent failed attemptsPrivileges
Score the attemptScore the attempt> 98%
Auto login?Remember me, etc
75% +Regular login
< 75%Show RecaptchaRequire 2FA
This all sounds superThis all sounds supercomplexcomplex
https://giphy.com/embed/5dYeglPmPC5lL7xYhs
How to get startedHow to get startedSo much potentialSo much potentialGet a free azure account & experiment
There are multiple competitions/challenges all come with brilliantdata sets to get started with
Steep learning curve, but look at sample solutions and work fromthere
ResourcesResourceshttps://studio.azureml.net/
https://azure.microsoft.com/engb/services/cognitiveservices/
https://gallery.azure.ai/
https://github.com/Microsoft/ProductRecommendations
https://docs.microsoft.com/enus/azure/machinelearning/studio/algorithmchoice
Thank youThank you