Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
IABM Copyright 2019@THEIABM www.theiabm.org
Enhancing Media Workflows with Amazon Machine
LearningShweta Jain
APAC Head of Business Development, Media & Entertainment
Amazon Web Services
IABM Copyright 2019@THEIABM www.theiabm.org
The New Media and Entertainment Reality
Viewer
Expectation
Content discoverability, recommendations,
personalization
Business
Opportunity
Learn about viewers, use the data to
increase engagement
Customer
Obsession
Customer satisfaction increases brand
value and drives revenue
IABM Copyright 2019@THEIABM www.theiabm.org
Challenges that have so-far prevented organizations from adopting Machine Learning quickly?
ML expertise
is rareBuilding and scaling ML
technology is
hard
Deploying and operating
models in production is
time-consuming
and expensive
Lack of cost-
effective,
easy-to-use, and
scalable ML services
IABM Copyright 2019@THEIABM www.theiabm.org
We set out to build aMachine Learning platform that is
accessible to every Developer,,Data Scientist & IT Professional
IABM Copyright 2019@THEIABM www.theiabm.org
Amazon ML : Broadest set of capabilities
M L F R A M E W O R K S
I N F R A S T R U C T U R E
A I S E R V I C E S
R E K O G N I T I O N
I M A G E
A M A Z O N
P O L L Y
A M A Z O N
T R A N S C R I B E
A M A Z O N
T R A N S L A T E
A M A Z O N
C O M P R E H E N D
A M A Z O N
L E X
R E K O G N I T I O N
V I D E O
Vision Speech Chatbots
A M A Z O N S A G E M A K E R
B U I L D T R A I N
A M A Z O N
F O R E C A S T
A M A Z O N
T E X T R A C T
A M A Z O N
P E R S O N A L I Z E
D E P L O Y
Pre-built a lgorithms & notebooks
Data label ing (G R O U N D T R U T H )
One-cl ick model training & tuning
Opti mizati on (N E O )
One-cl ick deployment
& hosti ng
M L S E R V I C E S
F ra m e w o r ks I nt e r fa ce s I n f ra st r u c t u re
E C 2 P 3 E C 2 C 5 F P G A s G R E E N G R A S S E L A S T I C
I N F E R E N C E
Reinforcement learningAlgori thms & models
( A W S M A R K E T P L A C E
F O R M A C H I N E L E A R N I N G )
Language Forecasti ng Recommendati ons
IABM Copyright 2019@THEIABM www.theiabm.org
Sample ML Use Cases in M&EAutomate creation of rich metadata (object, scene, activity, faces etc.), extracted from
audio visual content and integrated into digital and media asset management systems
AMAZON REKOGNITION, AMAZON TRANSCRIBE, AMAZON COMPREHEND
Media Metadata
Tagging
Closed Captioning
Detect potentially inappropriate content to avoid compliance issues in global markets, and
to increase brand safety for advertisers
AMAZON REKOGNITION, AMAZON TRANSCRIBE, AMAZON SAGEMAKER
Automated
Compliance Marking
Identify objects and emotion in content that enables users to integrate personalized ads
into subscriber video streams
AMAZON REKOGNITION, AMAZON SAGEMAKER, AWS MEDIATAILOR
Ad Personalization
Automated captions, transcription and translation of audio content
AMAZON TRANSCRIBE, AMAZON TRANSLATE
IABM Copyright 2019@THEIABM www.theiabm.org
Sample ML Use Cases in M&EIdentify and path player activity in the field or identify the precise timecodes when actors
enter and leave a scene, both visually and in spoken dialog (off-screen voice)
AMAZON REKOGNITION, AMAZON TRANSCRIBE
Player/ Actor
identification &
activity pathing
Translate transcripts and metadata. Improve localization workflows and search experience
AMAZON TRANSLATE
Language Translation
Analyze disparate data and enable the consumer to make personalized content choices and
the business to predict behavior
AMAZON PERSONALIZE, AMAZON SAGEMAKER
Content
Recommendations
Detect and pixelate faces captured incidentally to preserve the privacy of non-persons of
interest in news feeds and security footage
AMAZON REKOGNITION, AMAZON SAGEMAKER
Automated
Redaction
IABM Copyright 2019@THEIABM www.theiabm.org
Media Enrichment and Analysis
IABM Copyright 2019@THEIABM www.theiabm.org
Automated Metadata GenerationVideo
Rekognition
Video
Search Engine:
Amazon ElasticsearchAsset Management System
1. Video is uploaded and
stored to the Data lake
2. Create metadata for
celebrities, emotions, scene
time, objects, voices in video
3. The output is sent to the
digital/media asset
management system and be
available in a search engine
Dynamic search indexing
Transcribe
Data Lake on AWSStorage | Archival Storage | Data Catalog
IABM Copyright 2019@THEIABM www.theiabm.org
AI based taggingBackground
• Over 200,000 hours of content
• Only half of content is manually tagged
• How can we enrich our metadata in AWS?
• How can we unleash the value of content we already
own once in AWS?
Challenge
• Large scale video library
• High accuracy required
• Limited budget
• Ability to extract from video
• Keep up with daily increase in content
Results
• Solution developed within three weeks
• Live video frame based analysis
• Established, searchable baseline archive
• All content is now tagged and indexed
• Over 99,000 faces indexed and searchabl
• Saved ~9,000 hours a year in manual ef
IABM Copyright 2019@THEIABM www.theiabm.org
Event Broadcasting Enrichment
Facial recognition
combined with
additional
metadata – Bios,
News, Fashion,
Emotions.
Matching trained
by humans
IABM Copyright 2019@THEIABM www.theiabm.org
Tag, filter, and redact inappropriate content
Person 99.2% Gun 84.6% Handgun 73.5%
Drink 96.4% Alcohol 80.1% Wine 69.9%
IABM Copyright 2019@THEIABM www.theiabm.org
Formula1: Fan EngagementBackground
• Formula 1 is the worlds premier auto racing championship.
• Since the first gran prix in 1950, Formula 1 engages with
500 million fans watching 21 global races annually.
Challenge
120 car telemetry sensors create 3Gb of data per second,
how can Formula 1 bring analytics and insights to car data that
drives advances in the sport and an deeper fan experience
delivered by their next generation OTT/VOD platform. https://aws.amazon.com/f1insights/
Use of AWS
• AWS Kinesis streams data into S3 in real-time
• Amazon Rekognition, Amazon SageMaker, & Amazon
Transcribe analyze race data
• AWS Media Services powers next generation video platform
Results
• Deeper fan experience
• Monetize data and direct to consumer video platform
IABM Copyright 2019@THEIABM www.theiabm.org
Sports media tagging
IABM Copyright 2019@THEIABM www.theiabm.org
Personalization
IABM Copyright 2019@THEIABM www.theiabm.org
Users expect every interaction to be personalized
Activity & Content
Recommendation
Search
Personalization
Personalized
NotificationsEmails
IABM Copyright 2019@THEIABM www.theiabm.org
Personalization: multiple hard problems
Popularity Trap Naïve models give recommendations similar to popular items
Cold StartsNew users should get relevant recommendations, new items should show in recommendations
ScaleRecommendations should scale across millions of users and items
Real-TimePersonalization must be responsive to the changing user intent
Custom modelsPersonalization models must accurately reflect business context and user behavior
IABM Copyright 2019@THEIABM www.theiabm.org
Introducing Amazon PersonalizeBased on the same technology used at
amazon.com
IABM Copyright 2019@THEIABM www.theiabm.org
Amazon Personalize - Workflow
Prepare your data,
then upload with the
Amazon Personalize
API
Choose one of our
algorithms or tell AutoML
to find the best fit
Modify your code Retrain continually to
improve the model
IABM Copyright 2019@THEIABM www.theiabm.org
Real-time data can be consumed by Amazon Personalize
Historical user
activity
User
attributes
Item
catalog
Real-time data
Mobile
SDKs
(coming soon)
JavaScript SDK
Amazon S3
bucket
Server-Side SDKs
Offline data
IABM Copyright 2019@THEIABM www.theiabm.org
30% of page views on Amazon are from recommendations
IABM Copyright 2019@THEIABM www.theiabm.org
How we can help
Discovery Workshops - ML Solutions Lab – ML ProServBrainstorming
Custom modeling
Training
Work side-by-side with Amazon experts
Partner Ecosystem
AWS Media Solutions
https://aws.amazon.com/media/solutions/