Bringing Characters to Life with Amazon Polly Text-to-Speech - AWS Online Tech Talks

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Robin Dautricourt, Sr. Product Manager

Felix Duchesneau, Software Development Manager,

11/14/2017

Bringing Characters to Life with

Amazon Polly Text-to-Speech


Amazon Polly

• What is Amazon Polly?

• A short overview of how

Text-to-Speech is produced.

• Text-to-Speech features.

• Game development use case.

Amazon Lumberyard

• What is Lumberyard?

• A short overview of how video

games are made.

• Cloud assisted production tools.

• New experiences made possible.

Agenda for Today

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.


What is Amazon Polly?

• AWS service that converts text into lifelike speech

• 50 voices across 24 languages

• Low latency responses allow developers to build real-time

systems

• Developers can store, replay, and distribute generated

speech


Text-to-Speech Development

Text

Text normalization

Grapheme-to-phoneme conversion

Waveformgeneration

Speech

She has $20 in her pocket.

she has twenty dollars in her pocket

ˈ ʃ i ˈ h æ z ˈ t w ɛ n . t i ˈ d ɑ . ɫ ə ɹ z ˈ ɪ n ˈ h ɝ ɹ ˈ p ɑ . k ə t


Speech Synthesis Markup Language (SSML) Features

<speak>I can speak normally,

<prosody volume="x-loud">or I can speak in a louder voice.</prosody>

</speak>

<speak>When I get excited,

<prosody rate="x-fast">I start to speak very

fast.</prosody></speak>

<speak>Sometimes when I get nervous,

<prosody pitch="high">my pitch goes way up.</prosody></speak>


Speech Marks Feature

Synchronize Speech for an Enhanced Visual

Experience

• Speech Marks: additional metadata stream

• Timing information for visemes, words, sentences,

SSML

Sample use cases:

• Karaoke-style text highlighting

• Facial animation (lip-syncing)


Amazon Polly Use Cases

Telephony

/ IVR

Special Needs

AI Assistant

AI AnnouncerVideo

Voice-Over

Navigation Content

Creation

Education

Technology

Podcasts Game Development

• Pre-Production Speech

• Character Casting

• Directing (with SSML)

• Real-time Speech

• Store and Replay


Amazon Polly for Game Development -

Amazon Lumberyard Use Case


What is Lumberyard?

Amazon Lumberyard is a free AAA game engine deeply

integrated with AWS and Twitch – with full source.



Amazon Polly





Video game development in a nutshell


Producing a video game

Time

ProductionPre-productionConception

Gam

e L

aunches

Post-launch …



The environment will be under construction for most of the

production

The player, the NPCs, the gameplay mechanics, the story

telling need to be perfectly adapted to the environment

The assets: Characters, animations, props, audio will also

be under development for most of the production



Time

Produce 100s of characters

Put assets and characters in world

Build the world with tools in development

Produce 1000s of assets

Implement & improve gameplay mechanics

Playtest, Iterate, find and improve the fun

Story telling through cinematics and scripted events

Signs and feedback

Fix 1000s of bugs

Missions and level design

Assem

ble

every

thin

g

Hope f

or

the b

est

R&D and Experimentation

Develop new tools to build faster

Optimize

Prepare day1 patch


Placeholders will be used until the very end

- Reserve “physical” space

- Reserve memory budget

- Reserve time (timing-wise)

- Iterate and playtest

- Test if there is enough variety

- Test the localization

- …


Story telling - consistent story in a consistent world.

The story will be told by:

- Cinematics

- Scripted events

- Ambiance dialogues and audio

- Props, accessories

- Lighting

- Weather conditions


Game ≠ Story

Find the fun through play tests and iteration


Game mechanics

Player must understand what is going on.

Player must understand gameplay mechanics

Gameplay mechanics == computer algorithms

Player understands computer algorithms

How?

Signs and feedback


Signs and feedback

- UI elements

- Sounds

- Ambient dialogues

- NPC dialogues

- FX


Gameplay mechanics – An example


A stealth game with a gameplay loop of 4 states

CombatSearch

Alert Relaxed


Relaxed state

Wait Patrol

Enemy seen Enemy heard


Player has been momentarily seen or heard

Alert

“Did you hear something?”

“What was that?”

“What was that noise?”

“Huh?”


Alert state

Unseen

(--)

Seen/heard

(++)

Detected!


Player did not hide soon enough

Search

Show yourself!

Over there! I think I saw something.

Who’s there?


Search

Search

“Come out we don’t have all night.”

“Come out, come out wherever you are!”

“Hope you have good life insurance dude.”

“It takes some guts to come in here uninvited.”


Investigate last

known position

Search around

last known

position

Spread

Search state

Enemy

seen

Enemy

heard


Enemies are attacking – Time to run or fight

Combat

“Get him!”

“There he is!”

“Attaaaaaaack!”


Cloud assisted production tools




Three Parts of a Cloud Gem

Backend

infrastructure

Client Component Cloud Gem Portal

Plugin







New runtime game experiences made possible



Documents

Bringing Characters to Life with Amazon Polly Text-to-Speech - AWS Online Tech Talks