Cortana provides a whole new natural way of interacting ...sec.ch9.ms/slides/winHEC/CortanaandSpeechPlatformOverview.pdf · Cortana helps you be more productive by completing basic

Cortana provides a whole new natural way of interacting with your PC

And is the world’s most personal digital assistant.

The Microsoft Speech Platform is used to power all of the speech experiences in Windows 10 such as Cortana and dictation.

• Introduction to Cortana

• Windows Speech Platform and the Windows Audio Pipeline

• Hardware Specification and Test Guidance

MICROSOFT CONFIDENTIAL – for discussion purposes only. © 2015 Microsoft Corporation. All rights reserved.

Cortana1. More useful every day.

Cortana learns about you over time to become more useful every day. By paying

attention to what you like and how you do things, Cortana gives you an experience

where your individuality is celebrated not ignored. And you’re able to see and decide

what Cortana knows, so you’re always in control of what details you share.

2. Here on all your devices.

Cortana works across all your Windows 10 devices to save you time and effort.* Set a

reminder on your PC and have it pop on your phone. Start a search on your phone that

you access on your PC. So many examples, so many ways to get more done.

3. Get things done at home and at work.

Cortana helps you be more productive by completing basic tasks like sending emails,

scheduling events, and using the power of Bing to quickly search your devices, the

cloud, or the web. It couldn’t be easier to have Cortana help: just type into the taskbar

or say ‘Hey Cortana’ to get hands-free help without leaving what you’re doing.*

4. Best at reminders.

Cortana’s the best digital assistant for reminders, delivering them at the right time and

place so there’s less you forget and more you can get done. Set a reminder on your PC

and have it delivered on your phone, or vice versa.4MICROSOFT CONFIDENTIAL – for discussion purposes only. © 2015 Microsoft Corporation. All rights reserved.

• Cortana will be available on all Windows 10 devices reaching consumers on desktop,

tablet, and phones

• Cortana is the single Microsoft personal assistant across your Windows

10 devices

• Microsoft is committed to enabling our partners to build on the

magic of Cortana to differentiate their apps and help their users be more productive

• The Cortana development platform will have the tools, support and

capabilities our developers expect from Microsoft platforms


The Microsoft Speech Platform is used to power all of the speech experiences in Windows 10 such as Cortana and dictation

• Language support and speech engines for speech recognition

• Speech Recognition optimized to understand variations in speech patterns from a diverse population of users

• Windows Audio Pipeline includes DSP enhancements

• Voice activation functionality with “Hey Cortana”


A great speech experience begins with good acoustics and a high performing audio pipeline that can compensate for the ambient noise and deliver clean speech to the recognizer.

• If device exposes DSP enhancements for the speech mode, Windows will default to use those

• If device does not expose DSP enhancements, then the Windows inbox enhancements will be used by default

Important: The OEM must expose mic array geometry.

Mic EQ, Gain Speech Pipeline Speech Recognizer

Acoustic ModelsMulti-channel Echo Canceling

Noise Suppression

Beamforming

Mic Geometry

Voice Activation

OEM Microsoft

Automatic Gain Control


• The Microsoft Voice Activation algorithm achieves excellent Correct Accept (CA) and low False Accept (FA) performanceo Support for both staged commands (“Hey Cortana” <wait for beep> “What’s the weather?”) and

chained commands (“Hey Cortana, what’s the weather?”)

• Third-party Voice Activation solutions can be integrated with Cortana through the Voice Activation DDI (device driver interface)

Voice activation – “Hey Cortana”



Microphone + Digital Interface Recommendation

Max Level≤ -20 dBFS RMS

100-8000Hz

Min Level≥ -55 dBFS RMS

100-8000Hz


Microphone Array Hardware Guidance


Linear 2-element, 45-170 mm Circular array, 8 elements

Linear 4-element geometry L-shaped 4-element

Target test score = 90% or better

Target test score = 85% or better

Target test score = 82% or better -> moving to 85% or better for fall update

Recommended target score results based on Microsoft Speech platform tool output: Recorder and Scoring utility


• Designed to augment existing industry tools and test methodologies

• Driver Configuration Verification Tool• Verifies device capabilities, modes and microphone array details. The tool output provides a indicative

assessment of the audio pipeline.

• Recorder• Records the audio input(s) and audio output during a set of standardized tests and scores for Word

Error Rate which can be compared to the “Speech Platform: Input Device Recommendations.”

• Score Utility• Enables self-analysis of the recording tests for the recorder tool


Cortana provides a whole new natural way of interacting with your PC and is the world’s most personal digital assistant.

The Microsoft Speech Platform is used to power all of the speech experiences in Windows 10 such as Cortana and dictation.

• Specifications and tools available today

• Speech and Natural Language Ecosystem Labs targeted for Spring 2016

• Follow the guidance provided in the Speech Platform Input Device Recommendations Specification

• Design hardware solutions that enable high quality speech scenarios

• Ensure drivers expose microphone array geometry

• Engage your Microsoft ecosystem representatives for questions, concerns, or more information

• Send your Windows Speech Platform questions to [email protected]


mailto:[email protected]

(c) 2015 Microsoft Corporation. All rights reserved. This document is provided "as-is." Information and views

expressed in this document, including URL and other Internet Web site references, may change without notice. You

bear the risk of using it. This document does not provide you with any legal rights to any intellectual property in any

Microsoft product. You may copy and use this document for your internal, reference purposes.

Some information relates to pre-released product which may be substantially modified before it’s commercially

released. Microsoft makes no warranties, express or implied, with respect to the information provided here.

Documents

Cortana provides a whole new natural way of interacting ...sec.ch9.ms/slides/winHEC/CortanaandSpeechPlatformOverview.pdf · Cortana helps you be more productive by completing basic