Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted
음성 UI - 봇(Bot)과사물인터넷의설계및개발Voice User Interface & Assistant & Bots & IoT
황주필 ([email protected])유원조 ([email protected])
Sales Consultant, Infrastructure CloudAug 30, 2017
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
Confidential – Oracle Internal/Restricted/Highly Restricted
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
User Interface의변화CLI / GUI ??
Confidential – Oracle Internal/Restricted/Highly Restricted 3
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
HAL 9000
Confidential – Oracle Internal/Restricted/Highly Restricted 4
Computer
HAL & Computer
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
User Interface
Confidential – Oracle Internal/Restricted/Highly Restricted 5
?
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
User Interface
Confidential – Oracle Internal/Restricted/Highly Restricted 6
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
User Interface – CLI
Confidential – Oracle Internal/Restricted/Highly Restricted 7
CLI - Command Line Interface
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
User Interface
Confidential – Oracle Internal/Restricted/Highly Restricted 8
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
User Interface – GUI (마우스에서손가락으로)
Confidential – Oracle Internal/Restricted/Highly Restricted 9
GUI – Graphical User Interface
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Evolution of interaction
Confidential – Oracle Internal/Restricted/Highly Restricted 10
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
User Interface - Voice
Confidential – Oracle Internal/Restricted/Highly Restricted 11
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 12
Mobile Voice Interface
2011 2012 2014
Speech Recognition에서 AI로의발전
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Voice User Interface - Wars
스마트폰에서 Apple Siri와 Google Now같은음성인식 AI비서서비스로경쟁을했지만, Amazon이 2014년인공지능스피커 Echo를출시, 음성인식소프트웨어인 Alexa의개발자키트를외부에오픈 Skill이빠르게늘어남 (검색, 쇼핑, 뱅킹등등..) Echo 판매량증가
Confidential – Oracle Internal/Restricted/Highly Restricted 13
기업명 내용
Apple - 2010년 ‘Siri’ (음성인식소프트웨어) 인수- 2016년 VocalIQ (음성인터페이스) 인수
Amazon - 2011년 ‘Yap’ (STT : Speech-to-Text) 인수- 2014년 ‘Evi’ (지식기반 Semantic Search Engine), ‘INOVA’ (TTS : Text-to-Speech) 인수
Facebook - 2013년 ‘Jibbigo’ (모바일다국어음성번역) 인수- 2015년 ‘Wit.at’ (음성인식), ‘Two Big Ears’ (VR 오디오) 인수
Google - 2014년 ‘DeepMind technologies’ (AI 머신러닝) 인수- 2016년 API.AI (음성인식, 자연어처리) 인수, Limes Audio(음성처리) 인수
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Voice User Interface - Wars
Confidential – Oracle Internal/Restricted/Highly Restricted 14
회사 이름 API / SDK
Amazon Alexa (2014) Alexa voice service sdk 제공, 한국어지원안함
Google Assistant (2016) Google assistant sdk 제공, 한국어지원
Apple Siri (2011) Siri SDK for iOS, macOS 제공, 한국어지원
SKT Nugu (2016) X
KT GigaGeenie (2017) 제휴신청필요 (법인심사 2주소요)
Naver Wave (2017) 음성인식 ‘Clova’ API 공개(Android iOS SDK), 일 3600초까지무료
Kakao Kakao mini (2017) 3분기 AI앱/스피커출시예정, 음성인식 “뉴톤” Android,iOS SDK
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Voice User Interface - Wars
• Amazon – Alexa Voice Service (AVS)
• 2015 – Alexa Skills Kit (ASK) 공개, 2017년 1분기이용가능한 Skills 개수가 10,000 개 (Uber호출, Amazon물건구입, 스타벅스커피주문등..)
Confidential – Oracle Internal/Restricted/Highly Restricted 15
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
나만의음성인식비서를만들어보자
Confidential – Oracle Internal/Restricted/Highly Restricted 16
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
DIY Alexa !
Confidential – Oracle Internal/Restricted/Highly Restricted 17
집에 Raspberry Pi 하나쯤은있잖아요 !
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
DIY Alexa !
Confidential – Oracle Internal/Restricted/Highly Restricted 18
Step 1 … 10 따라하기
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
DIY Alexa !
Alexa-avs-sample-app
http://github.com/alexa/alexa-avs-sample-app
Alexa 개발자 – AVS 활성화 – 기기등록
Alexa! How is the weather today??
Confidential – Oracle Internal/Restricted/Highly Restricted 19
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Google Assistant 는?
Confidential – Oracle Internal/Restricted/Highly Restricted 20
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
• google-oauthlib-tool –client-serect/home/pi/client_id.json –scope https://www.googleapis.com/auth/assistant-sdk-prototype --save --headless
• googlesampeles-assistant-hotword
Confidential – Oracle Internal/Restricted/Highly Restricted 21
DIY Google Home !
• Raspberry Pi – Audio Enable
• Audio Lib 설치
• Python Module 설치 with pip– pip install –upgrade google-assistant-
sdk
– pip install –upgrade google-auth-oauthlib
• Google APIs > credentials > client id 등록및 client-secret json다운로드후 oAuth tool을통해서credentials.json 생성
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
한번보시죠Demo
Confidential – Oracle Internal/Restricted/Highly Restricted 22
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Home Automation with Google Assistant
Confidential – Oracle Internal/Restricted/Highly Restricted 23
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
AI Speaker - 아키텍처
AI. Speaker Cloud
How’s the weather?
기본기능시간,날씨,메모,음악
Intent 파악
응답 Speech 생성
인터넷검색
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
AI Speaker - 아키텍처
AI. Speaker Cloud
How’s the weather?
Intent 파악
종료
Custom 기능Alexa Skill, Google Action
Other 시스템
“Ask my skill” “Talk to my Home”
Webhook
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Home Automation - 아키텍처
음성인식,출력 Case에따른 Device 호출 Device 제어
자연어인식
의도(intent) 파악
Custom 기능 Action 호출
기기(TV, 에어컨, 조명)
명령 (On, Off)
기기제어 API 호출
기기제어 API
적외선(IR) 호출
Python Simple Http
Server
AI Speaker SDK
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Home Automation - 아키텍처
Confidential – Oracle Internal/Restricted/Highly Restricted 27
Google Assistant SDK
Google Actions
Google Api.AI
Python Simple Http
Server
BroadLinkRM3 Mini
Python API
ngrokWebhook
https
HomeNetwork
Server
TCP
IR (적외선)
IRCustomAction
Talk to“My Action”
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
ngrok 이란?
이미지출처 https://ngrok.com/
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
TV, 에어컨제어 - IR (적외선)
Confidential – Oracle Internal/Restricted/Highly Restricted 29
직접만들수있음Receiver, Sender, LIRC단점: 공구비용, 단방향
BroadLink RM3 Mini
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
TV, 에어컨제어 - IR (적외선)
Confidential – Oracle Internal/Restricted/Highly Restricted 30
TV
Audio
Aircon
사용자추가기기
192.168.1.111192.168.1.100
Send SIGNAL Code
Learn SIGNAL
IR Signal
TV ON Code : 26001399401923….Audio ON Code : 14454512345451….Aircon ON Code : 6234973491-384….
TCP
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
TV, 에어컨제어 - Code
Confidential – Oracle Internal/Restricted/Highly Restricted 31
pi@raspberrypi:~/rm3/python-broadlink/cli $ ls
README.md aircon.turnoff broadlink_discovery tv.channel1 tv.channel5 tv.channel9 tv.turnon
aircon.downtemp aircon.turnon MyHomeServer.py tv.channel2 tv.channel6 tv.channeldown tv.volumedown
aircon.morewind aircon.uptemp device.info tv.channel3 tv.channel7 tv.channelup tv.volumeup
aircon.smallwind broadlink_cli tv.channel0 tv.channel4 tv.channel8 tv.turnoff
pi@raspberrypi:~/rm3/python-broadlink/cli $ ./broadlink_cli -h
usage: broadlink_cli [--device DEVICE] [--type TYPE] [--host HOST] [--send SEND] [--sensors] [--learn] [--learnfile
LEARNFILE]
--send SEND send command
--learnfile LEARNFILE learn command and save to specified file
pi@raspberrypi:~/rm3/python-broadlink/cli $ cat tv.turnon
2600139940192398109823874098123098120398120974812738129739128673129038012983012938216498123213
BroadlinkRM3 Mini
Python API
01340891801…….
34923043123……
tv.channelup
tv.turnon
broadlink_cli
https://github.com/mjg59/python-broadlink
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
TV, 에어컨제어 - Code
Confidential – Oracle Internal/Restricted/Highly Restricted 32
if command == 'channel':
if level != '':
if level == '11':
exeCmd = './broadlink_cli --send @' + device + '.' + command + "1"
subprocess.call(shlex.split(exeCmd))
exeCmd = './broadlink_cli --send @' + device + '.' + command + "1"
subprocess.call(shlex.split(exeCmd))
elif command == 'volumedown' or command == 'volumeup':
for i in range(1,numlevel):
exeCmd = './broadlink_cli --send @' + device + '.' + command
subprocess.call(shlex.split(exeCmd))
...
...
def run(server_class=HTTPServer, handler_class=S, port=9876):
server_address = ('', port)
httpd = server_class(server_address, handler_class)
print("Starting httpd... 9876")
httpd.serve_forever()
Python Simple Http
Server
BroadLinkRM3 Mini
Python API
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Oracle Bot Integration with Alexa
Confidential – Oracle Internal/Restricted/Highly Restricted 33
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Live Demo - Pizza 주문
Confidential – Oracle Internal/Restricted/Highly Restricted 34
Oracle's Intelligent Bot
피자주문
크러스트
사이즈
피자종류
사이드
User결제수단
기존카드정보확인
접속요청
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Oracle Bot Integration with Alexa - 아키텍처
Confidential – Oracle Internal/Restricted/Highly Restricted 35
AmazonAVS SDK
Alexa Voice Service
Alexa Skill(Oracle Pizza)
ask“Oracle Pizza”
Oracle Intelligent Bot
Channel
Channel Configurator
DialogFlow
Execution
CustomComponents
AI EngineWelcome to
oracle pizza
Cloud
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 36
Alexa Skill 설정
{"intents": [
{"slots": [
{"name": "command","type": "LITERAL"
}],"intent": "CommandBot"
},{
"intent": "AMAZON.StopIntent"}
]}
Intent schemaSkill invocation name
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 37
Oracle Intelligent Bot 의 Alexa Channel 설정
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 38
Oracle Intelligent Bot 동작방식
“I want to order a pizza” “Can I order large thin cheese pizza?”
메시지수신
1
의미파악
2
처리를위한특정정보기억
3
답변수행
4
Intent
Utterance
Entity
Dialog Flow
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 39
Oracle Intelligent Bot 의 Intent, Entity 설정
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Web기반음성인식 with Oracle Bot
Confidential – Oracle Internal/Restricted/Highly Restricted 40
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Web기반음성인식 with Oracle Intelligent Bot
Confidential – Oracle Internal/Restricted/Highly Restricted 41
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 42
Live Demo - 한국어음성인식 Oracle Bot Demo
Oracle's Intelligent BotUser Chrome 브라우저
Backend System
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Web기반음성인식 with Oracle Intelligent Bot - Code
Confidential – Oracle Internal/Restricted/Highly Restricted 43
if (!('webkitSpeechRecognition' in window)) {
upgrade();
} else {
start_button.style.display = 'inline-block';
recognition = new webkitSpeechRecognition();
...
recognition.onresult = function(event) {
var interim_transcript = '';
for (var i = event.resultIndex; i < event.results.length; ++i) {
if (event.results[i].isFinal) {
var recData = event.results[i][0].transcript;
sendToBot(recData);
...
var u = new SpeechSynthesisUtterance();
speechSynthesis.speak(u);
Ajax로 Oracle Bot 호출
음성인식객체
음성출력
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Summary
Confidential – Oracle Internal/Restricted/Highly Restricted 44
• Machine Learning 의발달로점점완벽한음성인식가능
• AI Speaker 시장의확대
•다양한음성인식플랫폼의전성시대
• ChatBot 음성인식 Bot으로의전환
•음성인식시스템의 SDK의오픈개발자들의다양한기회
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Let’s go home to Code!!
Confidential – Oracle Internal/Restricted/Highly Restricted 45
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
Confidential – Oracle Internal/Restricted/Highly Restricted 46