Research Activities of AIRC on AI · 2017-09-08 · AI which cooperates with Human Established at 1st May, 2015 as a research center of AIST. Artificial Intelligence Research Centre

Research Activities of AIRC on AI -with emphasis on manufacturing-

DirectorAI Research Centre(AIRC)

AISTJapan

1

Junichi Tsujii

Plan of TalkBackground Research Topics

– Robots in Manufacturing– Self-Navigating Robots– Robot Scientist– Geo-Spatial Information– Health Care/Innovative Retailing– Deep Understanding

Infra-structureOutreach and Conclusions

2




3

4

AI Embedded in the Real World

AI which cooperates with Human

Established at 1st May, 2015 as a research center of AIST

Artificial Intelligence Research Centre

Established on 1/May, 2015

Industry

UniversitiesRiken, NICT,Institute ofStatisticalMathematics

BioinformaticsTranslational BiologyCancer ResearchPharmaceutics

6

Core Center of AI for Industry-Academia Co-operation

産総研人工知能研究センター

Application Domains

NLP, NLU Text mining Mining & Modeling

Behavior Mining & Modeling

ManufacturingIndustrial robots

Automobile

Innovative Retailing

Health CareElderly Care

Deployment of AI in real businesses and society

Data-Knowledge integration AIBrain Inspired AI

OntologyKnowledge

Model ofHippocampus

Model ofBasal ganglia

Logic & ProbabilisticModeling

Bayesian net ･･･

･･･

sSecurity

Network ServicesCommunication

Big SciencesBio-Medical Sciences

Material Sciences

Model ofCerebral cortex

Technology transferStarting Enterprises

Start-UpsInstitutionsCompanies

Technology transferJoint research Common AI Platform

Common ModulesCommon Data/Models

PlanningControl

PredictionRecommend 3D Object recognition

Image Recognition3D Object recognition

Planning/Business Team

･･･

Effective Cycles among Research and Deployment of AI

Standard TasksStandard Data

AI Research Framework

NEEDS

SEEDS

Planning/Business TeamDataPlatformsModulesModels

Simulation Machine Learning

8

Domain Knowledge

Recognition Modeling Planning

AI embedded in the real world – AI in contexts

Sensing Action

Inference

Data Acquisition Recognition

Action Planning& Execution

Real World Real WorldApplication

DomainApplication

Domain

Context Context

AI x IoT AI x Robotics

Computation Infra-structure -- ABCI


– Robots in Manufacturing– Self-Navigating Robots– Robot Scientist– Geo-Spatial Information– Human Life/Innovative Retailing– Deep Understanding


9

Plan of Talk

Background Research Topics

– Robots in Manufacturing– Self-Navigating Robots– Robot Scientist– Geo-Spatial Information– Health Care/Innovative Retailing


10

国立研究開発法人

AI for Manufacturing

Recognition Engine（Deep Learning）

Database of objects with picking

positions

Database of Action Sequences

Cloud Database Platform

Hardware Platform Software Platform

Mitsubishi Kawada

Choreonoid(AIST)

Picking Simulator(Mitsubishi)

Operation SequencesAcquisition of operation sequencesMimic learning

Reduction of cost in programming

Picking tasksSimulationmachine Learning

Improving picking actionsAI Platform (AIST)

Gathering of Human Operation Operation of Flexible Objects(Clothes, Scarfs, etc.)

R. Nakajo, S. Murata, H. Arie, and T. Ogata, ICDL-EpiRob 2015.

12

Imitation Learning

→to Reduce the cost of teaching robots

13

ＨａｎｄｌｉｎｇＦｌｅｘｉｂｌｅＯｂｅｊｃｔｓ@Cebit(2017.3)共同研究先

国立研究開発法人

AI for Manufacturing

Recognition Engine（Deep Learning）

Database of objects with picking

positions

Database of Action Sequences

Cloud Database Platform

Hardware Platform Software Platform

Mitsubishi Kawada

Choreonoid(AIST)

Picking Simulator(Mitsubishi)

Operation SequencesAcquisition of operation sequencesMimic learning

Reduction of cost in programming

Picking tasksSimulationmachine Learning

Improving picking actionsAI Platform (AIST)

Gathering of Human Operation Operation of Flexible Objects(Clothes, Scarfs, etc.)

Chubu Univ., Chukyo Univ., Tokyo Univ., AIST-CVRG

Kyushu Univ., AIST-MRG

Kyushu・Osaka・AIST-MRGShinshu・NAIST

Osaka・AIST-MRG

Waseda Univ.

Osaka・AIST-MRG(Mitsubishi Electric Co.）




15

Ogatsuma and IchiseKIT, NII, Waseda

強みと弱み

実用の見通し

デー

タ量

また

は複

雑度

人工ポテンシャル法（軌道生成・障害物回避）

ヒト由来情報処理，脳型コンピュータ（LSI化）

(基礎研究) (実用研究)

(甚大

) (中

程度

)

Deep Learning, 機械学習など（ベイズ統計則等）

データ量が多い程信頼性が高い

即時応答性は低い (オフライン学習)

人と同

等の

処理能

力

を期待

移動ロボットとの共通基盤化可能

データ量が多い程信頼性が高い

情報の複雑化 → 演算負荷大

ITSオントロジー（状況分析・知識DB処理）

未知の状況

注意

変化

・動作

など

生体計

測は

ある

程度

可能

状況の想定をフレームワーク化

即時性・オンラインセンシングとの整合性は課題

共通課題

相補

Region Observable by laser

Blind spots

Hard to observe

Exceptinal behaviour

16

Objective：Self-Driving with context understanding, Risk avoidance with prediction

Method：Hybrid system of data-driven AI and knowledge-driven AI

Implementation：Close cooperation on field tests with auto-makers・Use of diverse sensors, efficient hardware specialized for self-driving, real-time response (ADAS10ms-500ms)・Use of Ontology to measure the complexity index of context

Data-driven AI and Knowledge-driven AI which can explain, Efficient inferences for being feasible in the practical application

Knowledge-Driven AI

Data-Driven AI Brain-inspired Computation,Hardware Implementation(LSI)

Strength and Weakness

Deep LearningMachine LearningStatistical Model

ITS OntologyKnowledge bases

Potential Field MethodFor navigation

Demonstration at a Business Expo

17

Intelligent agent which cohabits and cooperates with human

18

Interface＆

Interaction

Model Construction

Human Model （faces, bodies, movements, location）

Model of Movements（map, paths, changes）

Model of Environment（shapes, locations, changes）

Recognition

Planning

Control

Self-Driving

環境EnvironmentContext

Self-Navigating Robot (Elderly Care) and Self-Driving

19

StimulationStimulation

ResponseResponse Physicalfactors

Physicalfactors

Decisionmodel ?Decisionmodel ?

Robust RoboticsFlexible Adaptation to

Environment

Eviornment

Navigation inHuman

Eviornment

Moving

Information Gathering while

Moving

・Construction of Dynamic 3D Maps・Real-time sensing of the self-posture and self-location・Map of human positions and Path-planning・Classification and Recognition by Nested iGMM

!

事故Planning

ControlRecognition

Models ofHuman &

Movements

Self-Navigation

Real-time 6D oF Robot Pose Localization 1st floor of Miraikan, Crowded Buisiness Expo Map size: 135 x 100 x 25 [m3]

Human Tracking using Laser Range Sensor

21

Object Recognition by Multiple Views

22

Robot Platform Deep Learning Robot

Conventional Methods

Better recognition rates with less training images

Our method

Images for training

Rec

ogni

tion

Rat

e

★12 categories, 132 objects, trained by 21,120 images of objects in everyday life

80

90%⇒92%、 80 images⇒less than 20

Competition of 3D object identification atStanford （SHREC2017） The best Performance




23

1,2-Diacyglycerol intracellular

AKT(PKB)

ALK

Androgen receptor

B-Raf

BETA-PIX

C/EBPbeta

C3G

CDC42

CDK2

CREB1

Ca('2+) cytosol

Cyclic AMP intrac

Cyclic GMP intrace

EGR1

ERK1/2

ESR1 (nuclear)

Elk-1

FMO3

FRS2

GAB1

GRB2

Galectin-1

H-Ras

HDBP1

HGF receptor (Met)

HIF1A

HSP27

IRS-1

IRS-2

JNK(MAPK8-10)

K-RAS

Lyn

MAP2

MEK1/2

MEK4(MAP2K4)

MEK6(MAP2K6)

MEKK1(MAP3K1)

MEKK4(MAP3K4)MLK3(MAP3K11)

N-Ras

NCK2 (Grb4)

NO intracellularNeurofibromin

PAK1

PDGF receptor

PDLIM3

PDZ-GEF1

PI3K cat class IA

PIP5KI

PKC

PR (nuclear)

Protein kinase G1

Pyk2(FAK2)

R-Ras

RASGRF2

RIPK4

Rac1

SHP-2

SOS

SP1

Shc

Slc39a14 (Zip14)

Tiam1

VEGFR-1

a-6/beta-4 integrin

c-Fos

c-Jun

c-Kit

c-Myc

c-Raf-1

cPLA2

p90Rsk

Big Mechanism: Reading-Assembly-Explanation

Reading Assembly Explanation


AKT(PKB)

ALK

Androgen receptor

B-Raf

BETA-PIX

C/EBPbeta

C3G

CDC42

CDK2

CREB1

Ca('2+) cytosol

Cyclic AMP intrac

Cyclic GMP intrace

EGR1

ERK1/2

ESR1 (nuclear)

Elk-1

FMO3

FRS2

GAB1

GRB2

Galectin-1

H-Ras

HDBP1

HGF receptor (Met)

HIF1A

HSP27

IRS-1

IRS-2

JNK(MAPK8-10)

K-RAS

Lyn

MAP2

MEK1/2

MEK4(MAP2K4)

MEK6(MAP2K6)

MEKK1(MAP3K1)


N-Ras

NCK2 (Grb4)


PAK1

PDGF receptor

PDLIM3

PDZ-GEF1

PI3K cat class IA

PIP5KI

PKC

PR (nuclear)

Protein kinase G1

Pyk2(FAK2)

R-Ras

RASGRF2

RIPK4

Rac1

SHP-2

SLC36A1

SOS

SP1

Shc

Slc39a14 (Zip14)

Tiam1

VEGFR-1

a-6/beta-4 integrin

c-Fos

c-Jun

c-Kit

c-Myc

c-Raf-1

cPLA2

p90Rsk


AKT(PKB)

ALK

Androgen receptor

B-Raf

BETA-PIX

C/EBPbeta

C3G

CDC42

CDK2

CREB1

Ca('2+) cytosol

Cyclic AMP intrac

Cyclic GMP intrace

EGR1

ERK1/2

ESR1 (nuclear)

Elk-1

FMO3

FRS2

GAB1

GRB2

Galectin-1

H-Ras

HDBP1

HGF receptor (Met)

HIF1A

HSP27

IRS-1

IRS-2

JNK(MAPK8-10)

K-RAS

Lyn

MAP2

MEK1/2

MEK4(MAP2K4)

MEK6(MAP2K6)

MEKK1(MAP3K1)


N-Ras

NCK2 (Grb4)


PAK1

PDGF receptor

PDLIM3

PDZ-GEF1

PI3K cat class IA

PIP5KI

PKC

PR (nuclear)

Protein kinase G1

Pyk2(FAK2)

R-Ras

RASGRF2

RIPK4

Rac1

SHP-2

SLC36A1

SOS

SP1

Shc

Slc39a14 (Zip14)

Tiam1

VEGFR-1

a-6/beta-4 integrin

c-Fos

c-Jun

c-Kit

c-Myc

c-Raf-1

cPLA2

p90Rsk

Very large conflicting(probabilistic) network

Smaller(relevant)groundedmodel

Computationalhypotheses/wet labExperimentscontrolling states of thenetwork

A.Rzhetsky(U.Chicago)

Open Science-The Need for Text Mining

Types of documents• Full papers• Abstracts• Reports, discharge summaries• EMR• Textbooks, monographs• Grey content, online

discussion forums

MEDLINE• 2005: ~14M• 2009: ~18M• 2013: ~22M• 2015: ~26M

25

Overwhelming information in textual, unstructured format

S.Ananiadou (U.Manchester)

Finding Evidence -EuropePubMed Central

• Currently: runs on 2,550, 328 full texts• 82,198,474 facts in 38,411,661 sentences• Full parsing used a version of Enju (Mogura) • Parsing pipeline run on 60 machines at EBI ~30

days

26

http://labs.europepmc.org/evf

By S. Ananiadou(U. Manchester)

Deep reading

custom componentsexisting components supplied with custom resources

existing components

By R. Batista(U. Manchester)

Hydroxide ion may make a nucleophilic attack on the phosphate

Agent Theme

NucleophicAttackEvent

Probable

SpeculationCue

FunctionalGroupFunctionalGroup NucleophicAttackTrigger

Neural Net

Embedding

Output

Prediction

Deep Structure Learning

Event Structure


AKT(PKB)

ALK

Androgen receptor

B-Raf

BETA-PIX

C/EBPbeta

C3G

CDC42

CDK2

CREB1

Ca('2+) cytosol

Cyclic AMP intrac

Cyclic GMP intrace

EGR1

ERK1/2

ESR1 (nuclear)

Elk-1

FMO3

FRS2

GAB1

GRB2

Galectin-1

H-Ras

HDBP1

HGF receptor (Met)

HIF1A

HSP27

IRS-1

IRS-2

JNK(MAPK8-10)

K-RAS

Lyn

MAP2

MEK1/2

MEK4(MAP2K4)

MEK6(MAP2K6)

MEKK1(MAP3K1)


N-Ras

NCK2 (Grb4)


PAK1

PDGF receptor

PDLIM3

PDZ-GEF1

PI3K cat class IA

PIP5KI

PKC

PR (nuclear)

Protein kinase G1

Pyk2(FAK2)

R-Ras

RASGRF2

RIPK4

Rac1

SHP-2

SOS

SP1

Shc

Slc39a14 (Zip14)

Tiam1

VEGFR-1

a-6/beta-4 integrin

c-Fos

c-Jun

c-Kit

c-Myc

c-Raf-1

cPLA2

p90Rsk

Big Mechanism: Reading-Assembly-Explanation

Reading Assembly Explanation


AKT(PKB)

ALK

Androgen receptor

B-Raf

BETA-PIX

C/EBPbeta

C3G

CDC42

CDK2

CREB1

Ca('2+) cytosol

Cyclic AMP intrac

Cyclic GMP intrace

EGR1

ERK1/2

ESR1 (nuclear)

Elk-1

FMO3

FRS2

GAB1

GRB2

Galectin-1

H-Ras

HDBP1

HGF receptor (Met)

HIF1A

HSP27

IRS-1

IRS-2

JNK(MAPK8-10)

K-RAS

Lyn

MAP2

MEK1/2

MEK4(MAP2K4)

MEK6(MAP2K6)

MEKK1(MAP3K1)


N-Ras

NCK2 (Grb4)


PAK1

PDGF receptor

PDLIM3

PDZ-GEF1

PI3K cat class IA

PIP5KI

PKC

PR (nuclear)

Protein kinase G1

Pyk2(FAK2)

R-Ras

RASGRF2

RIPK4

Rac1

SHP-2

SLC36A1

SOS

SP1

Shc

Slc39a14 (Zip14)

Tiam1

VEGFR-1

a-6/beta-4 integrin

c-Fos

c-Jun

c-Kit

c-Myc

c-Raf-1

cPLA2

p90Rsk


AKT(PKB)

ALK

Androgen receptor

B-Raf

BETA-PIX

C/EBPbeta

C3G

CDC42

CDK2

CREB1

Ca('2+) cytosol

Cyclic AMP intrac

Cyclic GMP intrace

EGR1

ERK1/2

ESR1 (nuclear)

Elk-1

FMO3

FRS2

GAB1

GRB2

Galectin-1

H-Ras

HDBP1

HGF receptor (Met)

HIF1A

HSP27

IRS-1

IRS-2

JNK(MAPK8-10)

K-RAS

Lyn

MAP2

MEK1/2

MEK4(MAP2K4)

MEK6(MAP2K6)

MEKK1(MAP3K1)


N-Ras

NCK2 (Grb4)


PAK1

PDGF receptor

PDLIM3

PDZ-GEF1

PI3K cat class IA

PIP5KI

PKC

PR (nuclear)

Protein kinase G1

Pyk2(FAK2)

R-Ras

RASGRF2

RIPK4

Rac1

SHP-2

SLC36A1

SOS

SP1

Shc

Slc39a14 (Zip14)

Tiam1

VEGFR-1

a-6/beta-4 integrin

c-Fos

c-Jun

c-Kit

c-Myc

c-Raf-1

cPLA2

p90Rsk

Very large conflicting(probabilistic) network

Smaller(relevant)groundedmodel

Computationalhypotheses/wet labExperimentscontrolling states of thenetwork

A.Rzhetsky(U.Chicago)

MaholoＲＢＩ, Start-Up by Researchers of AIST

30




31

Archives of Satellite Images• ASTER@AIST ~ 1PB

– Images of all regions in the world from 1999

• Landsat-8@Amazon S3 ~ 500 TB– Images fro 20132013年

– Open data

ASTER on NASA’ TERRA OLI on Landsat-8

Data(Imagery)

Knowledge(Map)

Mapping from Images to Maps

Source oftraining data

Daily update

Landsat-8 Open Street Map

By sliding the bar, we can jump to the past

Images by Landsat 8http://landbrowser.geogrid.org/landbrowser/index.html

Comparison of the image with the past image of the same region

Red Circles - Significant changes

深層学習による茨城〜千葉のメガソーラー抽出

36

Database constructed by human specialist Classification by DNN

若林地区，七北田川河口（E140.99552 N38.25715）

流出

非流出

撮像日：2010/8/4 撮像日：2011/3/14




38

Integration of Images, Patient Reports, Medical Databaseshistopathologic

examination

Mammary GlandsEcho Colonoscopy

Patient DB

RecognitionRetrieval Engine

*Copyright 2001 - 2012 Given ImagingLtd. All Rights Reserved.

Endoscopy

PerformanceImprovement

Used bydoctors

Data Accumulation

Precision upSatisfaction upContribution up

40

Ultrasonic inspection

Endoscopy inspection

Diagnosis support for medical images

Capsule endoscopy inspection

*Copyright 2001 - 2012 Given Imaging

Ltd. All Rights Reserved.

Pathology examination Support System with

ImageRecognition

to reduce the burdens on doctors and improve the quality and the efficiency of diagnosis

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 63 65 67 69 71 731 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 63 65 67 69 71 731 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 63 65 67 69 71 73

Performance of the Cancer Detection System

Abnorm

al

Norm

al

Syste

m O

utp

uts

100 % of RecallFalse negative → 0 (0/24)False Positive → 4% (2/50)

■Normal Images■Cancer

Cancer

正常

NormalNormal

Cancer

41ID of Images

Detection results

42

Red rectangle : suspected area of abnormal tissue

AI for Human LifeLiving Intelligence Complex

AIST Living Lab.Living Lab at 8F AIST Annex. July 2016

44The mock one is used for fundamental test before field test at the satellite ones.

AI application system for the real service

Fellica cards

RF-IDlist bands

RF-ID members cards

In store digital signage systems

27inchTouch panelCore i7 Windows PC

CameraKinect

Rf-ID

printer

30 x 20 vending machine




46

Recognition of sequences of actions with fine-grained object detectionSignificant error reduction by sequence recognition

Video Captioning

Baseline method： A man is drinking.Proposed method：A girl is doing makeup.

Baseline Method： A dog is playing with a dog.Proposed Method：A boy is playing with a dog.

Baseline Method ： A man is riding a car.Proposed Method：A woman is riding a boat.

Baseline Method： A man is riding a bicycle.Proposed Method ：A man is riding a bike.

47

9/8/2017言語処理学会第23回年次大会 48

(1) 09:00 日経平均、続落で始まる

(2) 09:29 日経平均、上げに転じる

(3) 11:30 日経平均、続落前引けは５円安の19386円

(4) 12:30 日経平均、午後は上昇で始まる

(5) 13:54 日経平均、上げ幅100円超える

(6) 15:00 日経平均、反発大引けは102円高の19494円

Reports by humanTime Sequence of Nikkei Average

Characteristics of Market Reports

Stock market report generation

Text Generation

1. Changes in short term and long term affects choices of expressions

2. Context (Time of the day) affects choices of expressions

3. References to actual figures

Reference to the historical context

Reference to the context of utterances

Reference to numeriaclvalues

9/8/2017 言語処理学会第23回年次大会 49

<s>

(2) Context of Utternaces

(3)Numerical values

日経平均、上げ幅<price2>円超える </s>

12167.29

12278.83

…

12451.66

12461.36

Movement of a day

12116.57

12120.94

…

12145.7

12150.49

End-of-a-day Share prices of 7 days= ( , , … , )

concatenation

Pre

-pr

ocess

ing

Processing of numbers

(1) History of share movements

encoder= ( , , … , )

encoder

Pre

-pr

ocess

ing

ℎ

ℎ• encoder

• MLP• CNN• RNN

Input日経平均、上げ幅300円超える

Output

RNNLMShort-term

Long-term

Integration of Information, Context of UtterancesReference to

the history Time of

Utterance

Reference to Number




50

Research Prototype, Internal useBy NEDO project

2016.6-

NEDO Project

Nairobi（Tsukuba）

FY2015 Supplementary Budget

AAIC (Tsukuba)

FY2016 Supplementary BudgetGlobal Research Hub for AI研究ABCI（Kashiwa）

2017.4-2018.3

A System being used for Joint Projects with Industry and Other Research Institues

Open Innovation Hub used by Funded by Private and PublicSectors

DL

HPC

0.5 PFlops 8.6 PFlops >130 PFlops

2.1 PFlops >12 PFlops0.2 PFlops

16 times 15 times

10times

Storage 4.5 PiB >40 PiB23 TiB200 times 10times

Computing Environment

※半精度演算のピーク性能

51

6 times

Computation Infrastructures

AIST

Domestic Partners

Partners Abroad

AI Research Platform（NEDO）AI Platform（Supplementary Budget）

＠Tsukuba

Commercial Clouds

AI Research Center＠Rinkai

SINET5JGN-X

ResearchData Alliance

AIRC Network

PartnersPartners

Owners ofData

Linkages with commercial cloud services through VPN and Direct Connect

High-Speed Data Transmission (100Gbps) and TV conference facilities through dark fibres

Stable and secure linkagesWith owners of data and research partners abroad

52




53

NEC-AIST Joint Research Lab. on AI OrganizationLaboratory Leader: Takashi Washio (Professor, Osaka University）

Members：Researchers ３０（AIST、ＮＥＣ、Universities）Yoshinobu Tsuruoka （University of Tokyo）

Location: AIRC＠Waterfront, Tokyo Period：３ years

Research Topics➀ Integration of ML and Simulation

② Integration of ML, Logic and NLU

➂ Cooperation among Multi-Agents

Industrial Linkage

Conclusions and Comments

• Concerted Efforts, Beyond individual researches

• Public vs. Private, Medical care, Elderly care, etc.• Integration of Computation-Intensive and Data-

Intensive Architectures • Ownership of Data, Software, and Learned ML models• Incentives for providing data and software

55

Documents

Research Activities of AIRC on AI · 2017-09-08 · AI which cooperates with Human Established at 1st May, 2015 as a research center of AIST. Artificial Intelligence Research Centre