Layered Implementation of Robotic Middlewares to the ... › t.kotoku › conf › iros2006ws › WS4-7.pdf · Oct. 10, 2006, Beijing, China International Conference on Intelligent

Oct. 10, 2006,Beijing, China

International Conference on Iintelligent Robots and Systems 1

Layered Implementation of Robotic Middlewaresto the Physical Agent System

IROS’06 Workshop WS4Robotics Standardization

Makoto MIZUKAWA, [email protected]

Shibaura Institute of Technology, TokyoChair ORiN Forum/JARA, Project Leader ISO TC184/SC2 RAPI

Contact of OMG Robotics-DTF to ISO TC184/SC2

Oct. 10, 2006, Beijing, China International Conference on Intelligent Robots and Systems2

Outline

Needs for standard EnvironmentPhysical Agent SystemLayered Implementation of Middlewares

ORiN for Robot CollaborationLwRTC for Embedded controller

Future work


Needs for standard Environment


Robotics in future

Human work

Industrial robots

Service robotsEmbedded/Ubiquitous robots

HouseholdWork assistDaily assistRescueMedical

HouseholdWork assistDaily assistRescueMedical

symbiosisEntertainment robots

Pet/ healingPet/ healing

New media Communication media

Manufacturing

Daily life support

Unmanned systems

Hazardous task substitute in fields Autonomy

High speed/precision

Human-Robot-Interaction


I’REX2005 DAEJEON


Related Activities (examples)FIPA:The Foundations for Intelligent

Physical Agents

FIPA:The Foundations for Intelligent Physical Agents

MPHPT: Ministry of Public Management, Home Affairs, Posts and Telecommunications, Japan

http://www.soumu.go.jp/english/index.html

MIC: Ministry of Information and Communication, KOREA

www.vcl.uh.edu/~rcv03/materials/slides/SangRok.ppt

OROCOSSoftwares for

Robot controller

OROCOSSoftwares for

Robot controller

ISO TC-184, JARARobots, Device profiling.

middleware for applicationhttp://www.orin.jp/

ISO TC-184, JARARobots, Device profiling.

middleware for applicationhttp://www.orin.jp/

OMGSDODAIS

OMGSDODAIS

for IT+RT servicesfor IT+RT services

http://www.fipa.org/

http://www.orocos.org/

for Mfgfor Mfgfor Telecomfor Telecom

for controllers/OSfor controllers/OS


Simple IdeaRobot is the object that has physical entitywhich interacts with the real worldCollaboration with others through networkingFrom service-applications to devicesTasks: Computers in the logical world

vs. Robots in the real worldSensing and processingInteract with the real worldHard real-time issues in controllersVarious structure and configurationVarious architecturePhysical & logical interoperability with other devices and robots


Issues

InteroperabilityConnectivityModularityReusability

RT-middleware for robot servicesRT-component-service middleware

for controllersRT-component standard interface

Domain specificDomain independentDomain specificDomain independent

Keys:Keys:


RT middleware-layers

Robot A Robot B

common API

application

Common controllerI/F

Robot profiles

controller

mediator

Vision Ctrl.

Body Ctrl. Arm Ctrl.

Sensor Ctrl.

controller

Dev.Profile Dev.Profile


motors arm

camera sensorsdevices

RTMW1 RTMW2

PART2 PART3

motion levelModel, data-framework

task levelModel, data-framework

service levelModel, data-framework

application

PART1

Internal NW


Physical Agent System


Motivation

Daily activity support and collaboration using remotely controlled robot

From control issues to service issues

•Operation based on information mainly through video cameras mounted on the remotely controlled robot•Gesture transmission to assist collaboration between remote sites (laser pointer/arm)

Master-slave systemMedia to support equivalent/close

relationship


PAS: Physical Agent SystemA remotely controlled autonomous robot that acts as an AGENT to support human collaboration over distant placesPAS as media for

Sharing spatial information between remote sitesTransmitting intension/attention of operator via motion

direction/orderdirection/order

spatial/field info.spatial/field info.


PCVoice control

Joystick

Sensory Glove

Input devicesInput devicesInput devices

WEB Simulator

Info. Sharing PadAgent robotAgent robotAgent robot

The Internet

Activity support Activity support in hazardous in hazardous environmentsenvironments

Daily life Daily life supportsupport

Physical Agent SystemPPhysical AAgent SSystemRemote

siteRemote

site

PC •Distributed control•System description•Image processing

•Distributed control•System description•Image processing

・mobility・obstacle avoidance・posture stabilization

camera, LP

・mobility・obstacle avoidance・posture stabilization

camera, LP

autonomyautonomy

Agent with physical

interaction

Agent with Agent with physical physical

interactioninteractioncamera

arm

Laser pointer


Applications of PAS

Disaster/RescueDaily life support

Elderly/HandicappedCommunication within Family in distanceHome-electronics・personal robots

Distributed/Embedded PAS in EnvironmentsReflects human intensionDevice profiling using XML to allow inter-operability and flexible configuration


5DOF Arm

GimbalMechanism

CCD Camera

Laser Pointer

Crawler

Prototype of the PAR04R


PAS System Configuration< device >Keyboard

< device >Display

< device >Sensory Glove

< device >Joystick

< device >Mouse

< processor >< on board>･input deviceClient-Server

(windows2000)Client-Server

(windows2000)

< management >･ Simulator system･ Input control･ Display the interface to an operator

< management >･ Preservation of task and re-execution command･ Communication with Client-Server(TCP/IP)･ Communication with Data-Server(TCP/IP)･ Communication control of CAN bus･ Process of camera image

「TCP/IP」

< device >H8S for Camera/LP control

Real-time OSReal-time OS

< on board>･ Camera･ LP･ H8S

< management >･Camera/LP control･Get Angle’s value･Command in another device

< device >H8S for Crawler control


< on board>･ Machine ･ PSD･ Ultrasonic Sensor･ H8S

< management >･Machine control･Get Sensor’s value･Command in another device

< management >･Machine control･Get Sensor’s value･Command in another device

< device >H8S for Arm control


< on board>･ Arm･ H8S

< management >･Arm control･Command in another device

< device >H8S for Power management


< on board>･ H8S

< management >･Task control･Get Current’s value･Store Current/Voltage logs･Command in another device

「CAN」

< processor >

Agent-Server(windows2000)Agent-Server

(windows2000)

< on board>･wireless LAN･PCI･IP7000

Agent Robot Operation

CAN(Controller Area Network)


Physical Agent Robot (PAS)PAR04R

Oct. 10, 2006,Beijing, China

International Conference on Iintelligent Robots and Systems 18

Aichi Expo 2005Robot week, June 9-19


Collaboration using RT middleware-ORiNComponent based on RT Middleware from AISTApriAlpha using ORCA

Robot Week (Jun 9-19)Aichi Expo

Intelligent Wheel Chair, “Komawari-kun”the helper robotUEC

Echono-vehicleMeijo Univ

PAR04RShibaura Inst. Tech.

Apri-Alpha & Apri-AttendaToshiba & SUT


Layered Implementation of Middlewares

ORiN for Robot CollaborationLwRTC for Embedded controller



Robot A Robot B

common API

application


Robot profiles

controller

mediator

Vision Ctrl.


Sensor Ctrl.

controller



motors arm


RTMW1 RTMW2

PART2 PART3




application

PART1

Internal NW


Robot status management

Robot profiles XML(CRD)

cameras

operator

UEC robots Meijo Univ robots PAR04R

robot control & status acquisition using RAC

robot control & management system

Toshiba robot

Voice operation

robot status storage & display system

monitor and refer robot status and location

projectorvisitors


System integration using ORiNcontroldata

operator

operator

GUI operation

Voice operation

RAC requests

CRD provider RAC provider

Robot profilesXML(CRD)

Robots

Robot control applicationVB.NET

robot control & management system

CAO Engine

command table


Application

ORiN

Robot Controller

ORiN: Open Resource/Robot Interface for the Network

Provider


ORiN structure

App.Z

CAO EngineApplication IF

Device IF

A Co. B Co. CRD

CRD File(XML)

ProprietaryProtocols

Other Standards(ex. OPC Server)

Non-ORiN Device

App.X

App.Y

Non-CAO Application(ex. OPC Client, UPnP Control Point)

CAP

Internet

Application

Engine

Provider

Device

SOAP

DDEUPnPOPC

Dev.B

Abstract Device

http://mfile.akamai.com/2478/asf/ihc.download.akamai.com/2478/upnp/core_tools_upnp_2.asx


Controller Access Object, CAOA middleware providing a standard program interface for the applications

and the devices.

It’s based on the distributed object technology, DCOM or CORBA.

Key Technologies of ORiN（Open Robot Interface Model）

Controller Access Protocol, CAPA standard communication protocol for the Internet to allow data-exchange

over firewall.

It’s based on the SOAP technology.

Controller Resource Definition language, CRDA standard data schema for representing the data in a device.

It’s based on the eXtensible Markup Language, XML-Schema.


The Robot Information Framework The Robot Information Framework and Application Program Interfaceand Application Program Interface

RAPIRAPI

ISO TC184/SC2 New Work Item


to provide standard methods for accessing robot/device system information when developing a multi-vendor Mfg. system

Objective of RAPI

•a unified access method•a common application platform•a common API•a unified robot/device profile description•networking capability using distributed object


Relationship between ORiN and RAPI

ORiN: PC technology of the states of art

•Implementation depending on Microsoft Technologybut

• Available and Ready-to-apply technology

ORiN: PC technology of the states of artPC technology of the states of art

•Implementation depending on Microsoft Technologybut

• Available and Ready-to-apply technology

RAPI:The Robot Information Framework and Application Program InterfaceIndependent from software products from the specific vendors• Abstraction of ORiN• Subset of ORiN in functionality

Limit to functions the least requirementExclude RAP

RAPI:The Robot Information Framework and Application Program InterfaceIndependent from software products from the specific Independent from software products from the specific vendorsvendors• Abstraction of ORiN• Subset of ORiN in functionality

Limit to functions the least requirementExclude RAP


Merits of RAPI

RAPI StandardsCommon application platform and APIIndependent from the Applications and the Robots

RAPI StandardsRAPI StandardsCommon application platform and APIIndependent from the Applications and the Robots

Expand the Robot Application Field• Reduce development cost for

Robot communication Interface. • New robot market for various

types of application is generated.

Robot Makers

Common Packages for Industries• Cost reduction by standardizing

software package.• New robot software market is

generated.

Software Vendors

Easy customizing of FA services• Reduce development cost by

using a common platform.• Independent from the robot

makers.

System Integrators

Cost effective operation• Multi-vendor Mfg. system to meet

user needs.• Mfg. Monitoring and Remote

Maintenance is easier.

End Users


ISO TC184/SC2 Schedule3rd Q 4th Q 1st Q 2nd Q 3rd Q 4th Q 1st Q 2nd Q 3rd Q 4th Q 1st Q 2nd Q

2006 200920082007

Creating the 1st draft Updating Finalizing6/15 ISO meeting in Paris Commit the NWIP

RAPI draft by the ORiN forum

Start the discussion by the PT

RAPI 1st revision

Objective: To create a standard framework for Robot communications.

Term of Activity: June 2006 to June 2009, 3 Years

Project Leader: Dr. Makoto MIZUKAWA, Shibaura Institute of Technology,

Chair of ORiN Forum.

Project Sub-leader: Satoshi SAKAKIBARA, DENSO WAVE INC.

Project Secretariat: Japan Robot Association


ORiN Forum

ORiN forumThe Japan Robot Association

Open Resource Interface for the Network /Open Robot Interface for the Network

http://www.orin.jp/


LwRTC for Embedded controller



Robot A Robot B

common API

application


Robot profiles

controller

mediator

Vision Ctrl.


Sensor Ctrl.

controller



motors arm


RTMW1 RTMW2

PART2 PART3




application

PART1

Internal NW


Adapting CAN

CAN device

MPU (enough to work CORBA)

OSProxyRTC

ProxyRTC

ProxyRTC

ProxyRTC

CAN

CAN-RTC

CAN device

CAN-RTC

CAN device

CAN-RTC

CAN device

CAN-RTC

Ethernet(CORBA)

RTC-CAN Gateway

RTC-CAN Proxy software

Ethernet

Implementation of LwRTC Supporting Function

Intercommunication withstandard RTCDirect communicationbetween CAN devices

RTC-CAN System OverviewConstruction of Module based Robot with Embedded MPU,CAN, and RT-Middleware RTC-CAN System

RTC-CAN


Proxy-RTCProxy-RTC

InPort RTC

Proxy-RTCProxy-RTC

OutPortRTC

CAN-RTC ID:0000CAN-RTC ID:0000

InPortID:0001

RTCID:0000

0000-00010001-0001

Subscribing Table

CAN-RTC ID:0001CAN-RTC ID:0001

OutPortID:0001

RTCID:0000

Proxy CANData CommunicationID: 001-0000-0001

CAN ProxyCommand Send

ID: 010-0001-1111

Proxy CANCommand Send

ID: 010-1111-0000

CAN ProxyData Reply

ID: 011-0000-1111

Proxy CANCommand Reply

ID: 011-1111-0001

CAN CANCommand Send

ID: 010-0000-0001

CAN CANCommand Reply

ID: 011-0001-0000

CAN CANData CommunicationID: 001-0001-0001

CAN ProxyData CommunicationID: 001-0000-0001

CAN message identifier（11bit）at RTC-CAN system


Experiment of RTC-CANSystem (Crawler Control)


OS

CAN

Crawler device

CrawlerCAN-RTC

Dummy device

DummyCAN-RTC

RTC-CAN Gateway


Ethernet

CrawlerProxy-RTC

CrawlerProxy-RTC

DummyProxy-RTC

DummyProxy-RTC

Client PC

Confirming the operation of CAN-RTCConfirming the operation of Crawler

SH4

H8s

H8s


Experiment of RTC-CANSystem (Crawler Control)

The operation of CAN-RTCWorking crawler-subsystem as RTC

The operation of CrawlerConfirming Gateway FunctionDirect communication between CAN devices


OS

CAN

Crawler device

CrawlerCAN-RTC

Dummy device

DummyCAN-RTC

RTC-CAN Gateway


Ethernet

CrawlerProxy-RTC

CrawlerProxy-RTC

DummyProxy-RTC

DummyProxy-RTC

Client PC SH4

H8s

H8s


Results

Communication Time Dilley20msec (one way)It’s too late to control robot.It is a cause that the processing of Gateway MPU reached the limit.

Specification

MPU H8S/2638F（HD64F2638WF20)ROM：256KB RAM：16KB

clock 20MHz

SRAM512KB（256K×16bit ）R1RW0416DSB-2LR（Renesas

Technology）

CAN device


OSProxyRTC

ProxyRTC

ProxyRTC

ProxyRTC

CAN

CAN-RTC

CAN device

CAN-RTC

CAN device

CAN-RTC

CAN device

CAN-RTC

Ethernet(CORBA)

RTC-CAN Gateway


Ethernet


Future work:Middleware services for RT

tag

internal servicesembedded servicesDistributed processes, objects, processors , devices, components, controllersControl software, wiring, PnP

ORB servicesrelations, functions, processinginterfaces, profiles repository, naming services

Ubiquitous servicesSensor networkRFID tags, data mining

softwaredistributed objects, encapsulation

servicesservice, task, motion

Structured and intelligentenvironmentPhysical and informational consistency

middlewaremodels, PIM, PSMdescription, languages, system tools

ORiNCAMUS…


summary

Propose of PAS for daily-life servicesDemonstration in Aichi Expo.3-layer model for the RT middleware design and implementation for seamless RT system integrationCoordination with environment with embedded RT components/devices MDA for Robotics

Documents

Layered Implementation of Robotic Middlewares to the ... › t.kotoku › conf › iros2006ws › WS4-7.pdf · Oct. 10, 2006, Beijing, China International Conference on Intelligent