27
GlobalMMCS DS-RT 2005 Tutorial IEEE DS-RT 2005 Montreal Canada Oct. 9 2005 Geoffrey Fox CTO Anabas Corporation and Computer Science, Informatics, Physics Pervasive Technology Laboratories Indiana University Bloomington IN 47401 [email protected] http:// www.infomall.org

GlobalMMCS DS-RT 2005 Tutorial IEEE DS-RT 2005 Montreal Canada Oct. 9 2005 Geoffrey Fox CTO Anabas Corporation and Computer Science, Informatics, Physics

Embed Size (px)

Citation preview

Page 1: GlobalMMCS DS-RT 2005 Tutorial IEEE DS-RT 2005 Montreal Canada Oct. 9 2005 Geoffrey Fox CTO Anabas Corporation and Computer Science, Informatics, Physics

GlobalMMCS DS-RT 2005 Tutorial

IEEE DS-RT 2005 Montreal Canada Oct. 9 2005

Geoffrey Fox

CTO Anabas Corporation andComputer Science, Informatics, Physics

Pervasive Technology LaboratoriesIndiana University Bloomington IN 47401

[email protected]://www.infomall.org

Page 2: GlobalMMCS DS-RT 2005 Tutorial IEEE DS-RT 2005 Montreal Canada Oct. 9 2005 Geoffrey Fox CTO Anabas Corporation and Computer Science, Informatics, Physics

Collaboration and Web Services Collaboration has

a) Mechanism to set up members (people, devices) of a “collaborative sessions”

b) Shared generic tools such as text chat, white boards, audio-video conferencing

c) Shared applications such as Web Pages, PowerPoint, Visualization, maps, (medical) instruments ….

b) and c) are “just shared objects” where objects could be Web Services but rarely are at moment

• We can port objects to Web Services and build a general approach for making Web services collaborative

a) is a “Service” which is set up in many different ways (H323 SIP JXTA are standards supported by multiple implementations) – we should make it a WS

Page 3: GlobalMMCS DS-RT 2005 Tutorial IEEE DS-RT 2005 Montreal Canada Oct. 9 2005 Geoffrey Fox CTO Anabas Corporation and Computer Science, Informatics, Physics

Shared Event Collaboration All collaboration is about sharing events defining state changes

• Audio/Video conferencing shares events specifying in compressed form audio or video

• Shared display shares events corresponding to change in pixels of a frame buffer

• Instant Messengers share updates to text message streams

• Microsoft events for shared PowerPoint (file replicated between clients) as in Access Grid

Finite State Change NOT Finite State Machine architecture Using Web services allows one to expose update events of all

kinds as message streams Need publish/subscribe approach to share messages (NB) plus System to control “session” – who is collaborating and rules

• XGSP is XML protocol for controlling collaboration building on H323 and SIP

Page 4: GlobalMMCS DS-RT 2005 Tutorial IEEE DS-RT 2005 Montreal Canada Oct. 9 2005 Geoffrey Fox CTO Anabas Corporation and Computer Science, Informatics, Physics

WSDisplay

WSViewer

WS Display

WS ViewerEvent

(Message)Service

Master

WSDisplay

WS Viewer

Collaboration as a WSSet up Session with XGSP

WebServic

e

F

I

U

O

F

I

R

O

Shared Input Port (Replicated WS) Collaboration

OtherParticipants

WebServic

e

F

I

U

O

F

I

R

O

WebServic

e

F

I

U

O

F

I

R

O

Page 5: GlobalMMCS DS-RT 2005 Tutorial IEEE DS-RT 2005 Montreal Canada Oct. 9 2005 Geoffrey Fox CTO Anabas Corporation and Computer Science, Informatics, Physics

WSDisplay

WSViewer

WS Display

WS Viewer

Event(Message)

Service

Master

WSDisplay

WS Viewer

Web Service MessageInterceptor

Collaboration as a WSSet up Session with XGSP

Application orContent source

WSDL

Web Service

F

I

U

O

F

I

R

O

Shared Output Port Collaboration

OtherParticipants

Text ChatWhiteboardMultiplemasters

Page 6: GlobalMMCS DS-RT 2005 Tutorial IEEE DS-RT 2005 Montreal Canada Oct. 9 2005 Geoffrey Fox CTO Anabas Corporation and Computer Science, Informatics, Physics

GlobalMMCS Design Principles A unified, scalable, robust “overlay” network is

needed to support AV and data group communication over heterogeneous networking environments.

A common AV signaling protocol has to be designed to support interactions between different AV collaboration endpoints.

A core conference control mechanism is required for establishing and managing the multi-point conference.

Finally, we’d like to use Web (Grid) -services to integrate collaboration communities in different technologies and to allow use of powerful services (like schedulers and meta-data catalogs) from general Grid/Web activities. • XGSP defines WSDL and other interoperability

specifications

Page 7: GlobalMMCS DS-RT 2005 Tutorial IEEE DS-RT 2005 Montreal Canada Oct. 9 2005 Geoffrey Fox CTO Anabas Corporation and Computer Science, Informatics, Physics

Global-MMCS Community Grid This includes an open source protocol independent Web Service

“MCU” which will scale to an arbitrary number of users and provides support for thousands of simultaneous users of collaboration services.

The function of A/V media server is distributed using NaradaBrokering architecture.• Media Servers mix and convert A/V streams

Open XGSP MCU based on the following open source projects• openh323 is basis of H323 Gateway• NIST SIP stack is basis of SIP Gateway• NaradaBrokering is open source messaging• Java Media Framework basis of Media Servers• Helix Community http://www.helixcommunity.org for Real

Media http://www.globalmmcs.org open source release

Page 8: GlobalMMCS DS-RT 2005 Tutorial IEEE DS-RT 2005 Montreal Canada Oct. 9 2005 Geoffrey Fox CTO Anabas Corporation and Computer Science, Informatics, Physics

Global-MMCS Service Architecture

Media Delivery, Storage Services( QoS : Reliable and Secure Delivery,Transport Mechanism,

Massive Dependable Storage )

Media Processing Service( Adaptation, Mixing,

Transcoding ... )

Session Management Service( Membership, Role

Management, Floor control)

AudiovisualCollaboration

Shared DataApplication

InstantMessaging

Page 9: GlobalMMCS DS-RT 2005 Tutorial IEEE DS-RT 2005 Montreal Canada Oct. 9 2005 Geoffrey Fox CTO Anabas Corporation and Computer Science, Informatics, Physics

XGSP Web Service MCU Architecture

SIP H323 Access Grid Native XGSPAdmire

Gateways convert to uniform XGSP Messaging

High Performance (RTP)and XML/SOAP and ..

Media ServersFilters

Session ServerXGSP-based Control

NaradaBrokeringAll Messaging

Use Multiple Media servers to scale to many codecs and manyversions of audio/video mixing

NB Scales asdistributed

WebServices

NaradaBrokering

Page 10: GlobalMMCS DS-RT 2005 Tutorial IEEE DS-RT 2005 Montreal Canada Oct. 9 2005 Geoffrey Fox CTO Anabas Corporation and Computer Science, Informatics, Physics

Break up into Web Services Monolithic MCU becomes many different “Simple Services”

• Session Control• Thumbnail “image” grabber• Audio Mixer• Video Mixer• Codec Conversion• Helix Real Streaming• PDA Conversion• H323/SIP Gateways

As independent can replicate particular services as needed• Codec conversion might require 20 services for 20 streams

spread over 5 machines 1000 simultaneous users could require:

• 1 session controller, 1 audio mixer, 10 video mixers, 20 codec converters, 2 PDA converters and 20 NaradaBrokers

Support with a stream optimized Grid Farm in the sky• Future billion way “Video over IP” serving 3G Phones and home media

centers/TV’s could require a lot of computing

Page 11: GlobalMMCS DS-RT 2005 Tutorial IEEE DS-RT 2005 Montreal Canada Oct. 9 2005 Geoffrey Fox CTO Anabas Corporation and Computer Science, Informatics, Physics

GlobalMMCS and NaradaBrokering All communication – both control and “binary” codecs are

handled by NaradaBrokering Control uses SOAP and codecs use RTP transport Each stream is regarded as a “topic” for NB Each RTP packet from this stream is regarded as an “event” for

this topic Can use replay and persistency support in NB to support

archiving and late clients Can build customized stream management to administer replay,

and who gets what stream in what codec NaradaBrokering supports unicast and multicast Use firewall penetration and network monitoring services in NB

to improve Q0S

Page 12: GlobalMMCS DS-RT 2005 Tutorial IEEE DS-RT 2005 Montreal Canada Oct. 9 2005 Geoffrey Fox CTO Anabas Corporation and Computer Science, Informatics, Physics

Incorporating Support for Audio/Video Delivery into

NaradaBrokering

Added support for an unreliable transport protocol, UDP

Implemented a fixed size (fast) topic (8 bytes). Designed a new compact event with minimum headers. Added support for legacy RTP clients (both unicast

clients and multicast groups) Improved the routing algorithm to handle real-time

audio and video stream delivery.

Page 13: GlobalMMCS DS-RT 2005 Tutorial IEEE DS-RT 2005 Montreal Canada Oct. 9 2005 Geoffrey Fox CTO Anabas Corporation and Computer Science, Informatics, Physics

0

10

20

30

40

50

60

0 200 400 600 800 1000 1200 1400 1600 1800 2000

De

lay

(Mill

ise

con

ds)

Packet Number

Average delays per packet for 50 video-clients NaradaBrokering Avg=2.23 ms, JMF Avg=3.08 ms

NaradaBrokering-RTP JMF-RTP

Page 14: GlobalMMCS DS-RT 2005 Tutorial IEEE DS-RT 2005 Montreal Canada Oct. 9 2005 Geoffrey Fox CTO Anabas Corporation and Computer Science, Informatics, Physics

0

1

2

3

4

5

6

7

8

0 200 400 600 800 1000 1200 1400 1600 1800 2000

Jitte

r (

Mill

ise

con

ds)

Packet Number

Average jitter (std. dev) for 50 video clients.

NaradaBrokering Avg=0.95 ms, JMF Avg=1.10 ms

NaradaBrokering-RTP JMF-RTP

Page 15: GlobalMMCS DS-RT 2005 Tutorial IEEE DS-RT 2005 Montreal Canada Oct. 9 2005 Geoffrey Fox CTO Anabas Corporation and Computer Science, Informatics, Physics

Polycom, Access Grid and RealVideo views of

video-mixed streams using GlobalMMCS

Page 16: GlobalMMCS DS-RT 2005 Tutorial IEEE DS-RT 2005 Montreal Canada Oct. 9 2005 Geoffrey Fox CTO Anabas Corporation and Computer Science, Informatics, Physics

Improved JMF Performance VIC Old JMF Client Fast JMF Client

1 8% - 9% 15% - 16 % 6% - 7%

2 13% - 14% 24% - 25 % 9% - 10%

3 17% - 18% 33% - 34 % 15% - 16%

4 23% - 24% 40% - 41% 17% - 18%

5 26% - 27% 46% - 47% 23% -24%

6 32% - 33% 51% - 52% 27% - 28%

7 35% - 36% 58% - 59% 31% - 32%

8 40% - 41% 62% - 63% 34% - 35%

Fraction of CPU used versus number of received streamsThe CIF-size video sequence from a 30-second movie with a lot of motions is streamed to the clients. Each stream is encoded in H.261, and has average bandwidth of 400Kbps and 20 fps.

Page 17: GlobalMMCS DS-RT 2005 Tutorial IEEE DS-RT 2005 Montreal Canada Oct. 9 2005 Geoffrey Fox CTO Anabas Corporation and Computer Science, Informatics, Physics

MPEG-4 vs. H.261

We added MPEG4 video to Java Media Framework Higher quality and flexible video sizes including

distributed pixels

Page 18: GlobalMMCS DS-RT 2005 Tutorial IEEE DS-RT 2005 Montreal Canada Oct. 9 2005 Geoffrey Fox CTO Anabas Corporation and Computer Science, Informatics, Physics

Coupled Diverse Streams GlobalMMCS supports many diverse streams managed by “video

system”• Different audio and video codecs• Shared display using video codecs (MPEG4 or H261)• Motion JPEG – stream of images to and from PDA

NaradaBrokering represents these and other collaborative streams just a “topics”; collaboration from multiple clients subscribing to a topic• Text Chat• Traditional lossless codec based shared display• White boards• Control streams

Streams can be linked to provide composite topics• eSports project linking video streams and real time annotation

of any frame• Can rewind and choose any frame of a real-time stream

Page 19: GlobalMMCS DS-RT 2005 Tutorial IEEE DS-RT 2005 Montreal Canada Oct. 9 2005 Geoffrey Fox CTO Anabas Corporation and Computer Science, Informatics, Physics

eSports SnapshotMaster Video AnnotationWhiteboard

Collaborative Video AnnotationWhiteboard

Synchronized Replay of archived video and annotation

Page 20: GlobalMMCS DS-RT 2005 Tutorial IEEE DS-RT 2005 Montreal Canada Oct. 9 2005 Geoffrey Fox CTO Anabas Corporation and Computer Science, Informatics, Physics

eSports Architecture

NB

Replay/Archiving Service

eSports Player

RTSP Server/Proxy

XGSP ClientsAudio Video JMFWhiteboard

Stremaing Gateway

NB Storage

NB Storage

Two way RTP linkOne way RTP linkLocal Storage accessCommunication channelTopic

Replay/Archiving Service

Page 21: GlobalMMCS DS-RT 2005 Tutorial IEEE DS-RT 2005 Montreal Canada Oct. 9 2005 Geoffrey Fox CTO Anabas Corporation and Computer Science, Informatics, Physics

Esports with sport movies

Page 22: GlobalMMCS DS-RT 2005 Tutorial IEEE DS-RT 2005 Montreal Canada Oct. 9 2005 Geoffrey Fox CTO Anabas Corporation and Computer Science, Informatics, Physics

Integration of PDA, Cell phone and Desktop Grid Access

NB Support for optimizedPDA Communication

Page 23: GlobalMMCS DS-RT 2005 Tutorial IEEE DS-RT 2005 Montreal Canada Oct. 9 2005 Geoffrey Fox CTO Anabas Corporation and Computer Science, Informatics, Physics

PDA Download video as images I

Page 24: GlobalMMCS DS-RT 2005 Tutorial IEEE DS-RT 2005 Montreal Canada Oct. 9 2005 Geoffrey Fox CTO Anabas Corporation and Computer Science, Informatics, Physics

PDA Download video as images II

Page 25: GlobalMMCS DS-RT 2005 Tutorial IEEE DS-RT 2005 Montreal Canada Oct. 9 2005 Geoffrey Fox CTO Anabas Corporation and Computer Science, Informatics, Physics

GlobalMMCS Status/Futures I 1. New Collaboration tools

• Shared IDL (Visualization), PowerPoint, OpenOffice (Applications need a month or so more)

• SVG game ( stable )• Whiteboard ( stable )• e-Sport ( prototype)• Jabber IM client ( prototype)• XGSP needs extension to support

2. JMF Audio/Video client ( stable)• performance enhancement finished• new codec ( MPEG4 finished; try H.264)• support different platform ( Linux, Mac – Mac well

developed but need to chase bug(s))• support NAT/firewall transparently like Skype• screen codec for shared display ( prototype )

Page 26: GlobalMMCS DS-RT 2005 Tutorial IEEE DS-RT 2005 Montreal Canada Oct. 9 2005 Geoffrey Fox CTO Anabas Corporation and Computer Science, Informatics, Physics

GlobalMMCS Status/Futures II 3. Replay & Archive (prototype)

• Replay Engine based on NaradaBroker Storage Service

• XGSP-RTSP gateway

• Extend RTSP and NaradaBrokering for Instant Replay 4. Web Server Portal ( stable)

• Standard calendar service ( iCalendar, vCalendar)

• Flexible conference management

• Need to package UI’s as portlets 5. Conferencing Media Processing Service ( Stable)

• Fix the bugs and add scheduling

• Support new codec ( MPEG4 ) 6. H.323 Gateway ( Stable)

• Import it to Linux platform

Page 27: GlobalMMCS DS-RT 2005 Tutorial IEEE DS-RT 2005 Montreal Canada Oct. 9 2005 Geoffrey Fox CTO Anabas Corporation and Computer Science, Informatics, Physics

GlobalMMCS Status/Futures III 7. RealStreaming Gateway ( Stable )

• Import it to Linux• Support Mobile clients

8. Global-MMCS deployment & test• Core performance measurements complete• Test under the setting of multiple NaradaBroker and

NAT/Firewall• support deployment for AFRL, NASA, DOE portals• test with remote sites

9. PDA Clients (prototype) 10. Improved video codec-based shared display 11. Scheduler of dynamic services sensitive to streaming

bandwidth requirement as well as CPU use of codec conversion