Application-Defined Network

Hao Xu and Shu Huang, Yufeng Xin, Leesa Brieger, Reagan Moore and Arcot Rajasekar

iRODS Optimization of Data Routing on

Software-Defined Networks

iRODS Meets Software-Defined Networks• iRODS (integrated Rule-Oriented Data System)

– Data grid middleware– Policy-based data operations and management

• SDN (Software-Defined Network)– Separate the control from data forwarding to make the network

programmable

• Two rule-based systems communicating to optimize performance – A new paradigm: applications direct the network (Application-Defined

Network)

iRODS rules: if condition(data) do: copy(data, from, to)

Openflow rules: if matches(packet) do: forward(packet, output_port)

3

My Data:disk, filesystem,

site-specific storage, ...

My Data:tape, database, filesystem,

...

Partner’s Dataremote disk, tape,

filesystem, site-specific storage,…

User Client

• iRODS installs over heterogeneous data resources• Users can share & manage distributed data as a single

collection

User sees a single collection

iRODS View of Distributed Data

iRODS Data Grid

4

iRODS Policy Implementation Microservices and Rules

• Microservice – the functional unit of work (C programs)

• Rules – workflows of microservices (and rules)

• Provide server-side (data-side) services

• Event-triggered rule execution (PEP: Policy Enforcement Points)

Presentation title goes here 5

ExoGENI• A GENI testbed

• Links GENI to open cloud computing (OpenStack) and dynamic circuit fabrics

• Provides a networked infrastructure-as-a-service (NIaaS) platform

• flexible networking operations using traditional VLAN-based switching and OpenFlow

• Uses ORCA (Open Resource Control Architecture) control framework software

• Served as testbed for this demo

Resource Provisioning -- ExoGENI

iRODS meets SDN• From tradition waterfall model to workflow model: tie together

the network and the data management components of the infrastructure

Application Management

Network Management

Computation Resource

Management

Storage Resource

Management

Data ManagementPolices

Presentation title goes here 8

Multiple Possible Paths Between Nodes

Presentation title goes here 9

Allow iRODS to Control Routing(Parallel Transfer)

• Parallel transfer can be enhanced – independent routes for individual threads– adapt independently to network traffic

• New PEP (policy enforcement points) trigger the network control mechanism iRODS OpenFlow– iRODS gets network state information from the Graph DB– iRODS rules direct OpenFlow to optimize the routes of the threads

Rule Engine

Unification of data and network rules

GraphDB Data Policies

Network Policies

OF Controller

iRODS Server

iRODS Server

iRODS Server

iCAT DB

acPreProcForServerPortal

acPostProcForServerPortal

Transfer Thread

Compute Shortest Path

Generate OpenFlow Rules

Update Link Cost

Delete OpenFlow Rules

Update Link Cost

Save OpenFlow Rule Info

Load OpenFlow Rule Info

Data Rules

Network Rulesirepl -R demoResc 100m:for each thread:

New iRODS PEPs and Actions

$time irepl -R demoResc 100m : 1m28.198s

$time irepl -R demoResc 100m : 0m44.198s

Without the parallel trigger on network control, three active nodes:

With iRODSnetwork control,8 active nodes:

Demo: data movement