SLAAC/ACS API: SLAAC/ACS API: Control of Systems of Control of Systems of Adaptive Computing Adaptive Computing NodesNodes

Virginia TechConfigurable Computing LabSLAAC Retreat March 1999

The Virginia Tech SLAAC The Virginia Tech SLAAC TeamTeam

• Dr. Peter AthanasDr. Peter Athanas

• Dr. Mark JonesDr. Mark Jones

• Heather HillHeather Hill

• Emad IbrahimEmad Ibrahim

• Zahi NakadZahi Nakad

• Kuan Yao Kuan Yao

• Diron DriverDiron Driver

• Karen ChenKaren Chen

• Chris TwaddleChris Twaddle

• Jonathan Jonathan ScottScott

• Luke ScharfLuke Scharf

• Lou PochetLou Pochet

• John ShiflettJohn Shiflett

• Peng PengPeng Peng

• Sarah AireySarah Airey

• Chris LaughlinChris Laughlin

Problem DefinitionProblem Definition

• A single adaptive computing board is A single adaptive computing board is insufficient for many applicationsinsufficient for many applications– insufficient power & functionalityinsufficient power & functionality

• Difficult to move an application from Difficult to move an application from a research reference platform to a research reference platform to deployment in a field systemdeployment in a field system

• Need for application to move to new Need for application to move to new platforms as they become available platforms as they become available without unreasonable effortwithout unreasonable effort

Research Reference Research Reference PlatformPlatform

• Network of ACS-Network of ACS-accelerated accelerated workstations.workstations.– Inexpensive readily Inexpensive readily

available platform for available platform for ACS development.ACS development.

– Tracks performance Tracks performance advances in advances in workstations and workstations and cluster computing.cluster computing.

• ACS hardware is PCI-based.ACS hardware is PCI-based.

• OS is NT or Unix. OS is NT or Unix.

• Network is simple Ethernet Network is simple Ethernet or high speed such as or high speed such as Myrinet.Myrinet.

Representative Field Representative Field SystemSystem

• Embedded, distributed systemEmbedded, distributed system– sensor nodessensor nodes

– actuator nodesactuator nodes

– adaptive computing nodesadaptive computing nodes

• Limited OS/Limited OS/microprocessor microprocessor support on most nodessupport on most nodes

• Heterogeneous networkHeterogeneous network

• Simplest carrier is a cluster Simplest carrier is a cluster of single-board computers. of single-board computers.

• ACS hardware is VME-based.ACS hardware is VME-based.

• OS is VxWorks.OS is VxWorks.

• Network is Myrinet SAN. Network is Myrinet SAN.

VME

Solution ApproachSolution Approach

• Define a platform independent API Define a platform independent API that allows for configuration and that allows for configuration and control of a multi-board ACS control of a multi-board ACS

• Provide efficient implementations of Provide efficient implementations of the API for research & field platformsthe API for research & field platforms– exploit high speed networkingexploit high speed networking

– modular design that performs more modular design that performs more complex control tasks on a OS-equipped complex control tasks on a OS-equipped hosthost

Capabilities of SLAAC/ACS Capabilities of SLAAC/ACS APIAPI• Allows control of a distributed system of Allows control of a distributed system of

adaptive computing nodes from a single adaptive computing nodes from a single host through functions defined in the APIhost through functions defined in the API

• Allows migration between platforms w/o Allows migration between platforms w/o modification of host source codemodification of host source code

– lightweight runtime environment on nodeslightweight runtime environment on nodes

• Channel-based model of computation Channel-based model of computation allows for flexible, efficient combining of allows for flexible, efficient combining of high-performance networks & ACS nodeshigh-performance networks & ACS nodes

Programming ModelProgramming Model• ACS API defines a ACS API defines a

systemsystem of of nodes nodes and and channels.channels.

– System dynamically System dynamically allocated at runtime.allocated at runtime.

– Channels stream Channels stream data between FIFOs data between FIFOs on host/nodes.on host/nodes.

– API provides common API provides common control primitives to control primitives to a distributed systema distributed system

• configure, readback, configure, readback, set_clock, run, etc.set_clock, run, etc.

Hosts

Nodes

Network

Network ChannelsNetwork Channels• Use network channels Use network channels

in place of physical in place of physical point-to-point point-to-point connections.connections.

Crossbar

F

M

F

M

F

M

F

M

Crossbar

F

M

F

M

F

M

F

M

Crossbar

F

M

F

M

F

M

F

M

Crossbar

F

M

F

M

F

M

F

M

Network-channel

– Boards operate on Boards operate on individual clocks, but individual clocks, but are data-are data-synchronous.synchronous.

– Channels can apply Channels can apply back-pressure to stall back-pressure to stall producers.producers.

Programmable TopologyProgrammable Topology• Channels allow data Channels allow data

to flow through the to flow through the system with a system with a programmable programmable topology.topology.

Crossbar

F

M

F

M

F

M

F

M

Crossbar

F

M

F

M

F

M

F

M

Crossbar

F

M

F

M

F

M

F

M

Crossbar

F

M

F

M

F

M

F

M

– Adds multiple Adds multiple dimensions of dimensions of scalability.scalability.

– Channel topology can Channel topology can be changed be changed dynamically.dynamically.

1

2

3

0

System Creation System Creation FunctionsFunctions

• ACS_Initialize ACS_Initialize – Parses command line.Parses command line.

– Initializes globals.Initializes globals.

• ACS_System_Create ACS_System_Create – Allocates nodes and Allocates nodes and

channels.channels.

– Creates opaque Creates opaque system system objectobject in host program. in host program.

– Same host program can Same host program can manage multiple manage multiple systems.systems.

– NodesNodes and and channelschannels are logically are logically numbered in order numbered in order of creation.of creation.

– Host is node zero.Host is node zero.

0

1

2

3

Memory Access FunctionsMemory Access Functions

• ACS_Read()ACS_Read()– Gets block of Gets block of

memory from memory from (system, node, (system, node, address, count) into address, count) into user buffer.user buffer.

• ACS_Write()ACS_Write()– Puts block of Puts block of

memory from user memory from user buffer to (system, buffer to (system, node, address, node, address, count). count).

• ACS_Copy()ACS_Copy()– Copies memory Copies memory

from (node1, from (node1, address1) to address1) to (node2, address2) (node2, address2) directly.directly.

• ACS_Interrupt()ACS_Interrupt()– Generates an Generates an

interrupt signal at interrupt signal at node.node.

Streaming Data FunctionsStreaming Data Functions

• Each node/system Each node/system has a set of FIFO has a set of FIFO buffers.buffers.

• Channels connect Channels connect two FIFO buffers.two FIFO buffers.

• Arbitrary streaming-Arbitrary streaming-data topologies data topologies supported.supported.

• ACS_Enqueue()ACS_Enqueue()

– put user data into put user data into FIFOFIFO

• ACS_Dequeue()ACS_Dequeue()

– get user data from get user data from FIFOFIFO

1

0FIFO 0

FIFO 1

FIFO 2

FIFO 3

FIFO 0

FIFO 1

FIFO 2

FIFO 3

2

FIFO 0

FIFO 1

FIFO 2

FIFO 3

Implementation StrategyImplementation Strategy

• CommunicationCommunication– Rely on MPI for high-performance Rely on MPI for high-performance

communication where availablecommunication where available

– When MPI not available or convenient, tightly When MPI not available or convenient, tightly couple network & ACS hardwarecouple network & ACS hardware

• PortabilityPortability– limited new code is required to extend API limited new code is required to extend API

implementation for a new ACS boardimplementation for a new ACS board

– control program for compute nodes is simple control program for compute nodes is simple enough to run w/o complex OSenough to run w/o complex OS

API Implementation StatusAPI Implementation Status• Completed implementation of v1.0 of Completed implementation of v1.0 of

APIAPI

– Implemented in C++ (callable from Implemented in C++ (callable from C)C)

– Software: NT + MPI (WMPI & MPI-FM)Software: NT + MPI (WMPI & MPI-FM)

– Hardware: WildForceHardware: WildForce

• Runs on the Tower of PowerRuns on the Tower of Power

– 16-node cluster of PCs 16-node cluster of PCs

– WildForce board on each PCWildForce board on each PC

– Myrinet network connecting all PCsMyrinet network connecting all PCs

PerformancePerformanceMonitorMonitor

• Dynamic topology display• Performance Metrics• Playback (future)

• Use to confirm the configuration of the system• Use to identify performance bottlenecks

ACS Multiboard DebuggerACS Multiboard Debugger

• Based on Based on Boardscope and Boardscope and JbitsJbits

• Will provideWill provide– WaveformsWaveforms

– State StatusState Status

– Channel StatusChannel Status

• Interfaces through Interfaces through SLAAC APISLAAC API

Project TimelineProject Timeline

Nov 98 Aug 99Feb 99 May 99

Kickoff

Multiboard API

Intervention Free Operation

Multiboard Debugger

Applications

SLAAC-1 & 2 Integration

Why Use This API?Why Use This API?

• Single Board SystemsSingle Board Systems– API closely matches accepted API’s, e.g. AMS API closely matches accepted API’s, e.g. AMS

Wildforce & SplashWildforce & Splash

– Virtually no overheadVirtually no overhead

– Your application will port to SLAACYour application will port to SLAAC

• Multi Board SystemsMulti Board Systems– Single program for multi-node applicationsSingle program for multi-node applications

– Inherent management of the networkInherent management of the network• Zero sided communicationZero sided communication

• It’s FREEIt’s FREE

Future WorkFuture Work

• Support for Linux in addition to NTSupport for Linux in addition to NT

• Support for RunTime Reconfiguration Support for RunTime Reconfiguration (RTR)(RTR)

• Extension to SLAAC-1 & 2 boardsExtension to SLAAC-1 & 2 boards

• API implementation for embedded API implementation for embedded systemssystems

• System level management of multiple System level management of multiple programsprograms

SummarySummary

• Latest versions of source code and design documents available for download

• For more information visit TOP websitehttp://acamar.visc.ece.vt.edu/http://acamar.visc.ece.vt.edu/