Upload
keon-dibb
View
220
Download
0
Tags:
Embed Size (px)
Citation preview
Architectural Freedomwith
Serial Interconnects
NDIA ConferenceSan Diego, CA
October 20-23, 2003
Emmanuel Eriksson – Dy 4 Systems([email protected])
Interconnect TrendsBus vs. Switched
Device
DeviceDevice
DeviceDeviceDevice
DeviceDevice
1980’s
Device
DeviceDevice
Device
Device
DeviceDevice
Bridge 1990’s
2000’s
DeviceDevice
DeviceDevice
DeviceDeviceDevice Device
SwitchedInterconnect
Shortcomings of Parallel Buses
scale number of connections scale physically scale bandwidth support multiple traffic types support high availability
You only need to encounter one limitation to make the move to a new architecture
Buses Don’t:
Typical Embedded Distributed System Requirements
• Large amount of data movement• Large amount of data processing• Large number of processing elements• Often multiple processor types
– DSP, CPU and NPU• Wide range of I/O devices
– Sensors– Imaging and Video devices– Storage– Network Interfaces– Displays
Bus Based Architecture
CPU
IO
CARD
IO
CARD
IO
CARD
IO
CARD
IO
CARD
IO
CARD
IO
CARD
PCI Bus
CPU
Bus Bridge
Memory
SCSIEthernet
USB
Sensor
Interface
Bus
Bridge
• Single address map• Limited to a single processor or single bus owner at a time• Difficulty scaling processing requirements with I/O needs
LAN Based Architecture
• Internet Protocol is a sophisticated protocol that is aimed at a complex problem:
Interconnect hundreds of thousands of computers
– Handle extreme latencies– Handle lost packets– Handle corrupted packets– Handle out-of-order packet arrival
Limitations of Current Architectures
• Difficulty balancing computational power with I/O requirements– No support for both processor-to-processor and processor-to-device
communication (Ethernet and bus architectures can do either but not both)
• “Lossy” LAN transport do not meet lossless data collection requirements without additional software– Ethernet will drop packets if congestion occurs
• High latency or limited bandwidth scaling– Moving a 64 bytes packet takes > 100 microseconds with Ethernet versus a
couple of microseconds with a switch fabric
• Limited Class of Service capability– Ethernet has no separate queuing or buffering for CoS– Buses don’t have any CoS
Serial Switched InterconnectsScalability, Density, Flexibility Supports “disaggregated” and “virtual” systems with highly flexible upgradeability
High Availability & ReliabilityFailures detected, reported & managed.Lossless fabric with CRC protection.One failure can’t take down system
Low Cost Leverages PCI infrastructure and a wealth of products with lean, protocol-agnostic switching.
High Performance Unified Fabric Multiple types and classes of traffic supported on low latency, high bandwidth interconnect.
Device
DeviceCPU/Device
CPU/Device
DeviceCPU/Device
DeviceCPU/Device
SwitchedInterconnect
StarFabric & PCI Express Advanced Switching (AS)
• Scalable, Extensible Serial Switched Interconnects– Lean, High Performance, Low Cost Switching Architectures– Layered Approach Promotes Efficient Modularity
• StarFabric– Open standard switched interconnect (StarFabric Trade Association)– Based on 622Mb/s SERDES technology
• 4X data bandwidth of 2 Gb/s• Links can extend beyond single chassis up to 10 meters through standard
CAT5 cabling– In production since Spring 2002
• PCI Express - Advanced Switching (AS)– Based on the PCI Express™ architecture – Based on 2.5Gb/s SERDES technology
• 4X data bandwidth of 8 Gb/s– Available in late 2004
Multi-Layered Protocols
Software
Physical Point to point, serial, differential,Point to point, serial, differential,hot-plug, inter-op form factorshot-plug, inter-op form factors
Data IntegrityData IntegrityData Link
622Mb/s for StarFabric2.5Gb/s for AS
Transaction StarFabric & Advanced SwitchingStarFabric & Advanced SwitchingPacket SwitchingPacket Switching
Hardware
PCI Bus Driver PCI Bus Driver
(init, enum, config)(init, enum, config)
P-to-P DriverP-to-P Driver SF/AS DriverSF/AS Driver
Operating SystemOperating System
LegacyPeer-Peer
CRC, Link Synchronization, Auto negotiation, Link status
Data routing, Credit based flow control, Event and error handling, etc.
Example Interconnect: StarFabric
Bus
Bridge
Bridge Bridge
BridgeBridge
Bridge
Bus
Bu
s
Bu
s
Bu
s
Bu
s
With one Switch device, connect 6 Bus
segments
Switch
Redundant fabric interconnect
accomplished with a 2nd Switch device
Switch
StarFabric Frames at 2.5G Transmit and
2.5G Receive
Parallel Bus
PCI or TDM or Proprietary
New Degrees of Architectural Freedom
SF or ASFabric
Network Gateway
Processor Boards
Solid State Storage
Storage BladesStorage BladesSCSI/FCCntrl
SCSI/FCCntrl
BridgeBridge
SCSI/FCCntrl
Bridge
ProcessorProcessor
ChipsetChipset
MemoryMemory
BridgeBridge
CPU
Chipset
Memory
Bridge
GigEMAC
Bridge
DSPDSP
MemoryMemory
BridgeBridge
DSPDSP
DSP
Memory
Bridge
DSP
DSP Boards
I/O
uC
I/O Bridge
uC
I/O
uC
I/O Bridge
uC
Other IO:SensorsEO/IR VideoVoice commsEtc...
SwitchedInterconnect
I/O
CARD
CPU
SWITCH
I/O
CARD
SWITCH
STORAGE
STORAGE
CPU
1G
LAN
CPU
SWITCH
CPU
I/O
CARD
SWITCH
STORAGE
1G
LAN
STORAGE
CPU
CPU
SWITCH
CPU
SWITCH
STORAGE2
STORAGE2
10G
LAN
CPU2
CPU2
CPU2
CPU2
DSP
DSP
DSP
DSP
DSP
DSP
DSP
DSP
I/O
CARD
I/O
CARD
Adaptive InfrastructureLife Cycle Benefits
Initial Purchase Initial Upgrade
Scale capacity
Mid Life Kicker
Upgrade CPUs & Storage
Increase LAN speeds
Dy 4 System Solutions
StarFabric Interconnect
&Interprocessor
Communication Software
CHAMP-AVII(Quad PowerPC)
CHAMP-FX(Dual FPGA Board)
StarReach(PMC Carrier Board)
DMV-182(Single Board
Computer)
Advantages of Distributed SwitchingSeparate Switch Card
Requires extra slot Single point of failure Extra LRU item logistics
Switch card
X
Backplane switching
Four slots
Active circuits on backplane
Switch circuit on backplane
X
StarFabric
Passive Backplane
XX
Switching on card
No active backplane
Four slots used
Summary
• Switched Interconnects like StarFabric and PCI Express AS:– Provide low cost, high performance interconnects for multislot,
distributed architectures– Leverage the PCI/PCI-X infrastructure and volumes– Provide CPU-to-CPU and CPU-to-device communication– Reduces processor overhead and system latencies– Provides a reliable interconnect for multiple classes of service– Enables design flexibility and system disaggregation– Allows flexible fabric architecture, centralized or distributed switching
Thank You!
Contact information for follow up questions or discussions...
Emmanuel ErikssonTechnology Architect
Technology Innovation GroupDy 4 Systems Inc.
[email protected](613) 599-9199 x5462
or visit...
www.stargen.comwww.dy4.com