Next Gen Cluster Draft Hardware Schema

Executive Summary

ASU Senior Leadership has charged the Complex Adaptive Systems Network team with articulating the cyberinfrastructure necessary to successfully execute its portfolio of projects. It is the assessment of the team that a Next Generation Cyber Capability (NGCC) is required to meet the unique needs of CASN projects. Conceptually, the NGCC is composed of standing capabilities and dynamic, on-demand, virtual resources that grow and contract as CASN efforts require. The NGCC is a synergistic, systems whole that uniquely synthesizes physical and logical infrastructure which is fulfilled through dedicated staff resources working as a team. Each infrastructure component is itself a collection of diverse capabilities executed through staff with specialized skills. While components of the NGCC reside in various isolated niches throughout the ASU community, to successfully meet the needs of CASN projects requires a dedicated enterprise.

Dell Enterprise is a strategic partner of Dr. Buetow and Arizona State University

Scratch Space

HPC Parallel HPC SMABig Data

ContextOntologies

Data ElementsInformation Models

Middleware

AnalyticGeneral Purpose

GenomicBig Data

Data ResourcesFile SystemRelationalKey/Value

TransactClinical Research

Life Science researchQualitative Research

Capacity

TransCORE Framework Knowledge Engine

Transactional

Data Reservoir

TransactionalHPCC BIG DATA

Exp

an

sio

n to

Pu

blic

Clo

ud

Exp

an

sio

n to

Pu

blic

Clo

ud

Ne

xt G

en

era

tion

Co

mp

ute

Clu

ste

r

V.1

.5a

/ Pa

ge

1N

GC

C G

lob

al T

op

olo

gy D

esig

n0

9/0

6/2

01

3Ja

y E

tch

ing

s, S

olu

tion

s A

rch

itect, D

ell E

nte

rpris

e

TRANSACTIONAL CLUSTER COMPONENT

IS BUILT FROM A 20 NODE CLUSTER

OFFERING 240 CORES, 640GB OF DDR3

AND 40-200GB SOLID STATE DRIVES IN

AN M1000E CHASSIS WITH 40GBE

UPLINK CONNECTS, CONVERGED

NETWORK ADAPTERS FOR ISCSI

OFFLOAD.

HA STORAGE

40Gbe VLT Fabric

BIG DATA CLUSTER CONFIGURATION

CONSISTING OF 40 -10GBE

INTERCONNECTED NODES WITH DUAL 8

CORE SERVERS WITH 128GB DDR3

MEMORY, DUAL 10GBE INTERCONNECTS,

24-2TB DRIVES AND DUAL PSUS. HIGH SPEED 40GBE CONNECTIONS TO THE

SUPPORTING ENVIRONMENT ARE

PROVIDED.

100 NODE / 800 CORE CLUSTER WITH

32GB MEMORY PER NODE CONNECTED

VIA INFINIBANC QDR INTERCONNECTS.

CLUSTER HAS 250TB OF FAST SCRATCH

STORAGE SPACE IN THE LUSTRE FILE

SYSTEM. A HIGH SPEED 40GBE

INTERCONNECT TO THE SUPPORTING

ENVIRONMENT IS PROVIDED.

ContextOntologies

Data ElementsInformation Models

Middleware

AnalyticGeneral Purpose

GenomicBig Data

Data Resources

File SystemRelationalKey/Value

0202

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

0202

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

HIGHLY AVAILABLE COMPELLENT

STORAGE WITH CLUSTERED NAS/NFS/

CIFS CAPABILITIES, 30 ENCLOSURES

WITH 12-3TB (360) 7.2K DRIVE, 1PB

REPLICATED TO 2PB CAPACITY ARRAY VIA

FCIP FABRIC EXTENSION BRIDGE.

1080TB

TransactClinical Research

Life Science researchQualitative Research

Capacity

0202

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

0202

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

HIGHLY AVAILABLE COMPELLENT

STORAGE CONSISTING OF 21

ENCLOSURES WITH 12- 4TB (252) 7.2K

DRIVES. REPLICATION TARGET FOR

PRIMARY ETL INGESTION ARRAY.

1008TB

FCIP Replication via Fabric Extension

TransCORE Framework Knowledge Engine

Transactional

Scratch SpaceBig Data

HPC SMAHPC Parallel

Data Reservoir

High Speed Connectivity

16GB FC SAN

Connection(s)

Jay Etchings, Solutions Architect, Dell Enterprise

Big Data

New models of computing intimately combine processing, large data stores, and data transfer through a unique file system (Hadoop/MapReduce) that provides novel computational strategies.

Standing Big Data Capacity would consist of 10GE Hadoop 40 node cluster composed of dual 8 core servers each with 72GB DDR3 memory, 2 X 10GE, 24 x 2TB storage, with dual PSUs. To facilitate potential expansion the servers would be configured over multiple rack with appropriate switches added as growth might dictate. High speed L3 network connection to the transactional infrastructure is required.

Hadoop is the popular Open Source implementation of MapReduce, a powerful tool designed for deep analysis and transformation of very large data sets.

Big Data

Next Generation Compute Cluster -HPCC

V.1.5a / Page 3 Server Roles Definitions 09/06/2013 Jay Etchings, Solutions Architect, Dell Enterprise

PowerEdge R720 X 4 Units

Input Pow er: 545w atts 1859.6btu/h *

Maximum Potential Pow er:* 750.9w atts 2562.2btu/h *

Input Current: 2.5amps

Sound Pow er Level: 6.1bels *

Airf low Rate: 18.5l/s 39.1CFM

Weight: 29.5kg 65lbs

Air Temperature Rise: 25.1C 45.1F

PowerEdge R720XD X 40

Units

Input Pow er: 551w atts 1880.1btu/h *

Maximum Potential Pow er:* 789.5w atts 2694btu/h *

Input Current: 2.5amps

Sound Pow er Level: 6.3bels *

Airf low Rate: 17l/s 35.9CFM

Weight: 29.2kg 64.4lbs

Air Temperature Rise: 27.6C 49.7F

Workload: Computational

Pow erEdge R720xd Pow erEdge R720xd

Processor

Intel Xeon E5-2670 2.60GHz, 20M Cache,

8.0GT/s QPI, Turbo, 8C, 115W, Max Mem 1600MHz

Additional Processor


8.0GT/s QPI, Turbo, 8C, 115W

Memory Configuration Type Performance Optimized

Memory DIMM Type and Speed 1600 MHz RDIMMS

Memory Capacity

8 x 16GB RDIMM, 1600MT/s, Low Volt, Dual Rank,

x4 Data Width

Operating System No Operating System

OS Media Kits No Media Required

Chassis Configuration Chassis w ith up to 24, 2.5" Hard Drives

RAID Configuration No RAID for H310 (1-16 HDDs)

RAID Controller

PERC H710P Integrated RAID Controller, 1GB NV

Cache

24 x 1TB 7.2K RPM SATA 3Gbps 2.5in Hot-plug

Hard Drive

2 x 500GB 7.2K RPM SATA 3Gbps 2.5in Hot-plug

Hard Drive

Pow er Supply

Dual, Hot-plug, Redundant Pow er Supply (1+1),

750W

2 x NEMA 5-15P to C13 Wall Plug, 125 Volt, 15 AMP,

10 Feet (3m), Pow er Cord

No Pow er Cord

Pow er Management BIOS Settings Pow er Saving Dell Active Pow er Controller

Embedded Systems Management iDRAC7 Enterprise w ith Vflash, 16GB SD Card

Add-in Netw ork Adapter

Broadcom 57810 DP 10Gb DA/SFP+ Converged

Netw ork Adapter

Select Netw ork Adapter

Broadcom 57800 2x10Gb SR/SFP+ + 2x1Gb BT

Netw ork Daughter Card, w ith SR Optics

Rack Rails

ReadyRails Sliding Rails With Cable Management

Arm

Pow er Cords

Hard Drives

Workload: Computational

Pow erEdge R720 Pow erEdge R720

Processor


8.0GT/s QPI, Turbo, 8C, 115W, Max Mem 1600MHz

Additional Processor


8.0GT/s QPI, Turbo, 8C, 115W

Memory Configuration Type Performance Optimized

Memory DIMM Type and Speed 1333 MHz LRDIMMs

Memory Capacity

16 x 32GB LRDIMM, 1333 MT/s, Low Volt, Quad

Rank, x4 Data Width

OS Media Kits No Media Required

Chassis Configuration 2.5" Chassis w ith up to 16 Hard Drives

RAID Configuration RAID 1 for H710P/H710/H310 (2 HDDs)

RAID Controller

PERC H710P Integrated RAID Controller, 1GB NV

Cache

Hard Drives

10 x 300GB 15K RPM SAS 6Gbps 2.5in Hot-plug

Hard Drive

Pow er Supply

Dual, Hot-plug, Redundant Pow er Supply (1+1),

750W

Pow er Cords

2 x NEMA 5-15P to C13 Wall Plug, 125 Volt, 15 AMP,

10 Feet (3m), Pow er Cord

Pow er Management BIOS Settings Pow er Saving Dell Active Pow er Controller

Embedded Systems Management iDRAC7 Enterprise w ith Vflash, 8GB SD Card

Select Netw ork Adapter

Broadcom 57800 2x10Gb SR/SFP+ + 2x1Gb BT

Netw ork Daughter Card, w ith SR Optics

Add-in Netw ork Adapter

2 x Broadcom 57810 DP 10Gb SR/SFP+ Converged

Netw ork Adapter, w ith SR Optics

Rack Rails

ReadyRails Sliding Rails With Cable Management

Arm

Ha

do

op

Po

we

r Co

olin

g S

pe

cific

atio

ns

HD

FS

Hadoop H-Rack-ANGCC

Clu

stere

d N

od

e N

etw

ork

RS

-232

PSU0SYS ALM

LOC

IPMI ET

H

RESET PSU1Master

STACK ID USB-A

USB-B

SSD

60 61 62 6352 53 54 55 56 57 58 5948 49 50 5144 45 46 4736 37 38 39 40 41 42 4332 33 34 3528 29 30 3124 25 26 2720 21 22 2312 13 14 15 16 17 18 198 9 10 114 5 6 70 1 2 3

124 125 126 127116 117 118 119 120 121 122 123112 113 114 115108 109 110 111100 101 102 103 104 105 106 10796 97 98 9992 93 94 9588 89 90 9184 85 86 8776 77 78 79 80 81 82 8372 73 74 7568 69 70 7164 65 66 67

Force10 Z9000

RS

-232

PSU0SYS ALM

LOC

IPMI ET

H

RESET PSU1Master

STACK ID USB-A

USB-B

SSD

60 61 62 6352 53 54 55 56 57 58 5948 49 50 5144 45 46 4736 37 38 39 40 41 42 4332 33 34 3528 29 30 3124 25 26 2720 21 22 2312 13 14 15 16 17 18 198 9 10 114 5 6 70 1 2 3

124 125 126 127116 117 118 119 120 121 122 123112 113 114 115108 109 110 111100 101 102 103 104 105 106 10796 97 98 9992 93 94 9588 89 90 9184 85 86 8776 77 78 79 80 81 82 8372 73 74 7568 69 70 7164 65 66 67

Force10 Z9000

Sla

ve

No

de

sM

an

ag

em

en

tM

aste

rN

etw

ork

H-Rack-A10 Slave Nodes

R720XD2-Xeon E5-2670

128GB DDR324-2TB 7.2k

4-10Gb QFSP+640C/1280T

1.920PB

ManagementExpansion

FPM 185RM LED console

4020 Rack Frame2-Managed PDU

C13/L6-30A

Master/HA Master3 R720

2-Xeon E5-2670512GB DDR3

10-300GB 15k SAS6-10Gbe QSFP+

Rack Aware TOR2-S4810 10GB

40Gbe HA Stack

Ne

xt G

en

era

tion

Co

mp

ute

Clu

ste

r -Ha

do

op

/ Big

Da

ta

V.1

.5a

/ Pa

ge

5H

-Ra

ck-A

09

/06

/20

13

Ja

y E

tch

ing

s, S

olu

tion

s A

rch

itect, D

ell E

nte

rpris

e

QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

02

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

02

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

02

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

02

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

AC OK

100-240V ~ 6A50 60Hz

100-240V ~ 6A50 60Hz

AC OK

PSU0

PSU1

AC OK

100-240V ~ 6A50 60Hz

100-240V ~ 6A50 60Hz

AC OK

PSU0

PSU1

2

VLT Domain 1

COREVLT Domain 2

10G

be

SF

P+

2

Quad 40Gbe Trunk Links

A & B Managed Power28C13 / 2-L6-30A

Distributed Data ProcessingJob TrackerMap Reduce

Secondary Name Node

7.8kW Peak6.1kW Typ

CB

1C

B2

CB

5C

B6

CB

3C

B4

CB

1C

B2

L1

CB

3C

B4

L2

CB

5C

B6

L3

ON

OFF

ON

OFF

ON

OFF

ON

OFF

ON

OFF

ON

OFF

2

2

2

2

2

2

2

2

2

2

40GB LAG

10GB Link

10GB iSCSI

10GB Trunk

1GB iSCSI

16GB FC

1Gbe Mgmt.

KMM FPM185

Jay Etchings, Solutions Architect, Dell Enterprise H-RACK-A Power Peak Pwr Typical BTU Peak BTU Typ Max Pot.

Totals 7865 6156 26819.65 20991.96 8661

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

FDX

/HDX

LNK

/ACT

DIAG

STACK ID

G1 G2 G4G3

LNK/ACT FDX/HDX

MASTER RPS PWR

FAN TEMP

RESET

SELECT

LNK

/ACT

LNK

/ACT

STACKING

PORT

1 2 3 4

5 6 7 8

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

FDX

/HDX

LNK

/ACT

DIAG

STACK ID

G1 G2 G4G3

LNK/ACT FDX/HDX

MASTER RPS PWR

FAN TEMP

RESET

SELECT

LNK

/ACT

LNK

/ACT

STACKING

PORT

1 2 3 4

5 6 7 8

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

Hadoop H-Rack-BNGCC

Master

Ne

two

rkM

an

ag

em

en

tS

lav

e N

od

es

H-R

ac

kA

H-R

ac

kC

H-R

ac

kE

Ne

xt G

en

era

tion

Co

mp

ute

Clu

ste

r -Ha

do

op

/ Big

Da

ta

V.1

.5a

/ Pa

ge

6H

-Ra

ck-B

08

/26

/20

13

Ja

y E

tch

ing

s, S

olu

tion

s A

rch

itect, D

ell E

nte

rpris

e

0202

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

AC OK

100-240V ~ 6A50 60Hz

100-240V ~ 6A50 60Hz

AC OK

PSU0

PSU1

AC OK

100-240V ~ 6A50 60Hz

100-240V ~ 6A50 60Hz

AC OK

PSU0

PSU1

02

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

02

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

RS

-232

PSU0SYS ALM

LOC

IPMI ET

H

RESET PSU1Master

STACK ID USB-A

USB-B

SSD

60 61 62 6352 53 54 55 56 57 58 5948 49 50 5144 45 46 4736 37 38 39 40 41 42 4332 33 34 3528 29 30 3124 25 26 2720 21 22 2312 13 14 15 16 17 18 198 9 10 114 5 6 70 1 2 3

124 125 126 127116 117 118 119 120 121 122 123112 113 114 115108 109 110 111100 101 102 103 104 105 106 10796 97 98 9992 93 94 9588 89 90 9184 85 86 8776 77 78 79 80 81 82 8372 73 74 7568 69 70 7164 65 66 67

Force10 Z9000

RS

-232

PSU0SYS ALM

LOC

IPMI ET

H

RESET PSU1Master

STACK ID USB-A

USB-B

SSD

60 61 62 6352 53 54 55 56 57 58 5948 49 50 5144 45 46 4736 37 38 39 40 41 42 4332 33 34 3528 29 30 3124 25 26 2720 21 22 2312 13 14 15 16 17 18 198 9 10 114 5 6 70 1 2 3

124 125 126 127116 117 118 119 120 121 122 123112 113 114 115108 109 110 111100 101 102 103 104 105 106 10796 97 98 9992 93 94 9588 89 90 9184 85 86 8776 77 78 79 80 81 82 8372 73 74 7568 69 70 7164 65 66 67

Force10 Z9000

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

iDRAC1 2 3 4

ST

21

2

1

3750W 750W

5

4 6

1

1 2 3 4

iDRAC

1 2750W 750W

1

1 2 3 4

iDRAC

1 2750W 750W

H-Rack-B10 Slave Nodes


128GB DDR324-2TB 7.2k

4-10Gb QFSP+640C/1280T

1.920PB

ManagementExpansion

2-R620 ASM-HA4020 Rack Frame2-Managed PDU

C13/L6-30A


40Gbe HA Stack2-Z9000 10/40Gbe

HA CLOS Fabric Core 50-m QFSP+

HA Name Node Map Reduce

HD

FS

VLT Domain 1

VLT Domain 1CORE


9.1kW Peak7.2kW Typ

CB

1C

B2

CB

5C

B6

CB

3C

B4

CB

1C

B2

L1

CB

3C

B4

L2

CB

5C

B6

L3

ON

OFF

ON

OFF

ON

OFF

ON

OFF

ON

OFF

ON

OFF

2

2

2

2

2

2

2

2

2

2

ManagementVLAN

2

2

2

2


10G

be

SF

P+

2

PCT 3548

40GB LAG

10GB Link

10GB iSCSI

10GB Trunk

1GB iSCSI

16GB FC

1Gbe Mgmt.


H-RACK-B Power Peak Pwr Typical BTU Peak BTU Typ Max Pot.

Totals 9185 7228 31302.85 24647.48 9128

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

FDX

/HDX

LNK

/ACT

DIAG

STACK ID

G1 G2 G4G3

LNK/ACT FDX/HDX

MASTER RPS PWR

FAN TEMP

RESET

SELECT

LNK

/ACT

LNK

/ACT

STACKING

PORT

1 2 3 4

5 6 7 8

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

FDX

/HDX

LNK

/ACT

DIAG

STACK ID

G1 G2 G4G3

LNK/ACT FDX/HDX

MASTER RPS PWR

FAN TEMP

RESET

SELECT

LNK

/ACT

LNK

/ACT

STACKING

PORT

1 2 3 4

5 6 7 8

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

Hadoop H-Rack-CNGCC

HD

FS

Clu

stere

d N

od

e N

etw

ork

RS

-232

PSU0SYS ALM

LOC

IPMI ET

H

RESET PSU1Master

STACK ID USB-A

USB-B

SSD

60 61 62 6352 53 54 55 56 57 58 5948 49 50 5144 45 46 4736 37 38 39 40 41 42 4332 33 34 3528 29 30 3124 25 26 2720 21 22 2312 13 14 15 16 17 18 198 9 10 114 5 6 70 1 2 3

124 125 126 127116 117 118 119 120 121 122 123112 113 114 115108 109 110 111100 101 102 103 104 105 106 10796 97 98 9992 93 94 9588 89 90 9184 85 86 8776 77 78 79 80 81 82 8372 73 74 7568 69 70 7164 65 66 67

Force10 Z9000

RS

-232

PSU0SYS ALM

LOC

IPMI ET

H

RESET PSU1Master

STACK ID USB-A

USB-B

SSD

60 61 62 6352 53 54 55 56 57 58 5948 49 50 5144 45 46 4736 37 38 39 40 41 42 4332 33 34 3528 29 30 3124 25 26 2720 21 22 2312 13 14 15 16 17 18 198 9 10 114 5 6 70 1 2 3

124 125 126 127116 117 118 119 120 121 122 123112 113 114 115108 109 110 111100 101 102 103 104 105 106 10796 97 98 9992 93 94 9588 89 90 9184 85 86 8776 77 78 79 80 81 82 8372 73 74 7568 69 70 7164 65 66 67

Force10 Z9000

Sla

ve

No

de

sM

an

ag

em

en

tN

etw

ork

H-Rack-C10 Slave Nodes


128GB DDR324-2TB 7.2k

4-10Gb QFSP+640C/1280T

1.920PB

ManagementExpansion

4322DS KVM 32 Port4020 Rack Frame2-Managed PDU

C13/L6-30A


40Gbe HA Stack

Ne

xt G

en

era

tion

Co

mp

ute

Clu

ste

r -Ha

do

op

/ Big

Da

ta

V.1

.5a

/ Pa

ge

6H

-Ra

ck-C

09

/06

/20

13

Ja

y E

tch

ing

s, S

olu

tion

s A

rch

itect, D

ell E

nte

rpris

e

QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

02

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

02

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

02

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

02

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

AC OK

100-240V ~ 6A50 60Hz

100-240V ~ 6A50 60Hz

AC OK

PSU0

PSU1

AC OK

100-240V ~ 6A50 60Hz

100-240V ~ 6A50 60Hz

AC OK

PSU0

PSU1

VLT Domain 1

COREVLT Domain 2


6.2kW Peak4.8kW Typ

CB

1C

B2

CB

5C

B6

CB

3C

B4

CB

1C

B2

L1

CB

3C

B4

L2

CB

5C

B6

L3

ON

OFF

ON

OFF

ON

OFF

ON

OFF

ON

OFF

ON

OFF

2

2

2

2

2

2

2

2

2

2

232

31

To remove power completely, both AC line cords must be disconnected.

B

A

A

100-240V~, 1.25A , 50/60 Hz LAN 2 PDU 1 SETUP

MODEM

LAN 1 PDU 22

1

B


100-240V~, 1.25A , 50/60 Hz

ManagementVLAN

10G

be

SF

P+

2


2 PCT 3548Management Switch

ACL VLAN

40GB LAG

10GB Link

10GB iSCSI

10GB Trunk

1GB iSCSI

16GB FC

1Gbe Mgmt.

Jay Etchings, Solutions Architect, Dell Enterprise H-RACK-C Power Peak Pwr Typical BTU Peak BTU Typ Max Pot.

Totals 6237 4875 21268.17 16623.75 7890

Hadoop H-Rack-ENGCC

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

FDX

/HDX

LNK

/ACT

DIAG

STACK ID

G1 G2 G4G3

LNK/ACT FDX/HDX

MASTER RPS PWR

FAN TEMP

RESET

SELECT

LNK

/ACT

LNK

/ACT

STACKING

PORT

1 2 3 4

5 6 7 8

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

FDX

/HDX

LNK

/ACT

DIAG

STACK ID

G1 G2 G4G3

LNK/ACT FDX/HDX

MASTER RPS PWR

FAN TEMP

RESET

SELECT

LNK

/ACT

LNK

/ACT

STACKING

PORT

1 2 3 4

5 6 7 8

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

HD

FS

Clu

stere

d N

od

e N

etw

ork

RS

-232

PSU0SYS ALM

LOC

IPMI ET

H

RESET PSU1Master

STACK ID USB-A

USB-B

SSD

60 61 62 6352 53 54 55 56 57 58 5948 49 50 5144 45 46 4736 37 38 39 40 41 42 4332 33 34 3528 29 30 3124 25 26 2720 21 22 2312 13 14 15 16 17 18 198 9 10 114 5 6 70 1 2 3

124 125 126 127116 117 118 119 120 121 122 123112 113 114 115108 109 110 111100 101 102 103 104 105 106 10796 97 98 9992 93 94 9588 89 90 9184 85 86 8776 77 78 79 80 81 82 8372 73 74 7568 69 70 7164 65 66 67

Force10 Z9000

RS

-232

PSU0SYS ALM

LOC

IPMI ET

H

RESET PSU1Master

STACK ID USB-A

USB-B

SSD

60 61 62 6352 53 54 55 56 57 58 5948 49 50 5144 45 46 4736 37 38 39 40 41 42 4332 33 34 3528 29 30 3124 25 26 2720 21 22 2312 13 14 15 16 17 18 198 9 10 114 5 6 70 1 2 3

124 125 126 127116 117 118 119 120 121 122 123112 113 114 115108 109 110 111100 101 102 103 104 105 106 10796 97 98 9992 93 94 9588 89 90 9184 85 86 8776 77 78 79 80 81 82 8372 73 74 7568 69 70 7164 65 66 67

Force10 Z9000

Sla

ve

No

de

sM

an

ag

em

en

tN

etw

ork

H-Rack-E10 Slave Nodes


128GB DDR324-2TB 7.2k

4-10Gb QFSP+640C/1280T

1.920PB

ManagementExpansion

4322DS KVM 32 Port2-PCT 3548

96 Mgmt. Ports4020 Rack Frame2-Managed PDU

C13/L6-30A


40Gbe HA Stack

Ne

xt G

en

era

tion

Co

mp

ute

Clu

ste

r -Ha

do

op

/ Big

Da

ta

V.1

.5a

/ Pa

ge

7H

-Ra

ck-C

09

/06

/20

13

Ja

y E

tch

ing

s, S

olu

tion

s A

rch

itect, D

ell E

nte

rpris

e

QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

02

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

02

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

02

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

02

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

AC OK

100-240V ~ 6A50 60Hz

100-240V ~ 6A50 60Hz

AC OK

PSU0

PSU1

AC OK

100-240V ~ 6A50 60Hz

100-240V ~ 6A50 60Hz

AC OK

PSU0

PSU1

VLT Domain 1

COREVLT Domain 2


6.2kW Peak4.9kW Typ

CB

1C

B2

CB

5C

B6

CB

3C

B4

CB

1C

B2

L1

CB

3C

B4

L2

CB

5C

B6

L3

ON

OFF

ON

OFF

ON

OFF

ON

OFF

ON

OFF

ON

OFF

2

2

2

2

2

2

2

2

2

2

232

31


B

A

A


MODEM

LAN 1 PDU 22

1

B


100-240V~, 1.25A , 50/60 Hz

10G

be

SF

P+

2


PowerConnect 3548Management Switch

ACL VLAN

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

FDX

/HDX

LNK

/ACT

DIAG

STACK ID

G1 G2 G4G3

LNK/ACT FDX/HDX

MASTER RPS PWR

FAN TEMP

RESET

SELECT

LNK

/ACT

LNK

/ACT

STACKING

PORT

1 2 3 4

5 6 7 8

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

FDX

/HDX

LNK

/ACT

DIAG

STACK ID

G1 G2 G4G3

LNK/ACT FDX/HDX

MASTER RPS PWR

FAN TEMP

RESET

SELECT

LNK

/ACT

LNK

/ACT

STACKING

PORT

1 2 3 4

5 6 7 8

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

ManagementVLAN

40GB LAG

10GB Link

10GB iSCSI

10GB Trunk

1GB iSCSI

16GB FC

1Gbe Mgmt.

Jay Etchings, Solutions Architect, Dell Enterprise H-RACK-E Power Peak Pwr Typical BTU Peak BTU Typ Max Pot.

Totals 6277 4915 21404.57 16760.15 7890

VLT Layer 2 Network

Transactional Shared Storage

0202

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

0202

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

VLAN-000 0.0.0.0/0

VLAN-000 0.0.0.0/0HPCC

VLAN-531 / 10.128.42.0/23 VLAN-230 10.128.230.0/24

Hadoop Cluster

32

31


B

A

A


MODEM

LAN 1 PDU 22

1

B


100-240V~, 1.25A , 50/60 Hz

E S T

PowerEdge R620 6

7

5

3

4

2

1

0

E S T

PowerEdge R620 6

7

5

3

4

2

1

0

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

FDX

/HDX

LNK

/ACT

DIAG

STACK ID

G1 G2 G4G3

LNK/ACT FDX/HDX

MASTER RPS PWR

FAN TEMP

RESET

SELECT

LNK

/ACT

LNK

/ACT

STACKING

PORT

1 2 3 4

5 6 7 8

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

FDX

/HDX

LNK

/ACT

DIAG

STACK ID

G1 G2 G4G3

LNK/ACT FDX/HDX

MASTER RPS PWR

FAN TEMP

RESET

SELECT

LNK

/ACT

LNK

/ACT

STACKING

PORT

1 2 3 4

5 6 7 8

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

QSFP+

MASTER

SYSFANPSU

RS-232

ETHERNET LNKACT 46 44 42 40 38 36 34 32 30 28 26 24 22 20 18 16 14 12 10 8 6 4 2 04856

5260

SFP+

Force10 S4810P

QSFP+

MASTER

SYSFANPSU

RS-232

ETHERNET LNKACT 46 44 42 40 38 36 34 32 30 28 26 24 22 20 18 16 14 12 10 8 6 4 2 04856

5260

SFP+

Force10 S4810P

Ne

xt G

en

era

tion

Co

mp

ute

Clu

ste

r -Ha

do

op

/ Big

Da

ta

V.1

.5a

/ Pa

ge

8H

ad

oo

p N

etw

ork

To

po

log

y / C

om

ple

te

09

/06

/20

13

Ja

y E

tch

ing

s, S

olu

tion

s A

rch

itect, D

ell E

nte

rpris

e

QSFP+

MASTER

SYSFANPSU

RS-232

ETHERNET LNKACT 46 44 42 40 38 36 34 32 30 28 26 24 22 20 18 16 14 12 10 8 6 4 2 04856

5260

SFP+

Force10 S4810P

QSFP+

MASTER

SYSFANPSU

RS-232

ETHERNET LNKACT 46 44 42 40 38 36 34 32 30 28 26 24 22 20 18 16 14 12 10 8 6 4 2 04856

5260

SFP+

Force10 S4810P

QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

RS

-232

PSU0SYS ALM

LOC

IPMI ET

H

RESET PSU1Master

STACK ID USB-A

USB-B

SSD

60 61 62 6352 53 54 55 56 57 58 5948 49 50 5144 45 46 4736 37 38 39 40 41 42 4332 33 34 3528 29 30 3124 25 26 2720 21 22 2312 13 14 15 16 17 18 198 9 10 114 5 6 70 1 2 3

124 125 126 127116 117 118 119 120 121 122 123112 113 114 115108 109 110 111100 101 102 103 104 105 106 10796 97 98 9992 93 94 9588 89 90 9184 85 86 8776 77 78 79 80 81 82 8372 73 74 7568 69 70 7164 65 66 67

Force10 Z9000

RS

-232

PSU0SYS ALM

LOC

IPMI ET

H

RESET PSU1Master

STACK ID USB-A

USB-B

SSD

60 61 62 6352 53 54 55 56 57 58 5948 49 50 5144 45 46 4736 37 38 39 40 41 42 4332 33 34 3528 29 30 3124 25 26 2720 21 22 2312 13 14 15 16 17 18 198 9 10 114 5 6 70 1 2 3

124 125 126 127116 117 118 119 120 121 122 123112 113 114 115108 109 110 111100 101 102 103 104 105 106 10796 97 98 9992 93 94 9588 89 90 9184 85 86 8776 77 78 79 80 81 82 8372 73 74 7568 69 70 7164 65 66 67

Force10 Z9000

VLT 4VLT 2

ASM

PowerEdge M1000e

9 101 2

11 123 4

13 145 6

15 167 8

0

1

00

1

0 0

1

00

1

0 0

1

00

1

0 0

1

00

1

0

0

1

00

1

0 0

1

00

1

0 0

1

00

1

0 0

1

00

1

0

EST

EST

PowerEdge M1000e

9 101 2

11 123 4

13 145 6

15 167 8

0

1

00

1

0 0

1

00

1

0 0

1

00

1

0 0

1

00

1

0

0

1

00

1

0 0

1

00

1

0 0

1

00

1

0 0

1

00

1

0

EST

EST

RACK ERACK CRACK BRACK A

40GB LAG

10GB Link

10GB iSCSI

10GB Trunk

1GB iSCSI

16GB FC

1Gbe Mgmt.

40G 40G 40G 40G

Management

QSFP+

MAST

ER

SYS

FAN

PSU

RS-232

ETHERNET

LNK

ACT

46

44

42

40

38

36

34

32

30

28

26

24

22

20

18

16

14

12

10

8

6

4

2

0

48

56

52

60

SFP+

Force10 S4810P

VLT 5

0202

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

0202

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

40G

VLAN-000 0.0.0.0/0

PowerEdge M1000e

9 101 2

11 123 4

13 145 6

15 167 8

0

1

00

1

0 0

1

00

1

0 0

1

00

1

0 0

1

00

1

0

0

1

00

1

0 0

1

00

1

0 0

1

00

1

0 0

1

00

1

0

40G40G

Public Cloud Space(INTERNET 2 READY)

Router

IO DataCenterPrimary Storage

CONSOLE

33-3637-40

LNKACT

LNKACT

Force10 MXL 10/40GbE

41-48

49-56

CONSOLE

33-3637-40

LNKACT

LNKACT


41-48

49-56

VLT 6

Ja

y E

tch

ing

s, S

olu

tio

ns A

rch

ite

ct, D

ell

En

terp

rise

QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

VLT 1 VLT 3

VLT 5

High Performance

High Performance systems are necessary to address computationally intensive tasks. These can be broken into two subclasses tasks which large number of computations and those that require large memory stores. Each requires a different physical infrastructure. The first configuration supports embarrassingly parallel tasks that are either naturally parallel or have specialized code to utilize parallel computing. This configuration needs a large collection of compute node/cores connected via high speed internal connections with fast access storage space to read/write information and to store intermediate information.

The standing infrastructure to support this need would be 100 node/ 800 core cluster with 32GB RAM/node connected via Infinaband QDR interconnects. The cluster would have 250TB of fast scratch storage space in a Lustre file system. High speed network connection to the transactional infrastructure is required.

The second High Performance configuration requires large shared memory. This environment processes jobs with high complexity/interdependencies requiring processing of a common data space, or code that isnt or cant be straightforwardly parallelized.

The infrastructure to support this component is composed of a 32 core (4 x 8 core) symmetric multiprocessor system with 2TB RAM, and 2 x 10GE with iSCSI offload. High speed network connection to the transactional infrastructure is required.

Reserved Space

HSS Lustre Expansion

10U

Reserved Space for NSS NFS

Appliance Expansion 6U

100V-240VAC, 1.6A

LOCATOR

RPS

DC IN 12V,15A

01

02

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

01

02

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

01

02

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

01

02

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

01

02

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

01

02

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

100V-240VAC, 1.5A

LOCATOR

RPS

DC IN 12V,15A

AC OK

100-240V ~ 6A50 60Hz

100-240V ~ 6A50 60Hz

AC OK

PSU0

PSU1

LOCATOR

100V-240VAC, 8A

MPS

DC IN 48V,12.5A

8

9

10

4

5

6

12

13

14

(1) R910 SMP 2TB RAM

PowerEdge M1000e

9 101 2

11 123 4

13 145 6

15 167 8

0

1

00

1

0 0

1

00

1

0 0

1

00

1

0 0

1

00

1

0

0

1

00

1

0 0

1

00

1

0 0

1

00

1

0 0

1

00

1

0

PowerEdge M1000e

9 101 2

11 123 4

13 145 6

15 167 8

0

1

00

1

0 0

1

00

1

0 0

1

00

1

0 0

1

00

1

0

0

1

00

1

0 0

1

00

1

0 0

1

00

1

0 0

1

00

1

0

100V-240VAC, 1.6A

LOCATOR

RPS

DC IN 12V,15A

100V-240VAC, 1.6A

LOCATOR

RPS

DC IN 12V,15A

0

21

3

54

6

87

9

1110

Front

0

21

3

54

6

87

9

1110

Front

0

21

3

54

6

87

9

1110

Front

0

21

3

54

6

87

9

1110

Front

0

21

3

54

6

87

9

1110

Front

0 1 2 3 4 5 6 7 8 9 10 11

0 1 2 3 4 5 6 7 8 9 10 11

0 1 2 3 4 5 6 7 8 9 10 11

0 1 2 3 4 5 6 7 8 9 10 11

0 1 2 3 4 5 6 7 8 9 10 11

0

21

3

54

6

87

9

1110

Front

0

21

3

54

6

87

9

1110

Front

0

21

3

54

6

87

9

1110

Front

0

21

3

54

6

87

9

1110

Front

0

21

3

54

6

87

9

1110

Front

0 1 2 3 4 5 6 7 8 9 10 11

0 1 2 3 4 5 6 7 8 9 10 11

0 1 2 3 4 5 6 7 8 9 10 11

0 1 2 3 4 5 6 7 8 9 10 11

0 1 2 3 4 5 6 7 8 9 10 11

0

21

3

54

6

87

9

1110

Front

0

21

3

54

6

87

9

1110

Front

0

21

3

54

6

87

9

1110

Front

0

21

3

54

6

87

9

1110

Front

0

21

3

54

6

87

9

1110

Front

0 1 2 3 4 5 6 7 8 9 10 11

0 1 2 3 4 5 6 7 8 9 10 11

0 1 2 3 4 5 6 7 8 9 10 11

0 1 2 3 4 5 6 7 8 9 10 11

0 1 2 3 4 5 6 7 8 9 10 11

LOCATOR

100V-240VAC, 8A

MPS

DC IN 48V,12.5A

PowerEdge M1000e

9 101 2

11 123 4

13 145 6

15 167 8

0

1

00

1

0 0

1

00

1

0 0

1

00

1

0 0

1

00

1

0

0

1

00

1

0 0

1

00

1

0 0

1

00

1

0 0

1

00

1

0

PowerEdge M1000e

9 101 2

11 123 4

13 145 6

15 167 8

0

1

00

1

0 0

1

00

1

0 0

1

00

1

0 0

1

00

1

0

0

1

00

1

0 0

1

00

1

0 0

1

00

1

0 0

1

00

1

0

Ne

xt G

en

era

tion

Co

mp

ute

Clu

ste

r -HP

CC

V.1

.5a

/ Pa

ge

10

HP

CC

Ra

ck D

esig

n0

9/0

6/2

01

3T

om

Ra

iso

r, HP

C S

olu

tion

s A

rch

itect, D

ell R

ese

arc

h C

om

pu

ting

(2) R620 + MD3260 NSS 4.5 (1) R210 TERASCALA HSS Mgmt. Node(1) PCT 5548 HSS4.5 Mgmt. Switch(1) R320 CIFS Gateway(3) R320 ISB Storage Bridge ServersIntelligent Storage Bridge(1) R620 TERASCALA MetaData Servers MD3220 StorageOSS Pair / MD32XX (HSS Expansion)(1) R620 TERASCALA OSS MD3260 Storage(2) R720 GPU Node w/ 2x NVIDIA K20(2) R620 Head Nodes(1) R620 Login Node/ KMM(4) M1000e(16) M620 Compute Nodes(256 cores/Chassis)(2) Force10 MXL Switch(1) M4001T IB(2) CMC

AC OK

100-240V ~ 6A50 60Hz

100-240V ~ 6A50 60Hz

AC OK

PSU0

PSU1

Reserved Space

NSS NFS Appliance 6U

Expansion

Reserved Space

HSS Lustre Expansion

10U

Ne

xt G

en

era

tion

Co

mp

ute

Clu

ste

r -HP

CC

V.1

.5a

/ Pa

ge

10

HP

CC

Ra

ck D

esig

n0

9/0

6/2

01

3T

om

Ra

iso

r, HP

C S

olu

tion

s A

rch

itect, D

ell R

ese

arc

h C

om

pu

ting

01

02

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

01

02

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

01

02

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

01

02

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

01

02

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

01

02

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

B2C2 A2B1 C1A1

4

1

7

5

2

8

6

3

9

CMC2CMC1 KVM

1 2 3 4 5 6

GbGb 21

CMC

iKVM

GbGb 21

CMC

B1 C1A1 B2C2 A2

B2C2 A2B1 C1A1

4

1

7

5

2

8

6

3

9

CMC2CMC1 KVM

1 2 3 4 5 6

GbGb 21

CMC

iKVM

GbGb 21

CMC

B1 C1A1 B2C2 A2

iDRAC

iDRAC

iDRAC

CONSOLE

33-3637-40

LNKACT

LNKACT


41-48

49-56

CONSOLE

33-3637-40

LNKACT

LNKACT


41-48

49-56

LNK ACT

Stacking HDMI1 2 SFP+1 2

LNK ACT LNK ACT

Reset

LNK ACT

Stack No.

M RPSFan

PW

R

Status

26 28 30 32 34 36 38 40 42 44 46 48

25 27 29 31 33 35 37 39 41 43 45 47

2 4 6 8 10 12 14 16 18 20 22 24

1 3 5 7 9 11 13 15 17 19 21 23

QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

6 5 4 3 2 1

Gb 1

Gb 2

Gb 3

Gb 4

10Gb

10Gb

6 5 4 3 2 1

7

4

3

2

1

B2C2 A2B1 C1A1

4

1

7

5

2

8

6

3

9

CMC2CMC1 KVM

1 2 3 4 5 6

GbGb 21

CMC

iKVM

GbGb 21

CMC

B1 C1A1 B2C2 A2

CONSOLE

33-3637-40

LNKACT

LNKACT


41-48

49-56

B2C2 A2B1 C1A1

4

1

7

5

2

8

6

3

9

CMC2CMC1 KVM

1 2 3 4 5 6

GbGb 21

CMC

iKVM

GbGb 21

CMC

B1 C1A1 B2C2 A2

CONSOLE

33-3637-40

LNKACT

LNKACT


41-48

49-56

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

STATUS

PSU 1

PSU 2

FAN

RST

3433

3231

3635

2827

2625

3029

2221

2019

2423

1615

1413

1817

109

87

1211

43

21

65

IS 5025CONSOLE

STATUS

PSU 1

PSU 2

FAN

RST

3433

3231

3635

2827

2625

3029

2221

2019

2423

1615

1413

1817

109

87

1211

43

21

65

IS 5025CONSOLE

STATUS

PSU 1

PSU 2

FAN

RST

3433

3231

3635

2827

2625

3029

2221

2019

2423

1615

1413

1817

109

87

1211

43

21

65

IS 5025CONSOLE

LNK ACT


LNK ACT LNK ACT

Reset

LNK ACT

Stack No.

M RPSFan

PW

R

Status

2 4 6 8 10 12 14 16 18 20 22 24

1 3 5 7 9 11 13 15 17 19 21 23

LNK ACT


LNK ACT LNK ACT

Reset

LNK ACT

Stack No.

M RPSFan

PW

R

Status

26 28 30 32 34 36 38 40 42 44 46 48

25 27 29 31 33 35 37 39 41 43 45 47

2 4 6 8 10 12 14 16 18 20 22 24

1 3 5 7 9 11 13 15 17 19 21 23

2

1

2

1

DC AC

DC AC

PSU A

PSU B

SBB A

SBB B

SBB B

SBB A

PSU B

PSU A PSU A

PSU B

SBB A

SBB B

SBB B

SBB A

PSU B

PSU A

SAS IN 32

SAS IN 10 SAS OUT

SAS IN 32

SAS IN 10 SAS OUT

2

1

2

1

DC AC

DC AC

PSU A

PSU B

SBB A

SBB B

SBB B

SBB A

PSU B

PSU A PSU A

PSU B

SBB A

SBB B

SBB B

SBB A

PSU B

PSU A

SAS IN 32

SAS IN 10 SAS OUT

SAS IN 32

SAS IN 10 SAS OUT

iDRAC

iDRAC

iDRAC

iDRAC

iDRAC

Gb 2Gb 1eSATA

MEST

iDRAC

iDRAC

2

1

2

1

DC AC

DC AC

PSU A

PSU B

SBB A

SBB B

SBB B

SBB A

PSU B

PSU A PSU A

PSU B

SBB A

SBB B

SBB B

SBB A

PSU B

PSU A

SAS IN 32

SAS IN 10 SAS OUT

SAS IN 32

SAS IN 10 SAS OUT

MD32 Series

PPID

SAS OUT

SAS IN0 1

SAS IN2 3

MD32 Series

PPID

SAS OUT

SAS IN0 1

SAS IN2 3

MGMT

MGMT

7.1kW Peak5.7kW Typ

15.1kW Peak12.1kW Typ

13.2kW Peak10.5kW Typ

(2) Force10 S4810 10Gbe Core(3) Mellanox FDR Switches(2) Spine (1) Storage(1) PCT 5548 1Gbe/IPMI/Mgmt.(8) Managed PDU L6-30

QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

iDRAC

iDRAC

iDRAC

Public Cloud Space

HPC Shared Memory

HPCC Topology

(1) PCT 5548

Gbe / IPMI / Mgt

18 initial, 30 for expansion

APC Mgd PDU (2)

NSS 4.5 (2) R620 + MD3260

(1) Login Node - R620

(2) Head Nodes R620

HPC Parallel

FDR IB Spine

(2) Mellanox 36 port

40 ports initially

populated,

32 left for expansion

(4 more blade chassis)

(1) FDR IB Leaf Switch

Storage and Admin

18 ports initially populated

18 free for expansion

Ne

xt G

en

era

tion

Co

mp

ute

Clu

ste

r -HP

CC

V.1

.5a

/ Pa

ge

11

HP

CC

/ NG

CC

To

po

log

y0

9/0

6/2

01

3T

om

Ra

iso

r, HP

C S

olu

tion

s A

rch

itect, D

ell R

ese

arc

h C

om

pu

ting LNK ACT


LNK ACT LNK ACT

Reset

LNK ACT

Stack No.

M RPSFan

PW

R

Status

2 4 6 8 10 12 14 16 18 20 22 24

1 3 5 7 9 11 13 15 17 19 21 23

4

LNK ACT


LNK ACT LNK ACT

Reset

LNK ACT

Stack No.

M RPSFan

PW

R

Status

26 28 30 32 34 36 38 40 42 44 46 48

25 27 29 31 33 35 37 39 41 43 45 47

2 4 6 8 10 12 14 16 18 20 22 24

1 3 5 7 9 11 13 15 17 19 21 23

B2C2 A2B1 C1A1

4

1

7

5

2

8

6

3

9

CMC2CMC1 KVM

1 2 3 4 5 6

GbGb 21

CMC

iKVM

GbGb 21

CMC

B1 C1A1 B2C2 A2

2

1

2

1

DC AC

DC AC

PSU A

PSU B

SBB A

SBB B

SBB B

SBB A

PSU B

PSU A PSU A

PSU B

SBB A

SBB B

SBB B

SBB A

PSU B

PSU A

SAS IN 32

SAS IN 10 SAS OUT

SAS IN 32

SAS IN 10 SAS OUT

2

1

2

1

DC AC

DC AC

PSU A

PSU B

SBB A

SBB B

SBB B

SBB A

PSU B

PSU A PSU A

PSU B

SBB A

SBB B

SBB B

SBB A

PSU B

PSU A

1 2

LnkLnkLnkLnk

ID/Diag

Lnk Lnk

1 2

LnkLnkLnkLnk

ID/Diag

Lnk Lnk

iDRAC

iDRAC

iDRAC

iDRAC

2

1

2

1

DC AC

DC AC

PSU A

PSU B

SBB A

SBB B

SBB B

SBB A

PSU B

PSU A PSU A

PSU B

SBB A

SBB B

SBB B

SBB A

PSU B

PSU A

SAS IN 32

SAS IN 10 SAS OUT

SAS IN 32

SAS IN 10 SAS OUT

1

1 2 3 4

iDRAC

1 2750W 750W

1

1 2 3 4

iDRAC

1 2750W 750W

1

1 2 3 4

iDRAC

1 2750W 750W

1

1 2 3 4

iDRAC

1 2750W 750W

1

1 2 3 4

iDRAC

1 2750W 750W

1

1 2 3 4

iDRAC

1 2750W 750W

MD32 Series

PPID

SAS OUT

SAS IN0 1

SAS IN2 3

MD32 Series

PPID

SAS OUT

SAS IN0 1

SAS IN2 3

MGMT

MGMT

LNK ACT


LNK ACT LNK ACT

Reset

LNK ACT

Stack No.

M RPSFan

PW

R

Status

26 28 30 32 34 36 38 40 42 44 46 48

25 27 29 31 33 35 37 39 41 43 45 47

2 4 6 8 10 12 14 16 18 20 22 24

1 3 5 7 9 11 13 15 17 19 21 23

STATUS

PSU 1

PSU 2

FAN

RST

3433

3231

3635

2827

2625

3029

2221

2019

2423

1615

1413

1817

109

87

1211

43

21

65

IS 5025CONSOLE

8

3 2

2

1

2

44

4

4

2

2

2

2

1

4

2

2

QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

1

Gb 2Gb 1eSATA

MEST

2

CONSOLE

33-3637-40

LNKACT

LNKACT


41-48

49-56

16

2

6 5 4 3 2 1

Gb 1

Gb 2

Gb 3

Gb 4

10Gb

10Gb

6 5 4 3 2 1

7

4

3

2

1

2

1 2 3 4

ST

21

iDRAC

2

1

3750W750W

5

4

7

6

2

1

2

STATUS

PSU 1

PSU 2

FAN

RST

3433

3231

3635

2827

2625

3029

2221

2019

2423

1615

1413

1817

109

87

1211

43

21

65

IS 5025CONSOLE

STATUS

PSU 1

PSU 2

FAN

RST

3433

3231

3635

2827

2625

3029

2221

2019

2423

1615

1413

1817

109

87

1211

43

21

65

IS 5025CONSOLE

8

1

1

(4) M1000E each with:

(16) m620 Compute Nodes

(1) Force10 MXL Ethernet Switches

(1) M4001F IB (8 uplinks per chassis)

(2) CMC

(2) R620

Terascala OSS and Storage

(4) R320

Terascala Appliance

CIFS Gateway/Intelligent Storage Bridge

Terascala HSS Mgt Node

(1) R210

(1) PCT 5524

NSS Storage Mgmt.

Switch / HA

(1) PCT 5548

HSS Storage Mgmt.

Switch / HA

(2) GPU Compute Nodes R720each w/2x NVIDIA K20

(2) R620

Terascala MetaData Servers + MD3220

(1) SMP 2TB RAM R910

(1) Force10 4810 10Gb Core

26 initial, 22 for expansion

VLT Layer 2 Network

iDRAC

iDRAC

iDRAC

iDRAC

40G

SW SM Option I2

Transactional

Transactional systems are the central component of the NGCC. They represent the general day-to-day utility platforms and are used for tasks such as routine computing, hosting services (including web servers, database servers, applications), hosting data, and supporting middleware.

The standing physical infrastructure to support utility computing would be comprised of a server cluster composed of 20 commodity duel quad core servers, each with 32 GB of memory. The Transactional systems would have 1 petabyte of RAID configured 10GE iSCSI Network Storage to support large data resources (replicated to 2 PB). Network gear would include VPN concentrator, gateway, switches, routers, and NICs. This component requires access to high speed internet (Internet 2).

Internet 2 Public Cloud VLT Layer 2 Network

SAS/SATA

PCIeSSD

SAS/SATA

PCIeSSD

SAS/SATA

PCIeSSD

SAS/SATA

PCIeSSD

0202

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

PowerEdge M1000e

9 101 2

11 123 4

13 145 6

15 167 8

M1000e Chassis20 M420 Blade Servers

Intel Xeon Processor E5-2430

(15M Cache, 2.20 GHz, 7.20 GT/s Intel QPI)

(40S, 240C, 640Thread)

32GB DDR3 Memory

2*200GB SSD RAID 0

2 16GB FC Controllers for SAN

Brocade 6505 FC Fabric 16G

2 Force 10 MXL 10/40Gbe

Dual Flex IO Modules (iSCSI)

64 Internal 10 Ports

Bright Cluster Manager

CMC/OM/iDRAC

ASM Integration

Dual SonicWall SM10K (Option I2)

(Unassigned Rack Location)

Transactional ClusterNGCC RS-23

2

PSU0SYS ALM

LOC

IPMI ET

H

RESET PSU1Master

STACK ID USB-A

USB-B

SSD

60 61 62 6352 53 54 55 56 57 58 5948 49 50 5144 45 46 4736 37 38 39 40 41 42 4332 33 34 3528 29 30 3124 25 26 2720 21 22 2312 13 14 15 16 17 18 198 9 10 114 5 6 70 1 2 3

124 125 126 127116 117 118 119 120 121 122 123112 113 114 115108 109 110 111100 101 102 103 104 105 106 10796 97 98 9992 93 94 9588 89 90 9184 85 86 8776 77 78 79 80 81 82 8372 73 74 7568 69 70 7164 65 66 67

Force10 Z9000

RS

-232

PSU0SYS ALM

LOC

IPMI ET

H

RESET PSU1Master

STACK ID USB-A

USB-B

SSD

60 61 62 6352 53 54 55 56 57 58 5948 49 50 5144 45 46 4736 37 38 39 40 41 42 4332 33 34 3528 29 30 3124 25 26 2720 21 22 2312 13 14 15 16 17 18 198 9 10 114 5 6 70 1 2 3

124 125 126 127116 117 118 119 120 121 122 123112 113 114 115108 109 110 111100 101 102 103 104 105 106 10796 97 98 9992 93 94 9588 89 90 9184 85 86 8776 77 78 79 80 81 82 8372 73 74 7568 69 70 7164 65 66 67

Force10 Z9000

Ne

xt G

en

era

tion

Co

mp

ute

Clu

ste

r

V.1

.5a

/ Pa

ge

13

NG

CC

Co

mp

uta

tion

al B

righ

t Co

mp

utin

g C

luste

r0

9/0

6/2

01

3Ja

y E

tch

ing

s, S

olu

tion

s A

rch

itect, D

ell E

nte

rpris

e

B2C2 A2B1 C1A1

4

1

7

5

2

8

6

3

9

CMC2CMC1 KVM

1 2 3 4 5 6

GbGb 21

CMC

iKVM

GbGb 21

CMC

B1 C1A1 B2C2 A2

CONSOLE

33-3637-40

LNKACT

LNKACT


41-48

49-56

CONSOLE

33-3637-40

LNKACT

LNKACT


41-48

49-56

0

23

22

21

20

19

18

17

8Gbps FC

Brocade M5424

0

23

22

21

20

19

18

17

8Gbps FC

Brocade M5424

Ma

na

ge

me

nt

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

FDX

/HDX

LNK

/ACT

DIAG

STACK ID

G1 G2 G4G3

LNK/ACT FDX/HDX

MASTER RPS PWR

FAN TEMP

RESET

SELECT

LNK

/ACT

LNK

/ACT

STACKING

PORT

1 2 3 4

5 6 7 8

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

FDX

/HDX

LNK

/ACT

DIAG

STACK ID

G1 G2 G4G3

LNK/ACT FDX/HDX

MASTER RPS PWR

FAN TEMP

RESET

SELECT

LNK

/ACT

LNK

/ACT

STACKING

PORT

1 2 3 4

5 6 7 8

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

0202

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

0202

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

QSFP+

MASTER

SYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

PO

RT

S

PCIe x8

8Gb FC

8 4 2

8 4 2

8 4 2

8 4 2

12

34

PO

RT

S

PCIe x8

8Gb FC

8 4 2

8 4 2

8 4 2

8 4 2

12

34

40GB LAG

10GB Link

10GB iSCSI

10GB Trunk

1GB iSCSI

16GB FC

1Gbe Mgmt.

Storage Zone

Management Zone

Fabric B

4-16GB F

C

Fabric A4-40GBe


E S T

PowerEdge R620 6

7

5

3

4

2

1

0

E S T

PowerEdge R620 6

7

5

3

4

2

1

0

ASM

Replication Zone

01

23

45

67

89

1011

1213

1415

01

23

54

GE

FC

GE

1S

TA

T0

ST

AT

Bro

ca

de

78

00

01

23

45

67

89

1011

1213

1415

01

23

54

GE

FC

GE

1S

TA

T0

ST

AT

Bro

ca

de

78

00

FCIP Bridge

STORAGE FABRIC B

Tra

nsa

ctio

na

l

STORAGE FABRIC A

Du

al 6

GB

SA

S C

on

ne

cto

rs

Ne

xt G

en

era

tion

Co

mp

ute

Clu

ste

r

V.1

.5a

/ Pa

ge

14

Co

mp

elle

nt H

igh

Ava

ilab

ility S

tora

ge

with

Re

plic

atio

n0

9/0

6/2

01

3Ja

y E

tch

ing

s, S

olu

tion

s A

rch

itect, D

ell E

nte

rpris

e

0 1 2 34 5 6 7 8 9 10 11 12 13 14 15 0 1 2 3 54GEFC

GE

1ST AT

0ST ATBrocade 7800

0 1 2 34 5 6 7 8 9 10 11 12 13 14 15 0 1 2 3 54GEFC

GE

1ST AT

0ST ATBrocade 7800

Brocade 78F FCIP Bridge Brocade 78F FCIP Bridge

02

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

02

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

Technology

Next Gen Cluster Draft Hardware Schema