Agenda - University of California, Berkeleyinst.eecs.berkeley.edu/~cs61c/sp11/lectures/27LecSp11RAIDx6.pdf · 4/27/11 4 RAID&Summary& • LogicalPtoPphysical&block&mapping,&parity&striping,&

4/27/11

1

CS 61C: Great Ideas in Computer Architecture (Machine Structures)

Redundant Arrays of Inexpensive Disks Instructors: Randy H. Katz

David A. PaHerson hHp://inst.eecs.Berkeley.edu/~cs61c/fa10

1 Spring 2011 -‐-‐ Lecture #27 4/27/11 4/27/11 Spring 2011 -‐-‐ Lecture #27 2

Agenda

•  RAID •  Administrivia

•  Course Summary (Randy)

•  Cal Culture (Dave) •  Course EvaluaTon

4/27/11 Spring 2011 -‐-‐ Lecture #27 3

Agenda




4/27/11 Spring 2011 -‐-‐ Lecture #27 4

EvoluTon of the Disk Drive

4/27/11 Spring 2011 -‐-‐ Lecture #27 5 IBM RAMAC 305, 1956

IBM 3390K, 1986

Apple SCSI, 1986

Can smaller disks be used to close gap in performance between disks and CPUs?

Arrays of Small Disks

4/27/11 Spring 2011 -‐-‐ Lecture #27 6

14” 10” 5.25” 3.5”

3.5”

Disk Array: 1 disk design

ConvenTonal: 4 disk designs

Low End High End

4/27/11

2

Replace Small Number of Large Disks with Large Number of Small Disks! (1988 Disks)

4/27/11 Spring 2011 -‐-‐ Lecture #27 7

Capacity Volume

Power Data Rate I/O Rate

MTTF Cost

IBM 3390K 20 GBytes

97 cu. e. 3 KW 15 MB/s

600 I/Os/s 250 KHrs $250K

IBM 3.5" 0061 320 MBytes

0.1 cu. e. 11 W 1.5 MB/s

55 I/Os/s 50 KHrs $2K

x70 23 GBytes

11 cu. e. 1 KW 120 MB/s

3900 IOs/s ??? Hrs $150K

Disk Arrays have potenTal for large data and I/O rates, high MB per cu. e., high MB per KW, but what about reliability?

9X

3X

8X

6X

RAID: Redundant Arrays of (Inexpensive) Disks

•  Files are "striped" across mulTple disks •  Redundancy yields high data availability – Availability: service sTll provided to user, even if some components failed

•  Disks will sTll fail •  Contents reconstructed from data redundantly stored in the array ⇒ Capacity penalty to store redundant info ⇒ Bandwidth penalty to update redundant info

4/27/11 Spring 2011 -‐-‐ Lecture #27 8

Redundant Arrays of Inexpensive Disks RAID 1: Disk Mirroring/Shadowing

4/27/11 Spring 2011 -‐-‐ Lecture #27 9

• Each disk is fully duplicated onto its “mirror” Very high availability can be achieved • Bandwidth sacrifice on write: Logical write = two physical writes

Reads may be opTmized • Most expensive soluTon: 100% capacity overhead

recovery group

Redundant Array of Inexpensive Disks RAID 3: Parity Disk

4/27/11 Spring 2011 -‐-‐ Lecture #27 10

P

10010011 11001101 10010011 . . .

logical record 1 0 1 0 0 0 1 1

1 1 0 0 1 1 0 1

1 0 1 0 0 0 1 1

1 1 0 0 1 1 0 1

P contains sum of other disks per stripe mod 2 (“parity”) If disk fails, subtract P from sum of other disks to find missing informaTon

Striped physical records

Redundant Arrays of Inexpensive Disks RAID 4: High I/O Rate Parity

D0 D1 D2 D3 P

D4 D5 D6 P D7

D8 D9 P D10 D11

D12 P D13 D14 D15

P D16 D17 D18 D19

D20 D21 D22 D23 P

."

."

."

."

."

."

."

."

."

."

."

."

."

."

."Disk Columns

Increasing Logical Disk Address

Stripe

Insides of 5 disks

Example: small read D0 & D5, large write D12-‐D15

4/27/11 11 Spring 2011 -‐-‐ Lecture #27

InspiraTon for RAID 5

•  RAID 4 works well for small reads •  Small writes (write to one disk): – OpTon 1: read other data disks, create new sum and write to Parity Disk

– OpTon 2: since P has old sum, compare old data to new data, add the difference to P

•  Small writes are limited by Parity Disk: Write to D0, D5 both also write to P disk

4/27/11 Spring 2011 -‐-‐ Lecture #27 12

D0 D1 D2 D3 P

D4 D5 D6 P D7

4/27/11

3

RAID 5: High I/O Rate Interleaved Parity

4/27/11 Spring 2011 -‐-‐ Lecture #27 13

Independent writes possible because of interleaved parity

D0 D1 D2 D3 P

D4 D5 D6 P D7

D8 D9 P D10 D11

D12 P D13 D14 D15

P D16 D17 D18 D19

D20 D21 D22 D23 P

.

.

.

.

.

.

.

.

.

.

.

.

.

.

. Disk Columns

Increasing Logical Disk Addresses

Example: write to D0, D5 uses disks 0, 1, 3, 4

Problems of Disk Arrays: Small Writes

D0 D1 D2 D3 P D0'

+

+

D0' D1 D2 D3 P'

new data

old data

old parity

XOR

XOR

(1. Read) (2. Read)

(3. Write) (4. Write)

RAID-‐5: Small Write Algorithm

1 Logical Write = 2 Physical Reads + 2 Physical Writes

4/27/11 14 Spring 2011 -‐-‐ Lecture #27

Tech Report Read ‘Round the World (December 1987)

4/27/11 Spring 2011 -‐-‐ Lecture #27 15

RAID-‐I

• RAID-‐I (1989) – Consisted of a Sun 4/280 workstaTon with 128 MB of DRAM, four dual-‐string SCSI controllers, 28 5.25-‐inch SCSI disks and specialized disk striping soeware

4/27/11 Spring 2011 -‐-‐ Lecture #27 16

RAID II

•  1990-‐1993 •  Early Network AHached

Storage (NAS) System running a Log Structured File System (LFS)

•  Impact: –  $25 Billion/year in 2002 –  Over $150 Billion in RAID

device sold since 1990-‐2002 –  200+ RAID companies (at the

peak) –  Soeware RAID a standard

component of modern OSs

4/27/11 17 Spring 2011 -‐-‐ Lecture #27

RAID II

4/27/11 Spring 2011 -‐-‐ Lecture #27 18

4/27/11

4

RAID Summary

•  Logical-‐to-‐physical block mapping, parity striping, read-‐modify-‐write processing

•  Embedded caches and orchestraTng data staging between network interfaces, parity hardware, and file server interfaces

•  Failed disk replacement, hot spares, background copies and backup

•  Embedded log-‐structured file systems, compression on the fly

•  Soeware complexity dominates hardware!

4/27/11 Spring 2011 -‐-‐ Lecture #27 19

Agenda




4/27/11 Spring 2011 -‐-‐ Lecture #27 20

Administrivia

•  Final Review: Mon 5/2, 5 – 8PM, 2050 VLSB •  Final Exam: Mon 5/9, 11:30-‐2:30PM, 100 Haas Pavilion – Designed for 90 minutes, you will have 3 hours – Comprehensive (parTcularly problem areas on midterm), but focused on course since midterm: lecture, lab, hws, and projects are fair game

– 8 ½ inch x 11 inch crib sheet like midterm

4/27/11 21 Spring 2011 -‐-‐ Lecture #27

Some Survey Results

•  I felt the midterm was 9% Far too difficult

43% Somewhat harder than it should have been

46% Fair

1% A liHle too easy

1% Far too easy

4/27/11 Spring 2011 -‐-‐ Lecture #27 22

Some Survey Results

•  How much Tme per week to you spend on average in 61C (including lecture, discussion, and labs)?

18% <10 hours per week 26% 11-‐12 hours per week 13% 13-‐14 hours per week 24% 15-‐16 hours per week 10% 17-‐20 hours per week 10% >21 hours per week

4/27/11 Spring 2011 -‐-‐ Lecture #27 23

Berkeley guidelines 3 hours/unit

(EECS classes oeen > guidelines)

Some Survey Results

•  Rated as “Enjoyed and learned a lot”: Project#4: Processor Design in Logisim, Part 2 (73%) & Part 1 (60%)

Logisim Labs (58%)

C memory management lab (55%)

TLP and DLP Labs (52%)

Project#3: Matrix MulTply Performance Improvement, Part 2 (36%) & Part 1 (35%)

4/27/11 Spring 2011 -‐-‐ Lecture #27 24

4/27/11

5

Some Survey Results

•  Did 61C material come up in interviews for internships or jobs? 47% Yes –  “experience with cloud compuTng and Amazon EC2 on an internship with the Houston Rockets in a job involving data analysis”

–  “Matrix mulTply was impressive to а video game company. Amazon loved that we had worked with AWS.”

–  “I'm currently deciding between doing an internship at a startup in SF or research over this summer because of my experience with map reduce/hadoop.”

4/27/11 Spring 2011 -‐-‐ Lecture #27 25

Some Survey Results

•  Did 61C material come up in interviews for internships or jobs? 47% Yes – “Got a job thanks to Proj1 (EC2)” – “I've got a $30/hr summer job. (40hr/wk) :)” – “basically, project 1 got me a summer internship because I could talk about mapreduce, and show that I had experience w/ EC2. thanks! but strangely I was not asked about how to simulate MIPS instrucTons...”

4/27/11 Spring 2011 -‐-‐ Lecture #27 26

Agenda




4/27/11 Spring 2011 -‐-‐ Lecture #27 27

CS61c is NOT really about C Programming

•  It is about the hardware-‐soeware interface – What does the programmer need to know to achieve the highest possible performance

•  Languages like C are closer to the underlying hardware, unlike languages like Scheme! – Allows us to talk about key hardware features in higher level terms

– Allows programmer to explicitly harness underlying hardware parallelism for high performance

4/27/11 Spring 2011 -‐-‐ Lecture #27 28

Old School View of Soeware-‐Hardware Interface

CS61C

I/O system Processor

Compiler

OperaLng

System (Mac OSX)

ApplicaLon (ex: browser)

Digital Design

Circuit Design

InstrucLon Set Architecture

Datapath & Control

transistors

Memory Hardware

SoWware Assembler

4/27/11 29 Spring 2011 -‐-‐ Lecture #27

New-‐School Machine Structures (It’s a bit more complicated!)

•  Parallel Requests Assigned to computer e.g., Search “Katz”

•  Parallel Threads Assigned to core e.g., Lookup, Ads

•  Parallel InstrucTons >1 instrucTon @ one Tme e.g., 5 pipelined instrucTons

•  Parallel Data >1 data item @ one Tme e.g., Add of 4 pairs of words

•  Hardware descripTons All gates funcToning in

parallel at same Tme 4/27/11 Spring 2011 -‐-‐ Lecture #27 30

Smart Phone

Warehouse Scale

Computer

So7ware Hardware

Harness Parallelism & Achieve High Performance

Logic Gates

Core Core …

Memory (Cache)

Input/Output

Computer

Main Memory

Core

InstrucTon Unit(s) FuncTonal Unit(s)

A3+B3 A2+B2 A1+B1 A0+B0

Project 2

Project 1

Project 3

Project 4

4/27/11

6

6 Great Ideas in Computer Architecture

1.  Layers of RepresentaTon/InterpretaTon 2.  Moore’s Law

3.  Principle of Locality/Memory Hierarchy

4.  Parallelism

5.  Performance Measurement & Improvement

6.  Dependability via Redundancy

4/27/11 Spring 2011 -‐-‐ Lecture #27 31

Levels of RepresentaTon/InterpretaTon

lw $t0, 0($2) lw $t1, 4($2) sw $t1, 0($2) sw $t0, 4($2)

High Level Language Program (e.g., C)

Assembly Language Program (e.g., MIPS)

Machine Language Program (MIPS)

Hardware Architecture DescripLon (e.g., block diagrams)

Compiler

Assembler

Machine Interpreta?on

temp = v[k]; v[k] = v[k+1]; v[k+1] = temp;

0000 1001 1100 0110 1010 1111 0101 1000 1010 1111 0101 1000 0000 1001 1100 0110 1100 0110 1010 1111 0101 1000 0000 1001 0101 1000 0000 1001 1100 0110 1010 1111 !

Logic Circuit DescripLon (Circuit SchemaLc Diagrams)

Architecture Implementa?on

Anything can be represented as a number,

i.e., data or instrucTons

4/27/11 32 Spring 2011 -‐-‐ Lecture #27

Moore’s Law

4/27/11 Spring 2011 -‐-‐ Lecture #27 33

Predicts: 2X Transistors / chip every 2 years

Gordon Moore Intel Cofounder B.S. Cal 1950!

# of transistors on an

integrated circuit (IC)

Year

•  Take advantage of the principle of locality to present the user with as much memory as is available in the cheapest technology at the speed offered by the fastest technology

Second Level Cache (SRAM)

Typical Memory Hierarchy

Control

Datapath

Secondary Memory (Disk)

On-‐Chip Components

RegFile

Main Memory (DRAM) D

ata Cache

Instr Cache

ITLB DTLB

Speed (%cycles): ½’s 1’s 10’s 100’s 10,000’s

Size (bytes): 100’s 10K’s M’s G’s T’s

Cost: highest lowest

4/27/11 34 Spring 2011 -‐-‐ Lecture #27

Memory Hierarchy

•  Caches – 3Cs: Compulsory/Capacity/Conflict misses – Direct-‐mapped vs. Set-‐associaTve – MulT-‐level caches for fast clock + low miss rate

•  Virtual Memory – Originally small physical memory that appears to be very large

– Modern: provide isolaTon through separated address spaces

4/27/11 Spring 2011 -‐-‐ Lecture #27 35

Parallelism

4/27/11 Spring 2011 -‐-‐ Lecture #27 36

4/27/11

7

Forms of Parallelism

•  InstrucTon Parallelism – Processor pipeline: mulTple instrucTons in execuTon at the same Tme

•  Task Parallelism – SynchronizaTon primiTves, openMP – Modern web services

•  Data Parallelism – Map-‐Reduce, SIMD instrucTon set – Data and numerically intensive processing

4/27/11 Spring 2011 -‐-‐ Lecture #27 37

Performance Improvement

•  Matching applicaTon to underlying hardware to exploit: –  Locality –  Parallelism –  Special hardware features, like specialized instrucTons (e.g., matrix manipulaTon)

•  Latency – How long to set the problem up – How much faster does it execute once it gets going –  It is all about Bme to finish

4/27/11 Spring 2011 -‐-‐ Lecture #27 38

Spring 2011 -‐-‐ Lecture #27

Great Idea #6: Dependability via Redundancy

•  Applies to everything from datacenters to storage to memory –  Redundant datacenters so that can lose 1 datacenter but Internet service stays online

–  Redundant disks so that can lose 1 disk but not lose data (Redundant Arrays of Independent Disks/RAID)

–  Redundant memory bits of so that can lose 1 bit but no data (Error CorrecTng Code/ECC Memory)

4/27/11 39

Randy’s Course Summary

•  As the field changes, cs61c has to change too! •  It is sTll about the soeware-‐hardware interface –  Programming for performance! – Understanding the memory hierarchy and its impact on applicaTon performance

– Unlocking the capabiliTes of the architecture for performance •  MulTcore programming and task parallelism •  Special instrucTons •  Special instrucTon sequences

•  Thanks for being our 2nd semester guinea pigs! – Wait unTl you interview for summer internships and tell the interviewers what you did this semester!

4/27/11 Spring 2011 -‐-‐ Lecture #27 40

Agenda




4/27/11 Spring 2011 -‐-‐ Lecture #27 41

What to Emphasize about Cal culture?

•  Top public university (US News) •  Top graduate program in the world?

–  35/36 departments in top 10 •  University doing the most public good

–  Washington Monthly 2009! •  Faculty Awards?

–  8 current Nobel Prize winners (21 all Tme)! –  3 Turing Award winners (“Nobel of CS”)! –  3 Fields Medalists (“Nobel of Math”)! –  86 in NaTonal Academy of Engineering! –  135 in NaTonal Academy of Science! –  28 “Genius” awards (MacArthur fellows)Source: hHp://

www.berkeley.edu/about/honors/

4/27/11 42 Spring 2011 -‐-‐ Lecture #27

4/27/11

8

Cal Cultural History: Football!

•  Started with “soccer” (aka football) –  11 on a team, 2 teams, 1 ball, on a field; object is to move ball

into “goal”; most goals wins. No hands! •  New World changes rules to increase scoring:

–  Make goal bigger! (full width of field) –  Carry ball with hands –  Can toss ball to another player backwards or laterally (called a

“lateral”) anyTme and forwards (“pass”) someTmes •  How to stop players carrying the ball? Grab them & knock

them down by making knee hit the ground (“tackle”) –  In soccer tackle the ball; football tackle the person

4/27/11 43 Spring 2011 -‐-‐ Lecture #27

ABCs of American Football

•  Score by... – Moving football into goal (“cross the goal line” or “into the end zone”) scoring a “touchdown” •  (6 points)

–  Kicking football between 2 poles (“goal posts”) scoring a “field goal” •  ( worth 3 points, unless aeer touchdown, then its just 1 point: “extra point” )

•  Kick ball to other team aeer score (“kickoff”) –  laterals OK

•  Game ends when no Tme lee (four 15 min quarters) and person with ball is stopped –  Soccer: two 45 min halves, Tme stops play

4/27/11 Spring 2011 -‐-‐ Lecture #27 44

Football Field

4/27/11 Spring 2011 -‐-‐ Lecture #27 45

50 40 30 20 10 40 30 20 10 Goal Line End

Zone

End Zone

Goal Line

100 yards (91.4 meters)

Spectacle of American Football

•  Cal’s archrival is Stanford –  Stereotype is Rich, EliTst Snobs –  E.g, derby Man City vs. Manchester United

•  Play nearby archrival for last game of season –  Called “The Big Game”: Cal vs. Stanford, winner gets a trophy (“The Axe”)

–  Oldest rivalry west of Mississippi; 100th in 1997 •  American college football is a spectacle –  School colors (Cal Blue & v. Red & ) –  Nicknames (Golden Bears v. Stanford Cardinal) –  School mascot (Oski the bear v. a tree(!)) –  Leaders of cheers (“cheerleaders”)

4/27/11 Spring 2011 -‐-‐ Lecture #27 46

Spectacle of American Football

•  “Bands” (orchestras that march) from both schools at games

•  March & Play – Before game, at haleime, aeer game

•  Stanford Band more like a drinking club (seen the movie “Animal House”?) – Plays one song: “All Right Now” (1970) – Cannot march and play

4/27/11 Spring 2011 -‐-‐ Lecture #27 47

1982 Big Game: “The Play”

“Top 20 favorite sports event in 20th century”, Sports Illustrated

“Greatest Football Play of All Time,” Best Damn Sports Show “…The Play, widely considered the most dramaTc ending in college football history” , AP news

“…widely considered the most famous play in college football history ,” Stanford Magazine

(“The Play” Has own entry in Wikipedia) •  Stanford

–  Quarterback is John Elway, who goes on to be a professional Hall of Fame football player (reTred 1999)

–  Possibly greatest quarterback in college history? •  In 1982, they had lost 4 games in last minutes

•  Stanford has just taken lead with 4 seconds lee in game; Cal team captain yells in huddle “Don’t fall with the ball!”; watch video

4/27/11 Spring 2011 -‐-‐ Lecture #27 48

4/27/11

9

Notes About “The Play” (1/2)

•  Cal only had 10 men on the field; last second another came on (170 pound Steve Dunn #3) and makes key 1st block

•  Kevin Moen #26: 6’1” 190 lb. safety, –  laterals to Rodgers (and doesn’t give up)

•  Richard Rodgers #5: 6’ 200 lb. safety, Cal captain “Don’t fall with that ball.” –  laterals to Garner

•  Dwight Garner #43: 5’9” 185 lb. running back –  almost tackled, 2 legs & 1 arm pinned, laterals

•  Richard Rodgers #5 (again): “Give me the ball!” –  laterals to Ford

4/27/11 Spring 2011 -‐-‐ Lecture #27 49

Notes About “The Play” (1/2)

•  Mariet Ford #1: 5’9”, 165 pound wide receiver –  Smallest player, leg cramps; overhead blind lateral to Moen and

blocks 3 Stanford players •  Moen (again) cuts through Stanford band into end zone

(touchdown!), smashes Trombonist •  On field for Stanford: 22 football players,

3 Axe commiHee members, 3 cheerleaders, 144 Stanford band members (172 for Stanford v. 11 for Cal) –  “Weakest part of the Stanford defense was the woodwinds.” -‐-‐

Cal Fan •  Cal players + Stanford Trombonist (Gary Tyrrell) hold

reunions; Stanford revises history (Changes score on Axe to 20-‐19); claims Garner’s knee was down – see video

4/27/11 Spring 2011 -‐-‐ Lecture #27 50

Special Thanks to the TAs: Andrew Gearhart, Conor Hughes, Yunsup Lee, Ari Rabkin,

Charles Reiss, Vasily Volkov

Andrew Waterman

4/27/11 51 Spring 2011 -‐-‐ Lecture #27

The Future for Future Cal Alumni

•  What’s The Future? •  New Century, Many New OpportunTes: Parallelism, Cloud, StaTsTcs + CS, Bio + CS, Society (Health Care, 3rd world) + CS

•  Cal heritage as future alumni –  Hard Working / Can do a�tude –  Never Give Up (“Don’t fall with the ball!”) –  Smallest on field, 3 big guys charging you: you make a play!

•  “The best way to predict the future is to invent it” – Alan Kay (inventor of personal compuTng vision)

•  Future is up to you!

4/27/11 Spring 2011 -‐-‐ Lecture #27 52

Notes About “The Play”

•  “Allright here we go with the kick-‐off. Harmon will probably try to squib it and he does. Ball comes loose and the Bears have to get out of bounds. Rogers along the sideline, another one... they're sTll in deep trouble at midfield, they tried to do a couple of....the ball is sTll loose as they get it to Rogers. They get it back to the 30, they're down to the 20...Oh the band is out on the field!! He's gonna go into the endzone!!! He got into the endzone!! … THE BEARS HAVE WON!!! THE BEARS HAVE WON!!! Oh my God, the most amazing, sensaTonal, dramaTc, heart rending... exciTng thrilling finish in the history of college football!” – KGO’s Joe Starkey

4/27/11 Spring 2011 -‐-‐ Lecture #27 53

Documents

Agenda - University of California, Berkeleyinst.eecs.berkeley.edu/~cs61c/sp11/lectures/27LecSp11RAIDx6.pdf · 4/27/11 4 RAID&Summary& • LogicalPtoPphysical&block&mapping,&parity&striping,&