The Next Generation Internet: Unsafe at any Speed?

The Next Generation Internet: Unsafe at any Speed?

Ken BirmanDept. of Computer ScienceCornell University

Convergent TrendsExisting Internet exhibiting brownouts, security and quality-of-service problemsTalk of a next generation Internet offering 10 to 100-fold performance improvementsA new generation of networked applications includes large numbers of critical ones

Typical Critical ApplicationsMedical monitoring and clinical databases. Community health information networks. Remote home care and Remote telesurgery Integrated modular avionics systems. Air traffic control. Free flight, 4-D navigation

Medical NetworksContacted a number of technical and business people in this field (HP Careview, EMTEK, Hospital for Sick Children)Asked: What are the trends? How are networks changing healthcare?How are these systems made secure & reliable?Got any good stories for me?

An ICU Computer SystemBedsideClinical data server Digital library and online PDRLaboratories PharmacyDoctors office

a field in transitionDuring 1980s, hospitals used largely dedicated systemsClient-server architectures now becoming dominant, but trend is a recent oneSystems ran in physical isolation and had limited, mission-specific functionality

Important distinctionMedical monitoring equipment, computer controlled devicesThese practice medicineFDA regulated, like a drugSoftware subjected to extreme verification methods, safety certification is costly and hardExtends to the IEEE medical information bus for connecting bedside devices

Important distinctionMedical monitoring equipment, computer controlled devicesClinical data systemsBy definition, not considered safety criticalMaintain the legally binding patient recordThink of a database system. Human checks all entries, even data obtained directly from devices or lab reports.

Traditional ApproachEach runs as a separate networkDeveloped completely independentlyNo interconnection of any kind

Networking technology?Monitoring network is increasingly a dedicated real-time LAN, this permits configuration flexibility, remote telemetry, even adjustment of monitoring devicesClinical database system increasingly connected to laboratories, community health information networks (CHINs), physicians office, insurance and HMOs

Platform choices?Overwhelming trend is to introduce standard PCs and workstations, standard Internet technologiesForced migration from dedicated platforms to shared, standard network platformsWeb access now common from PCs that run clinical database software

bluring the distinctionIncreasingly, see monitoring network cross-connected to the clinical data networkSome physical isolation: not yet common to see an IV perfusion drip controllable over an internet within a hospitalPerimeter security using passwords, firewalls. But medical security needs are unusual; mismatched to standard solutions.

Creep of critical roleTechnically, clinical data system is non-criticalBut increasingly, the system actually is critical: doctors and nurses depend upon theAccuracy and timeliness of reportingCorrect data for lab results, vitals, medicationsFDA is simply late to catch up with trendsMoreover, already seeing Windows 95 and MS Access as basis for such systems

Consumer / society pullIntensive and growing cost pressuresDesire for freedom from medical system, home careConsolidation of hospitalsHMOs want to control care plan create trend towards remote telemedicine, even robot telesurgery, CHINs

Vision: A Virtual Private NetworkApplication shares the network with untrusted agents but is isolated from them.

Reality?Current VPN support approximates this, but configuration potentially awkward, slowMany CHINs wont use VPNsBy running over the Internet, CHINs are exposed to bandwidth fluctuations and denial of service from many causes

Good stories?Many cases of security or privacy violations (EMTEK has a good one). HP told me that some hackers accidently disrupted a cardiac monitor in the Boston area a few years ago (trying to track this down)Nutty nursing aide in Britain changed orders, discharged patients, scheduled testsHP Careview, starved for bandwidth, flickers on and offline in some critical care units...

Broad picture?Application trends outstripping technologyDecision making is by societal consensus, cost pressures, reflects HMO needs.Hospital executives insisting on standardsHospital network of future: PCs, off-the-shelf Internet software, standard Web stuff. Critical or not, like it or not, its happening!

What about aviation?Much use of computer technologiesFlight management system (controls airplane)Flaps, engine controls (critical subsystems)Navigational systemsTCAS (collision avoidance system)Air traffic control system on groundIn-flight, approach, international hand-offAirport ground system (runways, gates, etc)

ATC system componentsControllersAir Traffic Database (flight plans, etc)X.500 DirectoryRadarOnboard

similar turmoilOn-board systems moving to COTS, integrated modular avionicsBoeing 777 SafeBus a success storyUnlikely it could be replicated with standard O/S and standard ATM or LAN hardwareEmergence of 4-D navigation (free flight) systems: ground network penetrates level-A critical cockpit components.

Free flightGround systemOn-board conflict alertand resolution systemTransponder and GPS

Future avionics systemsGround systems rely increasingly on automation, have form of a highly available, highly critical network. Built using standard PCs, software toolsGround network becomes critical to flight safetyOn-board avionics are basically a dedicated real-time LAN built with standard PCs but perhaps non-standard O/S. One platform, many apps.Safety validation of components replaces current validation of system. Think plug n play

The list goes onDisaster warning and response coordinationPower management (grid control)Banking, stock markets, trading systemsComputer-controlled vehiclesMilitary intelligence, command and controlCritical business applications

Commercial Off The ShelfBuild using COTS Standard componentsBuy off the shelf, then harden themIntended to be cheaper, easier to maintainAs a practical matter, there is nothing else on the shelf!Roll-your-own solutions abandon powerful tools that make modern computing great!

Technology MountainCOTS

Reliable Technology MountainCOTS

Next Generation InternetCurrent Internet looks frailOnly government investment can address security, reliability, scalability and performance problems of the InternetExpectation is that well build it quickly, hence that we basically know how today

Next Generation InternetConcrete details?Seeks 10 to 100-fold performance improvementOriginally expected to provide IP-v6 interfaceOriginally expected to implementLong IP addressesIPSec, DNSecQuality of Service options over some form of Diffsrv (or RSVP) mechanism

Reality checkBoth IPv6 and RSVP now uncertain due to resistance from mainstream IPv4 crowdRSVP resource use on routers grows as O(n2)IPv6 would outmode a huge existing investmentHow likely is it that the NGI will solve the practical problems identified earlier?How does one build a secure, reliable, scalable, high performance network application, anyhow?Do we in fact know how to do this?

Glimpse of the IPv4 crowdThey gave us TCP/IP, core internet services, stuff on which we run email, webThey elevate the end-to-end argument to a religion (basically: packets, not circuits)Little experience with critical applications

What about QoS?Best scheme: DiffsrvUses an edge-classification of packets; routers look at just a byte or twoBut routers distort flow dynamicsYou send 50 packets per second but within the network, a router might see a burst of 100, then a second of silenceConsequence is that Diffsrv will be at best stochastic (and it also cant handle routing changes)

a troubling implicationIt seems unlikely that the NGI will easily support isolation of critical subsystems with the range of properties requiredMore likely: a tool for building virtual circuits (one-one connections) that run at very high speeds Missing connection is the step from the network to the robust application

What do we need?Isolation of functionsCritical functionality compartmentalizedComponents only interact through well-defined interfaces with well-defined semanticsDeveloper proves that implementation respects interface definition and semanticsOn the other hand, adequate performance is fundamental to providing robustness

Evidence for these claims?This is how modern avionics modules are built (wing flap and engine control, flight management system, inertial navigation)Process is extremely costly and works only for very small pieces of softwareSafeBus on Boeing 777 allows such software to share platform by creating very strong firewalls between components

Agenda emergesFind ways to divide and conquerTransform big nasty system into smaller independent modulesRun them in an environment that has strong properties, which the modules exploitResulting system has strong properties tooCan we apply this to familiar distributed computing problems?

PhilosophyImagine a network as an abstract data typeAn Overlay Network or ONWe can instantiate it multiple times, condition each copy with desired quality properties:A Virtual Overlay Network or VONHow to introduce properties?Mixture of resource reservation at routers, on a per-ON basis, and management actions at edges

A VONLooks like a dedicated Internet, although hosted on a shared infrastructureSupports guarantees of properties such asBandwidthNoise levelSecurity and freedom from denial of serviceTreated as an aggregate, not a set of pt-to-pt connections!

Making Vision a Reality1) NGI needs to give us the ON mechanism2) We need to implement VONs using fairly standard protocols over the base ONs3) Must be able to produce specialized solutions for reliability/security needs4) Solutions amenable to selective use of formal tools

NGI hooks?Diffsrv and RSVP wont do itCreates an O(n2) resource reservation problemProblem is that both schemes are fundamentally connection oriented, and VON concept is fundamentally multipoint in natureHence these point-to-point QoS mechanisms are not suitable for supporting VONsAny other options?

Switches supporting flows already existMCI, Sprint, AT&T already sell each other dedicated bandwidth with isolationThis is on a scale of perhaps 10s of flows and hence classification is easyVONs might mean that a switch would see thousands, but such scaling seems well within technical feasibility

Router understands flowsLooks like this

Router understands flowsLooks like thisLooks like thisLooks like thisActs like thisFlow 1Flow 2Flow 3Everything else

Things to noticeA flow in this sense aggregates all the traffic for one ON the identifier is for the ON not the endpointsClassification task is thus much smaller and resources needed to support this are linear in number of ONs that pass through the switch, not the number of potential connectionsEach ON is like a dedicated network

An ON hasA bandwidth guarantee (router sets resources aside on its behalf)Perhaps latency guaranteesCan offer isolation between flowsBut not much else

NGI part of the pictureNGI needs to give us raw ONs but also:Robust routing infrastructureNamingAbility to build an ON tolerant of one link or router failureMany building blocks are already in placeBut the core Internet community is balking on all forms of QoS: isolation or other guarantees seen as inconsistent with end-to-end philosophy

But suppose we get our wishNext President declares moral equivalent of war after continuing Internet siege shuts down his web site during election:Let there be Overlay Networks!Then what?

Our new goal?Create VONs by adding properties to OnsUser sees VON as a set of end-points with minimum guarantees, like isolation, between themWe need a way to strengthen these propertiesE.g. manage security keys, manage RSVP parameters, routing, network name spaceWe may also need ways to reliably communicate (1-1, 1-n patterns)

VONs as abstract data types

VONs as abstract data typesFocus on the processes and network

VONs as abstract data typesThink of the ON interface as an abstract typeONONON

VONs as abstract data typesAdd encryption by substituting a module that looks the same but encrypts messagesencryptencryptencrypt

Layered MicroprotocolsInterface to Horus is extremely flexibleHorus manages group abstractiongroup semantics (membership, actions,events) defined by stack of modulesencryptfiltersignftolHorus stacksplug-and-playmodules to givedesign flexibilityto developervsync

Layered Microprotocols in HorusInterface to Horus is extremely flexibleHorus manages group abstractiongroup semantics (membership, actions,events) defined by stack of modulesencryptfiltersignftolEnsemble stacksplug-and-playmodules to givedesign flexibilityto developervsync

Layered Microprotocols in HorusInterface to Horus is extremely flexibleHorus manages group abstractiongroup semantics (membership, actions,events) defined by stack of modulesencryptfiltersignEnsemble stacksplug-and-playmodules to givedesign flexibilityto developervsyncftol

Same stack under each endpoint

Multiple VONs in single applicationencryptvsyncftolencryptvsyncftolencryptvsyncftolYellow group for video communicationGreen forcontrol andcoordination

Examples of reliability modelsVirtual synchrony model: emerged from our work on Isis, now widely acceptedBimodal multicast model: probabilistic and has neat performance properties but weaker logical consistency guaranteesSecure group communicationMultimedia channels

Virtual Synchrony ModelcrashG0={p,q} G1={p,q,r,s} G2={q,r,s} G3={q,r,s,t}pqrstr, s request to joinr,s added; state xfert added, state xfert requests to joinp fails

Virtual Synchrony ToolsVarious forms of replication:Replicated data, replicate an object, state transfer for starting new replicas...1-many event streams (network news)Load-balanced and fault-tolerant request executionManagement of groups of nodes or machines in a network setting

Stock Exchange Problem: Vsync. multicast is too fragileMost members are healthy.

but one is slow

Measured Impact of PerturbationThroughput (msgs/sec)Amount Perturbed

_919232030.xls

Chart2

200.0400080016200

192.8640308582200

187.6876876877200

153.1862745098200

55.9565776957200

45.02476362200

24.6353961372200

15.2606519351200

8.5217346843200

Virtual Synchrony Protocol

Pbcast Protocol

amount perturbed

throughput (msgs/sec)

Effect of Perturbation

Sheet1

0.10.10.0049990.005200.04000800162000.1200.0400080016200

0.20.20.0051850.005192.86403085822000.2192.8640308582200

0.30.30.0053280.005187.68768768772000.3187.6876876877200

0.40.40.0065280.005153.18627450982000.4153.1862745098200

0.50.50.0178710.00555.95657769572000.555.9565776957200

0.60.60.022210.00545.024763622000.645.02476362200

0.70.70.0405920.00524.63539613722000.724.6353961372200

0.80.80.0655280.00515.26065193512000.815.2606519351200

0.90.90.1173470.0058.52173468432000.98.5217346843200

&A

Page &P

Sheet2

&A

Page &P

Sheet3

&A

Page &P

Sheet4

&A

Page &P

Sheet5

&A

Page &P

Sheet6

&A

Page &P

Sheet7

&A

Page &P

Sheet8

&A

Page &P

Sheet9

&A

Page &P

Sheet10

&A

Page &P

Sheet11

&A

Page &P

Sheet12

&A

Page &P

Sheet13

&A

Page &P

Sheet14

&A

Page &P

Sheet15

&A

Page &P

Sheet16

&A

Page &P

_919232303.xls

Chart2

200.0400080016200

192.8640308582200

187.6876876877200

153.1862745098200

55.9565776957200

45.02476362200

24.6353961372200

15.2606519351200

8.5217346843200


Pbcast Protocol

amount perturbed



Sheet1

0.10.10.0049990.005200.04000800162000.1200.0400080016200

0.20.20.0051850.005192.86403085822000.2192.8640308582200

0.30.30.0053280.005187.68768768772000.3187.6876876877200

0.40.40.0065280.005153.18627450982000.4153.1862745098200

0.50.50.0178710.00555.95657769572000.555.9565776957200

0.60.60.022210.00545.024763622000.645.02476362200

0.70.70.0405920.00524.63539613722000.724.6353961372200

0.80.80.0655280.00515.26065193512000.815.2606519351200

0.90.90.1173470.0058.52173468432000.98.5217346843200

&A

Page &P

Sheet2

&A

Page &P

Sheet3

&A

Page &P

Sheet4

&A

Page &P

Sheet5

&A

Page &P

Sheet6

&A

Page &P

Sheet7

&A

Page &P

Sheet8

&A

Page &P

Sheet9

&A

Page &P

Sheet10

&A

Page &P

Sheet11

&A

Page &P

Sheet12

&A

Page &P

Sheet13

&A

Page &P

Sheet14

&A

Page &P

Sheet15

&A

Page &P

Sheet16

&A

Page &P

_930556971.xls

Chart2

200.0400080016200

192.8640308582200

187.6876876877200

153.1862745098200

55.9565776957200

45.02476362200

24.6353961372200

15.2606519351200

8.5217346843200


Pbcast Protocol

amount perturbed



Sheet1

0.10.10.0049990.005200.04000800162000.1200.0400080016200

0.20.20.0051850.005192.86403085822000.2192.8640308582200

0.30.30.0053280.005187.68768768772000.3187.6876876877200

0.40.40.0065280.005153.18627450982000.4153.1862745098200

0.50.50.0178710.00555.95657769572000.555.9565776957200

0.60.60.022210.00545.024763622000.645.02476362200

0.70.70.0405920.00524.63539613722000.724.6353961372200

0.80.80.0655280.00515.26065193512000.815.2606519351200

0.90.90.1173470.0058.52173468432000.98.5217346843200

&A

Page &P

Sheet2

&A

Page &P

Sheet3

&A

Page &P

Sheet4

&A

Page &P

Sheet5

&A

Page &P

Sheet6

&A

Page &P

Sheet7

&A

Page &P

Sheet8

&A

Page &P

Sheet9

&A

Page &P

Sheet10

&A

Page &P

Sheet11

&A

Page &P

Sheet12

&A

Page &P

Sheet13

&A

Page &P

Sheet14

&A

Page &P

Sheet15

&A

Page &P

Sheet16

&A

Page &P

_919232070.xls

Chart2

200.0400080016200

192.8640308582200

187.6876876877200

153.1862745098200

55.9565776957200

45.02476362200

24.6353961372200

15.2606519351200

8.5217346843200


Pbcast Protocol

amount perturbed



Sheet1

0.10.10.0049990.005200.04000800162000.1200.0400080016200

0.20.20.0051850.005192.86403085822000.2192.8640308582200

0.30.30.0053280.005187.68768768772000.3187.6876876877200

0.40.40.0065280.005153.18627450982000.4153.1862745098200

0.50.50.0178710.00555.95657769572000.555.9565776957200

0.60.60.022210.00545.024763622000.645.02476362200

0.70.70.0405920.00524.63539613722000.724.6353961372200

0.80.80.0655280.00515.26065193512000.815.2606519351200

0.90.90.1173470.0058.52173468432000.98.5217346843200

&A

Page &P

Sheet2

&A

Page &P

Sheet3

&A

Page &P

Sheet4

&A

Page &P

Sheet5

&A

Page &P

Sheet6

&A

Page &P

Sheet7

&A

Page &P

Sheet8

&A

Page &P

Sheet9

&A

Page &P

Sheet10

&A

Page &P

Sheet11

&A

Page &P

Sheet12

&A

Page &P

Sheet13

&A

Page &P

Sheet14

&A

Page &P

Sheet15

&A

Page &P

Sheet16

&A

Page &P

_917772005.xls

Chart2

200.0400080016200

192.8640308582200

187.6876876877200

153.1862745098200

55.9565776957200

45.02476362200

24.6353961372200

15.2606519351200

8.5217346843200


Pbcast Protocol

amount perturbed



Sheet1

0.10.10.0049990.005200.04000800162000.1200.0400080016200

0.20.20.0051850.005192.86403085822000.2192.8640308582200

0.30.30.0053280.005187.68768768772000.3187.6876876877200

0.40.40.0065280.005153.18627450982000.4153.1862745098200

0.50.50.0178710.00555.95657769572000.555.9565776957200

0.60.60.022210.00545.024763622000.645.02476362200

0.70.70.0405920.00524.63539613722000.724.6353961372200

0.80.80.0655280.00515.26065193512000.815.2606519351200

0.90.90.1173470.0058.52173468432000.98.5217346843200

&A

Page &P

Sheet2

&A

Page &P

Sheet3

&A

Page &P

Sheet4

&A

Page &P

Sheet5

&A

Page &P

Sheet6

&A

Page &P

Sheet7

&A

Page &P

Sheet8

&A

Page &P

Sheet9

&A

Page &P

Sheet10

&A

Page &P

Sheet11

&A

Page &P

Sheet12

&A

Page &P

Sheet13

&A

Page &P

Sheet14

&A

Page &P

Sheet15

&A

Page &P

Sheet16

&A

Page &P

The problem gets worse as the system scales up00.10.20.30.40.50.60.70.80.9050100150200250Virtually synchronous Ensemble multicast protocolsperturb rateaverage throughput on nonperturbed membersgroup size: 32group size: 64group size: 96

Why does stability matter?Swiss Stock ExchangeExchange is fully electronic [FTCS-27 paper]Uses Isis SDK to distribute all bids/offers and all trades. Every node has the pictureBut this means that entire trading history available to 50 member banks & firms and hundreds or thousands of traders!Unstable node could bring exchange to its knees.Similar issues seen in many other settings

Pbcast has a probabilistic reliability modelEither almost all destinations receive the message or almost none do soThis is strong enough to use in applications with critical reliability needs (but not necessary for all their communication purposes -- put side by side with virtual synchrony)

Chart1

0

0.0291968046

0.000581897

0.0000102291

0.0000001783

0.0000000033

0.0000000001

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0.0000000005

0.0000000364

0.0000035333

0.0006171974

0

number of processes to deliver pbcast

p{#processes=k}

Pbcast bimodal delivery distribution

Sheet1

00.00E+00

206.6298838.34570312.19E-036.43E-01101.78E-081.77E-0612.92E-02

255.9189458.653321.53.02E-031.02E+00152.18E-111.62E-0525.82E-04

305.440437.3408221.47E-039.52E-01203.42E-135.71E-0731.02E-05

355.0712897.5048832.53.92E-045.38E-01257.87E-161.66E-0641.78E-07

404.7636726.77343837.05E-052.07E-01308.89E-184.96E-0853.29E-09

454.531256.8759773.59.48E-066.13E-02352.60E-209.97E-0866.67E-11

504.3193366.33593841.01E-061.51E-02402.17E-223.10E-0971.53E-12

4.59.00E-083.26E-03458.26E-255.38E-0984.05E-14

56.85E-096.40E-04505.80E-271.76E-1091.26E-15

5.54.60E-101.16E-04553.67E-292.82E-10104.63E-17

62.88E-111.98E-05601.65E-311.56E-11112.04E-18

6.51.91E-123.20E-06121.08E-19

71.94E-135.07E-07136.95E-21

7.54.57E-148.72E-08145.45E-22

82.09E-142.03E-08155.23E-23

8.51.35E-147.74E-09166.16E-24

91.07E-144.37E-09178.93E-25

9.59.55E-153.08E-09181.60E-25

109.01E-152.50E-09193.53E-26

209.65E-27

213.27E-27

221.37E-27

237.13E-28

244.59E-28

253.67E-28

263.62E-28

274.43E-28

286.68E-28

291.24E-27

302.85E-27

318.00E-27

322.75E-26

331.16E-25

345.91E-25

353.67E-24

362.76E-23

372.51E-22

382.75E-21

393.62E-20

405.74E-19

411.10E-17

422.53E-16

437.06E-15

442.40E-13

451.01E-11

465.30E-10

473.64E-08

483.53E-06

496.17E-04

500.00E+00

Sheet1

Predicate I for 1E-9 reliability

Predicate II for 1E-12 reliability

#processes in system

fanout

Fanout required for a specified reliability

Sheet2

Predicate I

Predicate II

fanout

P{failure}

Effects of fanout on reliability

Sheet3

Predicate I

Predicate II

#processes in system

P{failure}

Scalability of Pbcast reliability

number of processes to deliver pbcast

p{#processes=k}

Pbcast bimodal delivery distribution

Pbcast has stable throughputGets this from a mixture of gossip-style local repair with several innovations to avoid overload when some process failsWe implemented the protocol and experimentally confirmed this

Chart4

00

0.97019867550.9735099338

0.02317880790.0198675497

00.0066225166

0.00331125830

0.00331125830

00

00

00

00

00

00

00

00

&A

Page &P

Pbcast with .05 sleep probability


Inter-arrival spacing (ms)

Probability of occurence

Histogram of throughput for pbcast

Histograms

fifo/.05BinFrequencyfifo/.45BinFrequencyPbcast/.05BinFrequencyPbcast/.45BinFrequency

Traditional Protocol with .05 sleep probabilityTraditional Protocol with .45 sleep probabilityPbcast with .05 sleep probabilityPbcast with .45 sleep probability

0.001870.005680.0046280.00530.0059370.00500.0060.0050

0.0031150.012130.0047730.01310.0059480.012930.0060080.01294

0.003410.015170.0049740.015290.0060080.01570.0060210.0156

0.0037610.0200.0051870.02230.0060250.0200.0060470.022

0.004010.02500.005270.02580.0060290.02510.0060660.0250

0.0042520.0300.0054040.0350.006030.0310.0060660.030

0.0042830.03500.0055680.03530.0060370.03500.0060660.0350

0.0045610.0400.0057090.0410.0060370.0400.0060660.040

0.004570.04500.0058120.04520.0060470.04500.0060670.0450

0.0045740.0500.0059050.0510.006050.0500.0060680.050

0.0045790.05500.0059310.05500.0060520.05500.0060690.0550

0.0045980.0610.0059450.0600.0060530.0600.0060740.060

0.0046010.06500.0060270.06520.0060540.06500.0060760.0650

0.0046040.0700.0062470.0700.0060540.0700.0060770.070

0.004611More00.006373More00.006057More00.006079More0

0.0046160.0064670.0060570.006079

0.0046220.0066860.0060580.006083

0.0046370.006760.0060590.006083

0.0046420.0070870.006060.006084

0.0046420.0074380.0060620.006085Traditional Protocol with .05 sleep probabilityTraditional Protocol with .45 sleep probabilityPbcast with .05 sleep probabilityPbcast with .45 sleep probability

0.0046630.0079020.0060640.0060860.00568300

0.0046640.0080030.0060650.0060870.0121331293294

0.0046730.0080320.006070.0060890.015172976

0.0046740.0080760.0060710.0060910.0202302

0.0046740.0080930.0060710.0060920.0250810

0.0046790.0088210.0060740.0060970.030510

0.0046820.0091610.0060740.0060970.0350300

0.0046840.0094390.0060760.0061020.040100

0.0046840.0095980.0060760.0061020.0450200

0.004690.0096980.0060770.0061060.050100

0.0046950.0097020.0060770.0061070.0550000

0.0046990.0098620.0060780.0061070.061000

0.0047180.009930.0060790.0061110.0650200

0.0047240.0099570.0060790.0061110.070000

0.0047340.010050.0060830.006112

0.0047350.0101240.0060830.006113299108302302

0.0047390.0106250.0060840.006113

0.0047630.0106520.0060850.006116

0.0047650.0108220.0060850.006117


0.0047690.0112510.0060860.0061180.00568300

0.0047710.0115180.0060870.0061180.0121331293294

0.0047750.0117170.0060870.006120.015172976

0.004780.01190.0060880.006120.0202302

0.0047810.0120050.0060880.0061210.0250810

0.0047970.01210.0060890.0061210.030510

0.0048040.0121780.0060890.0061220.0350300

0.0048210.0123390.0060890.0061240.040100

0.0048240.0124610.006090.0061240.0450200

0.0048270.0126670.006090.0061240.050100

0.0048290.0127050.0060910.0061240.0550000

0.0048380.0127760.0060910.0061250.061000

0.0048470.012880.0060920.0061260.0650200

0.004850.0128810.0060920.0061260.070000

0.0048620.0129590.0060920.006126

0.0048630.013270.0060930.006126299108302302

0.0049160.0135170.0060950.006127


0.0049210.013810.0060970.0061310.0050.22742474920.027777777800

0.0049280.0142010.0060980.0061330.010.71237458190.2870370370.97019867550.9735099338

0.0049290.0143540.0060990.0061330.0150.05685618730.26851851850.02317880790.0198675497

0.0049310.0144220.00610.0061330.0200.21296296300.0066225166

0.0049380.0148380.00610.0061340.02500.07407407410.00331125830

0.0049420.0152750.0061010.0061360.0300.04629629630.00331125830

0.0049440.0153410.0061010.0061370.03500.027777777800

0.0049570.0156270.0061020.0061380.0400.009259259300

0.004960.0156410.0061030.006140.04500.018518518500

0.0049610.0157230.0061030.0061410.0500.009259259300

0.0050030.0157680.0061030.0061410.0550000

0.005010.0158320.0061040.0061420.060.0033444816000

0.0050160.0159540.0061040.0061430.06500.018518518500

0.0050320.0162180.0061050.0061430.070000

0.0050390.0162650.0061050.006144

0.0050460.0164180.0061050.006144

0.0050460.0168060.0061050.006144

0.0050590.0168380.0061060.006145

0.0050680.016980.0061060.006145

0.0050830.0174190.0061060.006145

0.0051180.0174540.0061070.006146

0.0051290.0176560.0061070.006146

0.0051310.0182410.0061070.006146

0.0051510.0186210.0061090.006147

0.0051710.0186810.0061090.006148

0.0051720.0189150.006110.006148

0.0051950.0191280.0061110.006149

0.0051960.0199140.0061110.006149

0.0051970.0206180.0061110.00615

0.0052340.0206230.0061110.00615

0.0052530.0209220.0061110.00615

0.005260.0209340.0061120.006151

0.0052820.0211980.0061130.006151

0.0052880.0213060.0061130.006152

0.0052940.0219240.0061140.006153

0.0053230.0222670.0061140.006153

0.0053360.0254720.0061140.006153

0.0053520.0264850.0061140.006154

0.0053610.0266770.0061150.006154

0.0054680.0284970.0061150.006154

0.0055610.0294520.0061150.006155

0.0055940.0316370.0061150.006155

0.0056340.0317270.0061150.006156

0.0056470.0320650.0061160.006157

0.0056550.0369850.0061170.006158

0.0056560.0417480.0061180.006158

0.0056690.0439830.0061180.006158

0.0056810.0450930.0061180.006159

0.0056930.0625090.0061180.006159

0.0057060.0628730.006120.00616

0.005720.006120.006162

0.0057250.006120.006163

0.0057270.0061210.006163

0.0057320.0061220.006164

0.0057410.0061220.006166

0.0057440.0061230.006166

0.0058090.0061230.006166

0.0058320.0061230.006166

0.005840.0061230.006166

0.0058430.0061240.006168

0.0058530.0061240.00617

0.0058550.0061250.00617

0.005870.0061250.00617

0.0058720.0061250.006171

0.0058840.0061250.006172

0.0058940.0061260.006172

0.0059070.0061270.006173

0.0059360.0061280.006173

0.005940.0061280.006173

0.0059580.0061290.006174

0.0059690.0061290.006175

0.0059860.0061290.006176

0.0060.0061290.006176

0.0060080.0061290.006177

0.0060150.0061290.006179

0.0060250.0061290.00618

0.0060340.006130.006181

0.0060440.006130.006181

0.0060510.0061310.006185

0.0060660.0061320.006186

0.0060820.0061320.006187

0.0060910.0061320.006187

0.0060960.0061340.006187

0.0061060.0061340.006188

0.0061140.0061360.006189

0.0061310.0061370.006189

0.0061490.0061380.00619

0.0061590.0061380.006191

0.0061710.0061380.006192

0.0061950.0061390.006193

0.0062010.006140.006193

0.0062160.006140.006193

0.0062190.0061410.006195

0.0062350.0061420.006196

0.0062420.0061420.006199

0.0062540.0061420.006201

0.0062630.0061420.006202

0.0062730.0061430.006202

0.0062790.0061430.006203

0.006280.0061430.006204

0.0062910.0061430.006204

0.0063210.0061450.006205

0.0063320.0061450.006205

0.0063390.0061460.006206

0.0063450.0061470.006206

0.0063470.0061470.006207

0.0063480.0061470.006208

0.006350.0061490.006208

0.0063550.006150.006208

0.0063560.0061510.006211

0.006370.0061510.006211

0.006380.0061520.006214

0.0063870.0061530.006216

0.0063880.0061530.006218

0.0063910.0061540.006221

0.0063960.0061540.006222

0.0064090.0061550.006223

0.0064180.0061550.006223

0.0064210.0061550.006224

0.0064220.0061550.006224

0.0064340.0061560.006224

0.0064450.0061560.006227

0.0064470.0061560.00623

0.0064470.0061570.006231

0.0064510.0061570.006234

0.0064690.0061570.006234

0.0064690.0061580.006235

0.0064740.0061590.006235

0.0064810.0061590.006236

0.0064810.0061610.006237

0.0064860.0061620.006238

0.0064940.0061620.00624

0.0064970.0061630.00624

0.0065090.0061640.006241

0.0065170.0061640.006242

0.006520.0061650.006247

0.0065280.0061650.006247

0.0065630.0061650.00625

0.0065770.0061670.00625

0.0065970.0061680.006252

0.006630.0061690.006252

0.0066380.0061690.006254

0.0066550.0061720.006254

0.0066640.0061720.006254

0.0066650.0061730.006255

0.0066650.0061740.006256

0.0066810.0061740.006258

0.006690.0061750.00626

0.0067040.0061760.006261

0.0067380.0061760.006262

0.006760.0061770.006263

0.0067620.0061770.006263

0.0067660.0061790.006264

0.0067850.006180.006266

0.0067960.006180.006268

0.0068020.0061810.006268

0.0068370.0061810.006268

0.0068770.0061820.006271

0.0068820.0061820.006272

0.0069320.0061820.006274

0.0069440.0061830.006274

0.0069620.0061840.006274

0.006970.0061840.006276

0.0069780.0061850.006276

0.0069840.0061860.006279

0.0070170.0061940.006281

0.0070320.0061950.006283

0.0070570.0061980.006283

0.0070910.0061980.006289

0.0070950.0061990.006289

0.0071230.0062030.006291

0.0071390.0062050.006292

0.0071480.0062060.006293

0.007150.0062070.006295

0.0071780.0062070.006299

0.0071780.0062080.006301

0.0071950.0062080.006302

0.0072520.0062090.006302

0.0072660.0062120.006303

0.0073430.0062120.006303

0.0073430.0062170.006308

0.0073470.0062180.006313

0.0073490.0062180.006318

0.0073570.0062210.006322

0.0074040.0062210.006323

0.0076160.0062250.006328

0.0076390.0062250.006335

0.0077020.0062250.006338

0.0077290.0062250.006342

0.0078370.0062280.006345

0.0079060.0062320.006348

0.0079070.0062330.006356

0.0079720.0062340.006367

0.0080220.0062340.006375

0.0080290.0062360.006381

0.0081680.0062490.006381

0.0081950.0062640.006384

0.0082360.006270.006386

0.0082380.0062770.006387

0.0082420.0062890.006389

0.0082580.0062980.006392

0.0082750.0063080.006435

0.0082990.0063270.00645

0.0083840.0063340.006461

0.0084050.0063370.006463

0.0084890.0063370.006477

0.0085030.0063420.006483

0.0085470.0063670.006513

0.0085970.0063740.006566

0.0085980.0063940.006603

0.0086010.0064260.006642

0.0087170.0064260.006692

0.008720.0064780.006711

0.008890.0064920.006847

0.0089120.006510.00686

0.0089650.0065450.00686

0.009020.0066230.006887

0.009260.006710.006957

0.0093070.0067880.007104

0.009590.0069060.007113

0.009640.0070740.007295

0.0099740.0070970.007297

0.0099940.0071120.007391

0.0100220.0072970.007416

0.010040.0073110.007757

0.0100790.0074690.00821

0.0101070.0074820.008302

0.010240.0075290.008333

0.010530.0079130.008349

0.0109420.007930.0085

0.0113920.007980.00876

0.0115780.0082320.008958

0.0116620.0086610.00903

0.0116820.0087670.009121

0.0118010.0092540.009175

0.0119930.0101640.009281

0.0137070.0104310.010412

0.0138190.0110130.011389

0.0140370.0127690.011573

0.0144880.0129210.012047

0.0566960.0135720.013404

0.0139560.0143

0.021560.018217

0.0280.01829

&A

Page &P

Histograms

00

00

00

00

00

00

00

00

00

00

00

00

00

00

&A

Page &P

Traditional Protocol with .05 sleep probability

Traditional Protocol with .45 sleep probability

Time to receive 100 messages


Histogram of throughput for Traditional Protocol

g10.003fifo

00

00

00

00

00

00

00

00

00

00

00

00

00

00

&A

Page &P



Time to receive 100 messages


Histogram of throughput for PBCast

1\1\g1\5\g2\5\g

Fifo/highPbcast/highFifo/lowPbcast/lowFIFO/hPbcast/hFIFO/lPbcast/lFIFO/hPbcast/hFIFO/lPbcast/l

Traditional w/1 sleeperPbcast w/1 sleeperTraditional w/1 sleeperPbcast w/1 sleeperTraditional w/5 sleepersPbcast w/5 sleepersTraditional w/5 sleepersPbcast w/5 sleepers

0.05151.262153.82599.999699.9981123.82150.75799.999999.999277.5588265.736161.412199.997

0.15126.037153.98699.999799.998174.0949153.16100.00299.997771.1499264.722136.632199.995

0.25101.96155.231100.00499.640765.026150.35796.038399.99967.6259262.719116.164199.997

0.3577.0642145.21999.999799.999750.6761150.62573.306199.995559.6272267.606106.311199.992

0.4563.9061153.02996.421199.994439.7611153.33158.5499.995752.2691260.94587.0828199.996

0.5550.3154152.36775.08999.998431.1254151.62743.980499.996431.1254151.62743.980499.9964

0.6539.4076153.88853.555599.99821.6599153.26533.497899.996921.6599153.26533.497899.9969

0.7526.2399149.92739.227299.999114.7746153.56323.037596.84914.7746153.56323.037596.849

0.8516.0008153.21722.373599.99799.07249152.90212.206499.99879.07249152.90212.206499.9987

0.955.67649153.628.4509999.99772.9353156.2562.935399.99922.9353156.2562.935399.9992

1/1/b

FIFO/hPbcast/hFIFO/lPbcast/l

1\3\gThroughput for traditional protocol, measured at faulty hostThroughput for Pbcast, measured at faulty host

Traditional w/3 sleepersPbcast w 3/sleepers151.261153.83199.9998100.003

154.991153.64399.9998153.25799.8381100.001

112.362151.774102.277149.679100.00499.2005

80.445151.8778.8386126.51799.999498.7813

64.5427149.9563.8418116.21895.878116.218

50.2844155.49250.027899.75174.765880.7126Throughput for traditional protocol, measured at correct host

41.3491151.82839.078177.962453.128863.896

25.3831153.22625.773953.385638.951343.8917Throughput for PBCast, measured at correct host

19.4144150.58415.333730.327821.954330.3278

9.07342153.4564.879068.210887.879528.21088

3.4324152.63

&A

Page &P

0000

0000

0000

0000

0000

0000

0000

0000

0000

0000

&A

Page &P

Throughput for traditional protocol, measured at correct host

Throughput for PBCast, measured at correct host

Throughput for traditional protocol, measured at faulty host

Throughput for Pbcast, measured at faulty host

Probability of Sleep Event

Average Throughput

High Bandwidth comparison of PBCast performance atfaulty and correct hosts

000000

000000

000000

000000

000000

000000

000000

000000

000000

000000

&A

Page &P

Traditional w/1 sleeper

Pbcast w/1 sleeper

Traditional w/3 sleepers

Pbcast w 3/sleepers


Pbcast w/5 sleepers

Probability of sleep event

Throughput measured at unperturbed process

High Bandwidth measurements with varying numbers of sleepers

0000

0000

0000

0000

0000

0000

0000

0000

0000

0000

&A

Page &P


Pbcast w/1 sleeper


Pbcast w/5 sleepers


Average Throughput

Low Bandwidth measurements with varying numbers of sleepers

Chart3

151.262153.825154.991153.643123.82150.757

126.037153.986112.362151.77474.0949153.16

101.96155.23180.445151.8765.026150.357

77.0642145.21964.5427149.9550.6761150.625

63.9061153.02950.2844155.49239.7611153.331

50.3154152.36741.3491151.82831.1254151.627

39.4076153.88825.3831153.22621.6599153.265

26.2399149.92719.4144150.58414.7746153.563

16.0008153.2179.07342153.4569.07249152.902

5.67649153.623.4324152.632.9353156.256


Pbcast w/1 sleeper


Pbcast w 3/sleepers


Pbcast w/5 sleepers




Sheet1

0.10.0049990.005200.040008200

0.20.0051850.005192.8640309200

0.30.0053280.005187.6876877200

0.40.0065280.005153.1862745200

0.50.0178710.00555.9565777200

0.60.022210.00545.02476362200

0.70.0405920.00524.63539614200

0.80.0655280.00515.26065194200

0.90.1173470.0058.521734684200

Sheet1

99.999699.998199.999999.9992

99.999799.9981100.00299.9977

100.00499.640796.038399.999

99.999799.999773.306199.9955

96.421199.994458.5499.9957

75.08999.998443.980499.9964

53.555599.99833.497899.9969

39.227299.999123.037596.849

22.373599.997912.206499.9987

8.4509999.99772.935399.9992


Pbcast w/1 sleeper


Pbcast w/5 sleepers


Average Throughput


Sheet2

99.999699.998199.9998100.003

99.999799.998199.8381100.001

100.00499.6407100.00499.2005

99.999799.999799.999498.7813

96.421199.994495.878116.218

75.08999.998474.765880.7126

53.555599.99853.128863.896

39.227299.999138.951343.8917

22.373599.99792130.3278

8.4509999.99777.879528.21088


Pbcast w/1 sleeper

Throughput for traditional protocol, measured at perturbed host

Throughput for Pbcast, measured at perturbed host


Average Throughput

Low Bandwidth comparison of PBCast performance atfaulty and correct hosts

Sheet3

99.999699.998199.999999.9992

99.999799.9981100.00299.9977

100.00499.640796.038399.999

99.999799.999773.306199.9955

96.421199.994458.5499.9957

75.08999.998443.980499.9964

53.555599.99833.497899.9969

39.227299.999123.037596.849

22.373599.997912.206499.9987

8.4509999.99772.935399.9992


Pbcast w/1 sleeper


Pbcast w/5 sleepers


Average Throughput


151.262153.825154.991153.643123.82150.757

126.037153.986112.362151.77474.0949153.16

101.96155.23180.445151.8765.026150.357

77.0642145.21964.5427149.9550.6761150.625

63.9061153.02950.2844155.49239.7611153.331

50.3154152.36741.3491151.82831.1254151.627

39.4076153.88825.3831153.22621.6599153.265

26.2399149.92719.4144150.58414.7746153.563

16.0008153.2179.07342153.4569.07249152.902

5.67649153.623.4324152.632.9353156.256


Pbcast w/1 sleeper


Pbcast w 3/sleepers


Pbcast w/5 sleepers




151.262153.825151.261153.831

126.037153.98699.9998153.257

101.96155.231102.277149.679

77.0642145.21978.8386126.517

63.9061153.02963.8418116.218

50.3154152.36750.027899.751

39.4076153.88839.078177.9624

26.2399149.92725.773953.3856

16.0008153.21715.333730.3278

5.67649153.624.879068.21088

Throughput for traditional protocol, measured at unperturbed host

Throughput for PBCast, measured at unperturbed host

Throughput for traditional protocol, measured at perturbed host

Throughput for Pbcast, measured at perturbed host


Average Throughput

High Bandwidth comparison of PBCast performance atfaulty and correct hosts

00

00

00

00

00

00

00

00

00


Ideal Behavior

amount perturbed



Now we have several styles...Each style or model yields a VON with different propertiesApplication might not see the multicast stackInstead, the environment in which the application runs could see the stack and use it on behalf of the applicationFor example, a library could use stack to maintain the keys with which it authenticates actions

Formal methodsWith so much riding on VON, we need strong guarantees that the stack works!If protocols can be formally proved correct, confidence will be far strongerCan we use formal tools on network protocols built in this compositional manner?

Exploiting formal methodsVan Renesse and Hayden: code stack with language having strong semanticsThey used OCaml dialect of MLNow we can bring formal tools to bear on issues of correctness:Using Nuprl system for thisBasically, it automates proofs and program transformations

Initial Progress?Presented in 1999 ACM SOSP paperHave formalized the transformations used to optimize stacks for high performanceWe show that from one initial stack, we can produce multiple optimized stacks for common cases. Yields big speedups!

StepsTransform Ensemble stack into a single function in a functional styleUse partial evaluation to produce optimized version for common cases Use theorem proving to establish that stacks provide desired propertiesTransform back to imperative styleResulting code is optimized yet retains properties of original stack

Optimization ExampleencryptvsyncftolOriginal code is simple but inefficientOptimized code for common case is provably equivalent yet inefficiencies are eliminatedencryptvsyncftol? Common case?

Optimization Exampleencryptvsyncftolencryptvsyncftol? Common case?? Common case?We do nearly as well as hand-optimization and can automatically handle much bigger stacks!

Wrapping things upBy building better networks, and isolating protocol components and system components and adopting a modular architecture and selectively using formal methods we make it more and more practical to gain both high performance and other desired properties, such as reliability, security, stability, etc.

Potential NGI lets critical applications share network with untrusted onesVONs

But will it happen?Current political agenda focuses on speed and e-commerce transactionsEnd-to-end community resists giving any guarantees no matter how simpleAnd NGI focus is exclusively on point-to-point QoS, which seems unscalabledenying us the one primitive building block on which the whole concept depends!

Conclusions?The world needs better networks!Improve them by improved opportunity for modularity, isolation, guarantees of security and quality of service VONs and layers built over themLacking this, we face very serious problems simply going forward in directions to which society is already committed.

More info

http://www.cs.cornell.edu/ken/unsafe.ps

Documents

The Next Generation Internet: Unsafe at any Speed?