The Collaborative Radar Acquisition Field Test (CRAFT): Next Steps Kelvin K. Droegemeier University of Oklahoma 2 nd Level II Stakeholders Workshop 26-27

The Collaborative The Collaborative Radar Acquisition Radar Acquisition

Field Test (CRAFT): Field Test (CRAFT): Next StepsNext StepsKelvin K. DroegemeierKelvin K. DroegemeierUniversity of OklahomaUniversity of Oklahoma

22ndnd Level II Stakeholders Workshop Level II Stakeholders Workshop26-27 September 200226-27 September 2002

Norman, OklahomaNorman, Oklahoma

NCDC

http://www.nsf.gov/start.htm

http://www.roc.noaa.gov/

Grant funding for CRAFT communication links and Grant funding for CRAFT communication links and personnel is nearly exhausted (data will stop flowing personnel is nearly exhausted (data will stop flowing from CAPS sometime in November)from CAPS sometime in November)

The private and academic sectors are finding value in The private and academic sectors are finding value in real time Level II datareal time Level II data

A real time Level II NWS collection system A real time Level II NWS collection system – is is likelylikely more than 1 year away more than 1 year away– maymay not provide the latencies and reliability needed by the not provide the latencies and reliability needed by the

private sector for the short termprivate sector for the short term– maymay be perfectly suited for meeting all needs in the longer term be perfectly suited for meeting all needs in the longer term

What options exist?What options exist? How can we maximize the benefits to all stakeholders: How can we maximize the benefits to all stakeholders:

Government, industry, academia?Government, industry, academia?

The Issues Before UsThe Issues Before Us

A wide range of potential options exists, A wide range of potential options exists, all of which require all of which require Government approvalGovernment approval– Shut CRAFT down and wait for the NWS systemShut CRAFT down and wait for the NWS system

Timeline not yet definedTimeline not yet defined Not clear the NWS system will meet non-Government user needsNot clear the NWS system will meet non-Government user needs We likely won’t know until the system is in placeWe likely won’t know until the system is in place If it does meet all user needs, we’re setIf it does meet all user needs, we’re set If it does not, no alternative will exist (might take months to create)If it does not, no alternative will exist (might take months to create)

– Continue the present collaborative system (58 radars) or expand to all Continue the present collaborative system (58 radars) or expand to all 120 NWS radars (lots of sub-options)120 NWS radars (lots of sub-options)

– Create a stand-alone system that includes all 120 NWS WSR-88D radars, Create a stand-alone system that includes all 120 NWS WSR-88D radars, serves as a back-up to whatever the NWS implements, and has 7x24 serves as a back-up to whatever the NWS implements, and has 7x24 support, improved reliability, etcsupport, improved reliability, etc

Must consider administration of system (later in talk)Must consider administration of system (later in talk) The ideal perhaps is a partnership among all groups, with The ideal perhaps is a partnership among all groups, with

“partnership” defined many ways“partnership” defined many ways

OptionsOptions

Suppose the NWS Deploys Suppose the NWS Deploys and Manages its Own and Manages its Own Level II Distribution Level II Distribution

System System (a very sensible approach)(a very sensible approach)

CRAFT as a Scalable CRAFT as a Scalable System: The Current System: The Current Concept Expanded for Concept Expanded for

“Operational” Deployment“Operational” Deployment

LDM Server

Logical Network TopologyLogical Network Topology

LDM Server


At the moment, OUAt the moment, OUis the only server –is the only server –

Single points of failureSingle points of failure(server and line from(server and line from

each radar)each radar)

OUOU


LDM Server

UniversitiesUniversitiesNOAA LaboratoriesNOAA LaboratoriesNOAA Joint InstitutesNOAA Joint InstitutesNCAR/UCARNCAR/UCARMIT/Lincoln LabMIT/Lincoln LabNWS Regional HQ,NWS Regional HQ, NCEP Centers, RFCsNCEP Centers, RFCs


LDM Server

These already exist!!

UniversitiesUniversitiesNOAA LaboratoriesNOAA LaboratoriesNOAA Joint InstitutesNOAA Joint InstitutesNCAR/UCARNCAR/UCARMIT/Lincoln LabMIT/Lincoln LabNWS Regional HQ,NWS Regional HQ, NCEP Centers, RFCsNCEP Centers, RFCs


LDM Server

via phone linesor commodity

Internet

Abilene Backbone(no commercial traffic)

Commodity Internet

LDM Server

LDM Server

LDM Server

LDM Server

LDM Server

LDM Server

Abilene NetworkLDM Server

LDM Server

LDM Server

LDM Server

LDM Server

LDM Server

Abilene Network

Each LDM “Hub Site” Carries all 88D Each LDM “Hub Site” Carries all 88D data on Abilene “bus”-- redundancydata on Abilene “bus”-- redundancy

LDM Server

LDM Server

LDM Server

LDM Server

LDM Server

LDM Server

HUB

HUB

HUB

HUB

HUB

HUB

HUB


LDM Server

LDM Server

LDM Server

LDM Server

LDM Server

Abilene Network

Commodity InternetCommodity Internet

Commodity InternetCommodity InternetCommodity InternetCommodity Internet


Com

mod

ity

Inte

rnet

Com

mod

ity

Inte

rnetC

omm

odity Internet

Com

modity In

ternet

LDM Server

LDM Server

LDM Server

LDM Server

LDM Server

LDM Server

Abilene Network




Com

mod

ity

Inte

rnet

Com

mod

ity

Inte

rnetC

omm

odity Internet

Com

modity In

ternet

LDM Server

LDM Server

LDM Server

LDM Server

LDM Server

LDM Server

Abilene Network




Com

mod

ity

Inte

rnet

Com

mod

ity

Inte

rnetC

omm

odity Internet

Com

modity In

ternet

LDM Server

LDM Server

LDM Server

LDM Server

LDM Server

LDM Server

Abilene Network




Com

mod

ity

Inte

rnet

Com

mod

ity

Inte

rnetC

omm

odity Internet

Com

modity In

ternet

LDM Server

LDM Server

LDM Server

LDM Server

LDM Server

LDM Server

Abilene Network




Com

mod

ity

Inte

rnet

Com

mod

ity

Inte

rnetC

omm

odity Internet

Com

modity In

ternet

Customers

LDM Server

PrivateCompany

Dedicated or

Comm

odity

LDM Server

LDM Server

LDM Server

LDM Server

LDM Server

LDM Server


LDM Server

LDM Server

LDM Server

LDM Server

LDM Server

NOAA runs its own operational ingest system but NOAA runs its own operational ingest system but allows connections to the BDDS of each NWS radar allows connections to the BDDS of each NWS radar

The CRAFT configurationThe CRAFT configuration– Is completely scalable to more nodes or radarsIs completely scalable to more nodes or radars– Is highly redundant (each major hub server contains all of Is highly redundant (each major hub server contains all of

the data)the data)– Is highly reliable (loss of a major hub has minimal impact)Is highly reliable (loss of a major hub has minimal impact)– Leverages existing infrastructure Leverages existing infrastructure – Links easily to other networks (e.g., AWIPS)Links easily to other networks (e.g., AWIPS)– Has significant capacity for future growth (dual-pol, Has significant capacity for future growth (dual-pol,

phased array)phased array)– Could have dual communication lines from each radarCould have dual communication lines from each radar– Could serve as a backup system for the NWSCould serve as a backup system for the NWS

Features of this ConceptFeatures of this Concept

Many variants existMany variants exist May require enhancements to LDM, e.g., multi-May require enhancements to LDM, e.g., multi-

castcast Must consider support of LDM to the Must consider support of LDM to the

commercial sectorcommercial sector Key point is to create a national hierarchical Key point is to create a national hierarchical

distribution system along the lines of the distribution system along the lines of the current Unidata IDDcurrent Unidata IDD

Features of this ConceptFeatures of this Concept

Source

LDM

Source

Source

LDM LDM

LDMLDM

LDM LDM

LDM

LDM

Internet

Primary Ingest Node

Source

LDM

Source

Source

LDM LDM

LDMLDM

LDM LDM

LDM

LDM

InternetRelay Node

Source

LDM

Source

Source

LDM LDM

LDMLDM

LDM LDM

LDM

LDM

Internet

Leaf Nodes

8 Scenarios (6 Detailed) 8 Scenarios (6 Detailed) and and

Provisional CostsProvisional Costs

Scenario #1Scenario #1: Maintain the current system : Maintain the current system of 58 radars with OU as the single ingest of 58 radars with OU as the single ingest nodenode– AssumptionsAssumptions

Line charges paid by same groups as now, at the Line charges paid by same groups as now, at the same ratessame rates

Possible ScenariosPossible Scenarios

Scenario #1Scenario #1: Maintain the current system of : Maintain the current system of 58 radars with OU as the single ingest node58 radars with OU as the single ingest node– AssumptionsAssumptions

Line charges paid by same groups as now, at the same Line charges paid by same groups as now, at the same ratesrates– 6 Sea Grant sites: 6 Sea Grant sites: $31K/year $31K/year– 6 SRP sites6 SRP sites $72K/year $72K/year– 21 MIT sites21 MIT sites $200K/year$200K/year– 4 Florida sites4 Florida sites $5K/year $5K/year– 10 OU sites10 OU sites $80K/year $80K/year– 11 other sites FSL, NASA, GTRI, SLC, RAP, SEA (no cost 11 other sites FSL, NASA, GTRI, SLC, RAP, SEA (no cost

estimates available)estimates available) Total leveraging is ~ $450,000 per yearTotal leveraging is ~ $450,000 per year


Scenario #1Scenario #1: Maintain the current system of 58 : Maintain the current system of 58 radars with OU as the single ingest noderadars with OU as the single ingest node– AssumptionsAssumptions

Line charges paid by same groups as now, at the same ratesLine charges paid by same groups as now, at the same rates No significant s/w development or 7x24 QOSNo significant s/w development or 7x24 QOS Maintain current OU staff levels (C. Sinclair at 1.0 FTE and S. Hill at Maintain current OU staff levels (C. Sinclair at 1.0 FTE and S. Hill at

0.5 FTE)0.5 FTE) $20K for h/w replacement, $10K for travel (per year)$20K for h/w replacement, $10K for travel (per year) $1K for supplies (per year)$1K for supplies (per year) KD, DJ, DE at 1 month each (1.0 FTE) (per year)KD, DJ, DE at 1 month each (1.0 FTE) (per year)

– Yearly cost: Yearly cost: $355,000$355,000 (could be reduced by shifting some (could be reduced by shifting some existing lines to cheaper alternatives)existing lines to cheaper alternatives)

– AdvantagesAdvantages No additional h/w costs (above replacement)No additional h/w costs (above replacement) Continue using a proven reliable infrastructureContinue using a proven reliable infrastructure


– DisadvantagesDisadvantages Not all radars are includedNot all radars are included Continue with heterogeneous communications Continue with heterogeneous communications

infrastructure, latency problemsinfrastructure, latency problems Relies on existing groups to continue paying their local Relies on existing groups to continue paying their local

costscosts Little increase in QOS (i.e., no 7x24)Little increase in QOS (i.e., no 7x24) 56K lines will continue to fall behind in weather56K lines will continue to fall behind in weather Single ingest system at OU provides no redundancySingle ingest system at OU provides no redundancy Reliance upon university for private sector mission-critical Reliance upon university for private sector mission-critical

needsneeds No clear path to deal with data volume increase; however, No clear path to deal with data volume increase; however,

this may not be critical if NWS system is available relatively this may not be critical if NWS system is available relatively soonsoon


Scenario #2Scenario #2: Same as Scenario #1, but add the : Same as Scenario #1, but add the remaining 64 NWS radarsremaining 64 NWS radars– Additional assumptionsAdditional assumptions

New CAPS technical staff member ($40K/year) for QOS and other New CAPS technical staff member ($40K/year) for QOS and other workwork

$100K in one-time costs for PCs$100K in one-time costs for PCs $200K for one-time line installation costs and routers$200K for one-time line installation costs and routers $50K in travel$50K in travel $5K for supplies$5K for supplies $50K in h/w replacement costs and hot spares$50K in h/w replacement costs and hot spares 30 new lines cost average of current OU lines; rest cost $50/month 30 new lines cost average of current OU lines; rest cost $50/month

based on DSL/cable modem based on DSL/cable modem

– Year-1 cost: Year-1 cost: $1.3M$1.3M (could be reduced by shifting some (could be reduced by shifting some existing lines to cheaper alternatives)existing lines to cheaper alternatives)

– Beyond Year-1: Estimate Beyond Year-1: Estimate $900,000/year$900,000/year


– AdvantagesAdvantages No additional h/w costs (above replacement)No additional h/w costs (above replacement) Continue using a proven reliable infrastructureContinue using a proven reliable infrastructure All 120 NWS radars availableAll 120 NWS radars available Improved QOS via 2Improved QOS via 2ndnd OU staff person OU staff person



infrastructure, latency problemsinfrastructure, latency problems Relies on existing groups to continue paying their Relies on existing groups to continue paying their

local costslocal costs Little increase in QOS (i.e., no 7x24)Little increase in QOS (i.e., no 7x24) 56K lines will continue to fall behind in weather56K lines will continue to fall behind in weather Single ingest system at OU provides no redundancySingle ingest system at OU provides no redundancy Reliance upon university for private sector mission-Reliance upon university for private sector mission-

critical needscritical needs


Scenario #3Scenario #3: Same as Scenario #2, but add : Same as Scenario #2, but add UCAR as a second Abilene ingest nodeUCAR as a second Abilene ingest node– Additional assumptionsAdditional assumptions

$100K in computer hardware at UCAR$100K in computer hardware at UCAR One new UCAR technical staff member One new UCAR technical staff member

– Year-1 cost: Year-1 cost: $1.5M$1.5M (could be reduced by shifting (could be reduced by shifting some existing lines to cheaper alternatives)some existing lines to cheaper alternatives)

– Beyond Year-1: Estimate Beyond Year-1: Estimate $1.2M/year$1.2M/year– Note: Could possibly add MIT/LL as third Note: Could possibly add MIT/LL as third

redundant node, but this has not been discussed redundant node, but this has not been discussed with themwith them


– AdvantagesAdvantages No additional h/w costs (above replacement)No additional h/w costs (above replacement) Continue using a proven reliable infrastructureContinue using a proven reliable infrastructure All 120 NWS radars availableAll 120 NWS radars available Improved QOS via 2Improved QOS via 2ndnd OU staff person OU staff person Greatly improved redundancy, reliability, Greatly improved redundancy, reliability,

latencieslatencies





critical needs (not clear that UCAR can provide critical needs (not clear that UCAR can provide needed QOS)needed QOS)


Scenario Summaries (1-3)Scenario Summaries (1-3)No. No.

RadarRadarss

Ingest Ingest NodesNodes

CommunicationsCommunications

InfrastructureInfrastructureQOSQOS Yearly CostYearly Cost

Scenario 1Scenario 1 5858 OUOU Current Current Heterogeneous Heterogeneous

Mix*Mix*

LowLow $0.36M$0.36M


Mix*Mix*

MedMed $1.3M (Yr 1)$1.3M (Yr 1)

$0.9M (Yr 2)$0.9M (Yr 2)

Scenario 3Scenario 3 122122 OU & OU & UCAR**UCAR**

Current Current Heterogeneous Heterogeneous

Mix*Mix*

HighHigh $1.5M (Yr 1)$1.5M (Yr 1)

$1.2M (Yr 2)$1.2M (Yr 2)* Leverages $450K/year paid by other organizations* Leverages $450K/year paid by other organizations** Could try and add MIT/LL as third node** Could try and add MIT/LL as third node

Scenario #4Scenario #4: Same as Scenario #3, but with a : Same as Scenario #3, but with a national telecommunications carrier providing national telecommunications carrier providing uniform delivery service to the additional 64 radars uniform delivery service to the additional 64 radars onlyonly– Additional assumptionsAdditional assumptions

AT&T line costs for 2-year contract for 64 additional radars is AT&T line costs for 2-year contract for 64 additional radars is $850,000/year. $850,000/year.

Mixture of T1, DSLMixture of T1, DSL Note that these costs have not been negotiated and likely could Note that these costs have not been negotiated and likely could

be reduced substantially (might also be able to eliminate T1 lines)be reduced substantially (might also be able to eliminate T1 lines) Removes need for one-time installation charges and router costsRemoves need for one-time installation charges and router costs Still have the costs of the 64 new LDM PCs Still have the costs of the 64 new LDM PCs

– Yearly cost: Yearly cost: $2.1M$2.1M (hope this could be brought down to (hope this could be brought down to $1.6 or $1.7M$1.6 or $1.7M with tough negotiation) with tough negotiation)


– AdvantagesAdvantages No additional h/w costs (above replacement)No additional h/w costs (above replacement) Continue using a proven reliable infrastructureContinue using a proven reliable infrastructure All 120 NWS radars availableAll 120 NWS radars available Improved QOS via 2Improved QOS via 2ndnd OU staff person OU staff person Greatly improved redundancy, reliability, Greatly improved redundancy, reliability,

latencieslatencies Uniform networking for 64 radarsUniform networking for 64 radars QOS should be much higher (AT&T rapid QOS should be much higher (AT&T rapid

response)response)


– DisadvantagesDisadvantages Not all radars are includedNot all radars are included PARTLYPARTLY heterogeneous communications heterogeneous communications



critical needscritical needs



RadarRadarss





Mix*Mix*

LowLow $0.36M$0.36M


Mix*Mix*

MedMed $1.3M (Yr 1)$1.3M (Yr 1)

$0.9M (Yr 2)$0.9M (Yr 2)



Mix*Mix*


$1.2M (Yr 2)$1.2M (Yr 2)


AT&T for New 64 AT&T for New 64 RadarsRadars

HighHigh $1.6 to $2.1M$1.6 to $2.1M* Leverages $450K/year paid by other organizations* Leverages $450K/year paid by other organizations** Could try and add MIT/LL as third node** Could try and add MIT/LL as third node

Scenario #5Scenario #5: Same as Scenario #4, but with a : Same as Scenario #4, but with a national telecommunications carrier providing national telecommunications carrier providing uniform delivery service to uniform delivery service to all radarsall radars– Additional assumptionsAdditional assumptions

AT&T line costs for 2-year contract for all radars is $1.4M/year. AT&T line costs for 2-year contract for all radars is $1.4M/year. Mixture of T1, DSLMixture of T1, DSL Note that these costs have not been negotiated and likely could Note that these costs have not been negotiated and likely could

be reduced substantially (might also be able to eliminate T1 be reduced substantially (might also be able to eliminate T1 lines)lines)

Removes need for one-time installation charges and router costsRemoves need for one-time installation charges and router costs Still have the costs of the 64 new LDM PCs Still have the costs of the 64 new LDM PCs

– Yearly cost: Yearly cost: $2.8M$2.8M (hope this could be brought down (hope this could be brought down to to $2.2 or $2.3M$2.2 or $2.3M with tough negotiation) with tough negotiation)


– AdvantagesAdvantages No additional h/w costs (above replacement)No additional h/w costs (above replacement) Continue using a proven reliable infrastructureContinue using a proven reliable infrastructure All 120 NWS radars availableAll 120 NWS radars available Improved QOS via 2Improved QOS via 2ndnd OU staff person OU staff person Greatly improved redundancy, reliability, latenciesGreatly improved redundancy, reliability, latencies Uniform networking for Uniform networking for ALLALL radars radars QOS should be much higher (AT&T rapid response)QOS should be much higher (AT&T rapid response) Increased bandwidth needs (e.g., dual-pol, new Increased bandwidth needs (e.g., dual-pol, new

VCP, ¼ km by ½ degree resolution) could be VCP, ¼ km by ½ degree resolution) could be handled by the telecomm carrier “automatically”handled by the telecomm carrier “automatically”


– DisadvantagesDisadvantages Not all radars are includedNot all radars are included PARTLY heterogeneous communications PARTLY heterogeneous communications


local costslocal costs Little increase in QOS (i.e., no 7x24)Little increase in QOS (i.e., no 7x24) 56K lines will continue to fall behind in weather56K lines will continue to fall behind in weather Single ingest system at OU provides no Single ingest system at OU provides no

redundancyredundancy Reliance upon university for private sector Reliance upon university for private sector

mission-critical needsmission-critical needs



RadarRadarss





Mix*Mix*

LowLow $0.36M$0.36M


Mix*Mix*

MedMed $1.3M (Yr 1)$1.3M (Yr 1)

$0.9M (Yr 2)$0.9M (Yr 2)



Mix*Mix*


$1.2M (Yr 2)$1.2M (Yr 2)


AT&T for New 64 AT&T for New 64 RadarsRadars

HighHigh $1.6 to $2.1M$1.6 to $2.1M


AT&T for ALL AT&T for ALL RadarsRadars

HighHigh $2.2 to $2.8M$2.2 to $2.8M* Leverages $450K/year paid by other organizations* Leverages $450K/year paid by other organizations** Could try and add MIT/LL as third node** Could try and add MIT/LL as third node

Scenario #6Scenario #6: Use NWS River Forecast : Use NWS River Forecast Centers as points of aggregationCenters as points of aggregation– MayMay make sense only if the NWS wishes to make sense only if the NWS wishes to

pursue a non-AWIPS collection strategypursue a non-AWIPS collection strategy– The general CRAFT concept still could be The general CRAFT concept still could be

appliedapplied Scenario #7Scenario #7: Use the Planned NWS : Use the Planned NWS

Distribution SystemDistribution System Scenario #8Scenario #8: Create a System Operated : Create a System Operated

Entirely by the Private Sector (no university Entirely by the Private Sector (no university or UCAR involvement)or UCAR involvement)

Other ScenariosOther Scenarios

Points of Reference (for the sake of argument)Points of Reference (for the sake of argument)– Must be able to ensure 7x24 service (high reliability)Must be able to ensure 7x24 service (high reliability)– Latency must be as low as possibleLatency must be as low as possible– Government receives data at no cost but could/should cost Government receives data at no cost but could/should cost

share overall expenses in light of benefits to NCDC (direct share overall expenses in light of benefits to NCDC (direct ingest for long-term archive), NCEP, FSL, NWS Offices (Level II ingest for long-term archive), NCEP, FSL, NWS Offices (Level II recorders)recorders)

– Educational institutions receive data at no costEducational institutions receive data at no cost– Presumably don’t want another “NIDS arrangement”Presumably don’t want another “NIDS arrangement”

OptionsOptions– For-profit private companyFor-profit private company– University-based consortiumUniversity-based consortium– Not-for-profit 501(c)3 Not-for-profit 501(c)3 – University-based center (e.g., Wisconsin for satellite data)University-based center (e.g., Wisconsin for satellite data)– Others?Others?

Administrative StructureAdministrative Structure

Sustaining the operation of CRAFT beyond NovemberSustaining the operation of CRAFT beyond November Establishing private sector requirementsEstablishing private sector requirements

– ReliabilityReliability– LatencyLatency– Hardware and software supportHardware and software support

Meeting private (and academic) sector needs in the Meeting private (and academic) sector needs in the short, medium and long termshort, medium and long term

Administrative issues (including data access rules)Administrative issues (including data access rules) Dealing with future data volumesDealing with future data volumes Further analysis of system capabilitiesFurther analysis of system capabilities

– Impact of weather on data reliability/latencyImpact of weather on data reliability/latency– Networking simulationNetworking simulation

Key Items for DiscussionKey Items for Discussion

Documents

The Collaborative Radar Acquisition Field Test (CRAFT): Next Steps Kelvin K. Droegemeier University of Oklahoma 2 nd Level II Stakeholders Workshop 26-27