Field Area Network for Multiple Grid Apps – Roadmap into ...€¦ · Selection: MESH 55 “takeout” points ... Fiber Backhaul available from each regional center Microwave from

your most powerful membership™

Tom Guttormson, PEPrinciple Technology Engineer

1

Field Area Network for Multiple Grid Apps –Roadmap into Deployment

Tom GuttormsonPrincipal Technology Engineer

MINNESOTA POWER SYSTEMS CONFERENCENovember 6-8, 2018__________________University of MN College of Continuing and Professional Studies


130,000 member customers1,000 square miles territory44 substations12.5 kV 9,000 miles of line92% residential by customerSystem peak demand 550 MWWholesale Power Supplier –

Great River Energy


REVIEW OF THE LANDSCAPE: SYSTEMS

Substation SCADA – GRE 700 MHz system Customer Owned Generation – 900 MHz to cellular radio Recent transition to digital LMR

• LMR skip – safety issue• 900 MHz propagation issues, subject to bandwidth narrowing• 700 MHz “tunneling” (a.m.)• Existing tower – 1949 vintage, not structurally

CURRENT SYSTEMS

ISSUES


REVIEW OF THE LANDSCAPE: APPS

VVC – need FAN to capacitor and line regulator banks AMI planning in the works… FAN network; mesh or PTMP? (towers) Existing tower needs to go Substation SCADA - GRE frequency shopping (existing 700 MHz

hardware sunset)

CURRENT APPS

MOVING FORWARD

More DG – solar? Remote switching? Line Sensors? GRE LM system sunset

Need a Plan!


ROADMAP

Resolve current issues Meet current and short term apps Set up to expand for future apps Scalable – sufficient for current, expandable for future Cyber Security

• FAN• Backhaul• Resolve tower – replace, relocate, or…. ??

GOALS

SCOPE


PLAN DEVELOPMENT

Review landscape; what do we have to start with? Define bandwidth needs Define reliability requirements (level of 9’s) Identify com systems to support requirements Inventory resources and options

Develop optional plans


Inventory


OPTI

ON

SC

ON

SIDERED:Tier 1 –

Backbone – primary looped pathway –key tower sites and substations

Tier 2 –Backbone – network to substations from Tier 1 loop

Tier 3 –“FAN” (Field Area Network) – Feeder communications for AMI (take-out points), DG, DA

Network Strategy


BANDWIDTH NEEDS – COM STUDY ESTIMATES:

Low End Usage High End Usage

Bandwidth Requirement during Comm Peak

All Applications

Theoretical Throughput SCADA,

DA and AMI Requirement - kbps

Bandwidth Requirement during Comm Peak

All Applications

Theoretical Throughput SCADA,

DA and AMI Requirement - kbps

230 130 1360 1030


AMI PICTURE…

Business Basis Based on best cost Primary applications – reading meters DA applications – considered, but not primary business case

Options considered / reviewed: P to MP (tower based) Mesh Call Radio

Selection: MESH 55 “takeout” points – all substations + a few tall poles Need backhaul…


FIBER COMBINATION


MICROWAVE


INITIAL STUDY OPTIONS - COSTS

Tier 1 & 2 Options Considered: Fiber (by others with lease agreements) Microwave GRE new 700 MHz system Best fit hybrid of the above

Tier 1 and 2 CombinedAlternatives

30 Year Capital Costs

All Fiber – Connexus and GRE Only $ 7,800

All Fiber – Connexus/GRE/Zayo $ 6,500

All Microwave $ 3,800

All Costs are in $1,000s





LEAST COST BROADBAND STRATEGY

Fiber or Microwave Broadband: Use GRE fiber where (and where it becomes) available Establish Regional centers West: Palmer tower East: MNDot tower (at MN Correctional Facility) Fiber Backhaul available from each regional center Microwave from regional centers as redundant link to fiber

Microwave to Select Subs from Regional Centers Lets try a few to start…

Subs Without Broadband: GRE 700 MHz for SCADA Cell radio for AMI

“Hybrid”


WEST REGIONAL CENTER

EAST REGIONAL CENTER

RAMSEY CENTER

REGIONAL CENTERS


REGIONAL SPOKES


RESOLVING ISSUES:

LMR – cellular radio backhaul incompatibility Replacement of cell radio backhaul with fiber/MW

GRE 700 MHz Hardware sunset – replaced with Full Spectrum system Tunneling – cell radio fail-over

Old Anoka Tower Replacement 199’ Monopole Construction Located “out the back door”


EASTHUB

–HGO

TOWERVS. MNDOT TOWERLEASE

Full scale scenario – maximum attachments 30 yr NPV comparison HGO alternative includes Purchase and resale of land at end or period Rent revenue from one tenant

“build-a-tower” “rent-a-tower”Up front capital $740K $320K

30 NPV $532K $574K

“Build-a-Tower” vs “ Rent-a-Tower”


MICROWAVE – SUBSTATIONS OR “AREA HOPS”

Pole, Antenna, Cabling• $20K/location

Connection Hardware:• Pad• Cabinet (climate control, battery backup, grounding)• Network switch• $35K/location

TOTAL - $55K / location


FAN – DA / DER APPLICATIONS

Current: No. Units Customer-Owned Generators ~40 VVC (Volt Var Control) – cap banks ~180

In Progress Solar / Storage 3

Future (potential) TBD Line Sensors Feeder Switching More DER Micro-grids??


FAN – OPTIONS CONSIDERED

“Full Spectrum” – fixed area network 220 MHz 770 MHz EPRI field trial with GRE900 MHz Licenses (we own 1 channel) Avoiding 900 MHz unlicensed – preferring fixed network option

Cell radio Short term – until cost competitive fixed network available Long term – if above not

Cap Ex Op ExFixed Network High LowCell Radio Low High


CELL RADIO NETWORK (CONT.)

Need to maintain low bandwidth for pricingAve Bandwidth Traffic per site: 47Mbits / month Polling rate – 5 minute intervals, or report by exception


CELL RADIO NETWORK – IMPLEMENTATION, DISCOVERIES….

Sierra Gateway modem at capacitor control Employing AT&T, Verizon, Sprint IP straight through headend to control – limited choices: Not many suppliers have controls with IP com port

IP from headend to modem, serial connection modem to control Heavy bandwidth usage – high overhead If errors going on

Using UDP Reduces bandwidth need, however Does not provide auto correction (IP provides more info on session and

provides auto-retry) Make sure up-to-date on firmware


FAN LESSONS LEARNED –

Standards / interoperability – Not all “standards” are compatible between vendors’ systems

Thoroughly vet equipment against requirements – document, challenge…

If using cell radio, set up to enable multiple carriers Apply structured testing in lab before full scale deployment Map internal processes – identify / define responsibilities in areas

of IT/OT overlap


NEXT STEPS…

Cyber Security Enhancements Reviewing Line Sensors Continuing conversation with AMI provider for DA interface Adding TIER 1 and 2 fiber as becomes cost effective & available Expand Tier 1 and 2 Microwave

Documents

Field Area Network for Multiple Grid Apps – Roadmap into ...€¦ · Selection: MESH 55 “takeout” points ... Fiber Backhaul available from each regional center Microwave from