Upload
others
View
7
Download
0
Embed Size (px)
Citation preview
Design and Planning Tools
Topic 1.4.18 - Computational Science for Grid Management
Mihai AnitescuArgonne National LaboratoryNovember 2016
Grid Modernization Initiative
The Energy Department’s Grid Modernization Initiative (GMI) represents a comprehensive effort to help shape the future of our nation’s grid and solve the challenges of integrating conventional and renewable sources with energy storage and smart buildings, while ensuring that the grid is resilient and secure to withstand growing cybersecurity and climate challenges.
Through the Grid Modernization Multi-Year Program Plan (MYPP), the U.S. Department of Energy (DOE) will coordinate a portfolio of activities to help set the nation on a cost-effective path to an resilient, secure, sustainable, and reliable grid that is flexible enough to provide an array of emerging services while remaining affordable to consumers.
11/22/2016
Argonne National Laboratory ---- Mathematics and Computer Science Division
2
Design and Planning Tools (from MYPP)
Design and Planning Tools Incorporate uncertainty and system dynamics into reliability planning tools to
accurately capture effects of renewable generation Coupling grid transmission, distribution, and communications models to
understand cross-domain effects Advanced production cost modeling used to understand cost-benefit tradeoff of
the mix and location of storage and solar
11/22/2016
Argonne National Laboratory ---- Mathematics and Computer Science Division
3
MYPP supported areas
Task 4.3.3: Develop efficient linear, mixed-integer, and nonlinear mixed-integer optimization solution techniques customized for stochastic power system models, novel bounding schemes to use in branch and bound, and structure exploiting algorithms. Demonstrate the cost-benefit achieved by these techniques relative to existing ones. [FY18-20]
Task 4.3.4: Demonstrate the application of parallel and distributed computing algorithms on existing (multiprocessor, multi-core, and GPU) and emerging (many-core and co-processors) computational platforms to significantly improve the performance of applications and methods proposed in Tasks 4.3.1 – 4.3.3 at speeds an order of magnitude faster than is possible presently. [FY19-20]
Task 5.3.5: Develop and distribute advanced libraries of algorithms, solvers, uncertainty quantification, and stochastic optimization modules. [FY16-FY20]
Task 5.3.6: Develop computing frameworks that enable the coupling of advanced computation tools, data, and visualization technologies with easy workflow management. [FY16-FY20]
After proposal round, decision to make this activity a primarily frameworks definition activity; supporting optimization+uncertainty+dynamics, with solvers to be adjusted as needed, but not major significant research component.
11/22/2016
Argonne National Laboratory ---- Mathematics and Computer Science Division
4
The need/Project vision
Drivers: – Increased renewable penetration results in variability that is best captured by stochastic
representations. – GMLC’s goal of a 33 percent decrease in cost of reserve margins, while maintaining
reliability, requires more accuracy, and thus more computation. – An example is the drive to move when possible off-line analyses online in order to
contribute to the margin reduction.
Vision: – Allowing for uncertainty in the analyses increases the problem size in proportion to the
number of scenarios; while aiming for more accuracy and online performance (where appropriate) requires high performance computing, and in particular massive parallelism.
– Create the environment where computational analyses involving optimization, uncertainty and dynamics are expressed to take full advantage of solver features (e.gdecomposition)
– Increase the accuracy and decrease the time to solution of such analyses to fulfill the objectives above.
11/22/2016
Argonne National Laboratory ---- Mathematics and Computer Science Division
5
Project structure/team
Advanced Modeling, Integration, and Computation Frameworks (AMICF) Henry Huang (PNNL) and Wesley Jones (data, NREL)
Optimization (Mihai Anitescu ANL, Cosmin Petra LLNL) Dynamics (Slaven Peles, LLNL) Uncertainty (Jean Paul Watson, SNL; Russell Bent LANL)
Concept: Define use cases spanning the space of the tasks 4.3.3, 4.3.4, 5.3.5.,and 5.3.6
Employ these use cases to define the framework and adjust the solvers to solve large problems efficiently.
Framework/Solvers should be: – Scalable– Portable (Multicore—smart), Open– Extensible, Expressive – Fast– Compact
11/22/2016
Argonne National Laboratory ---- Mathematics and Computer Science Division
6
Lab Capabilities
National Labs Have Excellent Computational Capabilities Numerical Linear Algebra Solvers: PETSc, Trilinos Time Stepping Solvers: SUNDIALS, PETSc, Stochastic Optimization PySP, PIPS, DSP (When solving integer programming, primarily calling a commercial solver, but for
nonlinear continuous optimization fully contained solution).
We aim to make them usable in this context and provide value to GMLC Usable on many architectures Able to scale to many processors to provide timely solutions. Open, Free.
11/22/2016
Argonne National Laboratory ---- Mathematics and Computer Science Division
7
Dynamics Solvers: Overview of SUNDIALSSUite of Nonlinear and DIfferential/ALgebraic Equation Solvers
Developed and supported by Lawrence Livermore National Laboratory (LLNL) Freely available, released under BSD license; Over 4,500 downloads per year. Forward looking, extensible object oriented design with clean linear solver and
vector interfaces. Designed to be incorporated into existing codes with modular structure that allows users to supply their own data structures.
Scales well in simulations on over 500,000 cores. Modules and functionality:
– ODE integrators: (CVODE) variable order and step stiff BDF and non-stiff Adams, (ARKode) variable step implicit, explicit, and additive Runge-Kutta for IMEX approaches.
– Differential-algebraic equation integrator: (IDA) variable order and step stiff BDF.– CVODES and IDAS include forward and adjoint sensitivity capabilities.– KINSOL nonlinear solver: Newton-Krylov and accelerated fixed point and Picard
methods.
Used in industrial and academic applications worldwide. Examples include:– Power grid modeling (RTE France, ISU)– Electrical and heat generation within battery cells (CD-adapco)– Power transmission modeling and simulations (Griddyn, LLNL)– Empirically based simulations of neuron networks (Neuron, Yale University)
11/22/2016
Argonne National Laboratory ---- Mathematics and Computer Science Division
8
Dynamics Solvers: PETSc-TS: Scalability results for 5-sec simulation of a 20000 node power grid:
9
0.00
2.00
4.00
6.00
8.00
1 2 4 8 16 24
Exec
utio
n tim
e (s
ec)
# Cores
1.00
4.00
16.00
64.00
1 2 4 8 16 24
Exec
utio
n tim
e (s
ec)
# Cores
Overlapping Schwarz preconditioningAdaptive time-stepping
Faster than real-time
Slower than real-time
Faster than real-time
Slower than real-time
Stable Scenario
Unstable Scenario
Optimization Solvers: Stochastic programming –computational patterns
Computational pattern
10
Optimization
Deterministic model Effects of uncertainty
OptimizationRecourse problem outcome 1
OptimizationRecourse problem outcome 2
OptimizationRecourse problem outcome N
1st stage computations
2nd stage computations
Mihai Anitescu - Optimization under uncertainty
PIPS-IPM on power grid models
11
Not used on high-end supercomputer for this app, but nice to know we can. Strong scaling == same efficiency on 100-1000 cores and new multicore architectures
4 billion decisions variables4.1 billion constraints(32,768 scenarios)
Relaxation of 24-hour unit commitmentSolution time: < 40 mins
Parallel efficiency > 90%
FLOPS peak performance 12%
How do we tap performance? – A good framework
Library designer’s view of an application Example MATLAB
– SYSTEM=SIMULINK– MATH MODEL FRAMEWORK=“f” encoding
dynamics, provides derivatives for implicit integration, encodes sparsity
– SOLVERS=ODESUITE/LINPACK/LAPACK
To achieve best performance, the model/framework block should allow algebraic access if possible.
It should provide the functionality required for performance
– Gradient– Sparsity – Annotating Decomposition Patterns– …..
Once a good model framework exists, customization to GUI is ”easy”
11/22/2016
Argonne National Laboratory ---- Mathematics and Computer Science Division
12
SYSTEM (DATA INPUT/VIZ)
MATH MODEL FRAMEWORK
LIBRARY
HARDWARE
Framework
Desktop, Cluster, Cloud, GPU, ….
Other Frameworks Examples
In operations/research optimization, we tend to express problems directly in the MODEL, without the SYSTEM block.
– SYSTEM == None (hookups to Matlab for data/viz and solver exist)– MODEL==E.g AMPL, GAMS, AIMMS, (commercial), PyOMO, JuMP”/Julia (Open Source);
all support algebraic expression representation. – LIBRARY=CPLEX,IPOPT, etc
PLEXOS – SYSTEM = PLEXOS GUI– MODEL == AMMO (proprietary, roughly functionally equivalent to AMPL, etc), – LIBRARY=ExpressMP, MOSEK, GUROBI, …– Deane JP, et al., Soft-linking of a power systems model to an energy systems model,
Energy (2012), doi:10.1016/j.energy.2012.03.052
Many (most?) focused computational solutions architectures have separate SYSTEM and MODEL layers
11/22/2016
Argonne National Laboratory ---- Mathematics and Computer Science Division
13
from coopr.pyomo import *
m = ConcreteModel()
m.x1 = Var()m.x2 = Var(bounds=(-1,1))m.x3 = Var(bounds=(1,2))
m.obj = Objective( sense = minimize,expr = m.x1**2 + (m.x2*m.x3)**4 +
m.x1*m.x3 + m.x2 +m.x2*sin(m.x1+m.x3) )
model = m
Pyomo Overview
Formulating optimization models natively within Python– Provides a natural syntax to describe mathematical models– Enables data import and export in commonly used formats– Supports Natively ODEs as constraints (e.g gas/electricity codispatch) – Annotates Decomposition for solvers (for usage with PySP).
Highlights:– Clean syntax– Python scripts provide
a flexible context forexploring the structureof Pyomo models
– Leverage high-quality third-party Python libraries, e.g., SciPy, NumPy, MatPlotLib
Supports ODEs and PDEs Gas PDE-Network Model
# General Declarations for Modelfrom coopr.pyomo import *from coopr.dae import *
m = ConcreteModel()
m.lp = Set(initialize=[0,1,2])m.la = Set(initialize=[2,3,4])m.l = m.lp | m.la # Union of the sets
m.A = Param(m.l, initialize=10)m.rho = Param(m.l, initialize=10)m.lda = Param(m.l, initialize=10)m.D = Param(m.l, initialize=10)
# General Declarations Continuedm.fin = Param(m.l, initialize=10)m.fout = Param(m.l, initialize=10)m.theta_rec = Param(m.l,initialize=10)m.theta_snd = Param(m.l,initialize=10)m.del_theta = Param(m.la,initialize=10)m.Lend = Param(initialize=20)
m.t = ContinuousSet(bounds=(0,10))m.x = ContinuousSet(bounds=(0,value(m.Lend)))
m.p = StateVar(m.l,m.t,m.x)m.f = StateVar(m.l,m.t,m.x)
Gas PDE-Network Model
# PDE Equation 1def _pde1(m,i,j,k):
return m.p[i,j,k].derivative(wrt=m.t) + 1/m.A[i]*m.p[i,j,k]/m.rho[i] * m.f[i,j,k].derivative(wrt=m.x) == 0
m.pde1 = Constraint(m.l,m.t,m.x,rule=_pde1)
# PDE Equation 2def _pde2(m,i,j,k):
return 1/m.A[i]*m.f[i,j,k].derivative(wrt=m.t) +m.p[i,j,k].derivative(wrt=m.x) +(8*m.lda[i])/(3.1415**2*m.D[i]**5)*(m.f[i,j,k]*abs(m.f[i,j,k]))/m.rho[i] == 0
m.pde2 = Constraint(m.l,m.t,m.x,rule=_pde2)
Gas PDE-Network Model
def _flowbound(m,i,j):return m.f[i,j,0] == m.fin[i]
m.flowbound = Constraint(m.l,m.t,rule=_flowbound)
def _fupbound(m,i,j):return m.f[i,j,value(m.Lend)] == m.fout[i]
m.fupbound = Constraint(m.l,m.t,rule=_fupbound)
def _lupbound(m,i,j):return m.p[i,j,value(m.Lend)] == m.theta_rec[i]
m.lupbound = Constraint(m.l,m.t,rule=_lupbound)
def _llowboundp(m,i,j):return m.p[i,j,0] == m.theta_snd[i]
m.llowboundp = Constraint(m.lp,m.t,rule=_llowboundp)
def _llowbounda(m,i,j):return m.p[i,j,0] == m.theta_snd[i] + m.del_theta[i]
m.llowboundp = Constraint(m.la,m.t,rule=_llowbounda)
A Julia Open Representation for the Modeling Layer
– Julia – a fresh approach to scientific and technical computing high-level, high-performance, open-source dynamic language for technical computing keeps productivity of dynamic languages without giving up speed (2x of C/C++/Fortran) “writes like Matlab runs like C”, parallel support via MPI is a breeze
Very fast to create AML – e.g JuMP for optimization
Macros instead of operator overloading (known to have poor performance) Also
18
StructJuMP modeling and solving scenario-like nonlinear optimization problems in parallel
19
Structured extreme-size nolinear optimization problems
JuMPAutoDiffForwardDiffReverseDiffSparse
parallel optimization solver(PIPS, etc.)
parallel evaluationStructJuMP
Algebraic modeling front-end Easy-to-express, compact
mathematical syntax Parallel model manipulation
and function/derivatives evaluation using MPI
Does AD with a revisited edge pushing
A novel data structure to deal with irregular memory access with multiple lookups for each nonzero in the Hessian
Julia/JuMP--almost-C performance
Yet, solver agnostic
Parallel computing back-end Parallelism is present through
the entire stack: instantiation, AD, and optimization solver
No knowledge on parallel computing is required from the user.
Runs on “Blues” @ ANL ….
20
• NLP SC-ACOPF model: 300 buses, 411 lines, 69 generators
• With new (and faster) derivative computation
#procs
Model initiation (seconds)
Structure building
(seconds)
Function & derivative
evaluation (seconds)
Total time (seconds)
1 6.42 2.09 20.56 390.342 4.70 1.59 10.19 279.154 4.21 1.58 5.96 230.168 4.10 1.49 3.5 208.48
16 4.14 1.46 1.86 192.8524 4.09 1.42 1.47 179.9648 3.96 1.31 0.72 191.75
Stochastic LP economic dispatch model for Illinois
.. and on the cloud too; (with parallel support)
21
So this is an example of extensible, expressive, compact, scalable and fast integrative framework
How about Modelica? Modelica is a popular open-source algebraic framework mostly for dynamical
simulations (sort of like simulink); car industry and power plant simulation. Factor of 10 in development time reduction reporter for Building apps (Wetter and
Haugstetter) However, in 2016 Hilding Elmqvist (Modelica lead designer), advocated move to julia
(the Modia project)… “The function part of Modelica is, however, not rich enough. There are no advanced data structures such as union types, no matching construct, no type inference, etc.” .. Need to move to a LANGUAGE.
He gives example of Julia compactness v. modelica (due to type and size inference)
11/22/2016
Argonne National Laboratory ---- Mathematics and Computer Science Division
22
Elmqvist, Hilding, Toivo Henningsson, and Martin Otter. "Systems modeling and programming in a unified environment based on Julia." In International Symposium on Leveraging Applications of Formal Methods, pp. 198-217. Springer International Publishing, 2016.
Summary
We aim to drive the advances and demonstrate value of HPC in grid with use cases.
Such use cases drive the framework design that enables maximum performance in conjunction with advanced solvers.
Framework (with solvers) aims to be – Scalable– Portable (Multicore—smart), Open– Extensible, Expressive – Fast– Compact
May run on cloud or cluster?
There is the reality of limited time and resources. Do we have the right use cases to drive frameworks for HPC in grid?
11/22/2016
Argonne National Laboratory ---- Mathematics and Computer Science Division
23
USE CASES
GMLC Computation – Project 1.4.18
11/22/2016
Argonne National Laboratory ---- Mathematics and Computer Science Division
24
Objectives of use cases
Span a broad space of optimization+dynamics+uncertainty challenges in grid applications; but ideally should not be too many.
Have a focus on problems that need high performance computing and large parallel computing potential.
Can demonstrate intrinsic value of the analysis. Have solvers that can be tailored to solve them efficiently with moderate effort.
After the analysis of these requirements, we settled on 3 uses cases: Use Case 1: Security-Constrained OPF under uncertainty Use Case 2: Transient Security Assessment under uncertainty Use Case 3: Optimization under Uncertainty with Transient Security Constraints.
11/22/2016
Argonne National Laboratory ---- Mathematics and Computer Science Division
25
Use Case 1:Security-Constrained OPF under uncertainty
Need In current practice, power is dispatched from generators based on economic
considerations and assumes everything is deterministic. Many elements, however, are unknown, like load, renewable energy outputs,
equipment failure, etc. This solution may result in inappropriate margins (possibly even too low under
some circumstances; but too high in most) when increasing renewable penetration.
There is a need to (re)frame the OPF problem so that it explicitly models the stochasticity and then develop approaches that can tractably solve those models
11/22/2016
Argonne National Laboratory ---- Mathematics and Computer Science Division
26
Use Case 1: The Problem
Minimize objective (power generation cost)– while fulfilling the demand – satisfying the power flow physics and generation– transmission technological constraints – for a given set of prescribed contingencies (line, bus, or generation failures) – and uncertainty realizations (wind and solar power real value, equipment state).
Mathematical Expression:
11/22/2016
Argonne National Laboratory ---- Mathematics and Computer Science Division
27
Use Case 1 SCOPF under uncertainty
Applicability Preventive control considering voltage stability and penetration of renewable
energy (extension of [4]). Computation of reactive reserve margins under renewable energy uncertainty
(extension of [5]). Effect of load voltage dependence under DER deployment on reliability and
cost/pricing;
Data Needs: power system data, cost coefficients, list of contingencies, intertie or generation/load uncertainty.
Challenges: Nonlinearity of the Equations; Derivative Computation; Large Problem Size;
11/22/2016
Argonne National Laboratory ---- Mathematics and Computer Science Division
28
Use Case 2: Dynamic Security Assessment with Uncertainties
Need: With the penetration of variable energy resources the impacts of the resulting
uncertainties cannot be ignored in dynamic security assessment for both operation and planning applications.
Errors in generation and load forecasts can have a random impact on the grid. Therefore, the traditional deterministic-based applications need to be upgraded to
consider the forecast errors and be performed in a stochastics manner
11/22/2016
Argonne National Laboratory ---- Mathematics and Computer Science Division
29
Use Case 2: The problem
The impact of uncertainties introduced by forecasts of energy and loads needs to be considered in dynamic security assessment (DSA) within the operation and planning study timeframes. Such uncertainties come from both transmission and distribution levels, particularly with the increasing penetration of intermittent generation and distributed energy resources (DERs).
Components: 1. Differential Equation:
2. Network Algebraic Connectivity
3. Contingency Modeling – Contingencies in the power network changes the Y matrix because of the change in
connectivity and impedance.
4. Uncertainty Modeling– Bulk generation uncertainties affect mechanical power– Load uncertainties affect the algebraic equations – Distributed generation uncertainties affect net load the system needs to support.
11/22/2016
Argonne National Laboratory ---- Mathematics and Computer Science Division
30
−=
−−−=
sii
siieimii
s
i
dtd
DPPdt
dH
ωωδ
ωωωω
)(2
EIEY =*
Use Case 2: Dynamic Security Assessment with Uncertainties
Applicability: (1) Effect of increasing penetration of renewable generation on system stability; (2) Effect of DERs and consumer participation on bulk system operation and
planning; and (3) Optimization of DERs for multiple value streams for the power system.
Data Needs: Historical actual data, forecast data, power system model, generator parameters, and contingency list.
Challenges: computationally expensive due to a large number of DAE simulations and uncertainty scenarios. Traditional Monte Carlo simulation has scalability issues; large-scale statistical data analysis to quantify the sources of uncertainties; seamless data communication; user-friendly visualization and information presentation.
11/22/2016
Argonne National Laboratory ---- Mathematics and Computer Science Division
31
Use Case 3: Optimization under Uncertainty with Transient Security Constraints.
Need: Renewable Energy Sources will change several features of current systems. Uncertainty in production will become an element that must be addressed Loss of traditional rotating inertia will require both its consideration in the
optimization and its inclusion in contingency analysis. Emerging need:
– Consider uncertainty, dynamics, and contingency and AC physics together in the real time
– Enable large penetrations of renewable energy without affecting grid stability or higher cost via larger reserves.
This use case contains Use Case 1 and 2. However, it has a higher development risk, so we aim to define and provide solvers for 1,2 first.
11/22/2016
Argonne National Laboratory ---- Mathematics and Computer Science Division
32
Use Case 3: The Problem
11/22/2016
Argonne National Laboratory ---- Mathematics and Computer Science Division
33
Minimize objective (power generation cost)– while fulfilling the demand – satisfying the power flow physics and generation w/o contingency– Satisfying the transient contingency limits– for a given set of prescribed contingencies (line, bus, or generation failures) – and uncertainty realizations (wind and solar power real value, equipment state).
Mathematical Expression:
Use Case 3: Optimization under Uncertainty with Transient Security Constraints
11/22/2016
Argonne National Laboratory ---- Mathematics and Computer Science Division
34
Applicability Preventive rescheduling of power systems subject to stability constraints for
contingencies and uncertainty in load/generation (extension of [1]). Stability constrained interchange limits under renewable energy uncertainty
(extension of [4]). Determination of optimal reserve margins on the renewable energy uncertainty in
operations while considering transient security. Data Needs: uncertainty scenario data ; cost function h() parameters/forecasts ; system steady-state and transient constraint and evolution functions, power flow and transients : (including generator parameters for e.g swing equation) ; post-contingency limit requirement functions (post-contingency voltage and frequency limit functions). Challenges: Scalability and Accuracy of the Framework/Evaluation and Computation. Details: Efficient Expression/Computation of the Model with/for Nonlinearity (Gradients/Hessians); Fast Transient Simulation Requirements; Memory Limitations of Direct Transcription; Cost of computation of adjoint capability for transient constraints.
Questions
Are the use cases appropriate/optimal for demonstrating value of HPC for power grid applications?
Are there better use cases that combine optimization+uncertainty+dynamics?
Are there better areas of applicability for the use cases (i.e that would more clearly demonstrate value for the community/industry?)
Do these use cases have the right features or should they be modified in some way to contain other elements for example?
11/22/2016
Argonne National Laboratory ---- Mathematics and Computer Science Division
35