Open Grid SystemsOGO
RESTful CIM using the Open Data Protocol
!1
Alan McMorran B.Eng Ph.D
Open Grid SystemsOGO
CIM Exchanges!2
Open Grid SystemsOGO www.opengrid.com
CIM Data Exchange• The exchange of data using CIM has traditionally been
used in two primary ways:
• Full Model exchange as CIM RDF XML
• Message based exchange as CIM XSD
• CIM RDF XML and CIM XSD cover two common use-cases:
• CIM RDF XML supports Network Model Management (NMM) exchanges both within and between companies and organisations
• CIM XSD exchanges support intra-system exchanges using an Enterprise Service Bus and/or web services
!3
Open Grid SystemsOGO www.opengrid.com
Hierarchical vs Graph
!4
Open Grid SystemsOGO www.opengrid.com
Hierarchical vs Graph• RDF is very well suited to
non-directed graph data
• XSD is better suited for hierarchical data
• Electrical network data rarely fits neatly into a hierarchy
!5
Open Grid SystemsOGO www.opengrid.com
CIM XSD Standards• The IEC 61968 series of standards is primarily
focussed on defining CIM XSD based messages
• These messages are explicitly defined hierarchies that can be used by Web Services
• Examples include:
• IEC 61968-3 Network Operations
• IEC 61968-6 Maintenance & Construction
• IEC 61968-9 Meter Reading and Control
!6
Open Grid SystemsOGO www.opengrid.com
Message Payloads• These standards define
fixed message payloads of hierarchical CIM data
• These payloads are defined within the standard and implemented by vendors
• They are combined with standard verbs (CREATE, UPDATE, DELETE etc.)
!7
Open Grid SystemsOGO www.opengrid.com
Services• This means that every payload must be defined and
agreed between all parties
• This approach has scalability issues
• As more parties connect; more services are interconnected; and more CIM data is being shared a more flexible approach is needed
• Systems should be capable of sharing CIM data without requiring a full system export or all parties to agree a fixed subset and hierarchy for every possible service
!8
Open Grid SystemsOGO
RESTful Services!9
Open Grid SystemsOGO www.opengrid.com
What is RESTful?
• “Representational State Transfer (REST) is an architectural style that defines a set of constraints and properties based on HyperText Transfer Protocol (HTTP)"
• “REST-compliant web services allow the requesting systems to access and manipulate textual representations of web resources by using a uniform and predefined set of stateless operations”— Wikipedia
!10
Open Grid SystemsOGO www.opengrid.com
RESTful Principles• All resources are uniquely addressable, usually
through Uniform Resource Identifiers (URI)
• All resources can be manipulated through a constrained set of well-known actions, usually CRUD (Create, Read, Update, Delete)
• The data is transferred through any of a constrained number of well-known representations (e.g. HTML, XML, JSON)
• The communications between client and application are stateless allowing for multiple layers of intermediaries, caching etc.
!11
Open Grid SystemsOGO www.opengrid.com
RESTFUL Example• Read (HTTP GET)
!12
HTTP GET https://opengrid.com/myservice/001
{ "type":"example", "name":"get", "id": "001", "company":"Open Grid Systems", "web":"https://www.opengrid.com"}
Result
<No Content> Payload
Open Grid SystemsOGO www.opengrid.com
RESTful Example• Create (HTTP POST)
!13
{ "type":"example", "name":"post", "company":"Statnett", “web":"http://www.statnett.no"}
HTTP POST https://opengrid.com/myservice
Payload
{ "id": "002",}
Result
Open Grid SystemsOGO www.opengrid.com
RESTful Example
!14
{ “web”:”http://www.statnett.no/en/"}
HTTP POST https://opengrid.com/myservice/002
Payload
Status: HTTP 204 No Content Result
• Update (HTTP POST)
Open Grid SystemsOGO www.opengrid.com
RESTful Example
!15
<No Content>
HTTP DELETE https://opengrid.com/myservice/002
Payload
Status: HTTP 204 No Content Result
• Delete (HTTP DELETE)
Open Grid SystemsOGO www.opengrid.com
RESTful Benefits• All communications go through a single uniform
interface
• With HTTP protocols and JSON/XML payloads the services can be accessed by any modern language
• Simple, easy to use architecture
• Web Browsers use RESTful services over HTTP
• Standard verbs, response codes, data types etc.
• Flexible service endpoint structures
!16
Open Grid SystemsOGO www.opengrid.com
Challenges• This flexibility allows for different but valid URI patterns:
https://opengrid.com/myservice/002 https://opengrid.com/myservice?id=002 https://opengrid.com/myservice(id='002')
• All valid URI structures
• The payload formats are commonly JSON or XML but there is no restrictions
• The contents of the payloads can follow any structure
• For interoperability, these must be restricted and standardised
!17
Open Grid SystemsOGO
OData!18
Open Grid SystemsOGO www.opengrid.com
OData Background• The Open Data Protocol (OData) is a protocol standard
that defines a set of best practices for building and consuming RESTful APIs
• OData was initiated by Microsoft in 2007 and since version 4.0 has been under the OASIS OData Technical Committee
• This was submitted to the ISO/IEC as an international standard
• Allows the creation and consumption of REST APIs using meta-data driven URLs and and standard HTTP protocols and messages
!19
Open Grid SystemsOGO www.opengrid.com
OData Enhancements• Standard URI structure for RESTful service endpoints
• Standard Query format
• Supports client-selectable JSON or XML formats for payloads/results
• Supported by existing BI and office applications (including Excel and Sharepoint)
• OData libraries for .NET, Java, Javascript, PHP, C++ (but can be access as plain HTTP RESTful service without a library in any language that supports HTTP)
• Fully meta-data driven with an Entity Data Model (EDM)
!20
Open Grid SystemsOGO www.opengrid.com
Entity Data Model• Standard UML components map one-to-one to the Entity
Data Model
!21
Package Schema
Class EntityType
SuperClass BaseClass
Attribute Property
Association NavigationProperty
UML EDM
• Full support for data typing (e.g. String, Boolean, Floating Point, Integer) that map to XSD types
Open Grid SystemsOGO www.opengrid.com
RESTful with MetaData• This provides a fully meta-model driven RESTful
endpoint for managing data
• The CIM can be fully mapped into the EDM including all inheritance, associations, attributes and data types
!22
Open Grid SystemsOGO www.opengrid.com
Extended Payloads• Synchronous Machine as plain JSON
!23
{ "UUID":"_f4852cc6-ca12-11e7-b499-4ee79879b30e", "name":"Turner V1 SM", "p":0.0, "q":0.8, "maxQ":50.0, "minQ":-17.0, "operatingMode":1, "type":1, }
{ "@odata.context":"http://localhost:8080/cimphony/concert/db/resources.svc$metadata#Cimphony.Concert.CIM16A.IEC61970.Base.Wires.SynchronousMachine/$entity", "@odata.type":"#Cimphony.Concert.CIM16A.IEC61970.Base.Wires.SynchronousMachine", "@odata.id":"ConcertResource(ID='6f4c8710-6995-11e8-b5cf-96b18064619b')/Contents/Cimphony.Concert.Resource.CIM16A/SynchronousMachine('_f4852cc6-ca12-11e7-b499-4ee79879b30e')", "@odata.editLink":"ConcertResource(ID='6f4c8710-6995-11e8-b5cf-96b18064619b')/Contents/Cimphony.Concert.Resource.CIM16A/SynchronousMachine('_f4852cc6-ca12-11e7-b499-4ee79879b30e')", "UUID":"_f4852cc6-ca12-11e7-b499-4ee79879b30e", "name":"Turner V1 SM", "p":0.0, "q":0.8, "maxQ":50.0, "minQ":-17.0, "[email protected]":"#Int32", "operatingMode":1, "[email protected]":"#Int32", "type":1, }
• Synchronous Machine as JSON with extended meta-data
Open Grid SystemsOGO www.opengrid.com
Meta-Data Defined Services• The EDM is used by OData aware clients and servers to
validate communications
• Clients can request the EDM from the endpoint in a standard format to perform auto-discovery and client-side validation
• Servers will validate all endpoint URIs and service payloads against the EDM
• This allows for meta-data driven validation of data at the interface before it reaches the application itself
!24
Open Grid SystemsOGO www.opengrid.com
OData Example• OData abstracts the interface from a particular format
so can provide XML or JSON representations
• Endpoints can also be configured to returning other formats e.g. SVG, PNG or RDF XML through an OData interface
• The standard RESTful URL structure is all based on the meta-model:
!25
http(s)://<host>/cimphony/concert/db/management.svc/ ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/ Contents/Cimphony.Concert.Resource.CIM15/ SubGeographicalRegion('_9497C00D7BFE44D59A57C65D8B76B725')
!26
{ "@odata.context":"http://localhost:8080/cimphony/concert/db/management.svc$metadata#ConcertResource", "value":[ { "@odata.mediaContentType":"application/rdf+xml", "@odata.type":"#Cimphony.Concert.VersionControl.Management.ConcertResource", "@odata.id":"ConcertResource(ID='de6931d0-4212-11e7-b565-96b6cd723f39')", "ID":"de6931d0-4212-11e7-b565-96b6cd723f39", "head":"0c1dc960-4213-11e7-b565-96b6cd723f39", "name":"Alstom 60", "description":null, "[email protected]":"#DateTimeOffset", "updated":"2017-05-26T12:58:43.318Z", "[email protected]":"#DateTimeOffset", "created":"2017-05-26T12:57:26.637Z", "[email protected]":"#Collection(String)", "path":[ "EMS Examples" ], "frameID":null, "checkedOut":null, "checkedOutBy":null, "owner":"mcmorran", "version":null, "[email protected]":"#Collection(String)", "schemas":[ "http://iec.ch/TC57/2010/CIM-schema-cim15" ], "[email protected]":"ConcertResource(ID='de6931d0-4212-11e7-b565-96b6cd723f39')/Branches", "[email protected]":"ConcertResource(ID='de6931d0-4212-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15" }, { "@odata.mediaContentType":"application/rdf+xml", "@odata.type":"#Cimphony.Concert.VersionControl.Management.ConcertResource", "@odata.id":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')", "ID":"d8815f90-41db-11e7-b565-96b6cd723f39", "head":"d9d2bec0-41db-11e7-b565-96b6cd723f39", "name":"ABB 40", "description":null, "[email protected]":"#DateTimeOffset", "updated":"2017-05-26T06:23:36.62Z",
http(s)://<host>/cimphony/concert/db/management.svc/ ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/ Contents/Cimphony.Concert.Resource.CIM15/ SubGeographicalRegion('_9497C00D7BFE44D59A57C65D8B76B725')
!27
{ "@odata.context":"$metadata#Cimphony.Concert.Resource.CIM15/$entity", "@odata.type":"#Cimphony.Concert.Resource.CIM15", "@odata.id":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15", "ID":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15", "[email protected]":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/GeographicalRegion", "[email protected]":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/DayType", "[email protected]":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/BaseVoltage", "[email protected]":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/RegularTimePoint", "[email protected]":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/PowerTransformer", "[email protected]":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/Analog", "[email protected]":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/ConformLoadSchedule", "[email protected]":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/ApparentPowerLimit", "[email protected]":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/ShuntCompensator", "[email protected]":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/Unit", "[email protected]":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/RatioTapChanger", "[email protected]":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/NonConformLoad", "[email protected]":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/PowerTransformerEnd", "[email protected]":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/VoltageLimit", "[email protected]":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/OperationalLimitSet", "[email protected]":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/VoltageLevel", "[email protected]":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/RegulationSchedule",
http(s)://<host>/cimphony/concert/db/management.svc/ ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/ Contents/Cimphony.Concert.Resource.CIM15/ SubGeographicalRegion('_9497C00D7BFE44D59A57C65D8B76B725')
!28
{ "@odata.context":"http://localhost:8080/cimphony/concert/db/management.svc$metadata#ConcertResource", "value":[ { "@odata.type":"#Cimphony.Concert.CIM15.IEC61970.Core.SubGeographicalRegion", "@odata.id":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/SubGeographicalRegion('_9497C00D7BFE44D59A57C65D8B76B725')", "@odata.editLink":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/SubGeographicalRegion('_9497C00D7BFE44D59A57C65D8B76B725')", "UUID":"_9497C00D7BFE44D59A57C65D8B76B725", "mRID":null, "aliasName":null, "name":"ABB40APC" }, { "@odata.type":"#Cimphony.Concert.CIM15.IEC61970.Core.SubGeographicalRegion", "@odata.id":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/SubGeographicalRegion('_B7CD48062F2A49358F35F54AEB2D4008')", "@odata.editLink":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/SubGeographicalRegion('_B7CD48062F2A49358F35F54AEB2D4008')", "UUID":"_B7CD48062F2A49358F35F54AEB2D4008", "mRID":null, "aliasName":null, "name":"ABB40North" }, { "@odata.type":"#Cimphony.Concert.CIM15.IEC61970.Core.SubGeographicalRegion", "@odata.id":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/SubGeographicalRegion('_04D6363BC30E4663B7AAF08AC930A9A9')", "@odata.editLink":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/SubGeographicalRegion('_04D6363BC30E4663B7AAF08AC930A9A9')", "UUID":"_04D6363BC30E4663B7AAF08AC930A9A9", "mRID":null, "aliasName":null, "name":"ABB40South" }, { "@odata.type":"#Cimphony.Concert.CIM15.IEC61970.Core.SubGeographicalRegion", "@odata.id":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/
http(s)://<host>/cimphony/concert/db/management.svc/ ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/ Contents/Cimphony.Concert.Resource.CIM15/ SubGeographicalRegion('_9497C00D7BFE44D59A57C65D8B76B725')
!29
{ "@odata.context":"$metadata#Cimphony.Concert.CIM15.IEC61970.Core.SubGeographicalRegion/$entity", "@odata.type":"#Cimphony.Concert.CIM15.IEC61970.Core.SubGeographicalRegion", "@odata.id":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/SubGeographicalRegion('_9497C00D7BFE44D59A57C65D8B76B725')", "@odata.editLink":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/SubGeographicalRegion('_9497C00D7BFE44D59A57C65D8B76B725')", "UUID":"_9497C00D7BFE44D59A57C65D8B76B725", "mRID":null, "aliasName":null, "name":"ABB40APC", "Names":[
], "DiagramObjects":[
], "ModelingAuthoritySet":null, "Substations":[ { "@odata.type":"#Cimphony.Concert.CIM15.IEC61970.Core.Substation", "@odata.id":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/Substation('_58F637D8B03A4B12A67DF2E5797F9B6A')", "@odata.editLink":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/Substation('_58F637D8B03A4B12A67DF2E5797F9B6A')", "UUID":"_58F637D8B03A4B12A67DF2E5797F9B6A", "mRID":null, "aliasName":null, "name":"CROSSPLAINS" }, { "@odata.type":"#Cimphony.Concert.CIM15.IEC61970.Core.Substation", "@odata.id":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/Substation('_7324D6723635494784A4D8A9578FCE8A')", "@odata.editLink":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/Substation('_7324D6723635494784A4D8A9578FCE8A')", "UUID":"_7324D6723635494784A4D8A9578FCE8A", "mRID":null, "aliasName":null,
http(s)://<host>/cimphony/concert/db/management.svc/ ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/ Contents/Cimphony.Concert.Resource.CIM15/ SubGeographicalRegion('_9497C00D7BFE44D59A57C65D8B76B725')
Open Grid SystemsOGO
OData in Action for DERMS!30
Open Grid SystemsOGO www.opengrid.com
DERMS Project• Open Grid Systems is providing the CIM data management
repository for an ongoing DERMS innovation project at a large UK distribution company
• Multiple internal and external systems all provide and consume CIM data (including real-time data)
!31
Open Grid SystemsOGO www.opengrid.com!31
• The distribution network model from the DMS • Geographical data integrated with the DMS model • DER market data including availability/pricing • TSO market queries and requests • Outputs from market/system simulation engines • Real-time SCADA data
CIM Data
Open Grid SystemsOGO www.opengrid.com
Expanding Interfaces• The number of requested interfaces is constantly
growing
• Requests for access to different streams of data based on application requirements
• The CIM repository provides an OData interface to the static as-built network data and the real-time data
• Role Based Access Control to allow different users to view/update different sets of data
• OData allows most data access requests to be met with the generic service
!32
Open Grid SystemsOGO www.opengrid.com
Example Queries• “We need a list of the current real and reactive power
set points for all the DER on the network”
• The CIM class is PowerElectronicsConnection
• Only need the p and q attributes values for each
!33
http(s)://<host>/cimphony/concert/db/resources.svc/ ConcertResource(ID=‘6f4c8710-6995-11e8-b5cf-96b18064619b’)/ Contents/Cimphony.Concert.Resource.CIM16A/ PowerElectronicsConnection? $select=p,q
Open Grid SystemsOGO www.opengrid.com
Example Queries• “Oh but can we only get DER that are solar?”
• We need to check the PowerElectronicsUnit association of the device is of type PhotoVoltaicUnit
!34
http(s)://<host>/cimphony/concert/db/resources.svc/ ConcertResource(ID=‘6f4c8710-6995-11e8-b5cf-96b18064619b’)/ Contents/Cimphony.Concert.Resource.CIM16A/ PowerElectronicsConnection? $select=p,q& $filter=isof(‘PowerElectronicsUnit’,Cimphony.Concert.CIM16A.Extension.Generation.Production.PhotoVoltaicUnit)
Open Grid SystemsOGO www.opengrid.com
Example Queries• “We need an endpoint to return all the SCADA
measurement points attached to circuit breakers that are monitoring the switch status”
• The CIM class is Discrete (subclass of Measurement)
• Need all instances that associate with a Breaker via the PowerSystemResource association and have a measurementType of SwitchStatus
!35
http(s)://<host>/cimphony/concert/db/resources.svc/ ConcertResource(ID=‘6f4c8710-6995-11e8-b5cf-96b18064619b’)/ Contents/Cimphony.Concert.Resource.CIM16A/Discrete? $filter=isof(‘PowerSystemResource’,Cimphony.Concert.CIM16A.IEC61970.Base.Wires.Breaker) and measurementType eq ‘SwitchStatus’
Open Grid SystemsOGO www.opengrid.com
Services as Queries• The same approach is used for updates
• Applications can run a CIM query for specific CIM object(s) then use the returned data to run subsequent queries to update the data (or run more queries)
• e.g. Updating a SCADA measurement involves querying for the CIM Analog with the desire ICCP name
• The query includes an instruction to also include the associated AnalogValue ($expand=AnalogValue)
• A subsequent update changes the value of the AnalogValue with the returned ID
!36
Open Grid SystemsOGO
Conclusion!37
Open Grid SystemsOGO www.opengrid.com
Summary• Increasing numbers of systems, applications, and services
want to access and update CIM data
• A generic, platform-independent, metadata-driven service provides flexibility for users to use CIM data
• OData solves many of the problems by defining standard best practices and libraries for RESTful HTTP services
• Existing support in common applications, programming languages and platforms
• Can still be used as a light-weight plain RESTful service for clients that are not OData-aware
• A single endpoint to support multiple services
!38
Open Grid SystemsOGO
Questions
www.opengrid.com
!39