Building RESTful Web Services withPresented by Steve Ives

We’ve talked about what it is

We’ve talked about what it does

Let’s talk about how to use it

Requirements & Obtaining• Visual Studio 2017

• 15.8 or later

• .NET Core SDK & Runtime• 2.1 or later

• Synergy/DE• 10.3.3f or later• SDE x86, x64 and SDI

• CodeGen• 5.3.6 or later

• Internet access• Initially, to get to NuGet.org

• Project templates• From NuGet

• Repository describing your data• For code generation

• .NET Core, everything’s a NuGet package!

• Harmony Core packages• Harmony.Core• Harmony.Core.AspNetCore• Harmony.Core.EF• Harmony.Core.Odata

• Other packages• Microsoft.AspNetCore.Cors• Microsoft.AspNetCore.HttpsPolicy• Microsoft.AspNetCore.Mvc• IdentityServer4.AccessTokenValidation• Swashbuckle.AspNetCore

• Harmony Core project templates• Harmony.Core.ProjectTemplates

What Code Gets Generated• Model classes

• Describes individual data entities

• Metadata classes• Extended information about entities

• Entity data model class• Describes available entities and

relationships between them

• DB context class• Exposes data model to applications

• Web API controller classes• Exposes RESTful API CRUD endpoints

• Web API Startup class• Configures the Web API and OData

environments

And optionally:• Self-hosting program

• Hosts a runtime instance of the services

• Unit testing environment• Sample data environment classes• Unit test environment classes• Unit test classes

Leveraging the Synergy EF Provider• DbContext exposes a collection of the data

• DBContext.Customers

• Methods provide access to specific data• DBContext.Customers.FindQuery(value)

• Returns an IQueryable• DBContext.Customers.Find(key)

• Returns a data object• DBContext.Customers.FindAlternate(“Segment”,value [,…])

• And to alter the data• DBContext.Customers.Add(dataObject)• DBContext.Customers.Update(dataObject)• DBContext.Customers.Remove(dataObject)• DBContext.SaveChanges()

Configuring Your Repository for• Comprehensive and accurate repository data

will produce excellent results• Structures• Fields• Tags• Keys• Relations• Files

• If your repository is set up well for xfODBC, you should already be in great shape!

Repository – Custom Field Options• Optional identifiers placed in user text or long description

• HARMONY_EXCLUDE• Exclude the field• Should not be used on key segment fields

• HARMONY_AS_STRING• Expose decimal field with format string as formatted string• Currently only works for reading data, not create / update

• HARMONY_AUTHENTICATE• Consumers must be authenticated to see the field

• HARMONY_ROLES=role[,role…];• Consumers must be members of a role to see the field

• SAMPLE_DATA=John Doe;• Custom sample data used in API documentation and tests

Repository – Key Definitions• Define all keys completely and accurately

• List keys in the correct order• List access keys before foreign keys• Fill out optional key of reference #

• Don’t use overlays to define multiple segments

• Provide segment type and ordering• Optional and often ignored• Helps EF provider work optimally

• Key names are used when naming relationships

Repository - Relations• Enables powerful OData features

• Drill into related data when querying the main entity

• E.g., customers and their open orders

• Relationships often go both ways, so define in both structures

• Use foreign keys if you don’t have an access key in the From structure

• Literal segments in a foreign key can help when targeting a “tag” field

• Relationships take time, but significant benefits

Files with Multiple Record Types• Tags identify records within multi-record format files

• Typically FIELD == value, but may be more complex• RECORD_TYPE .eq. “A”• RECORD_TYPE .eq. “B”• RECORD_TYPE .eq. “C”

• If your data set includes this scenario• Define one STRUCTURE per record type• Declare appropriate TAG in each structure• Make sure the keys match across all structures• Assign all structures to a single FILE

• Not included in our sample environment but fully tested

Getting Started with The easy way

• Use a project template at the command line. Package:• Harmony.Core.ProjectTemplates

• Download and install:• dotnet new –i Harmony.Core.ProjectTemplates

• Create solution• dotnet new harmonycore [–n name] [–o folder]

• Add your repository schema or files

The manual way

• Create a .NET Core class library

• Add the Harmony and other NuGet package references

• Create source folders• Models, controllers, and wwwroot

• Add your repository schema or files

• Get the CodeGen templates (GitHub)

Getting Started via Project Templates• .NET Core frequently uses DOTNET CLI tools

DOTNET NEW harmonycore -n projectName [ -o folderName ]

• Repository project with empty schema

• Services project with sample OData service

• Services.Host project with minimal self-host program

• Inter-project and NuGet references pre-configured

• Other starter files provided• CodeGen templates• User-defined tokens file• Code generation batch file

• The project builds and runs out of the box

Sample OData Service – Books & Presses

Adding Your Repository• Option 1

• Paste your repository schema into repository.scm

• Option 2• Check “Use existing repository”• Name current repository ISAM files

• Option 3• Remove repository references from Services and Services.Host projects• Remove repository project• Customize RPSMFIL and RPSTFIL in “common properties”

CodeGen Environment – User-Defined Tokens

• Customize various values used during code generation• Project namespaces• Data file locations (for generated tests)• Documentation titles, descriptions, versions, etc.• Server protocols and port numbers• Authentication server information

• Default values work out of the box with the sample environment

CodeGen Environment – Batch File• Preconfigured to create a basic read-only environment

• Model, metadata, and controller classes• EdmBuilder, DbContext, and Startup classes

• Configurable options for lots of optional features• Advanced query and full CRUD support• Alternate key, collection count, and single property endpoints• Relation support• Self-hosting program, sample data environment, and unit testing• API documentation and Postman test files• Authentication and authorization support• Support for CORS and case-sensitive URLs• IIS hosting support

Configure Code Generation Targets

• Project names and subfolders to create files in

• Namespaces to use in generated code

• Preconfigured for the default environment

Configuring Structures to Process

• List the structures to process in DATA_STRUCTURES

• If some files have multiple record types, configure FILE_STRUCTURES

Configuring Optional Components• Select optional features by

uncommenting the ENABLE_ options

• Some cause additional files to be generated

• Some cause additional code to be added to existing files

• Much more detail in the workshop

Default Hosting Environment• Two base URLs, configured via user-defined tokens

http://localhost:8085/odatahttps://localhost:8085/odata

• HTTPS-enabled using self-signed developer certificate• Configure OS to trust

dotnet dev-certs httpsdotnet dev-certs https --trust

• HTTP endpoint auto-redirects to HTTPS

Default Capabilities• Service home page

• https://localhost:8086

• List main entity endpoints• https://localhost:8086/odata

• Metadata endpoint• https://localhost:8086/odata/$metadata

• Main entity endpoints (per entity)• https://localhost:8086/odata/Customers • https://localhost:8086/odata/Customers(1)

Controllers Receive DBContext via DI

Dependency injection

All controllers have easy access to your entire data environment via EF

Controllers Use DBContext to Access Data

Swagger Documentation• Extensive toolset for designing, building,

documenting, and testing RESTful APIs• SmartBear

• Harmony Core leverages Swagger UI• API documentation and testing

• Currently generate a “swagger file”• Describes an API and exposed types• ENABLE_SWAGGER_DOCS in regen.bat

• Swashbuckle may provide a more automated solution• Parsing of controllers and models• Currently works with Web API• OData support being developed

Additional Options for Reading Data• ENABLE_ALTERNATE_KEYS

• Endpoints for each alternate key• https://localhost:8086/odata/Customers(State=‘CA’)

• ENABLE_COUNT• Enable retrieving only the count of matching entities• https://localhost:8086/odata/Orders(CustomerNumber=8)/$count

• ENABLE_PROPERTY_ENDPOINTS• Endpoints to retrieve individual properties and raw property values• https://localhost:8086/odata/Customers(8)/Name• https://localhost:8086/odata/Customers(8)/Name/$value

Obtaining Data by Alternate Key

Unique keys return a single entity; non-unique keys return a queryable collection

Individual Property Endpoints

More Options for Reading Data• ENABLE_SELECT

• Select constrains the properties returned• https://localhost:8086/odata/Items?$select=(ItemNumber,QtyOnHand)• Becomes a SPARSE select in the back-end SELECT

• ENABLE_FILTER• Filters constrain the records returned• https://localhost:8086/odata/Customers?$filter=CreditLimit gt 5000• Will become WHERE in the back-end SELECT

• ENABLE_ORDERBY• Specify custom ordering sequence(s)• https://localhost:8086/odata/Items?$orderby=UnitPrice desc• Becomes an ORDERBY in the back-end SELECT

Paging Options When Reading Data• ENABLE_TOP

• Consumers can restrict the number of rows returned• https://localhost:8086/odata/Items?$top=10

• ENABLE_SKIP• Consumers can request “pages” of data• https://localhost:8086/odata/Items?$top=10&$skip=10

• TOP is fine, SKIP is currently expensive—use with care• Future DBL enhancement will optimize SKIP

Retrieving Related Data• ENABLE_RELATIONS

• Consumers may request expansion of related data• https://localhost:8086/odata/Customers?$expand=REL_Orders• https://localhost:8086/odata/Customers(8)?$expand=REL_Orders

•

• Adds relational properties to model classes• REL_ prefix used to avoid collisions with field property names

Combining Multiple Query Options

https://localhost/odata/Customers(8)?$select=CustomerNumber,Name

&$expand=REL_Orders($select=OrderNumber,DateOrdered;

$expand=REL_OrderItems($select=ItemNumber,ItemOrdered,QuantityOrdered,UnitPrice;

$expand=REL_ItemOrdered($select=CommonName,Size;

$expand=REL_VendorNumber($select=Name))))

This is a URL—multi-line formatting is for readability only

And the result…

Customer data

Order data

Order line item data

Item data

Creating, Updating, and Deleting Entities• POST – create new entity

• Client provides complete copy of the new entity, except primary key• Key generated by custom code you write• New entity returned to consumer, along with Location header

• PUT – replace existing entity, create if not existing• Client provides complete copy of the new entity, including key• Entity returned if created, otherwise a “No Content” indicates success

• PATCH – partial update entity• Client provides “patch document” specifying changes• Very efficient

• DELETE – delete entity• Client provides key of entity to delete

Enabling Create and Replace (full update)• ENABLE_POST

• Consumers may create new entities without knowing key value• Requires custom code to generate new key values• POST to entity collection: https://localhost:8086/odata/Items• Include entity (JSON) in request body

• ENABLE_PUT• Consumers may replace (or create) entire entities• PUT to specific entity: https://localhost:8086/odata/Items(12345)• Include entity (JSON) in request body

Enabling Patch and Delete• ENABLE_PATCH

• Consumers may apply specific changes to an entity• PATCH to specific entity: https://localhost:8086/odata/Items(12345) • Include “patch document” in request body

• ENABLE_DELETE• Consumers may delete an entity• DELETE to specific entity: https://localhost:8086/odata/Items(12345)

Patch Document• A patch document is a JSON array

of individual change instructions to be applied to an entity

• Efficient way to make specific changes to an entity

[{

"op": "replace","path": “Contact","value": "Willy Lopez“

},{

"op": "replace","path": “CreditLimit","value": 5000

}]

Authentication & Authorization• ASP.NET Identity

• Highly customizable• Supports multiple persistence mechanisms

• ADFS, Azure AD, OAuth, etc.• Role provider supports custom access control• Claims-based authentication• Unit testable

• Consumer authenticates and obtains a “bearer token”• JSON Web Tokens (JWT)

• Bearer token presented back via Authorization header

• Capabilities for protecting APIs• Authentication at the controller or operation level• Role-based authorization at the endpoint level• Role-based authorization at the field level

• Authentication• Who are you

• Authorization• What can you do?

Enabling Authentication & Authorization• ENABLE_AUTHENTICATION

• Consumers must obtain an access token and present it to the API• JWT provided as a Bearer token via the Authorization header

• Requiring authentication• {Authorize} attribute • Apply to controller to restrict all actions• Apply to methods to restrict specific operations

• Requiring authentication and authorization• {Authorize(Roles=“role1[,role2,…]”)}• Access token required AND must contain specific role(s)

Default Behavior• Using ENABLE_AUTHENTICATION

• Startup code configures the server for authentication• Adds {Authorize} to each controller

• All clients must be authorized for all operations

• Optionally specify user role requirements per operation type via optional user-defined tokens

• <ROLES_GET> Employee, Manager </ROLES_GET>• <ROLES_POST> Manager </ROLES_POST>• <ROLES_PUT> Manager </ROLES_PUT>• <ROLES_PATCH> Manager </ROLES_PATCH>• <ROLES_DELETE> Manager </ROLES_DELETE>

Field-Level Security• ENABLE_FIELD_SECURITY

• Individual fields may be restricted based on user roles• Used in conjunction with ENABLE_AUTHENTICATION

• Require authentication• Specify authentication required in repository long description

• HARMONY_AUTHENTICATE• Attribute added to model class property

• {AuthorizeField}

• Require authentication and authorization• Specify role requirements in repository long description

• HARMONY_ROLES=role1 [,role2…] ;• Attribute added to model class property

• {AuthorizeField("User,Manager")}

Adding Custom Functionality• Controllers are implemented as PARTIAL CLASS

• Extend code-generated controllers• Or just write additional controllers with custom functionality

• Add Web API / OData endpoint methods• Code can do anything you want• EF provider (DBContext) available for use via DI

• Expose any method (SPROC mechanism)• Apply {Controller} attribute to any class• Route determined by class name• Public methods exposed as operations• Parameters marshalled via URL and body

Using Postman with a Harmony Core Service• ENABLE_POSTMAN_TESTS

• Generates a Postman export file

• Includes tests for generated controller actions• Includes tests to obtain JWT from an OAuth server

Unit Testing a Harmony Core Service• ENABLE_UNIT_TEST_GENERATION

• Generates a unit-testing environment into a test project

• Environment includes• Unit-test classes / methods for generated controllers / actions• Client-side model classes for data de-serialization• Mechanism to refresh test data every time tests run• Environment setup classes• Customized self-hosting program to allow tests to be debugged

• Tests run via dotnet test CLI• Visual Studio Test Explorer doesn’t currently support .NET Core

Demonstration

• Two running instances of a Harmony Core server• Local: IIS https://localhost• Remote: Azure https://harmonycoreapp.azurewebsites.net

• Demo the service home page• Show the API documentation• Use Postman to demonstrate the tests in the Harmony Core Demo collection

Who has the first question?