TX-2016.TIER New and Next - Internet2 · • Docker containers • Virtual machine images to run the containers • Focus on automation tools • Build containers and VMs ... TIER

[ 1 ]

Trust and Identity in Education and Research (TIER)What’s New and What’s Next

Jim Jokl (University of Virginia), Chair/TIER Packaging Working Group

Keith Hazelton (University of Wisconsin-Madison), Chair/TIER API and Data Structures Working Group

Bill Thompson (Lafayette College/Unicon), Lead/Grouper Documentation Project

Tom Jordan (University of Wisconsin-Madison), Architect, (Reference Architecture)

Kevin Morooney (Internet2), Vice President/Trust and Identity

Ann West (Internet2), Associate Vice President/Trust and Identity

Steve Zoppi (Internet2), Associate Vice President/Services Integration and Architecture

[ 2 ]

IntroductionAnn West

[ 3 ]

SUSTAIN

#1Requirement:

SustainabilityandStandardization

Foundation Principles (TIER Program)

Sustain

Integrate

Enhance

Extend

Integrate

Integrationofaconsistentlyconfiguredanddeployedenvironmentforrapidadoptionandeaseofupgrade/update.

ENHANCE

DefineaDevOpsenvironmentwhichsupportscontinuousintegration,enhancementandreliable/consistentconfigurationvalidation/release.

EXTEND

DefineaDevOpsEnvironmentwhichenablescampusesandcommunitycontributorstoextendcurrentfunctionalityinthecontextofaself-containedFederationfabric.

[ 4 ]

[ 5 ]

General Principles

• Support Campus and Software Engineering “Best Practices”– Abstract Provided

Components– Define Stable Interfaces

for Campuses• Provide Firm Foundation

for Future Development

• Leverage Existing Work Wherever Possible

• Narrow The Toolsets• Utilize Virtual Machine

Images and Containers• Fortify the Federation’s

Operation

[ 6 ]

Groundwork

• Amazon Web Services (Deployment Target)• Community Forum (Vanilla Forums)• Contact Management Systems (SalesForce)• Defect and Feature Tracking Systems(JIRA)• Financial and Accounting Systems (NetSuite/Zuora/Salesforce)• Mail List Managers (Sympa / 21 lists and

community groups)

[ 7 ]

Groundwork

• Project Management (Web Portal/Web2Project)• Service Management (ServiceNow)• Source Management (GitHub - Enterprise)• Support Databases and

Reporting Systems (Various)• Document Management (Confluence)• Packaging and Container

Generation (Packer and Docker)

[ 8 ]

The Foundations of TIER Infrastructure

• RESTful APIs are the primary interfaces between TIER IAM Components and between TIER Components and External Systems

• TIER-defined Resources are shared and manipulated by RESTful APIs

• The API first approach is a key element supporting campus autonomy

• If you want to keep your identity registry, just expose its functionality as implementations of TIER APIs, it will then plug and play with other TIER components

• Supports selective adoption of TIER components and hybrid infrastructure models

[ 9 ]

Consequential Early Choices

• Swagger 2 as the Primary TIER API Tool – Used for both Design and

documentation of APIs– The language in which TIER

APIs will be formally specified

• SCIM 2 as the anchor point for TIER API Style– New IETF standard (RFCs 7642,

7643 and 7644) – Provisioning and de-provisioning

are the design center for SCIM, and a high priority for potential TIER adopters

– It doesn’t cover all IAM functional needs, so TIER will end up defining new APIs: IdMatch is a good example

– A potential common language for higher education and commercial SaaS

[ 10 ]

Release: Progress ReportAnn West

Steve Zoppi

[ 11 ]

T&I Program/Community-Requested SolutionsCategories Timing / Status

1 Application ProgrammingInterfaces/Campus Near-Term / In Progress2 VODe/ProvisioningandInvitationSvcs (Comanage) Near-Term / In Progress3 StudentApplicantIdentityProvider(CommIT) “Beta” / Suspended4 CommunityIdP (IdentityProviderofLast Resort) Community Solution / External5 EntityRegistry(PersonandObjectRegistry) Near-Term / In Progress6 GroupManagementandGroupAdministration Near-Term / In Progress7 InCommon FederationOperationsandManagement Near-Term / In Progress8 ComponentPackagingandDeployment Near-Term / In Progress9 Component andOperationsSecurityandAudit Near-Term / Recommendations10 ScalableConsent Services Near-Term / Grant-Funded11 IdentityProvider andServiceProvider(Shibboleth) Near-Term / In Progress12 Community TrainingandTIERProgram Mid-Term / Under Evaluation

[ 12 ]

In Our Last Episode … (Global Summit 2016)Proposed Milestones

q Backbone usage scenario (BUS) proof of conceptq An open IAM testbed based on the proof of conceptq A containerized version of the IAM testbed for local

experimentationq Well-instrumented code that can reveal behavior and

health of the IAM infrastructure components and their interactions

q First edition of a living guidebook: architectural patterns for IAM integration

[ 13 ]

In ProgressPOC Ready

Ready

RequirementsGathering

GrouperOutline In Dev

Today … (TechEx 2016)Actual Progress

q Backbone usage scenario (BUS) proof of conceptq An open IAM testbed based on the proof of conceptq A containerized version of the IAM testbed for local

experimentationq Well-instrumented code that can reveal behavior and

health of the IAM infrastructure components and their interactions

q First edition of a living guidebook: architectural patterns for IAM integration

[ 14 ]

In Our Last Episode … (Global Summit 2016)Proposed Milestones

• Testing and enhancement of the container/vm distributions

• Exploring Instrumentation• Prioritize usability enhancements

– Setup automation and pre-configuration

– Campus metadata– etc., etc.

• Build on the packaging foundation– Automate the container and VM

builds– Operationalize the build process;

produce regularly scheduled updates

– Start the process of automating testing

– Complete the work with the initial components

[ 15 ]

In ProgressPOC Ready

PENDINGCAMPUS

ADOPTION

THIS RELEASE

NEXT PHASES

NEXT PHASES

THIS RELEASE

Today … (TechEx 2016)Actual Progress• Testing and enhancement of the

container/vm distributions• Exploring Instrumentation• Prioritize usability enhancements

– Setup automation and pre-configuration

– Campus metadata– etc., etc.

• Build on the packaging foundation– Automate the container and VM

builds– Operationalize the build process;

produce regularly scheduled updates

– Start the process of automating testing

– Complete the work with the initial components

[ 16 ]

DevOps Model: Enabling Autonomy

Workbench Path 1:Development and Iteration / Ideation

Workbench Path 2: UAT/QA/Security Audit/Performance Assessment/Release

[ 17 ]

Lessons Learned

• The community can accomplish a lot– Dedicated set of volunteers– Weekly calls– Problem space evaluation– Survey development and analysis– Technical packaging strategy and

prototyping– Initial testbed– If you have interest in this space,

please join us

• Significant software development requires dedicated resources– The foundational work done by

volunteers– Engaged commercial firm to build

packaging

[ 18 ]

Lessons Learned

• The Data Structures and APIs WG, along with the Entity Registry WG, attracted talented people from higher education institutions across the land (well over 120 on the mailing lists)

• The WG members will show up, work hard together and deliver what the WG charters asked for (our meeting notes alone amount to nearly 200 pages)

• Institutions will share their experience, their documents and their code: Penn State just released their Apache 2-licensed SCIM server and client libraries, significantly accelerating the API work.

[ 19 ]

Lessons Learned

• Institutions with significant IAM projects underway have skin in the game and will make sure the Working Groups stay focused on essentials

From Architecture to Construction and Back Again

© 2016 Internet2

[ 21 ][ 21 ]© 2016 Internet2

ARCHITECTURE : REFERENCEARCHITECTURE REFERENCEIMPLEMENTATIONS

INTEGRATION : API’SANDEVENTMESSAGING DATAMODELS

DOCUMENTATION : BESTPRACTICESDOCUMENTATION DEPLOYMENTGUIDES

IMPLEMENTATION : DEMOWORKBENCH PRODUCTIONWORKBENCH

TIER DELIVERABLES

[ 22 ][ 22 ]© 2016 Internet2

APIs

http://bit.ly/tierapi

[ 23 ][ 23 ]© 2016 Internet2

Event-Driven Messaging

[ 24 ][ 24 ]© 2016 Internet2

Data Models, Schema, Resources{

"userName": "keith","name": {

"middleName": ”D","givenName": "Keith","familyName": "Hazelton","displayName": "Keith Hazelton","formatted": "Keith D Hazelton"

},"role": [

{"value": "[email protected]"

},{

"value": "[email protected]"},


}],

"externalID": "[email protected]","emails": [


}],"schemas": [

"urn:ietf:params:scim:schemas:core:2.0:User"]

}

[ 25 ][ 25 ]© 2016 Internet2

Demo WorkBench: Backbone Usage ScenarioFrom onboarding of a new faculty or student to providing them access to an LMS

[ 26 ][ 26 ]© 2016 Internet2


[ 27 ][ 27 ]© 2016 Internet2


[ 28 ][ 28 ]© 2016 Internet2


[ 29 ][ 29 ]© 2016 Internet2


[ 30 ][ 30 ]© 2016 Internet2


[ 31 ][ 31 ]© 2016 Internet2


[ 32 ][ 32 ]© 2016 Internet2

Inception of metrics and instrumentation

Devops and Packaging

© 2016 Internet2

[ 34 ][ 34 ]© 2016 Internet2

TIER Development Model

[ 35 ][ 35 ]© 2016 Internet2© 2016 Internet2

https://spaces.internet2.edu/display/TPWG/Survey+Results

Preferences for TIER Software Installation

[ 36 ][ 36 ]© 2016 Internet2


Docker for Production Services

[ 37 ][ 37 ]© 2016 Internet2© 2016 Internet2


VM Appliance for Production Services

[ 38 ][ 38 ]© 2016 Internet2

Strategy for TIER Packaging• Component teams retain traditional installers

• These will continue to be needed well into the future

• Provide additional release types for the components• Docker containers• Virtual machine images to run the containers

• Focus on automation tools• Build containers and VMs• Automate testing• Over time, goal of weekly builds• Identify and deploy tooling that is able to deliver multiple formats

• Keep pace as technology changes

© 2016 Internet2

[ 39 ]© 2016 Internet2

Component Packaging Status• Initial Release – April 2016

• First-look version of the packaged components• Tightly coupled to the TIER testbed

• Current Release – September/October 2016• Focus on production service capability• Configuration management• Virtual Machine images

• Docker container build environment• Docker operations

• Shibboleth – nearing release testing status• COmanage and Grouper following

[ 40 ]© 2016 Internet2

Build Environment

• Shibboleth configuration tree• Simple tooling for initial IdP

configuration• Docker container build • Scripting for operations

Operational Environment

Dual Function VMs: Build and Operate

Docker Tomcat –ShibIdp_0

Docker Tomcat –ShibIdp_1

Docker HAproxy

[ 41 ][ 41 ]

Next Steps for Packaging

• Shibboleth IdP release• COmanage and Grouper releases

• Tools– Campus metadata management– Shibboleth ease of configuration tooling– Docker versions of the other TIER components

• Integration management

• Start on production build releases

© 2016 Internet2

[ 42 ][ 42 ]

How you can help

• Testing• For production use• Real world scenarios

• Adoption• Schools with active Docker efforts first• We’d like to hear from you if you are not already engaged

• What else should we be doing?

© 2016 Internet2

Reference Architecture

© 2016 Internet2

[ 44 ]

TIER Reference Architecture

• A model for considering relationships between functional IAM components– Audiences

• Business / Executive context setting• Technical architects / implementers

• A set of narrative walkthroughs– Illustrating specific business functions (enrollment, hire, job change, etc)– Describing interactions between components

• A component glossary– Defining each component and how it relates to others

[ 45 ]

Grouper Deployment GuideTIER Grouper Deployment Guide 1.0

● TIER Grouper Deployment Guide Proposal• Expand the Columbia Grouper Deployment Guide to cover TIER recommendations• Work with the TIER community on shared vocabulary, practice and strategies• Distill community best practices from Community Contributions into specific guidance• Expand IAM capabilities/use cases

Timeline/Milestones• TIER working group charter and community team formation - Aug 2016• 0.5 draft outline TIER Grouper Deployment Guide - Sep 25, 2016• 2016 Internet2 Technical Exchange presentation/discussion/feedback - Sep 25-28, 2016• 0.7 draft release Dec 2016• 0.9 draft release and final comments period Feb-Mar 2017 • 1.0 final release Internet2 Summit Apr 2017

© 2016 Internet2

[ 46 ]

Grouper Deployment GuideDiscussion at TechEx

• Grouper BOF - Tue 2:30pm @ Bayfront B• ACAMP Session• Find me in the hall :)

Discussions after TechEx• Grouper Deployment Guide Work - TIER Program• NIST 800-162 ABAC• Columbia Grouper Deployment Guide

– What do you wish you knew before you started your Grouper deployment?– What is missing from an initial deployment perspective?– What should we work on in the way of recommended / best practices?

• Grouper wiki examples (folders, community examples, etc)– Good examples of use cases? Good presentation/format?

© 2016 Internet2

[ 47 ]

TIER Demos at 2016 Technology Exchange

• To see TIER Demos, stop by the TIER Demo Booth

• Biscayne Ballroom on the second level

• During your breaks and lunch through Wednesday.

© 2016 Internet2

[ 48 ][ 48 ]

Staying Informed About TIER• Join the TIER-Discussion email list

– To subscribe: Email [email protected] with the subject (case insensitive): subscribe tier-discussion

• Sign up to receive the TIER Newsletter– http://www2.internet2.edu/email-preferences-center

• Follow the TIER Working Groups from this wiki page– https://spaces.internet2.edu/display/TWGH/TIER+Working+Gr

oups+Home

© 2016 Internet2

Visit the Tech-Ex TIER Community

Library

https://internet2.box.com/v/TechEx2016TIERCommunityLibrary

[ 49 ]

Questions and AnswersTIER Release Team

Documents

TX-2016.TIER New and Next - Internet2 · • Docker containers • Virtual machine images to run the containers • Focus on automation tools • Build containers and VMs ... TIER