SAML Conformance Sub-Group Report

Face-to-face meeting August 29, 2001

Bob Griffin

Topics• Status• BindingConformance issue• Browser partition issue• Testing optional elements • Building test clients• Interoperability testing

Status• Conformance Clause

– Version 0.006 in SAML spec; no change from June F2F– Describes goals, validation/certification, partitions

• Conformance Program– Version 0.005 available– Includes cut at test cases for Authentication Authority Partition, including traceability to

requirements; other test cases tdb

• Certification – Decision at June F2F to focus on validation for SAML V1

• Test suite– Detailed design to be included in test cases following F2F

• Traceability of SAML to S2ML, AuthXML…– Prateek to do SAML/S2ML traceability– Darren to do SAML/AuthXML traceability– Dave O. to do SAML/ITML traceability

Issue: BindingConformance• “Should protocol bindings be the subject of conformance?”• Concerns raised in issue write-up:

– Profile is “a complete specification of the SAML aspects of some use case” while binding is “a building block in one or more profiles”.

– Specifying conformance in terms of protocol/bindings increases the number of test cases – Conformance claims based on protocol/binding would not necessarily mean

interoperability.

• Discussion– Current model of “conformance partitions” does not define conformance in terms of use

cases, but in terms of system entities in the static domain model: (authentication authority, attribute authority, policy decision point, policy enforcement point)

– Conformance claims by SAML implementation or application would be for one or more partitions;

– Within partition, subsets further allowed– For each system entity, test cases related to profiles and to protocols/bindings are defined

Authentication Authority Partition Test Cases

HTTP Web Server Profile(s)– Valid Authentication Assertion Produced

– Valid Authentication Assertion Consumed (eg, from another domain)?

– Valid Authentication Assertion Artifact Produced

– Valid Authentication Assertion Artifact Consumed

Authentication Request/Response Protocol: HTTP, SOAP bindings– Valid Authentication Assertion Produced

– Valid Authentication Assertion Artifact Produced

– Valid Authentication Assertion Artifact Consumed

Authentication Authority Partition Tests

SAML Conformance Test Client

SAML implementation or application

Request for authentication assertion embedded in HTTP

Authentication assertion returned, embedded in HTTP, and checked by test client for validity

Test case 1-4: Web Server Profile

Request for authentication assertion, bound to HTTP

Authentication assertion returned via HTTP and checked by test client for validity

Test case 5-16: Protocol/Binding

Web Server

BindingConformance (cont’d)• Discussion:

– Should conformance be tested in terms of use cases rather than entities in domain and producer-consumer models?

– Are profiles equivalent to use cases and therefore the way conformance should be specified?

– Does testing protocols/bindings test interoperability?– Are protocols specific enough so that conformance can be defined in terms of

protocols rather than protocol + binding?

• Recommendation:– For now, continue with “conformance partitions”.– Include behavior related to both profiles and protocol/bindings in

conformance partition testing.. – Implementation or application claiming conformance to a given partition must

specify which bindings it supports for the protocol(s) in that partition– Not all bindings have to be supported to claim conformance to that partition.

Issue: Browser Partition• Should there be a conformance partition for browser, comparable

to those for authentication authority, attribute authority, PEP, PDP?

• Discussion:– The other partitions represent implementations (eg. Multi-domain single-

signon product) or applications (eg, application server as PEP accepting assertions) of SAML.

– Insofar as unmodified browser is goal, no comparable implementation/application related to browser?

– Will all SAML-specific behavior related to profiles be captured by the other partitions?

• Recommendation:– Do not define browser partition.– Check that all SAML behavior is captured by other partitions

Optional Elements• Issue: Test cases have to include sub-cases based on

optional elements, encryption/signing, etc.

• Discussion

– Test suite should validate that SAML implementation or application has correctly specified an extension, but should not test the extension itself

– No expectation of having registry for SAML extensions

• Recommendation: following F2F, develop sub-cases for variants and optional elements to explore this issue further.

Saml Conformance Test Client Design• Test client has to evaluate responses received from SAML

implementation or application to determine validity of assertions and (in case of protocols) response messages

• Validity can be determined in several ways:

– Inspection of returned assertion and/or message

– Submission of assertion and/or message to open-source reference implementation

– Follow-on action based on assertion or message (eg, test succesful consuming of authentication assertion by requesting protecting resource)

Test Client Design - 2

SAML Conformance Test Client

SAML implementation or application

Request for authentication assertion embedded in HTTP header

Authentication assertion returned, embedded in HTTP header, and checked by test client for validity

Test case 1: Web Server Profile

Web Server

SAML Reference Implementation

Alternative 3. Resource request submitted to SAML implementation, using authentication assertion

Reference Assertion

Alternative 2. Received authentication assertion compared to reference assertion

Alternative 1. Received authentication assertion submitted to reference implementation in resource request

Test Client Design – 3• Recommendation: investigate options in terms of

detailed design of test cases, to determine how feasible each approach is across all test cases

Interoperability Testing• Several options:

– Test implementations/applications against open-source reference implementation (eg, submit assertions to reference implementation PEP)

– Cross-vendor testing in customer situations

– Cross-vendor testing in independent lab, etc

– Cross-vendor testing at vendor sites