Upload
alijah
View
21
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Open Standard Voting Localization with CAM - Localization Mechanisms - Publishing Localizations - Leveraging Open Standards - XSD ingesting. David RR Webber Chair OASIS CAM TC (Content Assembly Mechanism) E-mail: [email protected] http://wiki.oasis-open.org/cam. Overview. - PowerPoint PPT Presentation
Citation preview
Open Standard Voting Localization with CAM
- Localization Mechanisms- Publishing Localizations- Leveraging Open Standards - XSD ingesting
David RR WebberChair OASIS CAM TC(Content Assembly Mechanism)E-mail: [email protected]://wiki.oasis-open.org/cam
slide 2
Overview Goal is to promote the use of open
public standards for voting systems Develop in-country localizations from
standard international base set Provide common open public lexicon for
elections management and verification Leverage XML and open standards
approach, including NIEM* Provide developers with tools to aid
delivery, documenting and testing of solutions
*NIEM – National Information Exchange Model
slide 3
Approach Voting Localizations built from OASIS EML
EML has 29 pre-built components (schema) for voting processes Use desktop visual editor to pick components
Trim and select only what you need from each standard schema Desktop toolkit makes templates for the XML required
Generate localization details Automatically build XML components to use Build your own dictionary, compare to EML standard Create readable documentation of details
Test, Package and Publish The CAM desktop toolkit environment supports NIEM IEPD approach
Available as Open Source on SourceForge.net Implementation of the public OASIS CAM standard for schema
templates (camprocessor)
Localization Mechanisms
Creating a Country Voting Localization Package
slide 5
Localization Key Components
XML Schema
2
1
3
4
DocumentationReadable information use details (HTML), EML cross-reference Spreadsheet
XML artifacts The localized schema and picklist.xml
Examples & Test CasesRealistic test XML instances for conformance and interoperability testing
Key components(aka IEPD*)
EML Schema XSD aligns voting information model, processes and domain dictionary
*IEPD – Information Exchange Package Documentation
slide 6
Localization Approach StepsEML
XMLXSD
Voting
Exchange
Manage
develop
1ingest
2 BaseCAM
Templatelocalize
3 RevisedCAM
Template
Generate
4 Local XMLXSD
Compare
5EML
Dictionary
Steps:1. OASIS develops standard international XSD schema
Implements the voting process, management and information exchange needs2. Ingest the standard XSD into base CAM template 3. Apply localizations to CAM template
Exclude unwanted optional components Align local exchange needs to standard Extend for local needs
4. Generate new localized XSD schema that conform to EML standard5. Build dictionary cross-reference of localization usage6. Publish documentation and examples
Publish
6 Documentation+
Examples
slide 7
Creating a Package for Localization Step 1 - Ingest EML XSD – rules, structure, annotations Step 2 - Select & mark out your use model in visual
editor - Save your “pick list” selections - Compress template to desired structure
Step 3 - Generate your localized use XSD schema subset
Step 4 - Generate and test rich concrete data examples (complete with content hints / pass / fail / random options)
- Run rules engine - Verify operation and outcomes
Step 5 - Build localization documentation of structure elements, usage, rules and cross-reference map
Step 6 – Package, Publish and Share (IEPD)
slide 8
Challenge: XSD schema has limitations! The schema contains the superset of voting exchange
component variations for every country XSD syntax does not have direct context mechanisms Hence people make everything in schema optional Dependencies are not clear It is difficult to understand the constructs and to document
the rules clearly for election authorities to verify It is hard to create test cases and instances (the “pick list”
tough to visualize) These are all limitations inherent in W3C Schema language
itself
slide 9
Completing Localization Picture:
Base EML XSD schema structure model CAM template + rules / refinements Documentation of use patterns (aka “pick list” +
subset XSD) Test cases and examples Content hinting (localization use data) Validation engine for unit and regression testing Open standard, open platform and open source
allows consistent agreements between participants
slide 10
Localization Package Use Scenario
Testing,Agreement /Alignment.
Validate
Templates
Report
DeveloperUses &Creates
Results
Test
Pass / Fail
XMLhtml
32
4
Test Cases
XML
Package
StructureRulesContextExamplesDocumentation
Publish
1
Verify Result Outcomes
XML
LocalizationContent
Hints
slide 11
Tutorial Base CAM templates – (ingesting XSD schema )
step by step example
Documenting the Localization Patterns Creating “pick list” selections Compressing template structure Subset XSD generation
Publishing Localization package Cross-reference spreadsheet Documentation reporting options Generate use dictionary
Testing and Conformance Creating Test Case examples Content Hinting Running Test Cases
Advanced Techniques
Base CAM Templates
(ingesting XSD Schema)Using jCAM editor Wizard
(http://www.jcam.org.uk)
slide 13
EML V6.0 provides base CAM templates Templates come pre-packaged in the EML V6
download
However, for completeness we show how to generate one…
slide 14
Ingesting XSD to CAM template format
EML xml
EML xsd
Localization
Commonxsd
Namespace
XML Schema
Components CIQ xsd
INGEST
Structure
Rules
Documentation
XSLT toolsprocess
entire EML XSDcollection
CAM = Content Assembly Mechanism
OASIS EML XSD Collection
slide 15
Step 1 & 2 – Pick the XSD schema to ingest
1
Choose File / New Template
First location is the rootfolder for the XSD collection.
Typically this is the same as the location for the XSD schema you want to ingest.
(Complex XSD can have this in separate folder; hence need for option)
2
Pick XSD
Specify XSD Location root directory
slide 16
Step 3 – Choose the XSD parent element
From the dropdown list pick the correct root element you wish to use (EML).
XSD’s may have more than one collection in them (as shown here) or may not have the parent node element cleanly defined.
The Wizard shows you the list of all possible ones it finds – so you can select and confirm the right one.
3Choose XSD parent element
Confirm and start the XSD ingesting
slide 17
Step 4 – Ingesting complete – Save Results
4
Completed template is loaded and save dialogue appears
Processing usually takes a few seconds, some larger EML XSD can take a few minutes to process.
Tip: jCAM runs the ingesting as a background task – so you can continue to use the computer while such long ingesting is proceeding.
4
slide 18
Step 5 – Review ingested structure
5
Completed template is shown in the structure navigator panel for review and editing
Template contains all the default content model and structure rules ingested from the XSD.
All annotations and documentation from XSD also ingested (show as “paperclip” symbol).
Code lists and typical content values inserted for easy visual reference.
Documenting the Localization Patterns
“Pick lists”, documentation and XSD subset generation
slide 20
Building a pick list
MARK WHATIS NOT
NEEDED
Structure
Rules
Documentation
Make pick list
DESIRED RESULTS
excludeditems
EML structure is
all inclusive!!!
slide 21
Marking Items for exclude - pick list Can exclude at all levels within the
structure excludeTree() excludeElement() excludeAttribute()
Use XPath operators to control scope: Specific node Group of nodes Anywhere occurs in structure Contextually based on condition
Can add new domain elements with own namespace to EML template
slide 22
Using Editor to mark Exclude items
Structure Editor Tools
Select focus on item
Invoke action menu (right mouse click)
Pick action
Tip: exclude rule display is context sensitive and only available on optional items
Tip: use “Add New Rule” mode to specify different XPath for exclude (quick mode assumes “current path”)
slide 23
Export and Save completed pick list
File Menu Option
Select Export / Pick List and then specify filename of destination.
Excluded items are designated with red “dot” in structure editor and italics font with no bold highlight
slide 24
Pick List Details
EXCLUDE FLAGVALUE
Exported Example
Pick lists provide a handy way to catalogue the localization model and can be re-used later by importing into other templates
slide 25
Compress OperationFile Menu Option
Select option and specify filename for new copy of your template.
Compress process removes all rules and structure items marked with an exclude statement.
Note: ignores excludes that have a conditional context expression (advanced section technique).
Note: You can retain both original and new compressed templates
slide 26
Extending the base EML schema In addition to excluding existing optional
components you may need to add new pieces
New pieces should be assigned their own extension namespace prefix – to identify them from the base EML components
Typically the OASIS EML committee will review new pieces as candidates for adding to future versions of the standard
• Send comments and suggestions to [email protected]
slide 27
Adding New Domain Elements
Select root element node in structure; right mouse click for context menu; select “Add Namespace”; then enter prefix and URI
1
Select element node in structure; right mouse click for context menu; select “Add Child Attribute / Element”; then enter prefix and name
2
slide 28
Generating your EML use schema Once your localization details are
completed you are ready to complete the remainder of the process
This starts with generating your own new subset schema
Then once schema details are OK – creating test cases, cross-reference spreadsheet and documentation
slide 29
Generating subset use schema
Select Export CAM as XSD menu option
1
File Menu Option
Confirm the location and filename, and namespace mode.
Select ‘default’ for Mode to write a regular import set of XSD schema for each namespace
2
slide 30
Schema subset generated
Set of XSD files with filename and namespace suffix
Each namespace file is import for those specific type definitions
Reviewing XSD results in a schema visual editor tool
slide 31
Information Restriction Considerations The CAM template allows full use of
XPath conditional expressions and a rich set of over 30 functions including: setNumberRange(), setLength(), setLimit(),
setDateMask(), setNumberMask(), restrictValues(), lookup()
Those that are compatible with XSD constraints will cause constraint schema assertions to be written out when exporting to schema
In the advanced topics section we will look at cross field validations using XPath conditional rules
Generating Testing and
Conformance Examples
Generating base examplesCustomizing content with HintsSelecting valid and invalid modesRunning rules validation check
slide 33
Test Case Generation Quick StartFile Menu Option
Default directory to write examples into
Name to be used for the examples
How many examples to createRepeating elements count
for Quick Test – just click “OK” to use default settings
slide 34
Test Case Results
Active links to view the generated examples
slide 35
Advanced Generation Options
How to handle optional items: all | random | none
If you want deliberate errors for fail testing; (will give variety of data and structure errors)
Use namespaces or not; if ‘false’ is
selected – then XML instances are
created with minimized
namespace usage. Optional content hints
(explained next )
Use slider to pick a specific seed value – or leave blank for random seed
Optional schema file validation link; use this to have example validate with EML schema or local subset schemaUse content type
or item name (name is useful for checking backend transform processing)
slide 36
Test Case Generator Feature Summary Make both Pass / Fail testing examples Content hinting so examples use real not fake data Test optional item logic with: all / random / none Uses exclude() assertions so does not include those
items – makes realistic examples of your use pattern Can pass in seed value – use when adding and testing
hints (each test case is labelled with its seed value) Make hundreds of test cases without manual editing Can link test case to XSD schema for structure tests You can modify XSLT tool for own testing needs
slide 37
Run CAM Rules Check on Examples
Pick Test Case Example to VALIDATE; click Finish to run validation rules
Review validation results
Run Menu Option
1
3
2
slide 38
Content Hinting Mechanisms Designed to create realistic data
examples Hints can be provided in two ways Firstly - using ‘Value’ notes in annotations
on specific items in the structure editor Second – create your own Hints XML file
and add matching rules to globally apply across your template(s) – e.g. FirstName, LastName, Address, BirthDate, etc.
Can export from one template, import into another
slide 39
First Approach: annotation Value Hints
Select focus on structure item
Invoke action menu (right mouse click)
Choose “Edit Annotations”
slide 40
Then add Value annotation item
Click on “Add New”, then enter “Value” as Type and confirm “OK”
Select “Value” tab
Enter values terminated with “|” character
1
2
3
4
5 Re-run Example export to see results
slide 41
Second: Hints File Mechanism (XML file)
like / with partial name matching
key matching on tag name
key / parent path matching
use when same name occurs within different parents – e.g. Country and Person / NameElement with different content and context
use for exact match of items
3
2
1use for component match on items – e.g. first with name matches <nxnl:first_name>
Note: matching is case sensitive but ignores namespaces
TIP: can use Export Hints to create initial XML file for editing
slide 42
A- Using Examples Generator with Hints
Select XML hints file to be used here
slide 43
B- Import Hints into Annotations (merge)
set and select as needed
Option to auto-reload new anno file into current template
Documentation
Cross-Reference SpreadsheetDefault reporting options
slide 45
Generating Cross-Reference Spreadsheet Use Compare to Dictionary tool
Compare to default EML-V6-dictionary Drag and Drop XML file into Microsoft
Excel Review and save spreadsheet
workbook
slide 46
Dictionary Compare Tool
Select compare tool menu option
Browse and choose EML dictionary
slide 47
Drag and Drop to Excel Spreadsheet
ClickOK
slide 48
Documentation Layouts Five options
Source XML Component details (XML) Tabular format (HTML) Interactive web page (wiki) Code list
Tabular format page layout for data analyst use and designed to make rules and use patterns clear
Each documentation layout XSLT script can be customized as desired
slide 49
Tabular DocumentationDocumentation Menu Option
Select Format, resulting HTML opens in browser viewer
slide 50
Tabular HTML Content Details
EnhancedData typeLogic
Extended Code listhandling
Clear use pattern
XPath referencesandfunctions
Annotations
Summary
Ingesting XSD Creating use pattern (aka pick list) Generating test examples Hints system Generate XSD schema subset Running tests
slide 52
Localization Checklist Review
XML Schema
2
1
3
4
DocumentationReadable information use details (HTML), EML cross-reference Spreadsheet
XML artifacts The localized schema and picklist.xml
Examples & Test CasesRealistic test XML instances for conformance and interoperability testing
EML Schema XSD aligns voting information model, processes and domain dictionary
slide 53
Summary Capabilities covered
Ingesting XSD to Template Creating use pattern (aka pick list) Generating test examples Hints system Generate local use XSD schema subset Running tests Generating documentation and spreadsheets
Allows rapid localization development Enables consistent implementations Documentation provides clear details
on structures, content, rules and definitions
Cross-reference to EML standard
Advanced Techniques
Extending rules for actual use cases Using XPath selector wizard Handling ingestion recursion issues SourceForge XSLT repository
slide 55
Importing pick list operation
File Menu Option
Select Import and specify filename of your existing pick list xml.
Import process matches the path expressions in your pick list to the XPath expressions in the template.
Exclude statements generated for matching items.
Makes it easy to re-apply a pick list on new versions of schemas, or on similar schemas with same blocks of content – address, company, person, etc.
slide 56
Illustrative EML requirements
When Ballot type > 0 - AND Language code > 1
Require Ballot State = CA, NV, NM
When Registered Voters Total < 1000 - AND Ballots > 1000 Audit alerting
Resource Matching
Region Restriction
slide 57
Template rules syntax<as:BusinessUseContext> <as:Rules> <as:default> <as:context>
<as:constraint condition="//CastBallots /Total > ‘1000’ and( // RegisteredVoters /Total < ‘1000’)"
action="restrictValues(// CastBallots/Total, ‘Alert – possible audit required’) "/>
<as:constraint action="restrictValues(//eml:AdministrativeArea /eml:NameElement , 'CA|NV|NM')"/>
</as:context> </as:default> </as:Rules> </as:BusinessUseContext>
slide 58
XPath selector wizard
Check boxes allow user to select scope of rule action
Conditional context rules can be optionally used
Documentation notes can be added here
XPath syntax is automatically generated for you
slide 59
“CAM Kit” of XSLT tools used XSD 2 CAM
extracts structure and rules and builds template from schema
XML 2 pick list builds pick list from any XML instance (uses Level Depth
setting) CAM 2 Examples
Generates a collection of XML instance test cases Import/Export Hints
Manage and apply content hinting across CAM templates CAM 2 XSD export
Creates a subset XSD from CAM template
slide 60
SourceForge XSLT svn repository Using any svn client the XSLT scripts
can be retrieved from: https://camprocessor.svn.sourceforge.net/svnroot/
camprocessor/camed/uk.org.jcam.camed/trunk/xsl/
Resources
Selection of useful links and additional technical details
slide 62
Simple EML 440 Example Example using the EML 440 cast vote
schema Simplified to minimum requirements Example can be downloaded here:
http://markmail.org/message/qyrpfq75mtctblyq
slide 63
www.oasis-open.org/committees/election
docs.oasis-open.org/election
www.oasis-open.org/committees/cam
wiki.oasis-open.org/cam
docs.oasis-open.org/cam
www.jcam.org.uk
Resources: