Embed Size (px)
Citation preview
GJXDM User Conference Atlanta
8-10 June 2005
NLETS Lessons Learned
NLETS is a sophisticated high-speed message switching system created for and dedicated to the criminal justice community.
What is NLETS?What is NLETS?
Non-profit corporation chartered by the States – funded by user fees
Members are all 50 states, most federal agencies & territories, and the RCMP
Nearly 40 years in operation
Links 30,000 agencies and over 500,000 access devices in the U.S. and Canada
Topping 40 million transmissions per month
NLETS is:NLETS is:
Facts and Figures
• 41 Million transactions per month! Only way to: State Criminal Histories Electronic “HIT” confirmation Canadian Files (CPIC) Homeland Alert Messages Screen Officers Flying Armed INS databases at LESC – IAQs Driver records and vehicle registrations Aircraft Registrations HAZMAT, GSA Federal registrations, NICB Push administrative messages to all CJ agencies
ONE WAY
Services:Services:
Private T1 Network, NO InternetVPN – Triple DES encryptionFire-walled at every connectionHot Disaster Recovery Site - IdahoStore and Forward – Guaranteed Message DeliveryOperation center 7 x 24 x 365 - Arizona99.6% uptime
Requirements
• Represent precise data• Reliability• Performance• Flexibility• Maintainability
NLETS XML Operational Capabilities
• XML Message Router (XMR) since 2001• All NLETS transactions available in GJXDM 3.0
format plus full legacysystem translation capabilities since September
• 750,000 XML rap sheets alone per month All FBI, WI and KY rapsheets are in XML
• All NLETS states/agencies have access to XML and web services
XML Formats and Transport
• NLETS allows users to send XML via Web Services, MQ Series or TCP/IP Allowing XML over various protocols is crucial to
users in our environment
• NLETS transforms between XML and legacy text to allow interoperability between users
• All new data providers will be connecting via Web Services
Validates userTransforms XML to Text
`
XML Message
Text Message
XMR
Determines Destination &
FormatValidates XML
XML Message
Text Message
Text/CDATA XML
Message
Web Services
• Within a secured network, NLETS exposes a web service to which users connect and send XML as a string
• States wishing to receive a response via web services host own service for NLETS to connect to NLETS specifies name and fingerprint of service,
how data is handled is up to state
NLETS Standardized GJXDM XML Transactions
• All Inquiries Standard text format is easy to convert to/from XML
• Criminal History Responses Leveraging the JTF Criminal History Rapsheet
Specification• Hazardous Materials Responses
Standardized single-source responses from Operation Respond
• Drivers and Registration Responses Standardized responses from state DMVs based on
CANDLE grant products
So, how did we do it?
• Mapped elements to GJXDM• Documented XML Instances• Created applications to consume XML• Created schemas• Get users to start implementing• Change management
Mapping NLETS Transactions
• First, created a
generic message
envelope, in NLETS
namespace (prefix n)• Header information
remains in the payloadinstead of SOAP Multiple transports Legacy header data
<n:NLETS>
<n:NLETSInquiryMessage>
<n:NLETSInquiryHeader>
….. NLETS header elements ……
</n:NLETSInquiryHeader>
<n:NLETSInquiryData>
….. NLETS transaction data ……
</n:NLETSInquiryData>
</n:NLETSInquiryMessage>
</n:NLETS>
Mapping NLETS Transactions
• Next, within the generic NLETS message envelope we defined each transaction in XML Going from a legacy system provided us a strictly defined set of
information
• Used GJXDM elements where available• Extended GJXDM elements and types when possible
Example: Agency
• Participated in GJXDM development by utilizing feedback mechanisms (Bugzilla, XSTF)
NLETS GJXDM Example<n:NLETS> <n:NLETSMailHeader>
<n:MessageKeyCode>RQ</n:MessageKeyCode><j:DocumentSource.Organization>
<j:OrganizationORIID> <j:ID>GA0250300</j:ID>
</j:OrganizationORIID> </j:DocumentSource.Organization> <n:DocumentDestinationID> <j:ID>AZ</j:ID> </n:DocumentDestinationID> <n:DocumentControlFieldText>TERM000000</n:DocumentControlFieldText> </n:NLETSMailHeader>
<n:NLETSInquiryData Key=“RQ”><n:NLETSInquiryData Key=“RQ”>
<j:VehicleRegistrationPlateID><j:VehicleRegistrationPlateID>
<j:ID>LJB934</j:ID><j:ID>LJB934</j:ID>
</j:VehicleRegistrationPlateID></j:VehicleRegistrationPlateID>
<j:VehicleRegistrationPlateTypeCode>PC</j:VehicleRegistrationP<j:VehicleRegistrationPlateTypeCode>PC</j:VehicleRegistrationPlateTypeCode >lateTypeCode >
<n:IDExpirationDateText>1983</n:IDExpirationDateText><n:IDExpirationDateText>1983</n:IDExpirationDateText>
</n:NLETSInquiryData></n:NLETSInquiryData></n:NLETS>
RQ.GA0250300.AZ.*TERM000000.TXT
LIC/LJB934.LIY/1983.LIT/PCLIC/LJB934.LIY/1983.LIT/PC
NLETS GJXDM Header
<n:NLETSMailHeader><n:MessageKeyCode>RQ</n:MessageKeyCode><j:DocumentSource.Organization>
<j:OrganizationORIID> <j:ID>GA0250300</j:ID>
</j:OrganizationORIID> </j:DocumentSource.Organization> <n:DocumentDestinationID> <j:ID>AZ</j:ID> </n:DocumentDestinationID>
<n:DocumentControlFieldText>TERM000000</n:DocumentControlFieldText>
</n:NLETSMailHeader>
RQ.GA0250300.AZ.*TERM000000.TXT
NLETS GJXDM Data
<n:NLETSInquiryData Key=“RQ”><n:NLETSInquiryData Key=“RQ”> <j:VehicleRegistrationPlateID><j:VehicleRegistrationPlateID> <j:ID>LJB934</j:ID><j:ID>LJB934</j:ID> </j:VehicleRegistrationPlateID></j:VehicleRegistrationPlateID> <j:VehicleRegistrationPlateTypeCode>PC</j:VehicleRegistrationPlateTypeCode ><j:VehicleRegistrationPlateTypeCode>PC</j:VehicleRegistrationPlateTypeCode > <n:IDExpirationDateText>1983</n:IDExpirationDateText><n:IDExpirationDateText>1983</n:IDExpirationDateText></n:NLETSInquiryData></n:NLETSInquiryData>
LIC/LJB934.LIY/1983.LIT/PC LIC/LJB934.LIY/1983.LIT/PC
• Supporting legacy text values Legacy text may have supported
nonconforming data values (example: Expiration Date)
• Newly standardized transactions Cannot find structure in older, non-
standardized formats Must support two versions of XML “3.0”
• Multiple ways to represent same data• GJXDM not meant to support certain types
of data Messaging/Header information
GJXDM Mapping Challenges
Schemas
• Each message type has its own schema “package” Combined Document/Extension schema
Defines overall structure of message – referencing GJXDM elements and defining NLETS-specific elements
Global JXDM constraint schema Defines GJXDM elements being used and specifies
constraints (min/max occurrences)
• Shared supporting schemas NCIC, NIBRS code enumerations, etc
Schemas<n:NLETS><n:NLETS>
……</n:NLETS></n:NLETS>
XML InstanceXML Instance
DQ.xsdDQ.xsd
Document/Extension Document/Extension SchemaSchema
Constraint SchemaConstraint Schema
JXDM.xsdJXDM.xsd
Supporting SchemasSupporting Schemas
xsd.xsdxsd.xsdncic_2000ncic_2000
.xsd.xsd
Validates againstValidates against
ImportsImports
ImportsImports
Schemas
• NLETS does not currently perform schema validation on each transaction Table-based validation of data Validate users’ XML in a test environment first, then
trust them
• Schemas are provided primarily as documentation for programmers
• Recommend use during development and testing• Schemas cannot communicate all rules for XML
conformance
Documentation of XML Specifications
• XML is defined in detail in Appendix A of the NLETS User Guide
• For each transaction, schemas, XML instances, stylesheets and business rules are provided
• NLETS processes, switches data but does not create, store or manipulate data
• Must parse text and XML in order to route, validate and convert XPath for XML, regular expressions for
text
Consuming XMLConsuming XML
Standardized Rap SheetsStandardized Rap Sheets
Currently Wisconsin, Kentucky, Maine and FBI sending and/or receiving rapsheets in XML – JTF 2.2 formatNew states will be sending JTF 3.0 XML rapsheets (compliant with GJXDM 3.0)All users receiving standard presentation format – most in legacy formatsAll III Inquires done - go through and are delivered by – NLETS
Mapping the Rapsheet
• Similar to the process followed in mapping NLETS transactions
• While we built from an existing specification, we had more flexibility Benefited from GJXDM data dictionary
Data we had not previously thought to include Provides more clear consistency throughout rapsheet
• Found a greater need to extend elements to restrict allowable values
CANDLECANDLE
National Institute of JusticeBureau of Justice AssistanceNLETS (Awardee)American Association of Motor Vehicle AdministratorsAdvanced Technology SystemsPilot States
BJA
Collaboration between AAMVA and NLETSfor Driver License Exchange
CANDLE
• Standardized GJXDM XML representation of DMV drivers license and registration responses
• Unlike NLETS Inquiries and Rapsheets, CANDLE specifications were built from scratch – not existing spec
• Cannot convert text driver and registration responses to XML
Operation RespondOperation Respond
HAZMAT Information - live
Trucking manifests and rollover data
Rail Car Data
Passenger Train Schematics
Aerial photography via NLETS – a possibility
Operation Respond
• Not a criminal justice agency, though information is pertinent to CJ agencies
• GJXDM did not contain hazardous material elements
• Created elements in Operation Respond’s namespace, utilizing same naming conventions and GJXDM elements when available
Change Management
• Even small changes affect entire user base
• User concern about a “moving target”• Only plan changes when necessary – or
optionally adding value• Plan to give states advance notice to the
extent possible• Work with states to minimize effects
User Implementation
• Allowed users to start slow Specify format preference by message key for each
ORI XML can be sent via multiple protocols; not limited to
web services• Provide support to users • Stylesheets, transformations were a necessity
because users are and will continue to be at different stages
• As new services are offered, they may only be available via XML and web services
• Typos, other minor mistakes Do you fix and risk disrupting users or leave incorrect?
• Namespaces Has become considerably more complicated than with
initial specifications New namespaces (example AA) Multiple GJXDM versions/namespaces
• Inheritance – differing requirements for same element
• Support of multiple transports Difficult to utilize SOAP header, WS-Attachments, etc
• Hindsight… Multiple ways to do everything – each with own pros, cons
Problems
Benefits
• Allows NLETS to share more information with users
• Allows users to customize data presentation via stylesheets
• XML is non-proprietary Presents enormous cost savings to states
Benefits
• Implementing new services quicker and easier If a data provider already has info in GJXDM
XML, we can quickly position ourselves to receive it and provide to users
• Sharing data within a state is simpler – eliminates the need for XML-to-XML transformations (example: Amber Alert)
Lessons Learned
• Be sensitive to early adopters
• Recognize that you will need careful change management(“ripple effect”)
• Consider large issues carefully before beginning message structure,
namespaces, schemas• Remember that the
GJXDM will not provide everything
Future Plans
• Expand membership and services offered
• Web Services opens new doors to new providers
• Standardize all NLETS transactions in GJXDM-compliant XML
