Upload
gervase-boone
View
213
Download
0
Tags:
Embed Size (px)
Citation preview
Unified Search and Information Integration for the ISS Mission Control Center
Richard M. Keller, Ph.D. Group Lead for Information Sharing and Integration
Intelligent Systems Division
NASA Ames Research Center
August 16, 2011
Outline• Motivation: Search for Mission Control• XSearch System: Functionality & Architecture
– Data Integration Subsystem– Link Generation Subsystem– Interface
• Wrap-up– Status– User Acceptance– Related Work– Lessons Learned– Summary
Primary Customer
International Space Station (ISS) Mission Control Center
ISS Flight Controllers• Monitor ISS systems
• Troubleshoot
• Perform routine maintenance procedures
• Initiate ISS system changes via telemetry commands
• Document status and actions taken
• Coordinate and execute complex operations tasks (e.g. docking, boosting)
Why is SEARCH fundamental to International Space Station operations?
• 24x7 Space Station operations staff generates huge volume of operations workflow records in multiple databases
• Flight controllers must access large number of engineering & operations documents and manuals
• Flight controllers search with very high frequency while on console; search is essential to accomplishing almost every type of operations activity
• Finding information quickly is critical for
– effective decision making
– mission safety and assurance
Search in the Mission Control Center
• Information fragmented across many systems
• Each search interface is different
• Searching is labor intensive, time-consuming, error prone
• Interrelated information across systems is not linked and therefore difficult to access
• Information fragmented across many systems
• Each search interface is different
• Searching is labor intensive, time-consuming, error prone
• Interrelated information across systems is not linked and therefore difficult to access
What are the problems?
Why is interlinked information important?• Obtaining a search result
is just the first step in information gathering
• The next step is to explore key related information needed to interpret the result
• Must understand the context and relationships among data to make effective decisionsLinking Open Data cloud diagram,
by Richard Cyganiak and Anja Jentzsch. http://lod-cloud.net/
Anomaly Report
Procedure
PartFlight Rule
Example: Mission Control Center Search
Chit
Search the Mission Action Request Tracking System(a.k.a. the “Chits” System) for:
“ALS heater”
Interlinked information provides context for decision-making
Outline• Motivation: Search for Mission Control• XSearch System: Functionality & Architecture
– Data Integration Subsystem– Link Generation Subsystem– Interface
• Wrap-up– Status– User Acceptance– Related Work– Lessons Learned– Summary
XSearch Functionality
• Consolidates search – searches multiple databases & displays integrated results
• Standardizes & improves search – uniform search interface – improved search functionality
• full text search, conjunctive search, acronyms, synonyms
• Interlinks & contextualizes search results
XSearch Components1. Data Integration System:
– integrates and warehouses ISS information to be searched on a single server
2. Information Web:– interlinks the warehoused information
3. Link Generation Engine: – dynamically scans new and revised information to detect
links and insert into web of data
4. Search Interface: – searches information stored in the warehouse and
displays both results and linked data
XSearch Architecture
Data Integration Subsystem
XSearch Web Interface
Link Generation Subsystem
Outline• Motivation: Search for Mission Control• XSearch System: Functionality & Architecture
– Data Integration Subsystem– Link Generation Subsystem– Interface
• Wrap-up– Status– User Acceptance– Related Work– Lessons Learned– Summary
Data Integration Subsystem
• Data warehousing approach
• “Core” data fields extracted, normalized & stored
• Highly configurable approach accommodates changes in data sources
Data sources Replicated sources Data Warehouse
periodic replication extract and normalize
XSearch server
databases
documents
web pages
web services
Normalization ExampleTARGET:
Anomaly Database
Core Fields• RecordType• RecordID• Title*• DisplayDate• Content*• ContentType• Status• DisplayName• OriginatorName• Org• Discipline• Activity
XSearch Warehouse
tblAnomaly• ItemID• RequestTitle• WhenModified• StatusText• ControlNumber• OriginatorNetID• RequestOrg• RequestDiscipline• Activity
tblAnomalyBody• DocumentPart• DocumentSect• ErrDescription• ErrImpact• ErrWorkaround• ErrResolution• ItemID tblAnomalyAttachment
• ATTKEY• UserFileName
SOURCE:
*Full-text indexed
Mapping Rules
Mapping Table
for Anomaly Database
RecordType SrcXForm CacheTable CacheCol FixCyrillic AddBOM FixMarkupANOMALY Activity tblARCHIT_Common Activity 0 0 0ANOMALY {CAT select '<div class="ANOMALY_Body" id="ANOMALY_' + cast(documentpart as nvarchar(max)) + '_' + cast(documentsect as nvarchar(max)) + '">' + case when len(cast(isnull(ErrDescription,'') as nvarchar(max))) < 1 then '' else '<div class="ANOMALY_Body_Column" id="ANOMALY_' + cast(documentpart as nvarchar(max)) + '_' + cast(documentsect as nvarchar(max)) + '_ErrDescription">' + cast(isnull(ErrDescription,'') as nvarchar(max)) + '</div>' end + case when len(cast(isnull(ErrImpact,'') as nvarchar(max))) < 1 then '' else '<div class="ANOMALY_Body_Column" id="ANOMALY_' + cast(documentpart as nvarchar(max)) + '_' + cast(documentsect as nvarchar(max)) + '_ErrImpact">' + cast(isnull(ErrImpact,'') as nvarchar(max)) + '</div>' end + case when len(cast(isnull(ErrWorkaround,'') as nvarchar(max))) < 1 then '' else '<div class="ANOMALY_Body_Column" id="ANOMALY_' + cast(documentpart as nvarchar(max)) + '_' + cast(documentsect as nvarchar(max)) + '_ErrWorkaround">' + cast(isnull(ErrWorkaround,'') as nvarchar(max)) + '</div>' tblARCHIT_Common Content 1 1 0ANOMALY '.htm' tblARCHIT_Common ContentType 0 0 0ANOMALY requestDiscipline tblARCHIT_Common Discipline 0 0 0ANOMALY WhenModified tblARCHIT_Common DisplayDate 0 0 0ANOMALY ControlNumber tblARCHIT_Common DisplayName 0 0 0ANOMALY RequestOrg tblARCHIT_Common Org 0 0 0ANOMALY OriginatorNetID tblARCHIT_Common OriginatorName 0 0 0ANOMALY ITEMID tblARCHIT_Common RecordID 0 0 0ANOMALY 'ANOMALY' tblARCHIT_Common RecordType 0 0 0ANOMALY statusText tblARCHIT_Common Status 0 0 0ANOMALY RequestTitle tblARCHIT_Common Title 1 0 0ANOMALY WhenModified tblARCHIT_Common UpdateDate 0 0 0
SourceDatabase
MappingRule
TargetDatatable
TargetField
• source field• constant• SQL
statement
Mapping Table
for Anomaly Database
RecordType SrcXForm CacheTable CacheCol FixCyrillic AddBOM FixMarkupANOMALY Activity tblARCHIT_Common Activity 0 0 0ANOMALY {CAT select '<div class="ANOMALY_Body" id="ANOMALY_' + cast(documentpart as nvarchar(max)) + '_' + cast(documentsect as nvarchar(max)) + '">' + case when len(cast(isnull(ErrDescription,'') as nvarchar(max))) < 1 then '' else '<div class="ANOMALY_Body_Column" id="ANOMALY_' + cast(documentpart as nvarchar(max)) + '_' + cast(documentsect as nvarchar(max)) + '_ErrDescription">' + cast(isnull(ErrDescription,'') as nvarchar(max)) + '</div>' end + case when len(cast(isnull(ErrImpact,'') as nvarchar(max))) < 1 then '' else '<div class="ANOMALY_Body_Column" id="ANOMALY_' + cast(documentpart as nvarchar(max)) + '_' + cast(documentsect as nvarchar(max)) + '_ErrImpact">' + cast(isnull(ErrImpact,'') as nvarchar(max)) + '</div>' end + case when len(cast(isnull(ErrWorkaround,'') as nvarchar(max))) < 1 then '' else '<div class="ANOMALY_Body_Column" id="ANOMALY_' + cast(documentpart as nvarchar(max)) + '_' + cast(documentsect as nvarchar(max)) + '_ErrWorkaround">' + cast(isnull(ErrWorkaround,'') as nvarchar(max)) + '</div>' tblARCHIT_Common Content 1 1 0ANOMALY '.htm' tblARCHIT_Common ContentType 0 0 0ANOMALY requestDiscipline tblARCHIT_Common Discipline 0 0 0ANOMALY WhenModified tblARCHIT_Common DisplayDate 0 0 0ANOMALY ControlNumber tblARCHIT_Common DisplayName 0 0 0ANOMALY RequestOrg tblARCHIT_Common Org 0 0 0ANOMALY OriginatorNetID tblARCHIT_Common OriginatorName 0 0 0ANOMALY ITEMID tblARCHIT_Common RecordID 0 0 0ANOMALY 'ANOMALY' tblARCHIT_Common RecordType 0 0 0ANOMALY statusText tblARCHIT_Common Status 0 0 0ANOMALY RequestTitle tblARCHIT_Common Title 1 0 0ANOMALY WhenModified tblARCHIT_Common UpdateDate 0 0 0
Fill DisplayDate in the warehouse record with the WhenModified field from the Anomaly database
WhenModified DisplayDatea field in Anomaly DB in Warehouse
a string constant in Warehouse
Mapping Table
for Anomaly Database
RecordType SrcXForm CacheTable CacheCol FixCyrillic AddBOM FixMarkupANOMALY Activity tblARCHIT_Common Activity 0 0 0ANOMALY {CAT select '<div class="ANOMALY_Body" id="ANOMALY_' + cast(documentpart as nvarchar(max)) + '_' + cast(documentsect as nvarchar(max)) + '">' + case when len(cast(isnull(ErrDescription,'') as nvarchar(max))) < 1 then '' else '<div class="ANOMALY_Body_Column" id="ANOMALY_' + cast(documentpart as nvarchar(max)) + '_' + cast(documentsect as nvarchar(max)) + '_ErrDescription">' + cast(isnull(ErrDescription,'') as nvarchar(max)) + '</div>' end + case when len(cast(isnull(ErrImpact,'') as nvarchar(max))) < 1 then '' else '<div class="ANOMALY_Body_Column" id="ANOMALY_' + cast(documentpart as nvarchar(max)) + '_' + cast(documentsect as nvarchar(max)) + '_ErrImpact">' + cast(isnull(ErrImpact,'') as nvarchar(max)) + '</div>' end + case when len(cast(isnull(ErrWorkaround,'') as nvarchar(max))) < 1 then '' else '<div class="ANOMALY_Body_Column" id="ANOMALY_' + cast(documentpart as nvarchar(max)) + '_' + cast(documentsect as nvarchar(max)) + '_ErrWorkaround">' + cast(isnull(ErrWorkaround,'') as nvarchar(max)) + '</div>' tblARCHIT_Common Content 1 1 0ANOMALY '.htm' tblARCHIT_Common ContentType 0 0 0ANOMALY requestDiscipline tblARCHIT_Common Discipline 0 0 0ANOMALY WhenModified tblARCHIT_Common DisplayDate 0 0 0ANOMALY ControlNumber tblARCHIT_Common DisplayName 0 0 0ANOMALY RequestOrg tblARCHIT_Common Org 0 0 0ANOMALY OriginatorNetID tblARCHIT_Common OriginatorName 0 0 0ANOMALY ITEMID tblARCHIT_Common RecordID 0 0 0ANOMALY 'ANOMALY' tblARCHIT_Common RecordType 0 0 0ANOMALY statusText tblARCHIT_Common Status 0 0 0ANOMALY RequestTitle tblARCHIT_Common Title 1 0 0ANOMALY WhenModified tblARCHIT_Common UpdateDate 0 0 0
Fill RecordType in the warehouse record with the string ‘ANOMALY’
‘ANOMALY’ RecordType
Mapping Table
for Anomaly Database
RecordType SrcXForm CacheTable CacheCol FixCyrillic AddBOM FixMarkupANOMALY Activity tblARCHIT_Common Activity 0 0 0ANOMALY {CAT select '<div class="ANOMALY_Body" id="ANOMALY_' + cast(documentpart as nvarchar(max)) + '_' + cast(documentsect as nvarchar(max)) + '">' + case when len(cast(isnull(ErrDescription,'') as nvarchar(max))) < 1 then '' else '<div class="ANOMALY_Body_Column" id="ANOMALY_' + cast(documentpart as nvarchar(max)) + '_' + cast(documentsect as nvarchar(max)) + '_ErrDescription">' + cast(isnull(ErrDescription,'') as nvarchar(max)) + '</div>' end + case when len(cast(isnull(ErrImpact,'') as nvarchar(max))) < 1 then '' else '<div class="ANOMALY_Body_Column" id="ANOMALY_' + cast(documentpart as nvarchar(max)) + '_' + cast(documentsect as nvarchar(max)) + '_ErrImpact">' + cast(isnull(ErrImpact,'') as nvarchar(max)) + '</div>' end + case when len(cast(isnull(ErrWorkaround,'') as nvarchar(max))) < 1 then '' else '<div class="ANOMALY_Body_Column" id="ANOMALY_' + cast(documentpart as nvarchar(max)) + '_' + cast(documentsect as nvarchar(max)) + '_ErrWorkaround">' + cast(isnull(ErrWorkaround,'') as nvarchar(max)) + '</div>' tblARCHIT_Common Content 1 1 0ANOMALY '.htm' tblARCHIT_Common ContentType 0 0 0ANOMALY requestDiscipline tblARCHIT_Common Discipline 0 0 0ANOMALY WhenModified tblARCHIT_Common DisplayDate 0 0 0ANOMALY ControlNumber tblARCHIT_Common DisplayName 0 0 0ANOMALY RequestOrg tblARCHIT_Common Org 0 0 0ANOMALY OriginatorNetID tblARCHIT_Common OriginatorName 0 0 0ANOMALY ITEMID tblARCHIT_Common RecordID 0 0 0ANOMALY 'ANOMALY' tblARCHIT_Common RecordType 0 0 0ANOMALY statusText tblARCHIT_Common Status 0 0 0ANOMALY RequestTitle tblARCHIT_Common Title 1 0 0ANOMALY WhenModified tblARCHIT_Common UpdateDate 0 0 0
{CAT select '<div class="ANOMALY_Body" id="ANOMALY_' + cast(documentpart as nvarchar(max)) + '_' + cast(documentsect as nvarchar(max)) + '">' + case when len(cast(isnull(ErrDescription,'') as nvarchar(max))) < 1 then '' else '<div class="ANOMALY_Body_Column" id="ANOMALY_' + cast(documentpart as nvarchar(max)) + '_' + cast(documentsect as nvarchar(max)) + '_ErrDescription">' + cast(isnull(ErrDescription,'') as nvarchar(max)) + '</div>' end + case when len(cast(isnull(ErrImpact,'') as nvarchar(max))) < 1 then '' else '<div class="ANOMALY_Body_Column" id="ANOMALY_' + cast(documentpart as nvarchar(max)) + '_' + cast(documentsect as nvarchar(max)) + '_ErrImpact">' + cast(isnull(ErrImpact,'') as nvarchar(max)) + '</div>' end + case when len(cast(isnull(ErrWorkaround,'') as nvarchar(max))) < 1 then '' else '<div class="ANOMALY_Body_Column" id="ANOMALY_' + cast(documentpart as nvarchar(max)) + '_' + cast(documentsect as nvarchar(max)) + '_ErrWorkaround">' + cast(isnull(ErrWorkaround,'') as nvarchar(max)) + '</div>' end + case when len(cast(isnull(ErrResolution,'') as nvarchar(max))) < 1 then '' else '<div class="ANOMALY_Body_Column" id="ANOMALY_' + …
Fill Content in the warehouse record with multiple fields concatenated from the Anomaly database
in Anomaly DB in Warehousemultiple fields Content
Outline• Motivation: Search for Mission Control• XSearch System: Functionality & Architecture
– Data Integration Subsystem– Link Generation Subsystem– Interface
• Wrap-up– Status– User Acceptance– Related Work– Lessons Learned– Summary
Link Generation Subsystem
• links generated by analyzing textual information in warehouse records and documents
• links and warehouse records are displayed in interface
Data Warehouse Information Web
XSearch Interface
link generationlinks
Link Generation Methods
Method 1 (for explicit cross-references): XSearch analyzes document/record text to identify citations to other documents
Method 2 (for implicit cross-references): XSearch performs cross-document text analysis to determine similarities among documents
• Pairs of documents in the corpus are compared for similarity – over 10 billion comparisons!
What do flight control documents/records look like?
• Combination of structured and free text fields authored by Space Station flight controllers while on duty
• Free text fields cross-reference other documents/records using established naming and numbering conventions
• NO markup used to identify cross-references in text
• Difficult to spot cross-references visually
Sanitized Excerpt from an ISS “Flight Note”“The purpose of this note is to outline the hazards to the EVA crew associated with the EVA repair of the SWFPF, for 22A EVA 7. Dynamic events may affect structural integrity of the damaged component – this hazard cause will be controlled by implementation of chit 5452, “Update to Docked Loads Constraints for SWFPF during XPF rotations”, currently open for review. Analysis has shown the Solar Array… …Crew-induced loads on the WIF extender are generally controlled by Flight Rule 22A_C2-D, “Electrical Load Constraints [HC] [RI]”; loads constraints for EVA Articulating Portable Foot Restraint (APFR) and Worksite Interface (WIF) Extender are defined in table 22A_C2-86-F. APFR/WIF loads constraints for EVA 4 have been updated in CHIT 5480. Note that for contingency use of the WIF Extender for EV work on P6, the only approved attitude control is SRM QFMT NO DTP as detailed in EFN F012457A. VRCS loads during LoAC recovery are acceptable as detailed in AR 001767. This hazard control must continue to be observed, to prevent structural failure of the WIF Extender.”
Cross-reference Detection: Mining for links
Chit 004361
Time
...This chit implements aResolution to the primaryALS heater failure described in Anomaly #447...Based oninformation described in FN F0121902,temperature instability began on GMT 125.
Chit 004374
...This chit revisesthe policy establishedin Chit 004361
...temperature instabilitywas observed shortly afterthe primary ALS heater failure was detected...
EFN F013094EFN F0121902
AR 00447
CITESx-reference
CITED-BYx-reference
similar terms SIMILAR-TOx-reference
Search for “ALS heater”
Analyze each search result to determine three types of cross-references hidden in the text
Method 1: Explicit Cross-reference Detection via Pattern Matching
• Step 1: Cross-reference Candidate Identification
Find the cross-references in the text
• Step 2: Cross-reference Validation
Make sure the cross-referenced records exist in the specified database
Step 1: Cross-reference Candidate Identification
1. For regularized naming schemes: Use specialized BNF Grammars to recognize cross-reference citation patterns
• ‘Anomaly Report 001245’, ‘Anomaly 1245’, ‘AR #1245’
• ‘Flight Note F013457B’, ‘EFN F013457B’, ‘F013457B’
2. For irregular naming schemes: Use exhaustive match against database of defined record identifiers:
• ‘SPS cable assembly’
Anomaly Reports XRef Detection GrammarARpattern ::= AnomalyTrigger {Digit,1,6}AnomalyTrigger ::= AnomalyReport | ARAnomalyReport ::= Anomaly Report{‘s’,0,1}Anomaly = “Anomaly” | “anomaly”Report = “Report” | “report”AR = “AR” | “ar”
“Anomaly” “anomaly” “Report” “report”
“AR” “ar”
{Digi,1,6}
OR OR ?
OR
OR
ANDBNF Grammar
Example Citations:
• ‘Anomaly Report 001245’
• ‘Anomaly 1245’
• ‘AR #1245’AND
Step 2: Cross-reference Validation
• Check that cross-referenced record actually existsto avoid creating dead-end links
• Query appropriate database to confirm
Method 2: Implicit Cross-Reference Detection via Similarity Analysis
• Model each record as a point in a multi-dimensional vector space • Each dimension represents a word (found within some record) • Each rec is described by a vector:
<w1, w2, w3, w4, …. wn> • Dimensions normalized by frequency of occurrence• Similarity between two points is measured by the cosine of the
angle between their two vectors.
TF-IDF Vector Space Method
Outline• Motivation: Search for Mission Control• XSearch System: Functionality & Architecture
– Data Integration Subsystem– Link Generation Subsystem– Interface
• Wrap-up– Status– User Acceptance– Related Work– Lessons Learned– Summary
XSearch Interface
DetailsPane
SearchQueryPane
SearchResultsPane
XSearch Interface: Query Pane
Time Period Selection
Information Sources to be Searched
Search Term(s)
Search Field Selection
“ALS heater”
XSearch Interface: Results Pane
Cross-References Summary (Line 2)
Identifying Information (Line 1)
Common set of fields displayed across all data sources
Individual Search Result
Record Type
XSearch Interface: Details PaneRecord Preview
Cross-References Detail Tabs
Identifying Information
Outline• Motivation: Search for Mission Control• XSearch System: Functionality & Architecture
– Data Integration Subsystem– Link Generation Subsystem– Interface
• Wrap-up– Status– User Acceptance– Related Work– Lessons Learned– Summary
Status
• Deployed in Space Station Mission Control Center since July 2008
• Integrates 15 Mission Control information sources containing >1M records, total
• In use daily• User base: 2200 Space Station & Space
Shuttle personnel
User Acceptance• Informal evaluation performed using search log files
and user interviews• Documented cases of value during problem solving• Strongest & most enthusiastic adoption by those
w/cross-cutting flight control responsibilities:– Group leads, management personnel, multi-functional
consolidated flight control positions
• Preference for legacy tool-specific search capabilities over XSearch in the absence of cross-cutting analysis needs
• Training environment challenging
Related Systems1. Search:
- Google Search Appliance, Lucene/Solr, Autonomy Search
2. Enterprise integration:- IBM’s InfoSphere, Informatica’s PowerCenter
3. Bibliographic Citation Analysis: CiteSeer, Google Scholar, Scopus
4. Linked data: Link Discovery Systems, Named Entity Resolution, DB Record Linkage
XSearch uniquely combines capabilities, while addressing specific Space Station Mission Control Center needs:
– Fine-grained access control– Negligible indexing latency
Lessons Learned
• Architect for ease of system reconfigurability• Mitigate impact of ‘data politics’ w/negotiated
data service agreements• Ensure adequate systems health monitoring• Understand user needs thoroughly• Guard against “requirements creep”
Summary• XSearch integrates Space Station Mission Control
data from multiple sources and enables cross-source search
• Text mining & information retrieval techniques are used to create linked “document networks” connecting related mission control documents
• Interconnected “document networks” provide Flight Controllers with rapid access to related documents, thus decreasing incident response times and improving overall situational awareness
Acknowledgments: XSearch Team
Ames Research Center
Design /Development :Intelligent Systems
Division
Dan BerriosMohana Gurram
Rich Keller*Chris Knight
Steve LinMay WindremShawn Wolfe
Infrastructure / Operations
Support:MCC Web Tools
Team
James HartTim Hall*
Jason ToschlogBrian UlmanJenny Yang
Customers & Requirements
Brian Kelly*ISS/STS Flight Controllers
ISS EngineeringISS Program Staff
ISS
Ames Research Center Johnson Space Center
Johnson Space Center
Questions?
?
R.M. Keller and C.D. Knight, "XSearch: A Unified Search and Cross-Reference Detection Engine for the ISS Mission Control Center", IEEE Aerospace Conference, Big Sky, MT, March 2011
R.M. Keller, S.R. Wolfe, M.N.Windrem, and D.C. Berrios, "XSearch: A System for Searching and Interrelating NASA Mission Operations Data", AIAA Space Ops 2008 Conference, Heidelberg, Germany, May 2008..
References
Backup
Authorization and Access Control
• Only users belonging to a flight controller group can access portal where XSearch resides
• XSearch is a read-only interface
• Most sources were read-accessible by all flight controllers
• Where required, flight controller groups utilized as a basis for access control (console logs, flight notes)
• Exceptions for flight director and “private” notes
• Multi-group users have union of permissions
Why not buy the Google Search Appliance?
• We analyzed pros & cons of GSA• GSA cannot provide contextual cross-references
– No commercial search product provides this capability
• GSA can handle basic search and cross-source integration– But GSA is not magic and requires configuration (firewalls, security) and
ongoing maintenance (index, database reconfiguration)
• GSA solution does not meet NASA needs in some key areas: – access control – privacy and restricted information– latency of indexing– interface customizability– indexing methods are ‘black box’