DCS Overview MCS/DCS Technical Interchange Meeting August, 2000

DCS OverviewDCS Overview

MCS/DCSTechnical Interchange

MeetingAugust, 2000

SOFIA Data Cycle System: OverviewRIT Center for Imaging Science

The “Mission”The “Mission”

“The DCS shall provide the infrastructure to support the operation of facility instruments on SOFIA using best current knowledge and tools, and supporting continuous improvement in an efficient, extensible, and modular architecture.”

-- DCS CoDR, June 1999.


DCS RequirementsDCS Requirements

“The recommendations of the CoDR committee are the top level requirements for the DCS.”

-- DCS PDR, March 2000.



The DCS shall record and archive all data necessary for complete analysis, calibration and future access to observational data obtained with the FSIs.

The DCS shall support a select number of observing modes with pipeline reduction, including quicklook analysis, for each of the FSIs.



The DCS shall provide on-line documentation necessary for the GI to propose and carry out observations with the FSIs.

The DCS shall be implemented as a system of distributed modules, integrated by an extensible hardware and software infrastructure.



Functionality mandated by SOFIA leaders covers two functional domains:

SOFIA: Data acquisition from live instrument, fast in-flight reduction (“quicklook”).

SSMOC: Data acquisition from simulators and other external sources, full reduction of FSI data.



Goal: Avoid building two separate systems if at all possible.

Observation: The functional role is the same, only the algorithms and equipment varies.

Solution: Two DCS systems, one in SOFIA and one at SSMOC. Each instance of the DCS has the pipelines and interfaces needed for its environment. The core DCS is identical and invariant.


DCS Design GoalsDCS Design Goals

“The DCS must support and benefit from continuous improvement, both in itself and within the SOFIA program, over a twenty year lifetime.”



DCS Design GoalsDCS Design Goals

The DCS design must be:– Modular– Extensible– Maintainable

This is the heart of “continuous improvement.”



DCS Design Goals: ModularDCS Design Goals: Modular

The DCS must not be a monolithic program.

The DCS must not be limited to a single system.

The DCS shall be a collection of small independent services, residing on one or more machines, providing functionality “as needed”.


DCS Design Goals: ExtensibleDCS Design Goals: Extensible

The DCS must support the easy incorporation of new procedures and techniques to its repertoire.

The DCS will provide for the safe test and evaluation of new components, while maintaining access to established “proven components”.


DCS Design Goals: MaintainableDCS Design Goals: Maintainable

The DCS must not be tied to any specific vendor or platform.

The DCS must be based on open, widely available standards and technologies.

The DCS must be well documented, both in design and implementation.


DCS Base TechnologiesDCS Base Technologies

Technologies on which the DCS system is based:

Distribution of and communication between objects across a system. (CORBA)

Extensible and flexible exchange of information, both in data and documentation. (XML)


DCS SubsystemsDCS Subsystems

UserInteraction

Layer

DataAcquisition

DataReduction

DCSStorage

TaskLibrary


DCS Subsystems: User InteractionDCS Subsystems: User Interaction

Common interface to all DCS resources The “DCS experience” is customizable on

a per user basis without affecting the rest of the DCS

Leverages off the web and related tools– Access regardless of geographic location– Relies only on commonly available technology

at user’s end (e.g. web browser)


DCS Subsystems: User InteractionDCS Subsystems: User Interaction

UserInteraction

Layer

DataAcquisition

DataReduction

DCSStorage

TaskLibrary

User profile

User ProxyRequests and Responses


DCS Subsystems: Task LibraryDCS Subsystems: Task Library

Home to all core functionality on user’s behalf.

Can build on simple tasks to provide sophisticated sequences of actions.

Easily extended with new activities and procedures.

“Once you know how to do it, we can automate it.”


DCS Subsystems: Task LibraryDCS Subsystems: Task Library

UserInteraction

Layer

DataAcquisition

DataReduction

DCSStorage

TaskLibrary

User ProxyRequests and Responses

Storage access is unique to

task’s needs

Initiate and monitor data reduction

Initiate, monitor, and control data acquisition


DCS Subsystems: Data StoreDCS Subsystems: Data Store

Includes retrieval and storage of raw science data, but also “much more”.

Captures “everything” (e.g. raw FSI data, reduced data, OBS plans, instrument modes, pipeline parameters, science personnel).

Enables easy import & export in variety of formats (e.g. general investigators, IPAC) by being platform neutral.


DCS Subsystems: Data StoreDCS Subsystems: Data Store

UserInteraction

Layer

DataAcquisition

DataReduction

DCSStorage

TaskLibrary

Storage access is unique to

task’s needs

User profile

Obtain OBS plan, store raw FSI data

Obtain raw FSI data, store

reduced data


DCS Subsystems: Data AcquisitionDCS Subsystems: Data Acquisition

Data acquisition, like the rest of the DCS, is composed of small modules working in concert together.

Modularity insulates the DCS from instrument specifics, and encourages re-use between similar instruments.

Effectively, the DCS “translates” an experiment to instrument specific commands and needs.


DCS Subsystems: Data AcquisitionDCS Subsystems: Data Acquisition

UserInteraction

Layer

DataAcquisition

DataReduction

DCSStorage

TaskLibrary

Obtain OBS plan, store raw FSI data

Initiate, monitor, and control data acquisition

quicklook


DCS Subsystems: Data ReductionDCS Subsystems: Data Reduction

Instrument science teams focus on algorithm development, not the DCS.

Reduction is automatic, distributed, pipelined, and relentless.

DCS Data Reduction removes the need for every GI to have their own compute engines, but does not prevent GI from doing their own reduction.


DCS Subsystems: Data ReductionDCS Subsystems: Data Reduction

UserInteraction

Layer

DataAcquisition

DataReduction

DCSStorage

TaskLibrary

Obtain raw FSI data, store

reduced data

Initiate and monitor data reduction


DCS SubsystemsDCS Subsystems

UserInteraction

Layer

DataAcquisition

DataReduction

DCSStorage

TaskLibrary


DCS TechnologiesDCS Technologies

Built on top of the COTS technologies CORBA and XML, the DCS team is developing four new technologies that enable the overall DCS:– DCS Registry– Data Typing– Data Connection Association Object

(DCAO)– XML Parser Generator


DCS Technologies: DCS RegistryDCS Technologies: DCS Registry

Enables various modules in the DCS to locate each other based on combinations of names, versions, and other criteria.

Allows simultaneous, non-interfering deployment of modules with the same functionality:– Modules that are proven and trusted– Modules undergoing evaluation and test


DCS Technologies: Data TypingDCS Technologies: Data Typing

Provides a “live” dictionary for use in the DCS that defines various data types and formats.

Best explained by example: data typing allows objects to interact with each other using “rawFrameAIRES” instead of “IEEEsingle[1024][1024]”.

More expressive than IDL; addresses more portability and platform issues.


DCS Technologies: DCAODCS Technologies: DCAO

Provides common language, interface, and linkages between groups of objects that must locate and communicate with each other at runtime.

Directly enables the Data Acquisition and Data Reduction subsystems, both of which rely heavily on dynamic collections of objects.


DCS Technologies: Parser GeneratorDCS Technologies: Parser Generator

Extends existing event-driven XML parser (expat) by generating the support code normally written by engineers to use a particular DTD.

“Software writing software”, that ultimately yields a parsed DOM-like object for any given combination of XML document and DTD.


DCS RundownDCS Rundown

Subsystems– User

Interaction– Task Layer– Data Store– Data

Acquisition– Data Reduction

Technologies– Registry– Data Typing– DCAO– XML Parser

Generator

Documents

DCS Overview MCS/DCS Technical Interchange Meeting August, 2000