Upload
edda
View
27
Download
0
Tags:
Embed Size (px)
DESCRIPTION
The State of NeXus. P.F. Peterson and F.A. Akeroyd on behalf of the NeXus International Advisory Committee. 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008. NIAC Berkeley, CA. NIAC Villigen, Switzerland. NIAC Berlin, Germany. NIAC Sydney, Australia. - PowerPoint PPT Presentation
Citation preview
The State of NeXus
P.F. Peterson and F.A. Akeroydon behalf of the NeXus International
Advisory Committee
NeSSI: Collaborative Software and Other Mythical Creatures
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
Soft
NeS
S’94
Soft
NeS
S’95
Soft
NeS
S’96
v1.1
.0, v
1.2.
0, v
1.2.
1, v
1.3.
0
v1.3
.1, v
1.3.
2, v
1.3.
3
v2.0
.0
v2.1
.0
v3.0
.0
v4.0
.0
NIA
C Pa
sade
na, C
A
NIA
C Vi
llige
n, S
witz
erla
nd
NIA
C Sa
nta
Fe, N
M
NIA
C G
reno
ble,
Fra
nce
NIA
C Be
rkel
ey, C
A
NIA
C Be
rlin,
Ger
man
y
NIA
C Sy
dney
, Aus
tral
ia
NeSSI: Collaborative Software and Other Mythical Creatures
NIAC Members• Frederick Akeroyd, Rutherford Appleton Laboratory, UK (ISIS, Executive Secretary) • Stephen Cottrell, Rutherford Appleton Laboratory, UK (Muon Representative) • Matthias Drochner, Forschungzentrum Jülich, Germany • Emmanuel Farhi, Institut Laue Langevin • Andrew Götz, European Synchrotron Radiation Facility, France • Nick Hauser, Australian Nuclear Science and Technology Organisation, Australia
(Documentation Release Manager, netCDF contact) • Jens-Uwe Hoffmann, Helmholtz Zentrum Berlin (renamed HMI), Germany • Pete Jemian, Advanced Photon Source, USA • Paul Kienzle, DANSE, USA • Mark Könnecke, Paul Scherrer Institut, Switzerland (Technical Committee Chair) • Nick Maliszewskyj - NIST Center for Neutron Research, USA • Raymond Osborn, Argonne National Laboratory, USA • Peter Peterson, Spallation Neutron Source, USA (Chair) • Stephane Poirier, Synchrotron Soleil, France • Thomas Proffen, Los Alamos National Laboratory, USA • Jiro Suzuki, KEK, Japan
Principles of NeXus
Mission: NeXus enables comprehension of data amongst the neutron, x-ray, and muon science communities through a standard data format.
Reduced need for local expertise
• [email protected]• [email protected]• http://www.nexusformat.org
• http://svn.nexusformat.org• http://trac.nexusformat.org
Simple Plotting
NXdataNXdata
Reduced need for conversion
Is it based on anything?
• Hierarchical Data Format (4 and 5)– NXopen(filename,NXACC_CREATE4,handle)– NXopen(filename,NXACC_CREATE5,handle)
• eXtensible Markup Language– NXopen(filename,NXACC_CREATEX,handle)
– NXopen(filename,NXACC_RDWR,handle)– NXopen(filename,NXACC_READ,handle)
Who is committed to NeXus
SNS at ORNLSNS at ORNLNCNR at NISTNCNR at NISTISIS at RAL (neutrons and muons)ISIS at RAL (neutrons and muons)DIAMOND at RALDIAMOND at RALBI at ANSTOBI at ANSTOFRM2 at TUMFRM2 at TUMKENS at KEKKENS at KEKJ-PARCJ-PARC
APS at ANLAPS at ANLHMI in BerlinHMI in BerlinMLNSC at LANLMLNSC at LANLSINQ at PSISINQ at PSIILL in GrenobleILL in GrenobleLLB at SaclayLLB at Saclay
Existing HDF4/5 tools
• HDFLook – HDF based plotter (???)• Java HDF Viewer (NCSA)• Scientific Data Browser – view data over the
web (NCSA)
• Any web browser• Eclipse with WTP plugin
Existing XML tools
NeXus Utilities• NXbrowse – CLI tool to interactively explore NeXus files• NXdir – CLI tool that lists contents of NeXus file and convert
selected portions to ASCII• NXextract• NXtoDTD – Creates a definition file from an existing NeXus file• NXtoXML – Writes an XML version of an existing NeXus file• NXtranslate – anything to NeXus converter NXvalid – GUI tool to
interactively explore, plot, and validate NeXus files• NXtree – Create a html based tree view of NeXus files• NXvalid – GUI tool to view and validate NeXus files• NXvalidate – command line tool for validating structure of
NeXus files
NXvalid
NXtranslate• Anything to NeXus converter
– Binary dump– FRM2– IPNS run– NeXus– Spec– XML
• Uses an XML-based translation file to determine where to get information from
• Additional information can be supplied in the translation file directly• CLI to encourage batch usage• Simple macro language to allow translating several files using one
translation file
Tools and Utilities• addit, subi – programs to add or subtract powder data and write files suitable for Rietveld programs (SINQ)• Amortool – simple CLI tool for Reflectometer data analysis (SINQ)• anatric, cami4pcd – four circle diffractometer data analysis (SINQ)• BerSANS – data analysis for SANS through adapter (SINQ)• DANSE – Data Analysis Architecture for Neutron Scattering Experiments (CalTech)• DAVE (future) - Data Analysis and Visualization Environment (NCNR)• FABLE (ESRF)• fit – home grown fitting program (SINQ)• GumTree – scientific workbench for instrument control (BI)• hdfb.sav and h5b.sav – browser for any 1D/2D/3D data saved in HDF (APS)• ISAW – Integrated Spectral Analysis Workbench (IPNS)• KUPLOT – General purpose plotting program (MLNSC)• LAMP – Large Array Manipulation Program (ILL)• MANTID – Manipulation Toolkit for ISIS Data (ISIS)• McStas – instrument modeling program (SINQ/ILL)• Nathan – TOF data analysis for FOCUS (IDL based) (SINQ)• NDS – NeXus data server, publish NeXus files on TCP/IP, read only (SINQ)• ninx – inx (ILL) adapted to read FOCUS TOF files (SINQ)• NNDB – Java program to browse NeXus files published through NDS (SINQ)• NXviewer – OpenDave based viewer (FRM2)• OpenGenie – General purpose neutron and muon scattering data reduction and analysis tool (ISIS)• Redas – data analysis for Reflectometer, based on Scilab (SINQ) • ReflPak – data analysis for Reflectometer (future) (NCNR)• Scilab – general matlab like data analysis tool - with a NeXus interface for that (SINQ)• SNS Portal (SNS)• UDA and WIMDA – General purpose muon scattering analysis packages (ISIS)
nexusformat.org
NeXus API v4.2
NeXusDefinitionLanguage
Validation of NeXus files using XMl Schema
Freddie AkeroydNIAC
Why Validation?
Standardised definitions make sharing files and analysis programs easier
However, developers need a way to confirm their files comply to the standard
NeXus Meta-DTD Format
Not really a true DTD, more a template XML file
However there are a few issues:Syntax is a little “cryptic” – required training
to readNot easily machine parsableValidated by humans
Base classes /dictionaries
Base classes /dictionaries
Application definitionsApplication definitions
DocumentationDocumentationValidation formValidation form
schemaschema
<definition name="NXfermi_chopper" type=“group” extends="NXchopper"> <field name="type" type="NX_CHAR">
<doc>fchopper type</doc> </field> <field name="radius" type="NX_LENGTH">
<doc>radius of chopper</doc> <attribute name=“colour” /></field> <group type="NXgeometry">
<doc>geometry of the fermi chopper</doc> </group> …
New Definition Format
NXDL post XSLT transform
NeXus Schema (XSD) File
Advantages of XML schemaHave built-in data types and allow new types
Support extension and restriction of typesMany tools exist for using schema to validate
XML filesTools exist for generating language bindings
and data reading libraries from schemaPossible to add constraints on values via
enumerations, patterns and rangesSupports keys and uniqueness constraints
How are definitions built? Each NX* base class is defined in a separate file
This separation will make life easier for future base class editors
Instrument definitions restrict NXentryType by marking required items as mandatory
Inheritance used to ensure consistency and avoid duplication: “object”, “component”, “chopper base”
Enumerations created for chopper types, units, shapes, etc.
Special “Types with units” defined, such as floatAngleDataType and floatLengthDataType
XML fileXML file
NXD Definition
file
NXD Definition
file
NeXus fileNeXus file
XML schema
XML schema
Validation (xmllint)
Validation (xmllint)
NXVALIDATE
Questions?Questions?
NeXus homepagehttp://www.nexusformat.org
Feature requestshttp://trac.nexusformat.org