NIDASNCAR In-Situ Data Acquisition Software

Gordon MacleanSoftware EngineerNCAR/EOL

Background

Developed at the NCAR Earth Observing Laboratory (EOL), starting in 2004.

Targeted as the primary data system for the NSF HIAPER, NCAR's Gulfstream V research aircraft.

Developers included software engineers with experience at EOL's Research Aviation Facility (RAF) and In-Situ Sensing Facility (ISF)

Design was expanded to include the acquisition of ground-based meteorological measurements.

Built on the past experience of EOL engineers:

Aircraft Data System: ( ADS, ADS-2), 1970s – 2009

Surface systems: Portable Automated Mesonet (PAM I,II,III), 1973-2004 and the Atmosphere-Surface Turbulence Exchange Facility (ASTER), 1988-2002

Targeted ResearchDiverse topics, primarily NSF funded

Aircraft

Boundary layer to 50K ft

Cloud physics, convection, severe weather, chemistry and pollution, remote sensing, sensor validation

20 to 200 flight hours per research project

Surface

PBL surface boundary conditions, energy balance, turbulence

Fluxes of momentum, heat, moisture and trace gases

Horizontal and vertical (0-50 meter) gradients

24x7 operations, often unattended

Short term, typically 1 to 3 months per project

Navigation GPS, inertial reference systems, radar altimeter

State parameters Pressure, temperature, humidity, dew point (various)

Winds Pitot tubesLaser Air Motion Sensor: LAMS (under development)

Laser absorption hygrometers Vertical Cavity Surface Emitting Laser Hygrometer: VCSELNCAR TDL Water Vapor

Cloud particles, aerosols 2D particle probes: PMS 2D-C, 2D-P, Fast2DCCloud Droplet probeForward Scattering Spectrometer Probes: FSSP 100,200,300Ultra-high Sensitivity Aerosol Spectrometer: UHSASCondensation Nuclei countersPMS King Hot Wire Liquid Water ProbeCloud Particle Imager: 3V-CPI

Ice detectors Rosemount Icing DetectorSmall Ice Detector: SID-II

Inlets Counter-Flow Virtual Impactor: CVIHIML

Chemistry Chemiluminescense: NCAR CO, O3

Gas Chromatographs: Panther, UCATS: H2O, O

3, N

2O,CH

4

Soot photometer: DMT SP2Picarro: CO

2,Methane,O18,H

2O,Deuterium

Radiation Pyranometers, pyrgeometers, UV, IR surface temp

Temperature profiles Microwave Temperature Profiler: MTP*

Engineering Accelerometers, CMIGITSAir Pressure Scanner: PSI-9116

Airborne Instrumentation

Aircraft Deployments

Surface InstrumentationState parameters Barometer, T, RH: (various)Time, location GPS with pulse-per-second2d anemometers RMYoung Propvane, 85004, Gill WindSonic, WindObserver,

MetOne, Vaisala WMT3d anemometers Campbell CSAT3

Applied Technologies: ATIKWater Vapor, CO2Other trace gases

Campbell Krypton HygrometerLI-COR 7000, 7500, Campbell EC150

Radiation Pyranometers, Pyrgeometers: K&Z, Eppley, Hukseflux NR01Net: Rebs Q7, Hukseflux NR02, KZ NRLite2Solar and Diffuse: SPN1, LI-COR, ApogeeSurface Temp: EverestPhotosynthetic Active Radiation: LI-COR 190SA

Soil Heat flux, temp, moisture, thermal prop: REBS, Decagon, Hukesflux

Mesh sensor networks Digi Zigbee, EOL WisardWeather stations, loggers Vaisala WXT, CR10

Precipitation ETI, MRI rain gauges, Decagon Leaf WetnessFast pressure Paroscientific 216BHot film anemometers Dantek 1D/3DEngineering Power monitor, battery and charging system status

Surface Deployments

Required Data System Interfaces

Type ImplementationRS 232/422/485 serial Diamond Systems 8 port (with digio)

WinSystems PCM-COM8

Analog to Digital, 16 bit NCAR 8 channel, differential, sigma-deltaDiamond MM-16, MM-32: 16 channel diff, multiplexed, SAR

USB 1,2 Processor motherboardEthernet Processor motherboardPulse counters, frequency Mesa Anything I/O FPGA

Diamond GPIO-MMDigital I/O, analog out, relays Arcom Viper CPU

Diamond MM, IR104ARINC 429 navigation Condor Engineering CEI-420AIRIG-B timing protocol Brandywine PC104-SGBluetooth RFCOMM serial USB/Bluetooth adaptersMesh Networks, sensor arrays Serial interface to EOL Wisard base

motes

Initial Design Choices Operating System: Linux

Open source, near real-time, wide choice of system vendor Same OS on embedded systems and processing and display servers Extensive network services Multithreaded, dynamic (run-time) object loading

Data Sampling Modules: ARM Processors and PC104 expansion bus Low power (~2 Watt) low heat generation Extensive vendor support for PC104 interface cards

Languages: C++: object oriented, compiled C: low level interface driver modules

Configuration: XML: flexible, tree structured, widely used, field editable (by NIDAS geek)

Software Tools: Version control: subversion Builds: scons Packaging: RPM, Debian (embedded)

Server Systems

Data Sampling Modules, DSMs

Networks

Low level driver modulesncar_a2d serial network

Processors

StatisticsNetCDF

networkfiles

Processing Pipeline

Sorter

SensorObjects

Sorter

NIDAS

Calibrationdata

Time-tagged, raw Time-tagged, converted

SensorHandlerSensor Objects

InstrumentationData System Hardware Interfaces PC104 A2D card Serial Port Ethernet

NIDAS Sample Flow

Sensor Object Hierarchy

Highlights Supports diverse instrumentation

Wide range of sampling rates: ~0 to ~1000 samples/sec

Highly configurable, distributed processing

Runnable on low power (~2 Watt) embedded processors

Networked, supporting remote login, serial sensor interaction, power control

Open source, including all low level driver code

Acquired data is archived in raw sensor format

Archived time tags have microsecond precision

System clocks have ~20 microsecond accuracy

Many deployments

Revision controlled, automated tests

GUI configuration editor for aircraft deployments

Future Plans, More Info• Kernel upgrades on DSMs, using newer real-time capabilities of Linux.

• Reduce serial sampling latency (line discipline kernel module)

• Improve network service discovery mechanism

• Enhancements to real-time displays, diagnostics

• Framework for instrumentation control

• Calibration database

• Other server platforms: MacOSX

• Other embedded hardware platforms: Eurotech Titan, gumstix, new ARM/Intel processors, pc104+

• Support and collaboration with outside users.

http://www.eol.ucar.edu/data/software/nidas (under development)

maclean@ucar..edu

Questions/Answers