Meeting Users Needs with OctaveA survey of the use of Matlab™ and Octave at the NIH

Tom HolroydTom Holroyd

NIMH MEG Core FacilityNIMH MEG Core Facility

Presented at thePresented at the

Octave 2006 WorkshopOctave 2006 WorkshopApril 21, 2006April 21, 2006

OutlineSurvey ResultsSurvey Results

Number of usersNumber of users

LicensesLicenses

UsesUses

Other toolsOther tools

Issues for usersIssues for users

Octave use at the NIHOctave use at the NIHMEG data analysisMEG data analysis

Biowulf cluster (2000+ CPUs)Biowulf cluster (2000+ CPUs)

Matlab Users at NIH

25 responses to the survey25 responses to the survey

9 out of 27 institutes9 out of 27 institutesNCI, NHGRI, NLMNCI, NHGRI, NLM

NHLBI, NIMH, NIAMSNHLBI, NIMH, NIAMS

NIAID, NIDDK, NIDANIAID, NIDDK, NIDA

Matlab Users at NIH

35 licenses of various sorts35 licenses of various sorts

Toolboxes:Toolboxes:Statistics 11Statistics 11

Image processing 9Image processing 9

Bioinformatics 6Bioinformatics 6

Signal processing 6Signal processing 6

Neural network 4Neural network 4

Optimization 4Optimization 4

Database 3Database 3

Curve fitting 3Curve fitting 3

Spline 1Spline 1

Symbolic math 1Symbolic math 1

Fuzzy logic 1Fuzzy logic 1

Compiler 1Compiler 1

Excel link 1Excel link 1

Wavelet 1Wavelet 1

Many UsesMicroarray analysis (6 users)Microarray analysis (6 users)

DesignDesign

VisualizationVisualization

Statistical analysisStatistical analysis

DatabaseDatabase

Web siteWeb site

Image processingImage processing

Content-based image retrievalContent-based image retrieval

SegmentationSegmentation

CompressionCompression

Optical and electron microscope Optical and electron microscope image analysis image analysis

MRI data analysisMRI data analysis

Functional mappingFunctional mapping

Information-based mappingInformation-based mapping

Voxel-based morphometryVoxel-based morphometry

MRI modelling, image reconstructionMRI modelling, image reconstruction

Diffusion tensor imagingDiffusion tensor imaging

StatisticsStatistics

General purpose statisticsGeneral purpose statistics

t-tests, ANOVA, regression, etc.t-tests, ANOVA, regression, etc.

Neural networksNeural networks

Pattern recognition in mass spectraPattern recognition in mass spectra

Feature extractionFeature extraction

Many, Many Uses

GeneticsGenetics

Novel motif discovery in Novel motif discovery in regulatory regions of genomes regulatory regions of genomes

Signal processingSignal processing

Mass spectrometryMass spectrometry

InstrumentationInstrumentation

Real time control and data Real time control and data acquisition acquisition

NeurophysiologyNeurophysiology

MatOFF: M-series OFFline MatOFF: M-series OFFline Analysis for NeurophysiologyAnalysis for Neurophysiology

MEG data analysisMEG data analysis

Time-frequency plotsTime-frequency plots

Multitaper spectral analysisMultitaper spectral analysis

Magnetic field simulationsMagnetic field simulations

StatisticsStatistics

Realtime BCIRealtime BCI

Fieldtrip: ported to OctaveFieldtrip: ported to Octave

Curve fittingCurve fitting

Fluorescence recovery after Fluorescence recovery after photobleachingphotobleaching

Simulations of differential equationsSimulations of differential equations

Other Tools

RR

ScilabScilab

matplotlib/pylabmatplotlib/pylab

The competition? Data The competition? Data import/export issues exist.import/export issues exist.

RR

JavaJava

C/C++C/C++

Extension packages written Extension packages written in other languages.in other languages.

Issues for UsersLicensingLicensing

Multi-processor and clusterMulti-processor and cluster

PlatformPlatformOS X compatibilityOS X compatibility

64-bit systems and large memory (> 2GB)64-bit systems and large memory (> 2GB)

OctaveOctaveLack of GUI support (not an IDE)Lack of GUI support (not an IDE)

Compatibility/portability issues with other analysis packagesCompatibility/portability issues with other analysis packages

Speed (we need to convert more .m files to C++)Speed (we need to convert more .m files to C++)

Support, e.g., bioinformatics expertsSupport, e.g., bioinformatics experts

The License Hog

This ran for a week.This ran for a week.

What the note doesn't say is that he What the note doesn't say is that he was also using a license; nobody else was also using a license; nobody else could use Matlab for a week.could use Matlab for a week.

I had a user who ran a long job which I had a user who ran a long job which failed right at the end when it needed failed right at the end when it needed to do a t-test and the stats toolbox to do a t-test and the stats toolbox was busy.was busy.

Multiple Processor / Cluster

As an example, the Biowulf has 11 licenses (22 CPUs) As an example, the Biowulf has 11 licenses (22 CPUs) and various toolboxes costing $96,000. MathWorks can and various toolboxes costing $96,000. MathWorks can sell us a “cluster” license on 128 CPUs for $39,000. sell us a “cluster” license on 128 CPUs for $39,000. Cheaper, but still about 1900 CPUs short. It works out to Cheaper, but still about 1900 CPUs short. It works out to about $273 / CPU or $500,000 for the whole cluster (they about $273 / CPU or $500,000 for the whole cluster (they have never sold a 2000 CPU license).have never sold a 2000 CPU license).

Octave use at the NIH

Magnetoencephalography (MEG) is the recording and Magnetoencephalography (MEG) is the recording and analysis of the magnetic field of the brain. NIH has a 275 analysis of the magnetic field of the brain. NIH has a 275 channel scanner, which produces very large multivariate channel scanner, which produces very large multivariate datasets.datasets.Analysis methods include time-frequency plots, magnetic Analysis methods include time-frequency plots, magnetic field calculations, and source localization via power field calculations, and source localization via power spectrum estimation. All can be done with Octave.spectrum estimation. All can be done with Octave.

Contour Plots

The time-frequency graph The time-frequency graph shown here was calculated shown here was calculated using Octave with a C++ using Octave with a C++ extension, then the dataextension, then the datawere exported to pylab for were exported to pylab for rendering in a manner similar rendering in a manner similar to that used for Gnuplot.to that used for Gnuplot.

Stockwell Transform

MEG Source Localization

Power spectral estimates Power spectral estimates are used to generate maps are used to generate maps of brain activity. We get a of brain activity. We get a timecourse for each voxel.timecourse for each voxel.

Octave is used as a Octave is used as a compute engine here; we compute engine here; we run hundreds of such run hundreds of such datasets on the cluster. datasets on the cluster. Voxels are all independent!Voxels are all independent!

Biowulf Cluster

Recent patches to Octave 2.9.5+ (not in CVS yet!) make it Recent patches to Octave 2.9.5+ (not in CVS yet!) make it possible to run in a very large cluster environment. The issue was possible to run in a very large cluster environment. The issue was that Octave's cache of .m files was broken, and it was hitting the that Octave's cache of .m files was broken, and it was hitting the NFS server every function call. Now it caches both successes and NFS server every function call. Now it caches both successes and failures. The new issue is that it won't see new files, although it will failures. The new issue is that it won't see new files, although it will pick up files whose modification time has changed.pick up files whose modification time has changed.

New issues: how to get 200 running Octaves to communicate New issues: how to get 200 running Octaves to communicate usefully with each other, for highly parallel computations.usefully with each other, for highly parallel computations.

Octave 2006 Workshop

Congratulations to all on a successful workshop!

See you all at Octave 2007