Development and deployment ofa cross-platform open-source software package:the Infobiotics Workbench
Jonathan Blakes26/10/2010Supervised by Professor Natalio Krasnogor
Outline
Introduction to InfobioticsOrganisation of software
Supporting technologies
Tour of the Infobiotics Dashboard UIInterface considerations
Deployment on Windows, Mac and Linux
Finishing touches and availability
Conclusions
Infobiotics Workbench
Infobiotics Models (syntax)
Infobiotics Models (semantics)
Infobiotics Experiments
Command line tools with a common parameter file format:simulation (mcss)
model checking (pmodelchecker)
model structure and parameter optimisation (poptimizer)
Desktop applications (not cloud-based)
Written in C++ for speed
Infobiotics Dashboard
User friendly graphical user interface for perform Infobiotics experimentssetting, loading and saving of parameters
running experiments and reporting progress
plotting output
Present options in a clear and understandable manner, setting limits on parameter values
Give feedback when things go wrong!
Written in Python
Why Python?
Dynamic language: rapid prototyping
Large stdlib: tempfile, XML parsers, itertools
3rd party Libraries:NumPy n-dimensional arrays and ufuncs
pytables HDF5 (simulator output)
PyQt Qt GUI bindings (GPL), QScintilla text editor
Matplotlib 2D plotting
Mayavi 3D VTK-based visualisation (TVTK)
Traits...
Traits = Reactive Programming
Simulation Parameters (file)
Simulation Parameters (GUI)
Model checker parameters (1)
MC2
Model checker parameters (2)
Simulator Results
Simulation output amounts of each species in each compartment at each timepoint for each run - quickly become very large (gigabytes), but only some data is of use to the modeller.selecting important information
scaling amounts (molecules) to biologist-friendly units (e.g. nanomolar)
calculating statistics across runs (and other dimensions)
plotting timeseries, exporting data, etc.
SimulatorResultsDialog
Selecting data (filtering)
Filter compartments by name using wildcards
Using set class difference method
Custom ListWidget is a factory for filter LineEdit
Selecting data (sorting)
By providing custom QListWidget with regular expression and function map
Sort species by name
Sort compartments by name and/or positions
Selecting data (sampling)
Reducing data size by sampling reduces memory requirements, speeds up calculations and unclutters plots100 of 10,000 runs
Every 100th recorded timepoint
Transforming data (scaling)
Converts time, substance quantities and volumes from simulator data units into user-friendly display units, using python-quantities
For calculating concentrations, use compartment volumes (at each timepoint) if recorded, otherwise use one volume for all compartments at all timepoints
Exporting data
Selected data can be exported in several common formats for use with other tools:text (.csv)
Excel (.xls)
NumPy (.npz)
Plotting timeseries (old)
Plotting timeseries (improved)
Exporting figures
Added ability to save figures resized to specific screen dimensions (pixels), as surprisingly, matplotlib only supports physical dimensions (inches)
Model checker results
Visualising simulations (Mayavi)
Cross-platform issues
A lot of communication in Workbench uses temporary files these behave differently on Windows to Linuxmust close temp files before use, which can delete them!
Pseudo-terminal emulation library pexpect used to gather progress information not for Windowswinpexpect port not functioning at time of release
fall back to 'working' progress indicatornot good for long experiments
Deploying Python apps
A lot of dependencies inc. Python
Python comes with most Linux distributions, but not with Windows (unlike JVM) and Mac system Python is crippled
Therefore we needed to bundle Python interpreter and all libraries for Windows/Mac
Possible solution: 'freezing' Python applications, creates a single executable (~100 megabytes!)
Need a solid base to start from and add missing libraries
also python(x,y)for Windows
Freezing Python applications
Freezing works well for small scripts, not so well for applications with many librariesuses static code analysis for determine libraries for bundling TraitsUI/Matplotlib use dynamic imports
Freezers: cx_freeze, bbfreeze, PyInstaller,
Wrap python setup.py py2exe/py2app with bash scriptscopy missed files, unzip zipped Python classes
patch Matplotlib
fix Mac library paths using install_name_tool
Windows and Mac non-Python
mcss, pmodelchecker and poptimizer need cross-compiling for 32/64-bit Linux (chroot), Windows (32-bit) and Mac (OSX Snow Leopard only)
pmodelchecker requires PRISM and MC2 which are both Java applications
InstallJammer builds cross-platform installer:
Linux (Debian)
Not trivial either no authorative packaging guideMake packages for mcss, ..., PRISM, MC2, libhdf5-1.8
Used python-support to install Dashboard for system Python(s)
(1) add apt repository to sources.list
(2) sudo apt-get install infobiotics-workbench
Automatic updates!
Fedora/Suse packages made using Alien
Documentation (User Guide)
Documentation (User Guide)
Produced using Sphinx:input is reStructuredText
output is high-quality formatted HTML, PDF,
Used Enthought reST editor for writing
Documentation (API)
HTML API generated from docstrings using Endo
Alternatives:Epydoc
Doxygen
Code availability
Soliciting user feedback
getsatisfaction.com Javascript added to Sphinx template
Conclusions
Cross-platform application development in Python is easy, deployment is hard
Need to become an expert in knowing the differences between platforms
50 / 50 development vs. the rest
Good to be part of a team :)
Publications
Acknowledgements
Jamie Twycross (mcss, packaging)
Fran Romero Campero (IML, pmodelchecker, docs)
Claudio Lima (poptimizer, docs)
Pawel (for turning me on to Python)
Natalio Krasnogor
Questions?
Slides available on web
Infobiotics
Muokkaa otsikon tekstimuotoa napsauttamalla
Muokkaa jsennyksen tekstimuotoa napsauttamallaToinen jsennystasoKolmas jsennystasoNeljs jsennystasoViides jsennystasoKuudes jsennystasoSeitsems jsennystasoKahdeksas jsennystasoYhdekss jsennystaso