niprov Documentation - Read the Docs · niprov Documentation, Release 0.5 Ever encountered a ﬁle of which you weren’t sure what analysis steps it had gone through? Ever wanted

niprov DocumentationRelease 0.5

Jasper J.F. van den Bosch

December 20, 2016

Contents

1 Commandline Usage 3

2 Code Examples 5

3 Configuration 7

4 Provenance Fields 94.1 path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104.2 hostname . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104.3 user . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104.4 location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104.5 added . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104.6 version-added . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104.7 size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104.8 hash . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114.9 created . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114.10 transient . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114.11 modality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114.12 acquired . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114.13 duration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114.14 subject . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114.15 dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114.16 project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114.17 protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124.18 subject-position . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124.19 water-fat-shift . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124.20 transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124.21 parent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124.22 code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124.23 logtext . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124.24 script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124.25 args . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124.26 kwargs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124.27 sampling-frequency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134.28 fif-type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134.29 lowpass . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134.30 highpass . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134.31 bad-channels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

i

4.32 projection-description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134.33 seriesuid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134.34 filesInSeries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134.35 technique . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134.36 repetition-time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134.37 field-of-view . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144.38 epi-factor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144.39 magnetization-transfer-contrast . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144.40 diffusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144.41 slice-thickness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144.42 slice-orientation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144.43 echo-time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144.44 flip-angle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144.45 inversion-time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

5 MongoDB setup 15

6 niprov package 176.1 Submodules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

6.1.1 niprov.adding module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176.1.2 niprov.approval module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176.1.3 niprov.basefile module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186.1.4 niprov.camera module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186.1.5 niprov.capturing module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196.1.6 niprov.clock module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196.1.7 niprov.cnt module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196.1.8 niprov.commandline module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196.1.9 niprov.comparing module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206.1.10 niprov.config module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206.1.11 niprov.context module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216.1.12 niprov.dcm module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226.1.13 niprov.dependencies module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226.1.14 niprov.diff module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236.1.15 niprov.discovery module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256.1.16 niprov.exceptions module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256.1.17 niprov.exporting module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256.1.18 niprov.externals module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266.1.19 niprov.fif module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266.1.20 niprov.filefilter module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266.1.21 niprov.files module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266.1.22 niprov.filesystem module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266.1.23 niprov.format module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276.1.24 niprov.formatdict module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276.1.25 niprov.formatfactory module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286.1.26 niprov.formatjson module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286.1.27 niprov.formatnarrated module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296.1.28 niprov.formatobject module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296.1.29 niprov.formatsimple module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296.1.30 niprov.formatxml module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306.1.31 niprov.hashing module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306.1.32 niprov.importing module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306.1.33 niprov.inheriting module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306.1.34 niprov.inspection module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306.1.35 niprov.jsonfile module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

ii

6.1.36 niprov.libraries module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316.1.37 niprov.location module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316.1.38 niprov.locationfactory module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326.1.39 niprov.mediumdirect module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326.1.40 niprov.mediumfactory module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326.1.41 niprov.mediumfile module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326.1.42 niprov.mediumstdout module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326.1.43 niprov.mediumviewer module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336.1.44 niprov.mnefun module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336.1.45 niprov.mongo module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336.1.46 niprov.nifti module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346.1.47 niprov.parrec module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346.1.48 niprov.pictures module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346.1.49 niprov.pipeline module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356.1.50 niprov.pipelinefactory module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356.1.51 niprov.plogging module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356.1.52 niprov.querying module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366.1.53 niprov.recording module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366.1.54 niprov.renaming module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376.1.55 niprov.reporting module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376.1.56 niprov.repository module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376.1.57 niprov.searching module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 396.1.58 niprov.users module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 396.1.59 niprov.views module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 396.1.60 niprov.webapp module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

6.2 Module contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

7 Indices and tables 41

Python Module Index 43

iii

iv

niprov Documentation, Release 0.5

Ever encountered a file of which you weren’t sure what analysis steps it had gone through? Ever wanted to know whattypes of data you have available for a subject in one overview? Automatically document an analysis pipeline?

Provenance is meta-data that tracks the ‘history’ of a file, and niprov is a python program to create, store and publishprovenance for brain imaging files.

To get started, install niprov using pip:

pip install niprov

Keep in mind that to open image files you may need other libraries, such as mne-python, nibabel or pydicom.

The next step is to look through your directories for image files:

provenance discover /my/data/dir

After which you can start looking at what you’ve collected:

provenance report

You can do the same in python code:

import niprovniprov.discover('.')print(niprov.report()) #this will return a list of dictionaries

Read on to see other features and options.

Get in touch if you have questions or suggestions, by submitting an issue on http://github.com/ilogue/niprov or viaemail to [email protected], or at tweet to @niprovenance

Contents:

Contents 1

http://github.com/ilogue/niprov

mailto:[email protected]

https://twitter.com/niprovenance


2 Contents

CHAPTER 1

Commandline Usage

Look for image files below the current directory, inspect them and store the obtained provenance metadata.

provenance discover .

Run a transformation command and log it as provenance for the new file:

provenance record mcflirt -in t1flip_all_orig -out t1all_reg -refvol 0

Alternatively, log the provenance after running the command:

provenance log 'motion correction' --new fmri-3dmc.nii --parent fmri.nii

Publish provenance of known files for subject ‘John Doe’ as an html file.

provenance report --subject "John Doe" --html

Register a file to the provenance database without inspecting it:

provenance add 'motionvars.mat'

Rename dicom files that don’t have an extension (or niprov ignores them):

provenance rename /raw_data/dicoms/

List files marked for approval:

provenance todo

Approve a file:

provenance approve /path/to/myfile.img

3


4 Chapter 1. Commandline Usage

CHAPTER 2

Code Examples

Discover and retrieve provenance:

import niprovniprov.discover('.')files = niprov.report(forSubject='John Doe')

5


6 Chapter 2. Code Examples

CHAPTER 3

Configuration

Niprov can be configured through a text file with configuration directives. This file is located at ~/niprov.cfg. Alterna-tively, the settings can be changed in code:

from niprov.config import Configurationconf = Configurationconf.verbose = Trueniprov.log(x,y,z, opts=conf)

See niprov.config.Configuration for details on the individual settings.

7


8 Chapter 3. Configuration

CHAPTER 4

Provenance Fields

Overview of provenance attributes collected:

Field superficial transformed PAR-REC DICOM FIF CNTpath yes yes yes yes yes yeshostname yes yes yes yes yes yesuser yes yes yes yes yes yeslocation yes yes yes yes yes yesadded yes yes yes yes yes yesversion-added yes yes yes yes yes yessize yes yes yes yes yes yeshash yes yes yes yes yes yescreated yes yes yes yes yes yestransient yes yes yes yes yes yesmodality yes inherited yes yes yes yesacquired inherited yes yes yes yesduration inherited yes yes yes yessubject inherited yes yes yes yesdimensions yes maybe yes yesproject inherited yesprotocol inherited yes yessubject-position inherited yes yeswater-fat-shift inherited yes yestransformation yesparent yescode yeslogtext yesscript yesargs yeskwargs yessampling-frequency inherited yes yesfif-type yeslowpass yeshighpass yesbad-channels yesprojection-description yesseriesuid yesfilesInSeries yes

Continued on next page

9


Table 4.1 – continued from previous pageField superficial transformed PAR-REC DICOM FIF CNTtechnique inherited yesrepetition-time inherited yesfield-of-view yesepi-factor inherited yesmagnetization-transfer-contrast inherited yesdiffusion inherited yesslice-thickness yesslice-orientation yesecho-time inherited yesflip-angle inherited yesinversion-time inherited yes

4.1 path

The last known path to the file.

4.2 hostname

Hostname of the computer on which the file resides.

4.3 user

Name of the user that created the file and/or registered its provenance.

4.4 location

A string that combines computer and filesystem path.

4.5 added

Date and time that the provenance for this file was registered.

4.6 version-added

A floating point number reflecting the niprov version used to create the provenance record.

4.7 size

File size.

10 Chapter 4. Provenance Fields


4.8 hash

An MD5 hash of the file’s binary contents.

4.9 created

Last known modified date of the file as reported by the OS.

4.10 transient

Whether the file is deemed temporary.

4.11 modality

Type of data; MRI, DWI, MEG, EEG or other.

4.12 acquired

When the data was collected.

4.13 duration

Duration of the acquisition in seconds.

4.14 subject

The participant whose brain was imaged.

4.15 dimensions

Dimensions of the image. Order of dimensions dependent on format; in principle follows (where applicable); in-slice,number of slices, time. E.g. x,z,y,t.

4.16 project

The name of the research project.

4.8. hash 11


4.17 protocol

The name of the pulse sequence used.

4.18 subject-position

The position and orientation of the subject during during the scan. E.g. head first supine.

4.19 water-fat-shift

Water fat shift value.

4.20 transformation

The name of the transformation applied to generate this derivative image.

4.21 parent

The file that this file is a transformed version of.

4.22 code

The command used to generate this image.

4.23 logtext

The commandline output of the transformation.

4.24 script

The path to the code file containing the transformation routine.

4.25 args

The positional arguments passed to a python-based transformation command.

4.26 kwargs

The keyword arguments passed to a python-based transformation command.



4.27 sampling-frequency

How many samples were acquired per second.

4.28 fif-type

Type of MNE fiff file. One of ‘raw’, ‘ave’, ‘epo’, ‘cov’, ‘fwd’ or ‘trans’.

4.29 lowpass

Cutoff frequency of the lowpass filter applied, in Hz.

4.30 highpass

Cutoff frequency of the highpass filter applied, in Hz.

4.31 bad-channels

Names of channels/sensors marked as bad.

4.32 projection-description

Description of projection vectors in the data file.

4.33 seriesuid

A unique identifier for files in a series. Corresponds to SeriesInstanceUID in the DICOM format.

4.34 filesInSeries

A list of paths to files that are part of this series.

4.35 technique

The imaging technique that the protocol uses.

4.36 repetition-time

Time to scan one volume

4.27. sampling-frequency 13


4.37 field-of-view

The extent of the observable world that is seen by the image. This is a list of three values.

4.38 epi-factor

Echo-planar-imaging factor.

4.39 magnetization-transfer-contrast

A technique that uses the transfer of nuclear spin polarization and/or spin coherence from one population of nuclei toanother population of nuclei.

4.40 diffusion

Whether this is a diffusion image. Boolean.

4.41 slice-thickness

Thickness in mm of the slices. In case the file contains multiple images, this applies to the first image.

4.42 slice-orientation

Spatial orientation of the slices. In case the file contains multiple images, this applies to the first image.

4.43 echo-time

The time in milliseconds between the application of the 90° pulse and the peak of the echo signal in Spin Echo andInversion Recovery pulse sequences.

4.44 flip-angle

The angle to which the net magnetization is rotated or tipped relative to the main magnetic field direction via theapplication of an RF excitation pulse.

4.45 inversion-time

The time period between the 180° inversion pulse and the 90° excitation pulse in an Inversion Recovery pulse sequence.


CHAPTER 5

MongoDB setup

This quickstart guide explains how to setup a MongoDB backend for niprov on Ubuntu.

...incomplete..

Requires 2.6

Follow installation instructions at

https://docs.mongodb.com/manual/tutorial/install-mongodb-on-ubuntu/

Configure mongodb:

sudo nano /etc/mongod.conf

# network interfacesnet:

port: 27017bindIp: 0.0.0.0

sudo service mongod restart

Allow access to mongodb from IP addresses in the range 123.45.67.0/24:

sudo ufw allow from 123.45.67.0/24 to any port 27017

15

https://docs.mongodb.com/manual/tutorial/install-mongodb-on-ubuntu/


16 Chapter 5. MongoDB setup

CHAPTER 6

niprov package

6.1 Submodules

6.1.1 niprov.adding module

niprov.adding.add(filepath, transient=False, provenance=None, dependen-cies=<niprov.dependencies.Dependencies object>)

Simply register the file.

Inspects the file and makes it known to the provenance data, such that image files can be logged that have beencreated using this file. Useful also for temporary files.

Example

(provenance, status) = niprov.add(‘/path/to/my.nii’)

Parameters

• filepath (str) – Path to the newly created file.

• transient (bool, optional) – Set this to True to indicate that the file is only tem-porary and future checks should not expect it to be physically present. Defaults to False,assuming that the file remains.

• provenance (dict, optional) – Add the key-value pairs in this dictionary to theprovenance record for the new file.

Returns Tuple of new provenance and status. Status is a string with one of the following values:‘new’: File was not known yet and has been added. ‘series’: The file was deemed part of aseries and has been added. ‘failed’: There was an error inspecting the file. ‘known’: The filewas already known to niprov, nothing happened. ‘dryrun’: Function called with config.dryrun,database not touched.

Return type tuple

6.1.2 niprov.approval module

niprov.approval.approve(filepath, dependencies=<niprov.dependencies.Dependencies object>)Mark this file as approved.

Parameters filepath (str) – Path to the tracked file that has been found valid.

17


niprov.approval.markForApproval(files, reset=False, dependen-cies=<niprov.dependencies.Dependencies object>)

Mark a list of files for approval by a human.

Parameters

• files (list) – List of paths of files tracked by niprov to mark for approval.

• reset (bool) – Also mark files that have already been approved. False by default.

niprov.approval.markedForApproval(dependencies=<niprov.dependencies.Dependencies ob-ject>)

List files marked for approval by a human.

niprov.approval.selectApproved(files, dependencies=<niprov.dependencies.Dependencies ob-ject>)

Return only files that have approval status ‘granted’.

Parameters files (list) – List of paths of files to check for approval status.

6.1.3 niprov.basefile module

class niprov.basefile.BaseFile(location, provenance=None, dependen-cies=<niprov.dependencies.Dependencies object>)

Bases: object

attach(form=’json’)Not implemented for BaseFile parent class.

Parameters form (str) – Data format in which to serialize provenance. Defaults to ‘json’.

compare(other)

getProtocolFields()

getProvenance(form=’dict’)

getSeriesId()

getSnapshotFilepath()

inspect()

keepVersionsFromPrevious(previous)

parents

versions

viewSnapshot()

6.1.4 niprov.camera module

class niprov.camera.Camera(dependencies)Bases: object

saveSnapshot(data, for_)Plot an overview of the image and store it.

Uses PictureCache as service that provides a file-like handle to save the plotted picture to.

Calls takeSnapshot() to do the actual plotting.

Parameters data (numpy.ndarray) – Array of 2, 3 or 4 dimensions with image data.

18 Chapter 6. niprov package


takeSnapshot(data, on)Plot an overview of the image using matplotlib.pyplot.

Parameters

• data (numpy.ndarray) – Array of 2, 3 or 4 dimensions with image data.

• on (str or file-like object) – Where to save figure to.

6.1.5 niprov.capturing module

class niprov.capturing.OutputCaptureBases: object

Context manager which records stdout writes.

Used when recording python-based transformations.

6.1.6 niprov.clock module

class niprov.clock.ClockBases: object

getNowString()

6.1.7 niprov.cnt module

class niprov.cnt.NeuroscanFile(location, **kwargs)Bases: niprov.basefile.BaseFile

Support for the Neuroscan .cnt format.

inspect()

6.1.8 niprov.commandline module

class niprov.commandline.Commandline(dependencies=<niprov.dependencies.Dependencies ob-ject>)

Bases: object

addUnknownParent(fpath)

discoveryFinished(nnew, nadded, nfailed, ntotal)

exportedToFile(fname)

fileAdded(image)

fileError(fpath)

filesMarkedForApproval(images)

interpretedRecording(new, transform, parents)

log(level, message, exceptionClass=None)

missingDependencyForImage(lib, fpath)

mnefunEventReceived(operationName)

6.1. Submodules 19


receivedBashCommand(command)

renamedDicom(fpath)

usingCopyAsParent(copy)

vlevels = [’debug’, ‘info’, ‘warning’, ‘error’]

niprov.commandline.ordinal(num)

6.1.9 niprov.comparing module

niprov.comparing.compare(file1, file2, dependencies=None)Compare the provenance of two files.

This creates a niprov Diff object that can be further interrogated or displayed for differences between the twofiles.

Example

Chain calls with methods of Diff or print the Diff object:

compare(file1, file2).assertEqual(ignore='path')print(compare(file1, file2))

Parameters

• file1 (BaseFile) – One of two niprov BaseFile objects to compare.

• file2 (BaseFile) – As file1

Returns A niprov Diff object which reflects differences between the two files.

Return type niprov.diff.Diff

6.1.10 niprov.config module

class niprov.config.Configuration(configFilePath=’~/niprov.cfg’)Bases: object

General settings for niprov.

Individual settings are documented as follows;

setting = default_value type - Explanation.

The settings can be changed in the configuration file, or in code.

All settings:

attach = Falsebool: Attach provenance to image files. For nifti files for instance, this means inserting a header extensionwith serialized provenance. See ‘attach_format’ to configure which data format is used.

attach_format = ‘json’string: Format in which to attach provenance to the file. One of ‘json’, or ‘xml’. For example, if set to‘json’ and the ‘attach’ option is True, this will add a header extension to nifti files created with the relevantprovenance data in json format.



database_type = ‘file’str: Type of backend in which to store provenance. Currently only ‘file’ or ‘MongoDB’

database_url = ‘~/provenance.json’str: URL of the database. If database-type is file, this is the path to the file.

discover_file_extensions = [’.PAR’, ‘.dcm’, ‘.fif’, ‘.cnt’]list: Discover uses this to determine which files to include. Not strictly extensions, can be any string thatappears in the file name. Use comma’s to separate items.

dryrun = Falsebool: Do not execute commands or make lasting changes to the provenance database.

user = ‘’string: Name of the user creating provenance. If not provided, will be determined based on OS informationor as passed as an argument to the provenance operation. See also niprov.users

verbosity = ‘info’string: Level of information to report. One of ‘debug’,’info’,’warning’, ‘error’. Any level includes higherlevels, i.e. ‘info’ will log messages of that are deemed ‘info’, ‘warning’ or ‘error’.

6.1.11 niprov.context module

class niprov.context.ProvenanceContextBases: object

add(filepath, transient=False, provenance=None)See niprov.adding

approve(filepath)See niprov.approval

backup()See niprov.exporting

compare(file1, file2)See niprov.comparing

discover(root)See niprov.discovery

export(images, medium, form, pipeline=False)See niprov.exporting

get()See niprov.querying

importp(fpath)See niprov.importing

inspect(location)See niprov.inspection

log(new, transformation, parents, code=None, logtext=None, transient=False, script=None,user=None, provenance=None, opts=None)See niprov.plogging

markForApproval(files)See niprov.approval

markedForApproval()See niprov.approval

6.1. Submodules 21


print_(images, pipeline=False)See niprov.exporting

record(command, new=None, parents=None, transient=False, args=None, kwargs=None, user=None,opts=None)

See niprov.recording

renameDicoms(dicomdir)See niprov.renaming

search(text)See niprov.searching

selectApproved(files)See niprov.approval

view(images, pipeline=False)See niprov.exporting

6.1.12 niprov.dcm module

class niprov.dcm.DicomFile(location, **kwargs)Bases: niprov.basefile.BaseFile

getSeriesId()Return the DICOM “SeriesInstanceUID” that all files in this series have in common.

Returns A string uniquely identifying files belonging to this series.

Return type str

hasFile(other)

inspect()Inspect the DICOM file attributes.

If a general AcquisitionDateTime attribute is not present, the SeriesDate and SeriesTime will be used toset the acquired provenance field.

Returns Provenance for the inspected file.

Return type dict

mergeWith(img)Add a single DICOM file object to this series.

The file will be stored in provenance in the ‘filesInSeries’ list.

6.1.13 niprov.dependencies module

class niprov.dependencies.Dependencies(config=None)Bases: object

getCamera()

getClock()

getConfiguration()

getExternals()

getFileFactory()



getFileFilter()

getFilesystem()

getFormatFactory()

getHasher()

getLibraries()

getListener()

getLocationFactory()

getMediumFactory()

getPictureCache()

getPipelineFactory()

getQuery()

getRepository()

getSerializer()

getUsers()

reconfigureOrGetConfiguration(newConfiguration)

6.1.14 niprov.diff module

class niprov.diff.Diff(file1, file2)Bases: object

Difference between two files.

This represents differences in provenance between two files.

See niprov.comparing

Parameters

• file1 (BaseFile) – One of two niprov BaseFile objects to compare.

• file2 (BaseFile) – As file1

NCHARSCOL = 20

areEqual(ignore=None, select=None)Whether there are any differences between the files.

Parameters

• ignore (list) – Optional. List of fields not to evaluate when determining differences.

• select (list) – Optional. List of fields that should be specifically evaluated. All otherfields will be ignored.

Returns True if no differences, False otherwise.

Return type bool

areEqualProtocol()Whether there are any differences for protocol fields.

Each BaseFile subtype has a getProtocolFields() method that is used here to selectively see if any ofthese are different.

6.1. Submodules 23


Returns True if no differences, False otherwise.

Return type bool

assertEqual(ignore=None, select=None)Raises exception if there are differences.

Parameters



Raises AssertionError – Message with differences in a table.

assertEqualProtocol()Raises exception if there are differences in protocol fields.

Each BaseFile subtype has a getProtocolFields() method that is used here to selectively see if any ofthese are different.

Raises AssertionError – Message with protocol differences in a table.

defaultIgnore = [’_id’]

getDifferenceString(ignore=None, select=None)Get table of differences as string.

Parameters



Returns A three-columns table listing provenance fields and their respective values for the twofiles.

Return type str

getDifferences(ignore=None, select=None)Get dictionary with fields that differ and how they differ.

Parameters



Returns A dictionary with provenance fields as keys and strings indicating how they differ.

Return type dict

getSame()Get dictionary with fields that have equal values.

Returns A dictionary with provenance fields as keys the string ‘same’ as value.

Return type dict

getSameString()Get table of values that are the same for the compared files.

Returns A three-columns table listing provenance fields and their respective values for the twofiles.



Return type str

6.1.15 niprov.discovery module

niprov.discovery.discover(root, dependencies=<niprov.dependencies.Dependencies object>)Search a directory for image files, and add them to your provenance collection.

Files are only included if they match the filters in the ‘discover_file_extensions’ settings. Refer to niprov.addfor details on what happens to individual files.

Parameters root (str) – The top directory in which to look for new files.

6.1.16 niprov.exceptions module

exception niprov.exceptions.NiprovErrorBases: exceptions.Exception

exception niprov.exceptions.UnknownFileErrorBases: niprov.exceptions.NiprovError

6.1.17 niprov.exporting module

niprov.exporting.backup(dependencies=<niprov.dependencies.Dependencies object>)Shortcut for export(medium=’file’, form=’json’) for all provenance.

niprov.exporting.export(provenance, medium, form, pipeline=False, dependen-cies=<niprov.dependencies.Dependencies object>)

Publish or simply return provenance for selected files.

To get provenance on one specific file, pass its path as the ‘forFile’ argument. Alternatively, to get all filesassociated with a certain subject, use the ‘forSubject’ argument. If none of these is used, provenance for themost recently registered files is reported.

Parameters

• provenance – Niprov BaseFile object or list of such.

• medium (str) – The medium in which to publish the provenance. One of:

‘stdout’ (print the provenance to the terminal), ‘direct’ (return object to caller), ‘file’(write to a text file), ‘viewer’ (open in the system image viewer).

• form (str) – The format in which to serialize the provenance. One of‘json’,’xml’,’narrated’,’simple’,’dict’,’picture’.

Returns Depends on medium selected.

niprov.exporting.print_(provenance, pipeline=False, dependen-cies=<niprov.dependencies.Dependencies object>)

Shortcut for export(medium=’stdout’, form=’simple’).

niprov.exporting.view(provenance, pipeline=False, dependencies=<niprov.dependencies.Dependenciesobject>)

Shortcut for export(medium=’viewer’, form=’picture’).

6.1. Submodules 25


6.1.18 niprov.externals module

class niprov.externals.ExternalsBases: object

Utility that wraps the python subprocess module to start other applications

run(command)Start a subprocess with the command provided.

Parameters command (list) – A list of command elements.

class niprov.externals.Result(succesful, output)Bases: object

6.1.19 niprov.fif module

class niprov.fif.FifFile(location, **kwargs)Bases: niprov.basefile.BaseFile

attach(form=’json’)Attach the current provenance to the file by appending it as a json-encoded string to the ‘description’header field.

This is only attempted if the file has been inspect()-ed and has been determined to be a raw fif file.


inspect()

6.1.20 niprov.filefilter module

class niprov.filefilter.FileFilter(dependencies=<niprov.dependencies.Dependencies object>)Bases: object

include(filepath)Whether the file is to be included in discovery.

Parameters filepath (str) – The full path of the file.

Returns True if the file should be included.

Return type bool

6.1.21 niprov.files module

6.1.22 niprov.filesystem module

class niprov.filesystem.FilesystemBases: object

Wrapper of filesystem access functionality such as that implemented by the os package in the standard library.

fileExists(path)

getctime(path)Get the creation time for the file at path.

Parameters path – Path to the file to read.



Returns Time when the file was last changed

Return type datetime

getsize(path)

read(path)Read the contents of a textfile.

Parameters path – Path to the file to read.

Returns Contents of the file

Return type str

Raises IOError – [Errno 2] No such file or directory: ‘xyz’

readlines(path)

walk(path)

write(path, content)Write string content to a textfile.

Parameters

• path – Path to the file to read.

• content (str) – What to fill the file with

6.1.23 niprov.format module

class niprov.format.Format(dependencies=<niprov.dependencies.Dependencies object>)Bases: object

Parent Format class from which specific formats are derived.

serialize(provenance)Publish provenance.

This determines if the provenance is for a single file or multiple, and then calls the appropriate morespecific serialize method.

serializeList(provenance)

serializePipeline(pipeline)

serializeSingle(provenance)

serializeStatistics(provenance)

6.1.24 niprov.formatdict module

class niprov.formatdict.DictFormat(dependencies=<niprov.dependencies.Dependencies object>)Bases: niprov.format.Format

serializeList(fileList)

serializeSingle(item)

6.1. Submodules 27


6.1.25 niprov.formatfactory module

class niprov.formatfactory.FormatFactory(dependencies=<niprov.dependencies.Dependenciesobject>)

Bases: object

create(formatName)

6.1.26 niprov.formatjson module

class niprov.formatjson.DateTimeAwareJSONDecoder(**kwargs)Bases: json.decoder.JSONDecoder

Converts a json string, where datetime and timedelta objects were converted into objects using the Date-TimeAwareJSONEncoder, back into a python object.

dict_to_object(d)

class niprov.formatjson.DateTimeAwareJSONEncoder(skipkeys=False, ensure_ascii=True,check_circular=True, allow_nan=True,sort_keys=False, indent=None, sep-arators=None, encoding=’utf-8’,default=None)

Bases: json.encoder.JSONEncoder

Converts a python object, where datetime and timedelta objects are converted into objects that can be decodedusing the DateTimeAwareJSONDecoder.

default(obj)

class niprov.formatjson.JsonFormat(dependencies)Bases: niprov.format.Format

Helper to convert provenance data to and from json encoded strings.

deserialize(jsonRecord)Convert one provenance item from its json string version to the native python dictionary format.

Parameters jsonRecord (str) – The provenance item to convert as json-encoded string.

Returns Python dictionary of the provenance.

Return type dict

deserializeList(jsonListOfRecords)Convert a list of provenance items from its json string version to the a list of the native python dictionaryformat.

Parameters jsonListOfRecords (str) – The provenance items to convert as json-encodedstring.

Returns Python list of dictionaries of the provenance.

Return type list

serializeList(listOfRecords)Convert a list of provenance items from its native list of python dict type to a json string.

Parameters listOfRecords (list) – The provenance items to convert.

Returns Json version of the provenance items.

Return type str



serializeSingle(record)Convert one provenance item from its native python dict type to a json string.

Parameters record (dict) – The provenance item to convert.

Returns Json version of the provenance.

Return type str

6.1.27 niprov.formatnarrated module

class niprov.formatnarrated.NarratedFormat(dependencies=<niprov.dependencies.Dependenciesobject>)

Bases: niprov.format.Format

Generates human-readable text that describes provenance objects.

serializeSingle(img)

6.1.28 niprov.formatobject module

class niprov.formatobject.ObjectFormat(dependencies=<niprov.dependencies.Dependencies ob-ject>)


serializeList(provenance)

serializePipeline(provenance)

serializeSingle(provenance)

serializeStatistics(provenance)

6.1.29 niprov.formatsimple module

class niprov.formatsimple.SimpleFormat(dependencies=<niprov.dependencies.Dependencies ob-ject>)


serializeList(images)Publish the provenance for several images on the commandline.

Parameters provenance (list) – List of provenance dictionaries.

serializePipeline(pipeline)Pretty-print a pipeline to the terminal

Parameters pipeline (Pipeline) – Pipeline object to publish.

serializeSingle(img)Publish the provenance for one image on the commandline.

Parameters provenance (dict) – Provenance for one image file

serializeStatistics(stats)Publish statistics for collected provenance in the terminal.

Parameters stats (dict) – Dictionary with summary values.

serializeSummary(image)Publish a summary of the provenance for one image as one line in the terminal.

6.1. Submodules 29


Parameters provenance (dict) – Provenance for one image file

6.1.30 niprov.formatxml module

class niprov.formatxml.XmlFormat(dependencies)Bases: niprov.format.Format

serializeList(itemOrList)

serializeSingle(item)

6.1.31 niprov.hashing module

class niprov.hashing.HasherBases: object

blocksize = 524288

digest(filename)Determine the unique hash digest for a file.

Note: takes 26s for a 14GB file.

Parameters filename (str) – Path to the file for which a hash digest should be made.

6.1.32 niprov.importing module

niprov.importing.importp(filepath, dependencies=<niprov.dependencies.Dependencies object>)Add provenance in bulk from a file, such as saved by export().

Named importp as opposed to import because the latter is a reserved word in Python.

This can serve as a backup, migration tool, or for exchange.

6.1.33 niprov.inheriting module

niprov.inheriting.inheritFrom(provenance, parentProvenance)

6.1.34 niprov.inspection module

niprov.inspection.inspect(location, dependencies=<niprov.dependencies.Dependencies object>)

6.1.35 niprov.jsonfile module

class niprov.jsonfile.JsonFile(dependencies=<niprov.dependencies.Dependencies object>)Bases: object

Stores provenance in a local text file encoded as json.

add(image)Add the provenance for one file to storage.

Parameters image (BaseFile) – Image file to store.



all()Retrieve all known provenance from storage.

Returns List of provenance for known files.

Return type list

byId(uid)

byLocation(locationString)Get the provenance for a file at the given location.

In the case of a dicom series, this returns the provenance for the series.

Parameters locationString (str) – Location of the image file.

Returns Provenance for one image file.

Return type dict

byLocations(listOfLocations)

byParents(listOfParentLocations)

getSeries(image)Get the object that carries provenance for the series that the image passed is in.

Parameters image (DicomFile) – File that is part of a series.

Returns Image object that caries provenance for the series.

Return type DicomFile

inquire(query)

latest(n=20)

search(text)

serializeAndWrite(images)

statistics()

update(image)Save changed provenance for this file..

Parameters image (BaseFile) – Image file that has changed.

updateApproval(fpath, approvalStatus)

6.1.36 niprov.libraries module

class niprov.libraries.LibrariesBases: object

hasDependency(libname)

6.1.37 niprov.location module

class niprov.location.Location(locationString)Bases: object

Represents the location of a file.

toDictionary()

6.1. Submodules 31


toString()

toUrl()

6.1.38 niprov.locationfactory module

class niprov.locationfactory.LocationFactory(dependencies=<niprov.dependencies.Dependenciesobject>)

Bases: object

Creates Location objects.

completeString(locationString)Validates a location string.

If locationString only contains a filesystem path, computer info will be added.

fromString(locationString)Creates a Location object.

Parameters locationString (str) – String with path and optionally computer id.

6.1.39 niprov.mediumdirect module

class niprov.mediumdirect.DirectMediumBases: object

Dummy Medium class which simply returns the serialized provenance passed.

export(formattedProvenance, form)

6.1.40 niprov.mediumfactory module

class niprov.mediumfactory.MediumFactory(dependencies=<niprov.dependencies.Dependenciesobject>)

Bases: object

create(mediumName)

6.1.41 niprov.mediumfile module

class niprov.mediumfile.FileMedium(dependencies)Bases: object


6.1.42 niprov.mediumstdout module

class niprov.mediumstdout.StandardOutputMediumBases: object




6.1.43 niprov.mediumviewer module

class niprov.mediumviewer.ViewerMedium(dependencies)Bases: object

Uses the system picture viewer to display an picture file.

export(formattedProvenance, form=None)

6.1.44 niprov.mnefun module

mnefun support module

This module provides handlers to attach to mnefun events.

import mnefunimport niprov.mnefunsupportparams = mnefun.Params()params.on_process = niprov.mnefunsupport.handler

niprov.mnefunsupport.handler(text, func, out, params, dependen-cies=<niprov.dependencies.Dependencies object>)

mnefun on_process handler

Responds to the following mnefun steps: fetch_raw_files: Runs discover on the subjects’ raw data dirs.fetch_sss_files, apply_preprocessing_combined, save_epochs: For these steps, runs log for the new files.

6.1.45 niprov.mongo module

class niprov.mongo.MongoRepository(dependencies=<niprov.dependencies.Dependencies object>)Bases: object





Return type list

byId(uid)





Return type dict

byLocations(listOfLocations)

byParents(listOfParentLocations)

deflate(img)

6.1. Submodules 33






inflate(record)

inquire(query)

latest()

search(text)

statistics()



updateApproval(locationString, approvalStatus)

6.1.46 niprov.nifti module

class niprov.nifti.NiftiFile(location, **kwargs)Bases: niprov.basefile.BaseFile

attach(form=’json’)Attach the current provenance to the file by injecting it as a json-encoded extension to the nifti header.


inspect()

6.1.47 niprov.parrec module

6.1.48 niprov.pictures module

class niprov.pictures.PictureCache(dependencies)Bases: niprov.format.Format

getBytes(for_)

getFilepath(for_)

keep(picture, for_)

new()

saveToDisk(for_)

serializeSingle(image)Provides file path to picture of image.

This is part of the Format interface.



6.1.49 niprov.pipeline module

class niprov.pipeline.Pipeline(files)Bases: object

asFilenameTree()

6.1.50 niprov.pipelinefactory module

class niprov.pipelinefactory.PipelineFactory(dependencies=<niprov.dependencies.Dependenciesobject>)

Bases: object

forFile(image)Create a Pipeline object based on known files ‘parents’ field.

6.1.51 niprov.plogging module

niprov.plogging.log(new, transformation, parents, code=None, logtext=None, transient=False,script=None, user=None, provenance=None, opts=None, dependen-cies=<niprov.dependencies.Dependencies object>)

Register a transformation that creates a new image (or several).

This will retrieve the primary parent’s provenance. if no provenance is availale for the primary parent, callslistener.unknownFile. Otherwise, some fields are copied from the primary parent, subject to availability. Forinstance, if the parent has no ‘subject’ field, the new file’s provenance won’t either.

Parameters

• new (str or list) – Path(s) to the newly created file(s).

• transformation (str) – Name of the operation that has been used.

• parents (str or list) – Path(s) to the file(s) that were used as the basis of the trans-formation. Assumes that the first file in the list is the primary parent for which basic prove-nance is known.

• code (str, optional) – Code that was used to generate the new file

• logtext (str, optional) – Any information about the transformation that waslogged.

• script (str, optional) – Path to the code file that contains the transformation code.


• user (string, optional) – Name of the user logging provenance.

• provenance (dict, optional) – Add the key-value pairs in this dictionary to theprovenance record for the new files.

• opts (Configuration) – General settings for niprov. See niprov.config

Raises IOError – ‘[Errno 2] File not found’ is raised if the new file does not exist on the filesystemand is not marked as transient.

Returns New provenance, if multiple files were created, this is a list of images, otherwise, it is asingle object.

6.1. Submodules 35


Return type BaseFile

6.1.52 niprov.querying module

class niprov.querying.Query(dependencies)Bases: object

all()

allModalities()

allProjects()

allUsers()

byApproval(val)

byLocation(val)

byModality(val)

byProject(val)

bySubject(val)

byUser(val)

copiesOf(target)

getFields()

latest()

statistics()

class niprov.querying.QueryField(name, value, all)Bases: tuple

__getnewargs__()Return self as a plain tuple. Used by copy and pickle.

__getstate__()Exclude the OrderedDict from pickling

__repr__()Return a nicely formatted representation string

allAlias for field number 2

nameAlias for field number 0

valueAlias for field number 1

6.1.53 niprov.recording module

niprov.recording.record(command, new=None, parents=None, transient=False,args=None, kwargs=None, user=None, opts=None, dependen-cies=<niprov.dependencies.Dependencies object>)

Execute a command and log it as provenance for the newly created file.



Parameters

• command (list or str or callable) – Command to be executed. Either a stringof executable system code, a list of components thereof, or a python function object.

• new (list or str, optional) – Override path to the new file(s), i.e. if they cannotbe parsed from the command.

• parents (list or str, optional) – Override paths to parent file(s), i.e. if theycannot be parsed from the command.


• args (list, optional) – Positional arguments to be passed to command.

• kwargs (dict, optional) – Keyword arguments to be passed to command.

• opts (Configuration) – General settings for niprov. See niprov.config

Returns New provenance

Return type dict

6.1.54 niprov.renaming module

niprov.renaming.renameDicoms(dicomdir, dependencies=<niprov.dependencies.Dependencies ob-ject>)

Add the .dcm extension to any non-hidden files without extension.

Parameters dicomdir (str) – Directory in which to rename files.

6.1.55 niprov.reporting module

6.1.56 niprov.repository module

class niprov.repository.RepositoryBases: object





Return type list

byApproval(approvalStatus)

byId(uid)Get the provenance for a file with the given id.

Parameters uid (str) – Unique id for the file.

Returns File with the given id.

Return type BaseFile

6.1. Submodules 37






Return type dict

byLocations(listOfLocations)Get any files that match one of these locations


Parameters listOfLocations (list) – List of image locations.

Returns List with BaseFile objects

Return type list

bySubject(subject)Get the provenance for all files of a given participant.

Parameters subject (str) – The name or other ID string.

Returns List of provenance for known files imaging this subject.

Return type list





knows(image)Whether this file has provenance associated with it.

Returns True if provenance is available for this image.

Return type bool

knowsByLocation(locationString)Whether the file at this location has provenance associated with it.

Returns True if provenance is available for that path.

Return type bool

knowsSeries(image)Whether this file is part of a series for which provenance is available.

Parameters image (BaseFile) – File for which the series is sought.

Returns True if provenance is available for this series.

Return type bool

latest(n=20)Get the images that have been registered last.

Parameters n (int) – The number of files to retrieve. Defaults to 20.

Returns List of BaseFile objects.



Return type list



updateApproval(locationString, approvalStatus)

6.1.57 niprov.searching module

niprov.searching.search(text, dependencies=<niprov.dependencies.Dependencies object>)Search for files with the given text in their provenance fields.

Parameters text (str) – Words to look for.

Returns List of BaseFile objects

Return type list

6.1.58 niprov.users module

class niprov.users.Users(dependencies)Bases: object

determineUser(passedUserValue)Determine the name of the user creating provenance.

The following methods will be tried to determine the username; 1) The passedUserValue argument if notNone 2) The configuration file or setting has a value for the key user 3) The OS username.

Parameters passedUserValue (str) – This value will override any other method to deter-mine the username.

Returns The username or handle of the user.

Return type string

6.1.59 niprov.views module

niprov.views.home(request)

niprov.views.latest(request)

niprov.views.location(request)

niprov.views.modalities(request)

niprov.views.modality(request)

niprov.views.pipeline(request)

niprov.views.project(request)

niprov.views.projects(request)

niprov.views.search(request)

niprov.views.short(request)

niprov.views.stats(request)

6.1. Submodules 39


niprov.views.subject(request)

niprov.views.user(request)

niprov.views.users(request)

6.1.60 niprov.webapp module

niprov.webapp.main(global_config, **settings)This function returns a Pyramid WSGI application.

niprov.webapp.serve()

6.2 Module contents


CHAPTER 7

Indices and tables

• genindex

• modindex

• search

41


42 Chapter 7. Indices and tables

Python Module Index

nniprov, 40niprov.adding, 17niprov.approval, 17niprov.basefile, 18niprov.camera, 18niprov.capturing, 19niprov.clock, 19niprov.cnt, 19niprov.commandline, 19niprov.comparing, 20niprov.config, 20niprov.context, 21niprov.dcm, 22niprov.dependencies, 22niprov.diff, 23niprov.discovery, 25niprov.exceptions, 25niprov.exporting, 25niprov.externals, 26niprov.fif, 26niprov.filefilter, 26niprov.filesystem, 26niprov.format, 27niprov.formatdict, 27niprov.formatfactory, 28niprov.formatjson, 28niprov.formatnarrated, 29niprov.formatobject, 29niprov.formatsimple, 29niprov.formatxml, 30niprov.hashing, 30niprov.importing, 30niprov.inheriting, 30niprov.inspection, 30niprov.jsonfile, 30niprov.libraries, 31niprov.location, 31niprov.locationfactory, 32niprov.mediumdirect, 32

niprov.mediumfactory, 32niprov.mediumfile, 32niprov.mediumstdout, 32niprov.mediumviewer, 33niprov.mnefunsupport, 33niprov.mongo, 33niprov.nifti, 34niprov.pictures, 34niprov.pipeline, 35niprov.pipelinefactory, 35niprov.plogging, 35niprov.querying, 36niprov.recording, 36niprov.renaming, 37niprov.repository, 37niprov.searching, 39niprov.users, 39niprov.views, 39niprov.webapp, 40

43


44 Python Module Index

Index

Symbols__getnewargs__() (niprov.querying.QueryField method),

36__getstate__() (niprov.querying.QueryField method), 36__repr__() (niprov.querying.QueryField method), 36

Aadd() (in module niprov.adding), 17add() (niprov.context.ProvenanceContext method), 21add() (niprov.jsonfile.JsonFile method), 30add() (niprov.mongo.MongoRepository method), 33add() (niprov.repository.Repository method), 37addUnknownParent() (niprov.commandline.Commandline

method), 19all (niprov.querying.QueryField attribute), 36all() (niprov.jsonfile.JsonFile method), 30all() (niprov.mongo.MongoRepository method), 33all() (niprov.querying.Query method), 36all() (niprov.repository.Repository method), 37allModalities() (niprov.querying.Query method), 36allProjects() (niprov.querying.Query method), 36allUsers() (niprov.querying.Query method), 36approve() (in module niprov.approval), 17approve() (niprov.context.ProvenanceContext method),

21areEqual() (niprov.diff.Diff method), 23areEqualProtocol() (niprov.diff.Diff method), 23asFilenameTree() (niprov.pipeline.Pipeline method), 35assertEqual() (niprov.diff.Diff method), 24assertEqualProtocol() (niprov.diff.Diff method), 24attach (niprov.config.Configuration attribute), 20attach() (niprov.basefile.BaseFile method), 18attach() (niprov.fif.FifFile method), 26attach() (niprov.nifti.NiftiFile method), 34attach_format (niprov.config.Configuration attribute), 20

Bbackup() (in module niprov.exporting), 25backup() (niprov.context.ProvenanceContext method), 21BaseFile (class in niprov.basefile), 18

blocksize (niprov.hashing.Hasher attribute), 30byApproval() (niprov.querying.Query method), 36byApproval() (niprov.repository.Repository method), 37byId() (niprov.jsonfile.JsonFile method), 31byId() (niprov.mongo.MongoRepository method), 33byId() (niprov.repository.Repository method), 37byLocation() (niprov.jsonfile.JsonFile method), 31byLocation() (niprov.mongo.MongoRepository method),

33byLocation() (niprov.querying.Query method), 36byLocation() (niprov.repository.Repository method), 37byLocations() (niprov.jsonfile.JsonFile method), 31byLocations() (niprov.mongo.MongoRepository

method), 33byLocations() (niprov.repository.Repository method), 38byModality() (niprov.querying.Query method), 36byParents() (niprov.jsonfile.JsonFile method), 31byParents() (niprov.mongo.MongoRepository method),

33byProject() (niprov.querying.Query method), 36bySubject() (niprov.querying.Query method), 36bySubject() (niprov.repository.Repository method), 38byUser() (niprov.querying.Query method), 36

CCamera (class in niprov.camera), 18Clock (class in niprov.clock), 19Commandline (class in niprov.commandline), 19compare() (in module niprov.comparing), 20compare() (niprov.basefile.BaseFile method), 18compare() (niprov.context.ProvenanceContext method),

21completeString() (niprov.locationfactory.LocationFactory

method), 32Configuration (class in niprov.config), 20copiesOf() (niprov.querying.Query method), 36create() (niprov.formatfactory.FormatFactory method), 28create() (niprov.mediumfactory.MediumFactory method),

32

45


Ddatabase_type (niprov.config.Configuration attribute), 20database_url (niprov.config.Configuration attribute), 21DateTimeAwareJSONDecoder (class in

niprov.formatjson), 28DateTimeAwareJSONEncoder (class in

niprov.formatjson), 28default() (niprov.formatjson.DateTimeAwareJSONEncoder

method), 28defaultIgnore (niprov.diff.Diff attribute), 24deflate() (niprov.mongo.MongoRepository method), 33Dependencies (class in niprov.dependencies), 22deserialize() (niprov.formatjson.JsonFormat method), 28deserializeList() (niprov.formatjson.JsonFormat method),

28determineUser() (niprov.users.Users method), 39DicomFile (class in niprov.dcm), 22dict_to_object() (niprov.formatjson.DateTimeAwareJSONDecoder

method), 28DictFormat (class in niprov.formatdict), 27Diff (class in niprov.diff), 23digest() (niprov.hashing.Hasher method), 30DirectMedium (class in niprov.mediumdirect), 32discover() (in module niprov.discovery), 25discover() (niprov.context.ProvenanceContext method),

21discover_file_extensions (niprov.config.Configuration at-

tribute), 21discoveryFinished() (niprov.commandline.Commandline

method), 19dryrun (niprov.config.Configuration attribute), 21

Eexport() (in module niprov.exporting), 25export() (niprov.context.ProvenanceContext method), 21export() (niprov.mediumdirect.DirectMedium method),

32export() (niprov.mediumfile.FileMedium method), 32export() (niprov.mediumstdout.StandardOutputMedium

method), 32export() (niprov.mediumviewer.ViewerMedium method),

33exportedToFile() (niprov.commandline.Commandline

method), 19Externals (class in niprov.externals), 26

FFifFile (class in niprov.fif), 26fileAdded() (niprov.commandline.Commandline

method), 19fileError() (niprov.commandline.Commandline method),

19fileExists() (niprov.filesystem.Filesystem method), 26

FileFilter (class in niprov.filefilter), 26FileMedium (class in niprov.mediumfile), 32filesMarkedForApproval()

(niprov.commandline.Commandline method),19

Filesystem (class in niprov.filesystem), 26forFile() (niprov.pipelinefactory.PipelineFactory

method), 35Format (class in niprov.format), 27FormatFactory (class in niprov.formatfactory), 28fromString() (niprov.locationfactory.LocationFactory

method), 32

Gget() (niprov.context.ProvenanceContext method), 21getBytes() (niprov.pictures.PictureCache method), 34getCamera() (niprov.dependencies.Dependencies

method), 22getClock() (niprov.dependencies.Dependencies method),

22getConfiguration() (niprov.dependencies.Dependencies

method), 22getctime() (niprov.filesystem.Filesystem method), 26getDifferences() (niprov.diff.Diff method), 24getDifferenceString() (niprov.diff.Diff method), 24getExternals() (niprov.dependencies.Dependencies

method), 22getFields() (niprov.querying.Query method), 36getFileFactory() (niprov.dependencies.Dependencies

method), 22getFileFilter() (niprov.dependencies.Dependencies

method), 22getFilepath() (niprov.pictures.PictureCache method), 34getFilesystem() (niprov.dependencies.Dependencies

method), 23getFormatFactory() (niprov.dependencies.Dependencies

method), 23getHasher() (niprov.dependencies.Dependencies

method), 23getLibraries() (niprov.dependencies.Dependencies

method), 23getListener() (niprov.dependencies.Dependencies

method), 23getLocationFactory() (niprov.dependencies.Dependencies

method), 23getMediumFactory() (niprov.dependencies.Dependencies

method), 23getNowString() (niprov.clock.Clock method), 19getPictureCache() (niprov.dependencies.Dependencies

method), 23getPipelineFactory() (niprov.dependencies.Dependencies

method), 23getProtocolFields() (niprov.basefile.BaseFile method), 18getProvenance() (niprov.basefile.BaseFile method), 18

46 Index


getQuery() (niprov.dependencies.Dependencies method),23

getRepository() (niprov.dependencies.Dependenciesmethod), 23

getSame() (niprov.diff.Diff method), 24getSameString() (niprov.diff.Diff method), 24getSerializer() (niprov.dependencies.Dependencies

method), 23getSeries() (niprov.jsonfile.JsonFile method), 31getSeries() (niprov.mongo.MongoRepository method), 33getSeries() (niprov.repository.Repository method), 38getSeriesId() (niprov.basefile.BaseFile method), 18getSeriesId() (niprov.dcm.DicomFile method), 22getsize() (niprov.filesystem.Filesystem method), 27getSnapshotFilepath() (niprov.basefile.BaseFile method),

18getUsers() (niprov.dependencies.Dependencies method),

23

Hhandler() (in module niprov.mnefunsupport), 33hasDependency() (niprov.libraries.Libraries method), 31hasFile() (niprov.dcm.DicomFile method), 22Hasher (class in niprov.hashing), 30home() (in module niprov.views), 39

Iimportp() (in module niprov.importing), 30importp() (niprov.context.ProvenanceContext method),

21include() (niprov.filefilter.FileFilter method), 26inflate() (niprov.mongo.MongoRepository method), 34inheritFrom() (in module niprov.inheriting), 30inquire() (niprov.jsonfile.JsonFile method), 31inquire() (niprov.mongo.MongoRepository method), 34inspect() (in module niprov.inspection), 30inspect() (niprov.basefile.BaseFile method), 18inspect() (niprov.cnt.NeuroscanFile method), 19inspect() (niprov.context.ProvenanceContext method), 21inspect() (niprov.dcm.DicomFile method), 22inspect() (niprov.fif.FifFile method), 26inspect() (niprov.nifti.NiftiFile method), 34interpretedRecording() (niprov.commandline.Commandline

method), 19

JJsonFile (class in niprov.jsonfile), 30JsonFormat (class in niprov.formatjson), 28

Kkeep() (niprov.pictures.PictureCache method), 34keepVersionsFromPrevious() (niprov.basefile.BaseFile

method), 18knows() (niprov.repository.Repository method), 38

knowsByLocation() (niprov.repository.Repositorymethod), 38

knowsSeries() (niprov.repository.Repository method), 38

Llatest() (in module niprov.views), 39latest() (niprov.jsonfile.JsonFile method), 31latest() (niprov.mongo.MongoRepository method), 34latest() (niprov.querying.Query method), 36latest() (niprov.repository.Repository method), 38Libraries (class in niprov.libraries), 31Location (class in niprov.location), 31location() (in module niprov.views), 39LocationFactory (class in niprov.locationfactory), 32log() (in module niprov.plogging), 35log() (niprov.commandline.Commandline method), 19log() (niprov.context.ProvenanceContext method), 21

Mmain() (in module niprov.webapp), 40markedForApproval() (in module niprov.approval), 18markedForApproval() (niprov.context.ProvenanceContext

method), 21markForApproval() (in module niprov.approval), 17markForApproval() (niprov.context.ProvenanceContext

method), 21MediumFactory (class in niprov.mediumfactory), 32mergeWith() (niprov.dcm.DicomFile method), 22missingDependencyForImage()


mnefunEventReceived() (niprov.commandline.Commandlinemethod), 19

modalities() (in module niprov.views), 39modality() (in module niprov.views), 39MongoRepository (class in niprov.mongo), 33

Nname (niprov.querying.QueryField attribute), 36NarratedFormat (class in niprov.formatnarrated), 29NCHARSCOL (niprov.diff.Diff attribute), 23NeuroscanFile (class in niprov.cnt), 19new() (niprov.pictures.PictureCache method), 34NiftiFile (class in niprov.nifti), 34niprov (module), 40niprov.adding (module), 17niprov.approval (module), 17niprov.basefile (module), 18niprov.camera (module), 18niprov.capturing (module), 19niprov.clock (module), 19niprov.cnt (module), 19niprov.commandline (module), 19niprov.comparing (module), 20

Index 47


niprov.config (module), 20niprov.context (module), 21niprov.dcm (module), 22niprov.dependencies (module), 22niprov.diff (module), 23niprov.discovery (module), 25niprov.exceptions (module), 25niprov.exporting (module), 25niprov.externals (module), 26niprov.fif (module), 26niprov.filefilter (module), 26niprov.filesystem (module), 26niprov.format (module), 27niprov.formatdict (module), 27niprov.formatfactory (module), 28niprov.formatjson (module), 28niprov.formatnarrated (module), 29niprov.formatobject (module), 29niprov.formatsimple (module), 29niprov.formatxml (module), 30niprov.hashing (module), 30niprov.importing (module), 30niprov.inheriting (module), 30niprov.inspection (module), 30niprov.jsonfile (module), 30niprov.libraries (module), 31niprov.location (module), 31niprov.locationfactory (module), 32niprov.mediumdirect (module), 32niprov.mediumfactory (module), 32niprov.mediumfile (module), 32niprov.mediumstdout (module), 32niprov.mediumviewer (module), 33niprov.mnefunsupport (module), 33niprov.mongo (module), 33niprov.nifti (module), 34niprov.pictures (module), 34niprov.pipeline (module), 35niprov.pipelinefactory (module), 35niprov.plogging (module), 35niprov.querying (module), 36niprov.recording (module), 36niprov.renaming (module), 37niprov.repository (module), 37niprov.searching (module), 39niprov.users (module), 39niprov.views (module), 39niprov.webapp (module), 40NiprovError, 25

OObjectFormat (class in niprov.formatobject), 29ordinal() (in module niprov.commandline), 20OutputCapture (class in niprov.capturing), 19

Pparents (niprov.basefile.BaseFile attribute), 18PictureCache (class in niprov.pictures), 34Pipeline (class in niprov.pipeline), 35pipeline() (in module niprov.views), 39PipelineFactory (class in niprov.pipelinefactory), 35print_() (in module niprov.exporting), 25print_() (niprov.context.ProvenanceContext method), 21project() (in module niprov.views), 39projects() (in module niprov.views), 39ProvenanceContext (class in niprov.context), 21

QQuery (class in niprov.querying), 36QueryField (class in niprov.querying), 36

Rread() (niprov.filesystem.Filesystem method), 27readlines() (niprov.filesystem.Filesystem method), 27receivedBashCommand()


reconfigureOrGetConfiguration()(niprov.dependencies.Dependencies method),23

record() (in module niprov.recording), 36record() (niprov.context.ProvenanceContext method), 22renamedDicom() (niprov.commandline.Commandline

method), 20renameDicoms() (in module niprov.renaming), 37renameDicoms() (niprov.context.ProvenanceContext

method), 22Repository (class in niprov.repository), 37Result (class in niprov.externals), 26run() (niprov.externals.Externals method), 26

SsaveSnapshot() (niprov.camera.Camera method), 18saveToDisk() (niprov.pictures.PictureCache method), 34search() (in module niprov.searching), 39search() (in module niprov.views), 39search() (niprov.context.ProvenanceContext method), 22search() (niprov.jsonfile.JsonFile method), 31search() (niprov.mongo.MongoRepository method), 34selectApproved() (in module niprov.approval), 18selectApproved() (niprov.context.ProvenanceContext

method), 22serialize() (niprov.format.Format method), 27serializeAndWrite() (niprov.jsonfile.JsonFile method), 31serializeList() (niprov.format.Format method), 27serializeList() (niprov.formatdict.DictFormat method), 27serializeList() (niprov.formatjson.JsonFormat method),

28

48 Index


serializeList() (niprov.formatobject.ObjectFormatmethod), 29

serializeList() (niprov.formatsimple.SimpleFormatmethod), 29

serializeList() (niprov.formatxml.XmlFormat method), 30serializePipeline() (niprov.format.Format method), 27serializePipeline() (niprov.formatobject.ObjectFormat

method), 29serializePipeline() (niprov.formatsimple.SimpleFormat

method), 29serializeSingle() (niprov.format.Format method), 27serializeSingle() (niprov.formatdict.DictFormat method),

27serializeSingle() (niprov.formatjson.JsonFormat method),

28serializeSingle() (niprov.formatnarrated.NarratedFormat

method), 29serializeSingle() (niprov.formatobject.ObjectFormat

method), 29serializeSingle() (niprov.formatsimple.SimpleFormat

method), 29serializeSingle() (niprov.formatxml.XmlFormat method),

30serializeSingle() (niprov.pictures.PictureCache method),

34serializeStatistics() (niprov.format.Format method), 27serializeStatistics() (niprov.formatobject.ObjectFormat

method), 29serializeStatistics() (niprov.formatsimple.SimpleFormat

method), 29serializeSummary() (niprov.formatsimple.SimpleFormat

method), 29serve() (in module niprov.webapp), 40short() (in module niprov.views), 39SimpleFormat (class in niprov.formatsimple), 29StandardOutputMedium (class in niprov.mediumstdout),

32statistics() (niprov.jsonfile.JsonFile method), 31statistics() (niprov.mongo.MongoRepository method), 34statistics() (niprov.querying.Query method), 36stats() (in module niprov.views), 39subject() (in module niprov.views), 39

TtakeSnapshot() (niprov.camera.Camera method), 18toDictionary() (niprov.location.Location method), 31toString() (niprov.location.Location method), 31toUrl() (niprov.location.Location method), 32

UUnknownFileError, 25update() (niprov.jsonfile.JsonFile method), 31update() (niprov.mongo.MongoRepository method), 34update() (niprov.repository.Repository method), 39

updateApproval() (niprov.jsonfile.JsonFile method), 31updateApproval() (niprov.mongo.MongoRepository

method), 34updateApproval() (niprov.repository.Repository method),

39user (niprov.config.Configuration attribute), 21user() (in module niprov.views), 40Users (class in niprov.users), 39users() (in module niprov.views), 40usingCopyAsParent() (niprov.commandline.Commandline

method), 20

Vvalue (niprov.querying.QueryField attribute), 36verbosity (niprov.config.Configuration attribute), 21versions (niprov.basefile.BaseFile attribute), 18view() (in module niprov.exporting), 25view() (niprov.context.ProvenanceContext method), 22ViewerMedium (class in niprov.mediumviewer), 33viewSnapshot() (niprov.basefile.BaseFile method), 18vlevels (niprov.commandline.Commandline attribute), 20

Wwalk() (niprov.filesystem.Filesystem method), 27write() (niprov.filesystem.Filesystem method), 27

XXmlFormat (class in niprov.formatxml), 30

Index 49

Documents

niprov Documentation - Read the Docs · niprov Documentation, Release 0.5 Ever encountered a ﬁle of which you weren’t sure what analysis steps it had gone through? Ever wanted