Vistas for hardware implementation of SOFI: High speed imaging for rapid biological processes

Imaging fast, user-friendly and integrate with ease

Dirk HähnelIII. Institute of Physics – BiophysicsGeorg-August-University Göttingen

SOFI Developer MeetingGöttingen 28th March 2015

Göttingen 28th March 2015

2

Making it easy and user-friendly?

Göttingen 28th March 2015

6. price < 10thd. USD

1. physicist

2. chemicist

3. artifacts

4. image stacks

5. dynamical biosystems

CSDISMRequirements Palm Storm

SIM SSIM

Sted Tirf SOFI

3

Why speed is important?

Göttingen 28th March 2015

atomic scale0.1 - 1.0 nm

dynamic data0.1 - 10 ns

molecular dynamics

molecular scale1.0 - 10 nm

interaction dataKon, Koff, Kd

10 ns - 10 msinteractions

cellular scale10 - 100 nm

concentrationsdiffusion rates

10 ms - 1000 sfluid dynamics

4

• tier 1: interatome– which molecules talk to each other in networks?

• tier 2: deterministic– what is the average case behavior?

• tier 3: stochastic– what is the variance of the system?

Why integration is important?

Göttingen 28th March 2015

6

Fast SOFI: crucial challenges

• subpixel resolution

• linearize brightness

• multiplane imaging

Göttingen 28th March 2015

• timing / speed

• memory allocation

• cumulants computation

• integration

• scaling

physical and experimental challenges: implementation challenges

7

Implementation: timing challenge

Göttingen 28th March 2015

acquisition

image reconstruction

final SOFI image

acquisition image reconstruction

Final SOFI Image

Imaging today: no dynamics Imaging dynamical biological processes

live imagingreal time reconstruction

8

Implementation: memory challenge

Göttingen 28th March 2015

subpixel = more gates subpixel = more timecumulants => data swappinglinearization => very complex

start

input frame

SCMOS input:3 Gpixel => SOFI image1,5 GByte => SOFI image

SCMOS subpixel:12 Gpixel => SOFI image6 GByte => SOFI image

…

end

9

Implementation: cumulants computation challenge

Göttingen 28th March 2015

challenge:memory space < data 3D stack

cumulants are moments corrected by lower order moments

Tremendous matrix operations

10

Implementation: integration challenge

Göttingen 28th March 2015

features:• no intermediate data• open API• micromanager integration

11

Implementation: scaling system architecture challenge

Göttingen 28th March 2015

hardware schematic software schematic

cam n

cam n-1

hardware bus

cam 0

cam 1

data layer

Buffer

Reconstructioncomputation

originaldataintermediate

constant cam(s)stream

Intermediate

data

application layer interface

scali

ng

12

Integration and development roadmap

Imaging of dynamical biological systems:acquisition time > image reconstruction

Göttingen 28th March 2015

camera link

FFTcumulants

linearizationmicromanagerintegration

IIIQ 2015

integration• standard FPGA card • beta software mid 2015• setup recipe• complexity blackbox

13

Acknowledgements

Göttingen 28th March 2015