An FPGA based Phased Array Processor for the Sub ...jweintro/vlbi/forbob/Vinayaks Graphics... · An FPGA based Phased Array Processor for the Sub Millimeter ... CASPER team at UC

An FPGA based Phased ArrayProcessor for the Sub Millimeter

ArrayVinayak Nagpal

Chalmers University of Technology, Sweden

advised by

Jonathan Weintroub

Smithsonian Astrophysical Observatory

Vinayak Nagpal, CTH – p. 1/28

Outline

Introduction: What we plan to build and why?


Outline


Infrastructure: What did we have in hand?


Outline



Design: How we went around building it?


Outline




Results: Does it work?


Outline





Future: Where to go from here?


Outline





Future: Where to go from here?

Demonstration in M247 for those who want tosee it work and discuss in more detail.


Motivation

(VLBI) at 0.8 mm with sufficiently long baselines →

≈ 20 µas resolution.

In sub millimeter electron scattering is reduced.

Imaging observation of the event horizon in a blackhole e.g. SgrA∗, M87.

Sub-millimeter telescopes: JCMT, CSO, HHT, (ALMA)etc and SMA.

SMA full collecting area + JCMT + CSO → we need aphased array with interface to standard VLBIrecorders, i.e. Mark V.


Project Objectives

Proof of Concept

Development time → 10 months.

INPUT: 8 Antennas (SMA or JCMT/CSO),Single Polarization, 500 MHz Bandwidth.

OUTPUT: Real Time Phased Sum of 8antennas spooled to Mark Vb VLBI datastorage unit.

Build a scalable system using state of artsampling and FPGA technology.


Why FPGAs?

Field Programmable Gate Arrays → Like aPLA but can do much more!

FPGA ASICShort development cy-cle.

Long cycles

High cost. Low cost for large vol-umes.

Reconfigurable Use or throw.Design can be up-graded

Designs carved in sili-con.


Partners

(1) CASPER team at UC Berkeley led by DanWerthimer building FPGA based radioastronomy signal processing technology.


Partners


(2) MIT/Haystack using (1) to build DigitalBack End (DBE) for Mark Vb VLBI storageequipment.


Partners


(2) MIT/Haystack using (1) to build DigitalBack End (DBE) for Mark Vb VLBI storageequipment.

Let us use (1) combine with (2), customizeand build the SMA beam former.


CASPER Technology

iBOB Board based onXilinx Virtex II Pro.Heart of SMA PhasedArray Processor


CASPER Technology


iADC boards plugdirectly into iBOBs andprovide high speedsampling.


CASPER Technology


iADC boards plugdirectly into iBOBs andprovide high speedsampling.

Other boards which wedidn’t need → BEE2CASPER flagship.


iBOB-iADC

Atmel ADC

(AT84AD001),

2Gsamples/sec or

1Gsample/sec


iBOB-iADC

Atmel ADC

(AT84AD001),

2Gsamples/sec or

1Gsample/sec

Xilinx Virtex II

Pro. 5MB SRAM,

≈ 50K logic cells,

2 PPC, Rocket

I/O


iBOB-iADC

Infiniband, VSI,

RS-232, 100BaseT

Ethernet, External

SRAM.


iBOB-iADC

Infiniband, VSI,

RS-232,

100BaseT

Ethernet, External

SRAM.

Symbolic Repre-

sentation.


BEE Design Flow

VHDL → Low

Level


BEE Design Flow

VHDL → Low

Level

Simulink → Higher

Level


BEE Design Flow

VHDL → Low

Level

Simulink → Higher

Level

Fixed Platform →

Details hidden in

design flow


BEE Design Flow

VHDL → Low

Level

Simulink → Higher

Level

Fixed Platform →

Details hidden in

design flow

Berkeley Libraries:

Interface, Radio

Astronomy


Mark Vb DBE

ADC 4x8bits

1024Mhz

8bits

256MHz

FourierTransform32 point

Gain Adjustper bin

VSIBusInterface

Mark Vb Recorder

Infiniband Link

iADC iBOB

Time Domain Frequency Domain

Analog InFromDownconverter

iBOB Based Mark Vb Re cording Interface

VLBI Station 1

VSI

MarkVb


Single Baseline Phased Array

τ

τ

τ

φ

f

Delay adjust.(Geo,Atm,Inst)



τ

τ

τ

φ

φ

f

f

MixMixLO LO

φ1φ2

φ1 − φ2


Delay andphase adjust.



τ

τ

τ

φ

φ

f

f

MixMixLO LO

φ1φ2

φ1 − φ2


Delay andphase adjust.

Delay andphase adjustwith fringerotation.


IF Subsystem

0.5 GHz 1.5 GHz1 GHz

SMA 1st DCV Block Filters (MHz)

528768 1008

10401280 1520

1024 MHz1024 MHzMM

MM

ff

ff


Phased Array Processor

M5 Recorder

DBE

VSI

VLBI IniBOB-1

iBOB-2

Ant 1

Ant 2

Ant 3

Ant 4

Ant 5

Ant 6

Ant 7

Ant 8

≈ 8 Gbps per antenna



M5 Recorder

DBE

VSI

VLBI IniBOB-1

iBOB-2

Ant 1

Ant 2

Ant 3

Ant 4

Ant 5

Ant 6

Ant 7

Ant 8

8 Gbps

8 Gbps



Time Domain Approach

Delay τ1

Delay τ2

Delay τ3

Delay τn

Simple

Accuracy:τmin <<Tsample

Cannot adjustphase


Frequency Domain Approach

FFT N

FFT N

FFT N

FFT N

φ adjust

φ adjust

φ adjust

φ adjust

Complex

Accuracy:

N → Large

Can adjust phase,

hence LO phase

compensate and

fringe rotation can

be done digitally.



4x

Average

Average

Average

Average

Chunk FilterBW=480MHzCenter=760MHz

or Center=1280MHz


or Center=1280MHz


or Center=1280MHz


or Center=1280MHz


or Center=1280MHz

4x

4x

4x

4x1024Mhz

8bits

1024Mhz

8bits

1024Mhz

8bits

1024Mhz

8bits Digital DelayLineAccuracy 0.1ns

XAUI

Xilinx Virtex II Pro vp50iADC 2

iADC 1

Antenna 1Polarization P

Polarization P

Polarization P

Polarization P

Antenna 2

Antenna 3

Antenna 4

BPF

BPF

BPF

BPF

ADC

ADC

ADC

ADC

Demux by 4

Demux by 4

Demux by 4

Demux by 4

8bits

256MHz

8bits

256MHz

8bits

256MHz

8bits

256MHz

Digital DelayLineAccuracy 0.1ns



PPC

Delay Control

Delay Control

Delay Control

Delay Control

8bits

8bits

8bits

8bits

8bits

8bits

8bits

8bits

256MHz

256MHz

256MHz

256MHz

256MHz

256MHz

256MHz

256MHz

10bits

10bits

10bits

10bits

8bits

1024Mhz

To Infinibandconnector

Ts = 11024MHz

≈ 0.99ns

τmin = Ts

10

≈ 0.1ns


Coarse Delay

8bits

CONCAT32bits

FIFO

RD_PTR

WR_PTR

LOGICDELAY

DATA

Max Delay4000 ns

Delay Precision4 ns

Control via PPC.

Delay ↑. 4samples repeat.

Delay ↓. 4 sam-ples skipped.


Fine Delay

9101112

32bits

1234

5678

CONCAT

64bit

12345678

1234

2345

3456

4567

01

23DATA

SELECT

z−1

z−1

Barrel SelectorArrangement

Delay Precision1 ns


Super Fine Delay

1 2 3 4 5 6 7 8 9 10−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

Sample

Mag

nitu

de

Digital Filter for Delay

y(n) = x(n − D)

Hi(z) = z( − D)

hD(n) =sin π(n−D)

π(n−D)

If D is fractional hD(n) becomes afractional delay filter.

FIR approximations of Fractional Delayfilters are not symmetrical.

Precompute Coefficients for fractionaldelays from 0.1 ns to 0.9 ns in steps of0.1 ns.

Load coefficients on demand and imple-ment real time FIR filter using 10 taps.


Super Fine Delay

1 2 3 4 5 6 7 8 9 10−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

Sample

Mag

nitu

de

Digital Filter for Delay

y(n) = x(n − D)

Hi(z) = z( − D)

hD(n) =sin π(n−D)

π(n−D)

If D is fractional hD(n) becomes afractional delay filter.

FIR approximations of Fractional Delayfilters are not symmetrical.

Precompute Coefficients for fractionaldelays from 0.1 ns to 0.9 ns in steps of0.1 ns.

Load coefficients on demand and imple-ment real time FIR filter using 10 taps.


Super Fine Delay

DATA IN

DATA OUT

C1 C2 C3 C4 C5

z−1z−1z−1z−1z−1

Complex designof demux-by-4FIR filter

4 Multiplicationsand 4 partialsums computedin every stage

No. of stages =

No. of Taps


Super Fine Delay

C1

C1

C1

C1

C1

C2

C2

C2

C2

C2

C3

C3

C3

C3

C3

C4

C4

C4

C4

C4

C5

C5

C5

C5

C5

s1s1

s2s2s2

s3 s3s3s3

s4s4 s4s4s4

s5s5s5s5s5

s6s6 s6s6

s7s7s7

s8s8

s9

s10

s11

s12

Complex designof demux-by-4FIR filter

4 Multiplicationsand 4 partialsums computedin every stage

No. of stages =

No. of Taps


It Works too!

0 10 20 30 40 50 60 70 80 90 100

−20

−10

0

10

20

Sample

Mag

Channel 0

0 10 20 30 40 50 60 70 80 90 100−30

−20

−10

0

10

20

Sample

Mag

Channel 1

0 10 20 30 40 50 60 70 80 90 100−30

−20

−10

0

10

20

Mag

Sam

ple

Average


It Works too!

−50 −40 −30 −20 −10 0 10 20 30 40 50

−1

−0.5

0

0.5

1

x 106

Lag

Cor

rela

tion

Correlation


It Works too!

0 10 20 30 40 50 60 70 80 90 100

−20

0

20

Channel 0

Sample

Mag

0 10 20 30 40 50 60 70 80 90 100

−20

0

20

Channel 1

Sample

Mag

0 10 20 30 40 50 60 70 80 90 100

−20

0

20

Average

Sample

Mag


It Works too!

−140 −120 −100 −80 −60 −40 −20 0 20

−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

x 106

Lags

Cor

rela

tion

Correlation


It Works too!

−60 −58 −56 −54 −52 −50

5

6

7

8

9

10

11

x 105

Lags

Cor

rela

tion

Correlation


How to get delays?

Geometry → knownwell enough.


How to get delays?


Atmosphere → needto track


How to get delays?



Noisy Data.


How to get delays?



Noisy Data.

SMA Correlator?


How to get delays?

SMACORRELATOR

SMAPHASEDARRAY PROCESSOR

ANT 1ANT 2ANT 3ANT 4

τ1τ2τ3τ4

τ5τ6τ7τ8



Noisy Data.

SMA Correlator?

Instrumental →different paths.


How to get delays?

SMACORRELATOR

SMAPHASEDARRAY PROCESSOR

ANT 1ANT 2ANT 3ANT 4

τ1τ2τ3τ4

τ5τ6τ7τ8



Noisy Data.

SMA Correlator?

Instrumental →different paths.

Another Correlator?


What sort of Correlator?

7 Delay measurements.




δτ is small! Order of few minutes.





Time Multiplex 7 Measurements.






Single Baseline Correlator!






Single Baseline Correlator!

Berkeley Library.


Correlator

PFB

PFB FFT

FFT

CONJ

X Avg

DATA_IN

DATA_IN

RS_232

DATA_OUT

FX.


Correlator

PFB

PFB FFT

FFT

CONJ

X Avg

DATA_IN

DATA_IN

RS_232

DATA_OUT

FX.

64 Complex/32Real Channel


Correlator

PFB

PFB FFT

FFT

CONJ

X Avg

DATA_IN

DATA_IN

RS_232

DATA_OUT

FX.


Use PFB-FFTfrom CASPER.


Correlator

PFB

PFB FFT

FFT

CONJ

X Avg

DATA_IN

DATA_IN

RS_232

DATA_OUT

FX.



Lots of challenges→ lots of time.


Correlator

PFB

PFB FFT

FFT

CONJ

X Avg

DATA_IN

DATA_IN

RS_232

DATA_OUT

FX.




Full frequencyoperation.


Correlator

PFB

PFB FFT

FFT

CONJ

X Avg

DATA_IN

DATA_IN

RS_232

DATA_OUT

FX.





Dynamic Range.


Correlator

PFB

PFB FFT

FFT

CONJ

X Avg

DATA_IN

DATA_IN

RS_232

DATA_OUT

FX.





Dynamic Range.

Sensitivity


Does it work?

Figure 1: Autocorrelation Vinayak Nagpal, CTH – p. 25/28

Does it work?

Figure 2: SNR = − 2dB Vinayak Nagpal, CTH – p. 25/28

Does it work?

Figure 3: SNR = − 9dB


Does it work?



Does it work?



Does it work?

−500 −400 −300 −200 −100 0 100 200 300 400 5000

10

20

30

40

50

60

70

MHz

dB

−500 −400 −300 −200 −100 0 100 200 300 400 500−3

−2

−1

0

1

2

3

MHz

Pha

se

Figure 6: No Delay Vinayak Nagpal, CTH – p. 25/28

Complete Picture

M5 Recorder

DBE

VSI

VLBI IniBOB-1

iBOB-2

Ant 1

Ant 2

Ant 3

Ant 4

Ant 5

Ant 6

Ant 8

8 Gbps

8 Gbps


Correlator

Correlator

More work needed!

XAUI Linkintegration

DBE interface

CorrelatorSensitivity

Automatic delayextraction

Fringe Rotation

Bandwidth


Conclusion

The SMA Phased Array Processordevelopment has come a long way.

Most major blocks are ready and working.

Still some work is needed before the systemcan be ready for a sky observation.

Latest trends in FPGA technology and theCASPER paradigm are effective in makingdevelopment cycle times shorter!


Acknowledgements

SMA Team: Jonathan Weintroub, Bob Wilson,John Test, Taco, Jim Moran, Ray Blundell,Lincoln Greenhill.

CASPER Team: Dan Werthimer, Melvyn Wright,Aaron Parsons, Pierre Droz, Henry Chen,Patrick Crescini.

MIT/Haystack: Shep Doeleman, Brian Fanous,Alan Rogers, Alan Whitney.

Jon Conway: Onsala Space Observatory, Sweden.

Xilinx Inc.

Synopsys Inc.Vinayak Nagpal, CTH – p. 28/28

Documents

An FPGA based Phased Array Processor for the Sub ...jweintro/vlbi/forbob/Vinayaks Graphics... · An FPGA based Phased Array Processor for the Sub Millimeter ... CASPER team at UC