MHz Digital Frequency Domain Multiplexer - CMBPol

MHz Digital Frequency Domain MHz Digital Frequency Domain Multiplexer Multiplexer Matt DobbsMatt Dobbs

McGill UniversityMontreal, Canada

Berkeley (LBNL/UC) (original Analog fMux, cold components for all systems)• John Clarke, Bill Holzapfel, Adrian Lee, Paul Richards, Helmuth Spieler• Sherry Cho ( NIST)• Trevor Lanting ( McGill), Martin Lueker, Tom Plagge• John Joseph, Chinh VuMcGill (new Digital Backend, high BW SQUIDs)• Francois Aubin, Peter Hyland, Kevin MacDermid• Eric Bissonnette, Graeme SmecherMinnesota (cooling, striplines, bolo integration) NIST (SA SQUIDs, Inductors)• Shaul Hanany, Hannes Hubmayr ● Gene Hilton, Kent Irwin

[email protected]@McGill.ca, , CMBpolCMBpol Aug 2008Aug 2008 2

fMuxfMux in the Fieldin the Field

EBEX1500 TES, 2009Digital fMux

SPT1000 TES, 2007

Analog fMux

POLARBEAR1200 TES, 2009

Digial fMux

APEX320 TES, 2006

Analog fMux


fMUX Readout SystemfMUX Readout SystemStaged Development and DeploymentStaged Development and Deployment

APEX-SZ320 Channels2005, Chile

POLARBEAR1200 Channels

South Pole Telescope1000 Channels

2006/7EBEX

1500 ChannelsLD Balloon

AC-bias, Scalable SystemMUX Demonstration

Full MultiplexerLow Power, Flight Worthy2nd Generation

Digital


SQUID

250 mK

TES

L

G

Optical Power

Basic TES ReadoutBasic TES Readout

Vbias

Warm Bi-polaramplifier

Vout


SQUID

250 mK

TES

L

G

Optical Power

Basic TES ReadoutBasic TES Readout

Vbias

Warm Bi-polaramplifier

Vout

AC


MHz Frequency Domain MultiplexerMHz Frequency Domain Multiplexer

f500 kHz

f500 kHz

200 Hz

80 dBc

Amp-Modulated by BoloDDSAmplified

600 kHz 600 kHz f100 Hz

Demodulated

500 kHz

600 kHz

f100 Hz

f500 kHz

200 Hz

80 dBc

600 kHz

Shunt FBSQUID


MHz Frequency Domain MultiplexerMHz Frequency Domain MultiplexerAdvantage #1: Simple architecture.

Continuous readout, no switching transients.Only 1 stage of SQUIDs. 1 SA-Squid per module.No SQUIDs in focal plane – straight forward magnetic shielding.

Easy SQUID setup/tuning, easy to understand.

Wire count:Two sub-Kelvin wiresper module.


fMuxfMux –– Cryostat ComponentsCryostat Components

MUX Inductors

TES WedgeSQUID board

Cryoperm SQUID Shield

4K room Wiring

SQUID Controller(room temp)


Cold Component AdvantagesCold Component AdvantagesNO power dissipated on subNO power dissipated on sub--Kelvin stageKelvin stage..

Readout system components are LReadout system components are L’’s and Cs and C’’s only.s only.Optimized for Optimized for RRnormalnormal=0.5=0.5--1 1 ΩΩ

Small impedance of connectors between bolos & SQUIDs not an issue.

No bump bonds or complexity in the interface.Individual bias levels for each bolometerIndividual bias levels for each bolometer

Shifts system complexity away from bolo Shifts system complexity away from bolo fabfab..Sky signals are modulated at MHzSky signals are modulated at MHz

Well above Well above microphonicsmicrophonicsNo direct readout sensitivity at baseband, where SQUID is sensitNo direct readout sensitivity at baseband, where SQUID is sensitive to ive to magnetic fieldsmagnetic fields

Only Only one SQUID stageone SQUID stageLives at 4K, easy to shield.Lives at 4K, easy to shield.No SQUIDs on focal plane.No SQUIDs on focal plane.Easy to characterize. No multiple tunings.Easy to characterize. No multiple tunings.Gain is set by warm FLL feedback resistor Gain is set by warm FLL feedback resistor –– not necessary to characterize RO not necessary to characterize RO gain channel by channel.gain channel by channel.


11stst Generation Analog MUXGeneration Analog MUX

SQUID

2 modules of 8 channelseach

4 modules of 16 channelseach

New Digital MUXNew Digital MUXAdvantage: Complete modularity.

1 SA-Squid = 1 Module.No columns/rows or cross bars.

If one component fails, you loose one module only.


Channel Spacing/CountChannel Spacing/Count

LLMUXMUX dictated by bolometer time dictated by bolometer time constants.constants.

If If ττ TESTES << << ττopticaloptical, channels will , channels will need to be widely spaced.need to be widely spaced.Presently we use LPresently we use LMUXMUX = 16 = 16 μμHHThis can be reduced by factor 3This can be reduced by factor 3--5 5 by careful tuning of bolometer by careful tuning of bolometer time constants.time constants.

ΔΔff chosen to ensure thatchosen to ensure thatJohnson noise leakage from Johnson noise leakage from neighboring channels is small neighboring channels is small (presently 2%)(presently 2%)Neighbor cross talk is small Neighbor cross talk is small (presently < 10(presently < 10--33))Bias leakage from neighbors is Bias leakage from neighbors is <10% (spoils voltage bias)<10% (spoils voltage bias)

With 16 With 16 μμHH inductors, this inductors, this spacing is 75 kHz today.spacing is 75 kHz today.

Channel count is just BW/spacingChannel count is just BW/spacing

BW is limited by SQUID FLL, BW is limited by SQUID FLL, which presently uses a 4K SQUID which presently uses a 4K SQUID

300K amplifier feedback.300K amplifier feedback.

Room for substantial Room for substantial improvementimprovement

Optimize bolo Optimize bolo ττCould envision handling bias Could envision handling bias leakage with active feedbackleakage with active feedbackGreatly increase BW with Greatly increase BW with alternate SQUID FLL (factor 10 alternate SQUID FLL (factor 10 has already been achieve in the has already been achieve in the lab lab ““LISALISA””))

Advantage #3: System is “young”, with established directions for improvement on short timescale if adequately funded.

No fundamental limit to the number of MUX-ed channels per comb.(one might argue for at least a factor 10 in modularity – target 100?)


Electronics AnatomyElectronics Anatomy

Two MezzaninesPer MotherboardProvide 4 MUXCombs.

Advantage (AGAIN): Complete modularity.All control + smarts implemented onembedded processor.

His board needs NOTHING from the outside world except a clock and power to tune its module and keep it going. There is no special master board.


System SetupSystem SetupNetwork AnalysisNetwork Analysis

System measures the System measures the complex impedance of the complex impedance of the MUX combMUX comb

Accounts for changes in C as Accounts for changes in C as system coolssystem cools

Chooses bias points.Chooses bias points.

Takes about 15 minTakes about 15 minNo attempt has been made to No attempt has been made to make this fast.make this fast.

This algorithm needs to be This algorithm needs to be run ONCE in the lifetime of run ONCE in the lifetime of each comb.each comb.


System SetupSystem SetupSQUID TuningSQUID Tuning

Only one SAOnly one SA--SQUID to tune SQUID to tune per combper comb

Trace out VTrace out V--phi curve as a phi curve as a function of function of IIbiasbias..

Choose optimal bias point Choose optimal bias point and lock FLL.and lock FLL.

Long version (once per Long version (once per SQUID lifetime) takes 2 SQUID lifetime) takes 2 minutes.minutes.

Short version (once per Short version (once per SQUID thermal cycle) takes SQUID thermal cycle) takes ½½--1 min. Further 1 min. Further improvements possible.improvements possible.


System SetupSystem SetupBolo TuningBolo Tuning

Heat or zap bolos normal.Heat or zap bolos normal.

OverOver--bias bolometers, and bias bolometers, and null carriersnull carriers

Drop bolos into transition, Drop bolos into transition, mapping out Imapping out I--V curveV curve

Can also map out complex Can also map out complex impedance impedance Z(Z(ωω), to measure ), to measure response function and bolo response function and bolo time constants.time constants.


System Performance System Performance -- NoiseNoiseComponent of white noise from Component of white noise from Readout system is enhanced by Readout system is enhanced by factor factor √√22

true for any AC biased systemtrue for any AC biased systemIn a well optimized system the In a well optimized system the contribution of this noise should contribution of this noise should be negligible.be negligible.

Digital system has significant Digital system has significant improvement in 1/f noiseimprovement in 1/f noise

Knee <= 100 mHz, dominated by Knee <= 100 mHz, dominated by DAC output DAC output transisitorstransisitors..This noise is COMMON mode This noise is COMMON mode across a comb.across a comb.Can monitor it with dark channel Can monitor it with dark channel or subtract with the typical or subtract with the typical common mode subtraction used common mode subtraction used for temperature fluctuations and for temperature fluctuations and atmosphere.atmosphere.

SPT 2008, Measured noise on sky.(E. Shirokof, ASC 2008 talk)

fMuxfMux noise demonstrated noise demonstrated with SPT.with SPT.


System Performance System Performance -- NoiseNoiseComponent of white noise from Component of white noise from Readout system is enhanced by Readout system is enhanced by factor factor √√22

true for any AC biased systemtrue for any AC biased systemIn a well optimized system the In a well optimized system the contribution of this noise should contribution of this noise should be negligible.be negligible.

Digital system has significant Digital system has significant improvement in 1/f noiseimprovement in 1/f noise

Knee <= 100 mHz, dominated by Knee <= 100 mHz, dominated by DAC output DAC output transisitorstransisitors..This noise is COMMON mode This noise is COMMON mode across a comb.across a comb.Can monitor it with dark channel Can monitor it with dark channel or subtract with the typical or subtract with the typical common mode subtraction used common mode subtraction used for temperature fluctuations and for temperature fluctuations and atmosphere.atmosphere.

Noise spectra for EBEX bolometers(H. Hubmayr, see poster)

fMuxfMux noise demonstrated noise demonstrated with SPT with SPT Lab demonstrations of Lab demonstrations of Digital system.Digital system.


DfMuxDfMux Size/Form FactorSize/Form FactorOne 6U VME crate houses 20 One 6U VME crate houses 20 DfMuxDfMuxboards (80 modules)boards (80 modules)

16x MUX = 1280 channels16x MUX = 1280 channels

No wire length restrictionsNo wire length restrictions

Only connection to outside is via Ethernet.Only connection to outside is via Ethernet.

Interfaces to 10 SQUID Control boards Interfaces to 10 SQUID Control boards mounted on side of cryostat in small (7mounted on side of cryostat in small (7””cube) RF enclosurescube) RF enclosures

Present design requires short wire separation Present design requires short wire separation from SQUIDs, 20cm.from SQUIDs, 20cm.

Open loop SQUID operation (with active Open loop SQUID operation (with active nulling), or nulling), or LinearizedLinearized SQUID arrays (LISA) SQUID arrays (LISA) would remove this wire length requirement.would remove this wire length requirement.


Tech Challenge: Wiring InductanceTech Challenge: Wiring InductanceFor passive voltage bias, RFor passive voltage bias, RBOLOBOLO

must dominate.must dominate.

Wiring Wiring LLstraystray cannot be tuned cannot be tuned away.away.

Analog systems used short lengths Analog systems used short lengths of of NbNb/Ti twisted pair (/Ti twisted pair (bad,badbad,bad).).

New system uses New system uses NbNb striplinesstriplinesdeveloped at Minnesota.developed at Minnesota.

Keeps Keeps LLstraystray negligible for negligible for RRbolobolo ≈≈ ½½ ΩΩ

Allows for long wires from Allows for long wires from subKelvinsubKelvinto 4K.to 4K.


Tech Challenge: Wire LengthsTech Challenge: Wire LengthsPresently SQUID FLL includes Presently SQUID FLL includes components at 300K and 4Kcomponents at 300K and 4Kphase delay along these wires restricts phase delay along these wires restricts wiring to 20cm (ouch wiring to 20cm (ouch –– big heat load).big heat load).Real issue: preReal issue: pre--amplifier bandwidthamplifier bandwidth

(my favorite) Strategies:(my favorite) Strategies:Move some (or all) of the feedback to 4KMove some (or all) of the feedback to 4K

SQUID OpSQUID Op--ampampLinearizedLinearized SQUID Array SQUID Array

Run the squids open loop Run the squids open loop –– and keep the bias and keep the bias carriers tiny.carriers tiny.

Digital feedback to control nulling amplitudeDigital feedback to control nulling amplitude


Tech Challenge: Power ConsumptionTech Challenge: Power Consumption

Power scales with number of modules(!)Power scales with number of modules(!)DSP power per Watt for DSP power per Watt for FPGAsFPGAs is increasing very fast.is increasing very fast.

Substantial power savings could be achieved by:Substantial power savings could be achieved by:Reducing data conversion rate from 25 MHz to 5 MHz.Reducing data conversion rate from 25 MHz to 5 MHz.Increasing number of channels/Modules. Increasing number of channels/Modules.

32 32 chanchan/module = 130 W//module = 130 W/kilopixelkilopixel128 128 chanchan/module = 33 W//module = 33 W/kilopixelkilopixel

Achieving this requires careful control of bolo time constants aAchieving this requires careful control of bolo time constants and/or a nd/or a larger scale engineering effort on par with the UCB TDM effort.larger scale engineering effort on par with the UCB TDM effort.

Predicted Measured Per Mod [W]# MUX Mods [W] [W] [W]

DfMUX FPGA Motherboard 4 5.8 6.1 1.5DfMUX Analog Mezzanine 2 4.0 4.3 2.2MUX SQUID Controller 8 4.1 4.0 0.5

total per MUX module 4.175

total for 32 modules (384 channels assuming 12x MUX) 134Power consumption per kilopixel 348


Space QualificationSpace QualificationSQUIDs:SQUIDs:

Flight qualified for Gravity Probe B.Flight qualified for Gravity Probe B.

Need radiation hardness test of NIST SANeed radiation hardness test of NIST SA--SquidsSquids

FPGAsFPGAs::Xilinx V4 family already has flight qualified line, V4QV. IncludXilinx V4 family already has flight qualified line, V4QV. Includes es device 20% larger than that used at McGill (LX200).device 20% larger than that used at McGill (LX200).

McGill/COM DEV funded through CSA to develop firmware SEL/SEU McGill/COM DEV funded through CSA to develop firmware SEL/SEU error detection and correction technology.error detection and correction technology.

A/D and D/A are biggest challengeA/D and D/A are biggest challenge25 MHz devices used in McGill 25 MHz devices used in McGill DfMuxDfMux system not flight qualified.system not flight qualified.

In principle, no features exclude this possibility In principle, no features exclude this possibility –– perhaps even for the perhaps even for the devices we use.devices we use.

our requirements are different from most commercial/communicatioour requirements are different from most commercial/communications ns markets.markets.

Technical readiness (via Technical readiness (via BenfordBenford method) method) ≡≡ 4.84.8


SummarySummaryAdvantagesAdvantages

Simple. True modularity.Simple. True modularity.No subNo sub--Kelvin power dissipationKelvin power dissipation

Set bolo biases individuallySet bolo biases individuallySkySky--signals modulated above signals modulated above microphonicsmicrophonics and magnetic and magnetic pickuppickupOnly one SQUID stage (4K)Only one SQUID stage (4K)No formal limit on # MUX No formal limit on # MUX channels/module.channels/module.

Primary challenge is FLL Primary challenge is FLL bandwidth/wire lengthbandwidth/wire length

ChallengesChallenges

Power consumptionPower consumption1/f D/A noise at 100mHz1/f D/A noise at 100mHz

But purely common mode across But purely common mode across comb.comb.

11stst Generation Analog MUX in Generation Analog MUX in field for SPT/APEXfield for SPT/APEX

achieving excellent noise achieving excellent noise performance.performance.

New warm electronics deploying New warm electronics deploying for EBEX/for EBEX/PolarbearPolarbear

Relatively young.Relatively young.Scales with Scales with numbrnumbr of of MuxMuxModulesModulesImproved 1/f noise (<0.1Hz)Improved 1/f noise (<0.1Hz)10x less power, 10x smaller10x less power, 10x smaller

Documents

MHz Digital Frequency Domain Multiplexer - CMBPol