how (not) to cross-correlate - or: the quest for an ...afialkov/talk_Chung.pdf · 26 27 28 29 30 31 32 33 34 freq (GHz) 0.6 0.4 0.2 0.0 0.2 0.4 0.6 y (deg) 0 2 4 6 8 10 temp (uK)

26 27 28 29 30 31 32 33 34

freq (GHz)

−0.6

−0.4

−0.2

0.0

0.2

0.4

0.6

y(d

eg)

0

2

4

6

8

10

tem

p(u

K)

CO

MA

P

how (not) to cross-correlateor: the quest for an optimal cross-correlation target

for the CO Mapping Array Pathfinder

Dongwoo Chung

Stanford University

Aspen—2018/02/06

(1 / 13) Dongwoo Chung how (not) to cross-correlate Aspen—2018/02/06 1a / 7

CO

MA

P why cross-correlate?extremely valuable to a novel, complex subfield

cross-correlating data fromindependent observations

improves confidence ina tentative detectionpotentially allowsscience not possiblewith either in isolation

21 cm × galaxies: HIcontent of galaxiesCO × Ly-α: molecularfraction of LAEsas many synergies asthere are pairings

4 E. R. Switzer, K. W. Masui, et al.

Figure 1. Temperature scales in our 21 cm intensity mapping survey. Thetop curve is the power spectrum of the input deep field with no cleaningapplied (the wide field is similar). Throughout, the deep field results aregreen and the wide field results are blue. The dotted and dash-dotted linesshow thermal noise in the maps. The power spectra avoid noisebias bycrossing two maps made with separate data sets. The points below showthe power spectrum of the deep and wide fields after the foreground clean-ing described in Section 2.1. Individual modes in the map aredominatedby thermal noise rather than residual foregrounds or signal. Errors are thethermal noise power divided by the number of modes in thek-bin, plussample variance. The negative values are shown with thin lines and hollowmarkers. The red dashed line shows the 21 cm signal expected from the am-plitude of the cross-power with the WiggleZ survey (forr = 1) and basedon simulations processed by the same pipeline.

matrix square root of the inverse of our covariance model ma-trix and normalize its rows to sum to one. This provides a set offunctions which decorrelates (Hamilton & Tegmark 2000) thepre-whitened power spectrum and boosts the errors. At large scales(k = 0.1 hMpc−1) where these effects are relevant, decorrela-tion and sample variance increase the errors by a factor of 1.5 inthe wide field and 4 in the deep field.

3 RESULTS

The auto-power spectra presented in Fig. 1 will be biased by an un-known positive amplitude from residual foreground contamination.These data can then be interpreted as an upper bound on the neutralhydrogen fluctuation amplitude,ΩHIbHI. In addition, we have alsomeasured the cross-correlation with the WiggleZ galaxy survey(Masui et al. 2013). This findsΩHIbHIr = [0.43 ± 0.07(stat.) ±0.04(sys.)] × 10−3, wherer is the WiggleZ galaxy-neutral hydro-gen cross-correlation coefficient (taken here to be independent ofscale). Since|r| < 1 by definition and is measured to be posi-tive, the cross-correlation can be interpreted as a lower bound onΩHIbHI. In this section, we will develop a posterior distribution forthe 21 cm signal auto-power between these two bounds, as a func-tion of k. We will then combine these into a posterior distributiononΩHIbHI.

The probability of our measurements given the 21 cm signalauto-power and foreground model parameters is

p(dk|θk) = p(dc|sk, r)p(ddeepk |sk, fdeepk )p(dwide

k |sk, fwidek ). (2)

Figure 2. Comparison with the thermal noise limit. The dark and lightshaded regions are the 68% and 95% confidence intervals of themeasured21 cm fluctuation power from equation (3). The dashed line shows the ex-pected 21 cm signal implied by the WiggleZ cross-correlation if r = 1.The solid line represents the best upper95% confidence level that we couldachieve given our error bars in both fields, in the absence of foreground con-tamination. Note that the autocorrelation measurements, which constrainthe signal from above, are uncorrelated betweenk-bins, while a singleglobal fit to the cross-power (in Masui et al. (2013)) is used to constrainthe signal from below. Confidence intervals do not include the systematiccalibration uncertainty, which is 18% in this space.

Here, dk = dc, ddeepk , dwidek contains our cross-power and

deep and wide field auto-power measurements, whileθk =sk, r, fdeep

k , fwidek contains the 21 cm signal auto-power, cross-

correlation coefficient, and deep and wide field foreground con-tamination powers, respectively. The cross-power variable dc rep-resents the constraint onΩHIbHIr from both fields and the range ofwavenumbers used in Masui et al. (2013). The band-powersddeepk

anddwidek are independently distributed following decorrelation of

finite-survey effects. We assume that the foregrounds are uncorre-lated betweenk-bins and fields, also. This is conservative becauseknowledge of foreground correlations would yield a tightercon-straint. We takep(dc|sk, r) to be normally distributed with meanproportional tor

√sk, andp(ddeepk |sk, fdeep

k ) to be normally dis-tributed with meansk + fdeep

k and errors determined in Sec 2.3(and analogously for the wide field). Only the statistical uncertaintyis included in the width of the distributions, as the systematic cali-bration uncertainty is perfectly correlated between cross- and auto-power measurements and can be applied at the end of the analysis.

We apply Bayes’ theorem to obtain the pos-terior distribution for the parameters,p(θk|dk) ∝p(dk|θk)p(sk)p(r)p(f

deepk )p(fwide

k ). For the nuisance pa-rameters, we adopt conservative priors.p(fdeep

k ) and p(fwidek )

are taken to be flat over the range0 < fk < ∞. Likewise, wetake p(r) to be constant over the range0 < r < 1, which isconservative given the theoretical bias towardsr ≈ 1. Our goalis to marginalize over these nuisance parameters to determine sk.We choose the prior onsk, p(sk), to be flat, which translates into aprior p(ΩHIbHI) ∝ ΩHIbHI. The signal posterior is

p(sk|dk) =

∫p(sk, r, f

deepk , fwide

k |dk) dr dfdeepk dfwide

k . (3)

This involves integrals of the form∫ 1

0p(dc|s, r)p(r) dr which,

given the flat priors that we have adopted, can generally be writ-ten in terms of the cumulative distribution function ofp(dc|s, r).Fig. 2 shows the allowed signal in each spectralk-bin.

Taking the analysis further, we combine band-powers into asingle constraint onΩHIbHI. Following Masui et al. (2013), we

c© 2013 RAS, MNRAS000, 1–??

4 E. R. Switzer, K. W. Masui, et al.

Figure 1. Temperature scales in our 21 cm intensity mapping survey. Thetop curve is the power spectrum of the input deep field with no cleaningapplied (the wide field is similar). Throughout, the deep field results aregreen and the wide field results are blue. The dotted and dash-dotted linesshow thermal noise in the maps. The power spectra avoid noisebias bycrossing two maps made with separate data sets. The points below showthe power spectrum of the deep and wide fields after the foreground clean-ing described in Section 2.1. Individual modes in the map aredominatedby thermal noise rather than residual foregrounds or signal. Errors are thethermal noise power divided by the number of modes in thek-bin, plussample variance. The negative values are shown with thin lines and hollowmarkers. The red dashed line shows the 21 cm signal expected from the am-plitude of the cross-power with the WiggleZ survey (forr = 1) and basedon simulations processed by the same pipeline.

matrix square root of the inverse of our covariance model ma-trix and normalize its rows to sum to one. This provides a set offunctions which decorrelates (Hamilton & Tegmark 2000) thepre-whitened power spectrum and boosts the errors. At large scales(k = 0.1 hMpc−1) where these effects are relevant, decorrela-tion and sample variance increase the errors by a factor of 1.5 inthe wide field and 4 in the deep field.

3 RESULTS

The auto-power spectra presented in Fig. 1 will be biased by an un-known positive amplitude from residual foreground contamination.These data can then be interpreted as an upper bound on the neutralhydrogen fluctuation amplitude,ΩHIbHI. In addition, we have alsomeasured the cross-correlation with the WiggleZ galaxy survey(Masui et al. 2013). This findsΩHIbHIr = [0.43 ± 0.07(stat.) ±0.04(sys.)] × 10−3, wherer is the WiggleZ galaxy-neutral hydro-gen cross-correlation coefficient (taken here to be independent ofscale). Since|r| < 1 by definition and is measured to be posi-tive, the cross-correlation can be interpreted as a lower bound onΩHIbHI. In this section, we will develop a posterior distribution forthe 21 cm signal auto-power between these two bounds, as a func-tion of k. We will then combine these into a posterior distributiononΩHIbHI.

The probability of our measurements given the 21 cm signalauto-power and foreground model parameters is

p(dk|θk) = p(dc|sk, r)p(ddeepk |sk, fdeepk )p(dwide

k |sk, fwidek ). (2)

Figure 2. Comparison with the thermal noise limit. The dark and lightshaded regions are the 68% and 95% confidence intervals of themeasured21 cm fluctuation power from equation (3). The dashed line shows the ex-pected 21 cm signal implied by the WiggleZ cross-correlation if r = 1.The solid line represents the best upper95% confidence level that we couldachieve given our error bars in both fields, in the absence of foreground con-tamination. Note that the autocorrelation measurements, which constrainthe signal from above, are uncorrelated betweenk-bins, while a singleglobal fit to the cross-power (in Masui et al. (2013)) is used to constrainthe signal from below. Confidence intervals do not include the systematiccalibration uncertainty, which is 18% in this space.

Here, dk = dc, ddeepk , dwidek contains our cross-power and

deep and wide field auto-power measurements, whileθk =sk, r, fdeep

k , fwidek contains the 21 cm signal auto-power, cross-

correlation coefficient, and deep and wide field foreground con-tamination powers, respectively. The cross-power variable dc rep-resents the constraint onΩHIbHIr from both fields and the range ofwavenumbers used in Masui et al. (2013). The band-powersddeepk

anddwidek are independently distributed following decorrelation of

finite-survey effects. We assume that the foregrounds are uncorre-lated betweenk-bins and fields, also. This is conservative becauseknowledge of foreground correlations would yield a tightercon-straint. We takep(dc|sk, r) to be normally distributed with meanproportional tor

√sk, andp(ddeepk |sk, fdeep

k ) to be normally dis-tributed with meansk + fdeep

k and errors determined in Sec 2.3(and analogously for the wide field). Only the statistical uncertaintyis included in the width of the distributions, as the systematic cali-bration uncertainty is perfectly correlated between cross- and auto-power measurements and can be applied at the end of the analysis.

We apply Bayes’ theorem to obtain the pos-terior distribution for the parameters,p(θk|dk) ∝p(dk|θk)p(sk)p(r)p(f

deepk )p(fwide

k ). For the nuisance pa-rameters, we adopt conservative priors.p(fdeep

k ) and p(fwidek )

are taken to be flat over the range0 < fk < ∞. Likewise, wetake p(r) to be constant over the range0 < r < 1, which isconservative given the theoretical bias towardsr ≈ 1. Our goalis to marginalize over these nuisance parameters to determine sk.We choose the prior onsk, p(sk), to be flat, which translates into aprior p(ΩHIbHI) ∝ ΩHIbHI. The signal posterior is

p(sk|dk) =

∫p(sk, r, f

deepk , fwide

k |dk) dr dfdeepk dfwide

k . (3)

This involves integrals of the form∫ 1

0p(dc|s, r)p(r) dr which,

given the flat priors that we have adopted, can generally be writ-ten in terms of the cumulative distribution function ofp(dc|s, r).Fig. 2 shows the allowed signal in each spectralk-bin.

Taking the analysis further, we combine band-powers into asingle constraint onΩHIbHI. Following Masui et al. (2013), we

c© 2013 RAS, MNRAS000, 1–??

Figure: Switzer+13 (arXiv:1304.3712)


CO

MA

P COMAP Phase ICO(1-0), z = 2.4–3.4, degree-scale field

10 15 20 25 30 35 40frequency (GHz)

0

2

4

6

8

10re

dshi

ft

CO(1-0)z=2.4-3.4

COMAPKa-band

19-pixel, single-pol, heterodyne, 26–34 GHztargeting initial detection of CO clustering signalnoise dominates errors, not sample variance


CO

MA

P available targetswhat to cross-correlate with at z ∼ 3?

line-intensity surveys

σz 0.01(1 + z)many examples represented here, many coming online soonpossible in future with coordination, but what’s available now?

spectroscopic galaxy surveys

σz 0.01(1 + z)excellent at z . 1, but depth/area fall off for z & 2should improve with future surveys on e.g. PFS, but for now ...

photometric galaxy surveys

e.g. COSMOS2015 (Laigle+16)sizeable catalogue with good source abundance even at z & 2, but ...σz & 0.01(1 + z)—how does this affect signal?

are these encouraging cross-correlation targets for COMAP?


CO

MA


line-intensity surveysσz 0.01(1 + z)many examples represented here, many coming online soonpossible in future with coordination, but what’s available now?

spectroscopic galaxy surveys

σz 0.01(1 + z)excellent at z . 1, but depth/area fall off for z & 2should improve with future surveys on e.g. PFS, but for now ...




(5 / 13) Dongwoo Chung how (not) to cross-correlate Aspen—2018/02/06 4b / 7

CO

MA



spectroscopic galaxy surveysσz 0.01(1 + z)excellent at z . 1, but depth/area fall off for z & 2should improve with future surveys on e.g. PFS, but for now ...




(6 / 13) Dongwoo Chung how (not) to cross-correlate Aspen—2018/02/06 4c / 7

CO

MA



spectroscopic galaxy surveysσz 0.01(1 + z)excellent at z . 1, but depth/area fall off for z & 2should improve with future surveys on e.g. PFS, but for now ...

photometric galaxy surveyse.g. COSMOS2015 (Laigle+16)sizeable catalogue with good source abundance even at z & 2, but ...σz & 0.01(1 + z)—how does this affect signal?


(7 / 13) Dongwoo Chung how (not) to cross-correlate Aspen—2018/02/06 4d / 7

CO

MA

P COMAP × galaxy surveys?presenting the photo-z problem

even limits inspectroscopic redshiftprecision can attenuatethe cross spectrum

but the effect is moresevere for galaxyredshift errors ofσz & 0.01(1 + z)

attenuating crossspectrum by 3–10×(enough for pointingCOMAP at COSMOS tono longer makeoverwhelming sense)

10−1 100

k (1/Mpc)

−0.2

0.0

0.2

0.4

0.6

0.8

1.0

r(k)

perfect

σz/(1 + z) = 0σz/(1 + z) = 0.0007

σz/(1 + z) = 0.003σz/(1 + z) = 0.02

Figure: DTC+ in prep


CO

MA


even limits inspectroscopic redshiftprecision can attenuatethe cross spectrum

but the effect is moresevere for galaxyredshift errors ofσz & 0.01(1 + z)


10−1 100

k (1/Mpc)

−0.2

0.0

0.2

0.4

0.6

0.8

1.0

r(k)

perfect

PFS-like

σz/(1 + z) = 0σz/(1 + z) = 0.0007

σz/(1 + z) = 0.003σz/(1 + z) = 0.02


(9 / 13) Dongwoo Chung how (not) to cross-correlate Aspen—2018/02/06 5b / 7

CO

MA


even limits inspectroscopic redshiftprecision can attenuatethe cross spectrumbut the effect is moresevere for galaxyredshift errors ofσz & 0.01(1 + z)


10−1 100

k (1/Mpc)

−0.2

0.0

0.2

0.4

0.6

0.8

1.0

r(k)

perfect

PFS-like

WFC3-like

σz/(1 + z) = 0σz/(1 + z) = 0.0007

σz/(1 + z) = 0.003σz/(1 + z) = 0.02


(10 / 13) Dongwoo Chung how (not) to cross-correlate Aspen—2018/02/06 5c / 7

CO

MA


even limits inspectroscopic redshiftprecision can attenuatethe cross spectrumbut the effect is moresevere for galaxyredshift errors ofσz & 0.01(1 + z)


10−1 100

k (1/Mpc)

−0.2

0.0

0.2

0.4

0.6

0.8

1.0

r(k)

perfect

PFS-like

WFC3-likephoto-z

σz/(1 + z) = 0σz/(1 + z) = 0.0007

σz/(1 + z) = 0.003σz/(1 + z) = 0.02


(11 / 13) Dongwoo Chung how (not) to cross-correlate Aspen—2018/02/06 5d / 7

CO

MA

P COMAP × HETDEX?both as LAE survey and as LIM survey

natural target forcross-correlation

redshift overlap (z ∼ 3)both instruments northof the equator

initial projectionspromising!

best detection may befrom using HETDEX asa Ly-α intensity mapperbut could still obtaindetection for COMAP ×HETDEX LAE

10−1 100

k (1/Mpc)

−0.2

0.0

0.2

0.4

0.6

0.8

1.0

r(k)

COMAP x perfect redshift surveyCOMAP x HETDEX LIM, 1/4 fillCOMAP x LAE (L ≥ 3.0× 1042 erg s−1), 1/4 fillCOMAP x photo-z catalogue, σz/(1 + z) = 0.02



CO

MA

P in summarya few general takeaways

designing our surveys to enable cross-correlation will becrucial for validation and for multi-tracer sciencemedium/broad-band photometric surveys form much of theancillary data currently available at z & 2, but line-intensitysurveys relying on finer line-of-sight information should insteadlook forward to near-future spectroscopic surveysnever too early to start conversations on coordinating efforts toenable/propose wide-field (& deg2) spectroscopic surveys atintermediate redshifts (z ∼ 3) and nearer to reionisation (z & 6)


CO

MA

P backup slides!why? what’d I miss?


CO

MA

P photo-z: quick solutions?no such thing as a free lunch

coarsen CO cube in thez-direction?

gains signal by ‘integratingaway’ photo-z errors ...... but loses signal bydiscarding line-of-sightinformation

clustering redshifts?usually requiresspectroscopic samplemore sensible tocross-correlate againstthat sample to start with?

101 102 103

number of channels over 26-34 GHz

2

3

4

5

6

7

8

9

tota

lsig

nal-t

o-no

ise

CO auto spectrumCO-galaxy cross spectrum(σz/(1 + z) = 0.02)


CO

MA

P Lyman-alpha modelsimplistic L(Mh, z) relation w/ some physical motivation

basic form:

LLyα ∝ SFR ·fesc(SFR, z)

use Behroozi+13SFR(Mh, z) relationuse fine-tuned escapefraction function stronglyevolving with SFR/mass

tuned to Sobral+17 (z ∼ 2)and Gronwall+07 (z ∼ 3)

built for intermediateredshifts z ∈ (2, 5)

1010 1011 1012 1013

halo mass (Msol)

1040

1041

1042

1043

L(Ly

) (er

g/s)

z = 2.0z = 2.5z = 3.0z = 3.5

10 2 10 1 100 101 102

SFR (Msol/yr)

1040

1041

1042

1043

L(Ly

) (er

g/s)

z = 2.0z = 2.5z = 3.0z = 3.5

1010 1011 1012 1013

halo mass (Msol)

0.0

0.2

0.4

0.6

0.8

1.0

Ly/U

V es

cape

frac

tion

z = 2.0z = 2.5z = 3.0z = 3.5

10 2 10 1 100 101 102

SFR (Msol/yr)

0.0

0.2

0.4

0.6

0.8

1.0

Ly/U

V es

cape

frac

tion

z = 2.0z = 2.5z = 3.0z = 3.5


CO

MA

P Lyman-alpha modelcomparison against S-SC4K LAE LFs (Sobral+18)

1041 1042 1043 1044

LLy (erg/s)

10 5

10 4

10 3

10 2

10 1

(L) [

Mpc

3 (lo

gL)

1 ]

z = 2.2S-SC4KStanford model

1041 1042 1043 1044

LLy (erg/s)

10 5

10 4

10 3

10 2

10 1

(L) [

Mpc

3 (lo

gL)

1 ]


1041 1042 1043 1044

LLy (erg/s)

10 5

10 4

10 3

10 2

10 1

(L) [

Mpc

3 (lo

gL)

1 ]


1041 1042 1043 1044

LLy (erg/s)

10 5

10 4

10 3

10 2

10 1

(L) [

Mpc

3 (lo

gL)

1 ]


1041 1042 1043 1044

LLy (erg/s)

10 5

10 4

10 3

10 2

10 1

(L) [

Mpc

3 (lo

gL)

1 ]


1041 1042 1043 1044

LLy (erg/s)

10 5

10 4

10 3

10 2

10 1

(L) [

Mpc

3 (lo

gL)

1 ]



Documents

how (not) to cross-correlate - or: the quest for an ...afialkov/talk_Chung.pdf · 26 27 28 29 30 31 32 33 34 freq (GHz) 0.6 0.4 0.2 0.0 0.2 0.4 0.6 y (deg) 0 2 4 6 8 10 temp (uK)